Note: The job is a remote job and is open to candidates in USA. ITRex Group is a pioneering AI company that builds real-world systems and solutions. They are seeking an AI Inference Engineer with strong C++ expertise to deploy and optimize AI models for production, focusing on integrating and enhancing existing products with machine learning capabilities.

Responsibilities

• Work on deploying machine learning models to edge devices using the frameworks: llama.cpp, ggml
• Collaborate closely with researchers to assist in coding, training and transitioning models from research to production environments
• Integrate AI features into existing products, enriching them with the latest advancements in machine learning

Skills

• 4+ years of professional experience in Modern C++ (C++17/20)
• Strong knowledge of memory management, multithreading, profiling and performance optimization
• Experience debugging low-level issues (memory leaks, fragmentation, OOM, concurrency)
• Experience working with Linux development environments
• Experience integrating machine learning models into production applications
• Experience deploying and optimizing AI inference pipelines
• Hands-on experience with AI inference frameworks such as: llama.cpp (strong plus), ggml (strong plus), ONNX Runtime, TensorRT / TensorRT-LLM, OpenVINO, MLC LLM, ExecuTorch, TVM
• Experience profiling inference performance and optimizing memory usage and latency
• Strong understanding of modern AI model architectures, including: Transformer architecture, Large Language Models (LLMs), Diffusion Models, Tokenization, Attention mechanisms, KV Cache, Quantization techniques, Model conversion and deployment
• Experience working with one or more of the following: LLM deployment, Computer Vision models, OCR models, Multimodal models, Speech models, Image generation models
• Experience evaluating new models and integrating them into existing products is highly desirable
• CUDA
• Vulkan Compute
• Metal
• OpenCL
• Typescript
• Python
• Experience contributing to open-source AI infrastructure projects

Benefits

• Remote flexibility: Work where and how you work best - we trust you to deliver
• Fair compensation: Competitive salary + benefits that matter (medical, learning)
• Ownership opportunities: See a problem worth solving? Own it. We back smart risks over bureaucratic safety
• AI enhancement: We leverage AI to make you faster and stronger - complementing your abilities, not replacing them
• Learning investment: English classes, professional development
• Career progression: Real paths up, not just sideways shuffling
• Responsive teammates: No ignored Slacks, no "not my problem" attitudes
• Supportive culture: When you're stuck, people help. When things break, we fix them together
• Human connections: Regular meetups, tech talks, and actual relationships beyond work

Company Overview