Note: The job is a remote job and is open to candidates in USA. Steneral Consulting is seeking an AI/ML Engineer to design and implement secure, scalable AI solutions. The role involves optimizing AI models for local infrastructure and ensuring compliance with regulatory requirements.

Responsibilities

• Design, deploy, and optimize open-source LLMs and AI frameworks for on-premises, bare-metal/private hardware clusters
• Build, maintain, and secure local AI inference pipelines, advanced fine-tuning/embedding workflows, and RAG (Retrieval-Augmented Generation) systems
• Optimize AI models and workflows for maximum efficiency (latency, throughput, CPU/GPU/memory usage) on bare-metal infrastructure
• Ensure complete data isolation, integrating AI with internal data sources while adhering to compliance and regulatory requirements
• Evaluate and select open-source, local-first AI tools (vector databases, orchestration frameworks, model serving layers)
• Collaborate with engineering and compliance teams to align AI solutions with regulatory needs
• May perform additional duties as assigned

Skills

• Extensive local (non-cloud) AI deployment experience, especially with open-source LLMs (e.g., LLaMA, Mistral) in air-gapped/private environments
• Deep expertise in AI frameworks like PyTorch, Hugging Face, LangChain, LlamaIndex
• Strong experience with RAG architectures, embeddings, semantic search, and vector databases (e.g., FAISS, Qdrant, Milvus, Chroma)
• Proficient in containerizing AI workloads (Docker/Kubernetes) and managing GPU-based compute environments
• Solid understanding of advanced ML concepts (e.g., LoRA/QLoRA fine-tuning, prompt engineering, model quantization formats like GGUF, AWQ, EXL2)
• Ability to work autonomously in isolated, non-cloud development environments
• Experience in compliance-driven or regulated industries (e.g., fintech, legal-tech)
• Familiarity with local-first agentic workflows, Model Context Protocol, and developing internal developer copilots

Company Overview