Note: The job is a remote job and is open to candidates in USA. National General is a part of The Allstate Corporation, which means we have the same innovative drive that keeps us a step ahead of our customers’ evolving needs. They are seeking an Applied Machine Learning & Data Engineering Senior Consultant II to support high-impact AI and data automation initiatives, focusing on improving data ingestion, transformation, and mapping processes across the organization.

Responsibilities

• Design and build AI-powered data pipelines to process and transform structured and unstructured data (PDF, Excel, CSV, etc.)
• Develop and optimize LLM-based solutions (prompt engineering, mapping, extraction, and transformation)
• Translate complex data and technical outputs into actionable insights for business stakeholders
• Partner with cross-functional teams (product, engineering, business SMEs) to define and deliver solutions
• Validate and improve solution accuracy through testing, iteration, and stakeholder feedback
• Support integration of ML/AI solutions into enterprise systems and workflows (APIs, databases, cloud platforms)
• Document solutions to ensure scalability, maintainability, and long-term adoption
• Analyze and process carrier and CMS datasets
• Build AI/ML mapping logic using LLMs and prompts
• Develop data pipelines (Python/C# data processing logic, SQL, APIs, Azure)
• Validate outputs against expected results with business stakeholders
• Troubleshoot accuracy, performance, and edge cases
• Document pipelines and ensure production-readiness
• Support integration into enterprise systems (C#, APIs, Azure)

Skills

• 3+ years of related experience
• Strong proficiency in Python and SQL for data analysis and engineering
• Experience working with Generative AI / LLMs (prompt engineering, APIs, or similar)
• Hands-on experience with data transformation, file parsing, and pipeline development
• Knowledge of DevOps practices (CI/CD pipelines, version control, cloud deployment)
• Familiarity with cloud platforms (preferably Azure)
• Ability to communicate effectively with both technical and non-technical stakeholders
• Proven ability to work in ambiguous, fast-evolving environments
• Experience with C# / .NET integration or enterprise application development
• Familiarity with LLM frameworks (LangChain, etc.) or document processing tools
• Knowledge of API design, JSON processing, and system integration patterns
• Background in insurance, healthcare, or other regulated industries

Benefits

• Allstate provides a comprehensive technology setup, including a laptop, monitors, headset, keyboard, and mouse.
• Employees eligible to work from home also receive a monthly connectivity reimbursement to help offset internet costs.
• When working from home, you must have a dedicated, private workspace free from distractions, along with appropriate desk and seating.
• Reliable internet is required, with minimum speeds of 50 MB download and 5 MB upload.

Company Overview