Note: The job is a remote job and is open to candidates in USA. CodeGeniusRecruit is a company looking for a Data Engineer to work remotely on a contract basis. The role involves authoring complex benchmark tasks, designing evaluations for language models, and contributing to benchmark datasets for professional document understanding.

Responsibilities

• Author complex, multi-step benchmark tasks grounded in real-world workspace files including technical specs, architecture docs, API references, and codebases
• Design tasks that evaluate the ability of language models and systems to reason over technical documentation and follow precise instructions
• Pair each task with a clearly defined ground truth output and an objective evaluation rubric
• Incorporate web search and code execution into task design to reflect professional, real-world workflows
• Contribute to the development of benchmark datasets used to assess LLMs on professional document understanding and instruction following within the Technology domain

Skills

• Strong experience in software engineering, data science, or analytics in a hands-on professional capacity
• Strong experience in authoring or evaluating technical documentation, including architecture docs, API references, or codebases
• Strong experience in designing structured, multi-step tasks with measurable outputs and evaluation criteria
• Strong experience working independently in a remote, self-directed contract environment

Company Overview