Note: The job is a remote job and is open to candidates in USA. Meridial Marketplace, by Invisible, is seeking elite Subject Matter Experts (SMEs) to engage in high-impact AI evaluation and training projects. The role involves defining quality criteria for AI outputs, curating benchmark examples for model training, and providing expert analysis on model failures. This is a consulting opportunity for independent professionals with deep domain knowledge who want to shape the standards for AI in their fields.

Responsibilities

• Review and advise on the quality criteria used to evaluate AI outputs
• Create, edit, or certify "Golden Tasks"—benchmark examples of perfect prompts and responses that serve as the ground truth for model training
• Analyze model failures (hallucinations, logical fallacies) and provide technical breakdowns of why the model failed and how a human expert would have approached the problem
• Consult on the difficulty and nuance of specific prompts to ensure the AI is being tested against sufficiently rigorous scenarios

Skills

• Elite Domain Mastery: Advanced degrees (PhD, MD, JD) or equivalent industry-recognized status (e.g., Senior Principal Engineer, Published Author, Award-Winning Creative)
• Professional Autonomy: You currently operate as an independent consultant or freelancer and have your own workflows, equipment, and professional methodology
• Interest in Shaping Technology: While prior experience in AI evaluation is a bonus, it is absolutely not required. We want industry veterans who are eager to set the 'gold standard' for how AI should operate within their specific profession
• Articulate Reasoning: The ability to not just know the right answer, but to articulate the reasoning process clearly in writing
• Global Compliance: Candidates must be properly registered to operate as independent contractors in their country of residence

Company Overview