Note: The job is a remote job and is open to candidates in USA. Blueprint Technologies is a technology solutions firm headquartered in Bellevue, Washington, seeking a detail-oriented Labeler / Annotator to evaluate responses generated by AI systems in Italian. The role focuses on evaluating AI outputs across various scenarios to improve AI communication in Italian, requiring strong judgment and attention to detail.

Responsibilities

• Perform side-by-side (SBS) comparisons of AI-generated responses
• Evaluate outputs based on:
• Accuracy
• Relevance
• Clarity
• Instruction-following
• Identify nuances in tone, meaning, and cultural context across Italian
• Apply detailed, scenario-specific annotation guidelines
• Maintain consistency and high-quality evaluations

Skills

• Native or professional fluency in Italian
• Strong English reading comprehension (required for guidelines)
• Prior experience with side-by-side (SBS) labeling or annotation
• Excellent analytical thinking and attention to detail
• Ability to quickly understand and apply structured evaluation frameworks
• Background in linguistics, translation, or localization
• Experience in AI evaluation
• Experience in data annotation
• Experience in search relevance or content quality analysis

Benefits

• Statutory benefits (e.g., paid vacation, aguinaldo, and vacation premium)
• Social security and healthcare contributions (IMSS)
• Additional benefits aligned with local market practices

Company Overview