Study Domain Expert – Judge Calibration

• Apply professional judgment to realistic Sales work and evaluate model-generated outputs
• Sign an NDA before receiving study materials
• Complete one realistic Sales task using supplied source files
• Review model outputs, rank and score them, and write evidence-based rationales
• Provide structured feedback on the task and study design
• Complete 13 hours of work during September 12-15, 2026
• Work on projects focused on training and enhancing AI systems

Requirements

• More than 5 years of experience in Sales or a closely relevant field
• Experience at a high-caliber firm ideally
• Ideally held a manager, director, or principal-level Sales role; entry-level experience alone is insufficient
• Fluent or native English
• Prior work as an expert is a plus
• Must sign an NDA before receiving study materials
• No use of LLMs or other AI assistants during the study
• People who have worked on Project Alchemy in any capacity or accessed its source-world data are not eligible
• Must be engaged as an independent contractor
• H1-B and STEM OPT candidates are not supported
• No geographic restrictions

Core Competencies

Demonstrates extensive experience in Sales, particularly in high-caliber firms, with a focus on evaluating model-generated outputs and providing structured feedback. Proficient in applying professional judgment to enhance AI systems through realistic Sales tasks.

Highest-signal resume keywords

• 5+ Years Of Sales Experience
• Manager, Director, Or Principal-Level Sales Role
• Fluent Or Native English
• Expertise In Evaluating Model Outputs
• Independent Contractor Engagement

ATS Optimization Keywords

Industry Keywords

• Sales
• High-Caliber Firm
• Project Alchemy
• NDA
• AI Systems
• Model Outputs
• Evidence-Based Rationale
• Structured Feedback
• Independent Contractor
• Realistic Sales Task