Romanian Audio Generalist Evaluator Expert (SF or London, in person)

Mercor is seeking detail-oriented transcribing/writing experts to contribute to a high-impact audio AI research project with a leading lab. Freelancers will listen to and transcribe audios, annotate images and evaluate videos to help train advanced language models. This is a ~6 month short-term, flexible opportunity for professionals with strong academic backgrounds and a knack for instructional clarity. Ideal for those who enjoy distilling complex concepts into well-crafted text.

Job Details:

• Transcribe and Optimize Audio/Video: Create detailed audio transcriptions with multiple constraints and instructions from Romanian.
• Define and Document Evaluation Standards: Establish high-level expectations for correct responses in general audio consumer contexts, and develop comprehensive rubric.
• Conduct Model Testing and Grading: Run prompts through models and assess preliminary outputs against expectations.
• Support Benchmarking and Quality Assurance: Collaborate in QA review processes to ensure prompt tasks and rubrics meet rigor, maintaining consistency and reliability before integration into official benchmarks.

Minimum Qualifications:

• BS or BA from a reputable institution completed or in progress
• Strong writing and critical thinking skills.

Preferred Qualifications:

• College students/grads

Application & Onboarding Process:

• Complete an AI-led interview, this should take around 15 minutes.
• If selected, you will be invited to work on the project.

More Details About This Role:

• Expect to contribute at least 20 hours per week.
• Expect a commitment of around 2-4 weeks.