Internet-Native Bilingual Evaluator Expert (Spanish – Spain)

Mercor is seeking Internet-Native Bilingual Evaluator Experts (Spanish–English) to contribute to cutting-edge AI research focused on multilingual communication, internet culture, and real-world language use. In this role, you will evaluate, annotate, and improve AI model outputs across English and Spanish, helping ensure that models accurately understand modern online communication, cultural references, slang, and multilingual interactions. This role is specifically focused on Spanish internet culture, language use, and online communities in Spain.

Job Responsibilities

Evaluate Internet-Native Communication

• Analyse content originating from social media, forums, messaging platforms, online communities, and consumer-facing digital environments.
• Assess AI-generated responses for naturalness, fluency, cultural appropriateness, and internet authenticity.
• Identify issues involving slang, memes, abbreviations, code-switching, and evolving online language.
• Evaluate content specific to Spanish internet culture, trends, and online communities in Spain.

Develop Evaluation Standards

• Create detailed rubrics and guidelines for evaluating bilingual and internet-native communication.
• Define standards for tone, cultural relevance, contextual understanding, and linguistic accuracy.
• Document edge cases involving internet culture, humour, sarcasm, and platform-specific language.
• Identify distinctions between Peninsular Spanish and Latin American Spanish internet usage.

Conduct Model Testing and Feedback

• Test language models using real-world multilingual prompts.
• Evaluate outputs in both English and Spanish.
• Provide structured feedback to improve model performance and cultural understanding.

Support Quality Assurance

• Participate in review cycles and benchmark development.
• Collaborate with project leads to improve evaluation frameworks.
• Maintain consistency across datasets and evaluation processes.

Minimum Qualifications

• Native or near-native fluency in Spanish (Spain) and professional fluency in English.
• Deep familiarity with Spain-based internet culture and online communities.
• Active engagement with platforms such as YouTube, TikTok, Reddit, Discord, X, Instagram, Twitch, Forocoches, or similar communities.
• Strong understanding of Peninsular Spanish slang, memes, internet humor, and contemporary digital culture.
• Strong writing, analytical, and critical thinking skills.
• Available to commit 10–20 hours per week.

Preferred Qualifications

• Experience in content moderation, localization, translation, linguistics, social media, journalism, or AI evaluation.
• Experience evaluating user-generated content or digital communications.
• Familiarity with differences between Spain Spanish and Latin American Spanish online discourse.
• Interest in AI, language models, and internet culture research.