Internet-Native Bilingual Evaluator Expert (Japanese)

Mercor is seeking Internet-Native Bilingual Evaluator Experts (Japanese–English) to contribute to advanced AI research focused on multilingual communication, internet culture, and contemporary digital language use. This role is ideal for individuals who actively participate in both Japanese- and English-language online communities and understand how internet culture, slang, humour, and social norms differ across languages.

Job Responsibilities

Evaluate Internet-Native Communication

• Analyse content from social media, forums, video platforms, gaming communities, and digital communication channels.
• Assess AI-generated outputs for linguistic quality, cultural relevance, and authenticity.
• Evaluate internet slang, memes, abbreviations, emoji usage, honorific language, and code-switching.

Develop Evaluation Standards

• Create evaluation rubrics for internet-native bilingual communication.
• Document linguistic and cultural edge cases specific to Japanese and English online communities.
• Establish standards for tone, appropriateness, and contextual understanding.

Conduct Model Testing and Feedback

• Test AI systems using bilingual prompts and real-world internet scenarios.
• Evaluate outputs in Japanese and English.
• Provide detailed feedback to improve multilingual model performance.

Support Quality Assurance

• Participate in benchmark development and QA reviews.
• Ensure consistency and quality across evaluation datasets.

Minimum Qualifications

• Native or near-native fluency in Japanese and professional fluency in English.
• Deep familiarity with Japanese and English internet culture.
• Active engagement with platforms such as X, YouTube, Discord, Reddit, Twitch, Nico Nico, gaming communities, or similar online ecosystems.
• Strong writing, analytical, and critical thinking skills.
• Understanding of internet slang, memes, internet etiquette, and online communication norms in both languages.
• Available to commit 10–20 hours per week.

Preferred Qualifications

• Background in linguistics, translation, localisation, journalism, social media, humanities, or AI evaluation.
• Experience evaluating user-generated content or online communities.
• Familiarity with internet subcultures, fandoms, gaming communities, and contemporary digital trends.
• Interest in AI, language models, and multilingual communication research.