Query Vetting Specialist – AI/LLM Evaluation (Japanese/German/Portuguese)

Location Type: On-site

 

Hello Everyone,

Greetings from the Datamundi team!!

We are looking for highly motivated and detail-oriented Query Vetting Specialists with expertise in Japanese , German or Portuguese language evaluation and a strong understanding of Generative AI and Large Language Models (LLMs).

Please find the details below.

Job Type: Contract (Short-Term / Long-Term based on performance and project requirements)
Work Mode: Remote (WFH)
Location Requirement: Candidates must be located in the United States
Domain: IT / Generative AI
Working Hours: US Shift – 35 Hours per Week

In this role, you will support AI research and evaluation initiatives by designing and executing data collection, annotation, fact-checking, and query evaluation workflows. You will work closely with AI researchers and engineers to improve the quality, safety, and performance of LLM systems.

This opportunity is ideal for candidates with strong linguistic skills, research capabilities, analytical thinking, and an interest in cutting-edge AI technologies.

Key Responsibilities

• Vet, review, and evaluate user queries and AI-generated responses for quality, relevance, and factual accuracy
• Collaborate with GenAI researchers and engineers to understand project requirements and evaluation goals
• Expand high-level requirements into detailed evaluation workflows and documentation
• Conduct deep research using LLM tools and reliable external sources
• Perform fact-checking and accuracy verification to minimize misinformation risks
• Assess curated and AI-generated content for linguistic quality, cultural appropriateness, and informativeness
• Create and maintain annotation guidelines and quality standards
• Identify prompt quality issues and recommend improvements for better LLM performance
• Ensure high throughput while maintaining exceptional quality standards
• Collaborate with cross-functional teams in a fast-paced remote environment

Required Qualifications

Language Proficiency

Candidates must demonstrate Native, Bilingual, or Full Professional Proficiency in at least one of the following:

• Japanese (Native of Japan)
• German (Native of Germany)
• Portuguese – (Native of Brazil)

Educational Background

• Graduate with strong academic performance and deep cultural understanding
• Degree from a reputed US or international university/institute preferred

Experience

• Minimum 2+ years of professional experience in Japanese or German language-related roles
• Prior experience in AI/LLM evaluation, content moderation, annotation, localization, linguistic QA, or research preferred
• Understanding of AI, NLP, and Large Language Models is mandatory

Preferred Skills

• Excellent reading, writing, grammar, and communication skills
• Strong research and analytical capabilities
• Exceptional attention to detail and accuracy
• Familiarity with AI tools and prompt evaluation
• Understanding of ethical AI practices and bias mitigation
• Data literacy and experience handling structured/unstructured datasets
• Ability to work independently with minimal supervision

Interview Process

1. Resume Screening
2. Assessment Test
3. Take-Home Assignment / Interview
4. Managerial Round

Why Join Datamundi?

• Work on cutting-edge Generative AI projects
• Collaborate with global AI researchers and engineers
• Flexible remote work environment
• Opportunity for long-term engagement based on performance
• Exposure to advanced LLM evaluation and AI quality workflows

Ideal Candidate

We are looking for young, dynamic professionals who have demonstrated strong academic excellence, exceptional linguistic expertise, and the ability to thrive in a fast-paced AI-driven environment with minimal supervision.