AI Response Evaluator

Thoth AI · Posted 2026-05-01

Job Title: AI Response EvaluatorLocation: RemoteType: Contract / FreelanceLanguage: Arabic (MSA)OverviewWe are hiring multilingual annotators to support Supervised Fine-Tuning (SFT) projects by creating high-quality, step-by-step responses (“trajectories”) across multiple languages. This role focuses on ensuring reasoning clarity, linguistic accuracy, and cultural relevance for Arabic speakers.Key ResponsibilitiesCreate structured question → reasoning → answer annotationsWrite clear and logical step-by-step explanationsTranslate and adapt content while preserving meaning and reasoningReview and improve multilingual responses for clarity and consistencyEnsure outputs are natural, accurate, and culturally appropriateRequirementsFluency in Modern Standard Arabic for the task and English for understanding guidelinesStrong writing and comprehension skillsAbility to break down complex ideas into simple, structured stepsHigh attention to detail and consistencyNice to HaveExperience in annotation, translation, or content writingFamiliarity with AI training data or SFT workflowsBackground in linguistics, education, or communicationWhat Success Looks LikeClear, well-structured reasoning across all outputsConsistent quality across languagesStrong alignment with guidelines and toneCompensationCompetitive, based on language and experience

Apply for this role