Position: Agentic Coding Annotator Online / Offline TasksType: Short-Term Contract (5 weeks)Compensation: $15 per hourLocation: RemoteCommitment: 8 hours per day with 4 hours overlap with PSTRole ResponsibilitiesPerform online evaluations by interacting with models on predefined coding tasks and grading outputsConduct offline evaluations by designing realistic coding tasks and defining evaluation criteriaReview and analyze model-generated code by reading, debugging, and validating outputsRun tests, scripts, and terminal commands to verify correctness of solutionsWrite clear, evidence-based rationales for trajectory rankings and assessmentsDesign task-specific rubrics and ensure consistent evaluation across runsIdentify issues in outputs, environments, or instructions and escalate with supporting evidenceWork with agentic coding tools and evaluation frameworks to assess model performanceRequirementsStrong years of experience in software engineering, QA, or similar code-heavy rolesStrong proficiency in at least 1 2 programming languages (Python, JavaScript, Java, C/C++, etc.)Experience working with Linux/terminal, Git, and development toolsFamiliarity with coding-agent tools (e.g., Cursor, Claude Code, OpenCode, or similar)Ability to read unfamiliar codebases, debug issues, and evaluate correctnessStrong attention to detail and ability to follow structured evaluation processesComfortable with repetitive, high-precision evaluation workExperience with Docker or reproducible environments is a plusAbility to work independently in a remote environmentApplication ProcessApply/Easy Apply and check email for application formFill Google formAssessment Link (After shortlisting; candidates can choose between two options and complete within 24 hours)