This project focuses on the technical review and validation of coding tasks used to train AI models. Coding challenges are generated automatically by AI, and your role is to evaluate quality of the challenge.
The core questions you will be answering on every task:
Are the requirements well defined?
Are the associated tests correctly validating the requirements?
Tests are treated as the mechanism of truth in this context. Mistakes here propagate at scale into AI systems — error criticality is high.
Tasks in this project involve code written across multiple languages. You will be expected to review and evaluate tasks in any of the following:
Python
JavaScript / TypeScript
Go
Rust
Java
Evaluate whether the coding task itself is clearly and correctly defined
Validate whether tests accurately reflect whether the problem has been solved
Determine whether a solution that passes the tests genuinely solves the underlying problem
Apply strict technical criteria and quality standards consistently across tasks
This role is designed for mid/senior-level software engineers with real professional experience.
Technical requirements
Proven professional experience in software development (production environments)
Experience reviewing and evaluating code written by other engineers
Solid understanding of automated testing — how tests validate (or fail to validate) behavior
Experience contributing to or working with open source projects
High attention to detail and strong technical judgment
Comfortable working fully in English (reading and writing)
Nice to have
Experience with test-driven development (TDD) or test design
Familiarity with large or complex codebases
This is expert, task-based technical work focused on analysis, validation, and judgment — not code production.
USD 60-80 per hour.
~5-10 hours per week.
Terac is building the world's largest pool of vetted human experts for AI. Researchers, AI labs, and product teams use Terac to recruit, screen, and pay study participants across industries, languages, and skill sets.
Learn more at terac.com or on YouTube at @jointerac.