Job Description
Note: The job is a remote job and is open to candidates in USA. Rex.zone is hiring a Remote, Full-Time AI Training Specialist to support production AI training pipelines. The role involves creating high-quality training data and evaluation signals across NLP and computer vision, including various evaluation tasks and content safety labeling.
Responsibilities
- Produce labeled datasets for NLP tasks such as named entity recognition, text classification, summarization, and instruction-following
- Perform RLHF tasks: preference comparisons, rubric-based ratings, and consistency checks
- Run LLM evaluation and prompt evaluation for accuracy, helpfulness, harmlessness, and policy compliance
- Conduct QA evaluation, audits, and adjudication to improve training data quality and reduce label noise
- Apply annotation guidelines compliance, document edge cases, and propose clarifications to playbooks
- Support computer vision annotation (bounding boxes, polygons, segmentation, keypoints) when needed
- Perform content safety labeling (hate, harassment, self-harm, sexual content, violence) aligned to policy
- Collaborate with project leads on inter-annotator agreement, drift, throughput, and quality metrics
Skills
- Experience with data labeling, QA evaluation, RLHF, or LLM evaluation in production/high-throughput environments
- Strong written English for rubric-based judgment and prompt evaluation
- Familiarity with NLP concepts (NER, classification) and/or computer vision annotation
- Ability to follow precise guidelines and handle sensitive content for safety labeling
Company Overview
Apply To This Job