Job Description
- A highly skilled and detail-oriented Human Subject Matter Expert (SME) specializing in Clinical pre-annotation validation. The successful candidate will play a critical role in our incremental annotation process, focusing on human validation of pre-annotated data, refining annotation guidelines, and contributing to the development of high-quality golden datasets. While core clinical annotation skills are paramount, experience with the Johnson Labs (JSL) ecosystem and advanced NLP concepts are highly desirable.Key Responsibilities:
- Perform rigorous human validation on pre-annotated data generated by commercial clinical NLP models (e.g., Amazon Comprehend Medical) or internal LLM/encoder-based tools.
- Contribute to the iterative refinement of annotation guidelines and examples to improve inter-annotator agreement and overall data quality.
- Identify and resolve disagreements between pre-annotations and human validations, ensuring the accuracy and consistency of the annotated datasets.
- Collaborate closely with data science and NLP teams to provide feedback on model performance and contribute to the continuous improvement of pre-annotation models.
- Assist in the creation and maintenance of golden datasets for training and evaluating NLP models.
- Participate in discussions regarding data sensitivity and ensure adherence to all relevant data privacy and security protocols, especially for patient data.
- Work on diverse clinical annotation projects (e.g., alcohol and smoking use cases, oncology (ECOG, Karnofsky scores), biomarkers, SDO, and mental health). Qualifications:To help recruiters identify the best fit, we've outlined the essential skills for this role and additional valuable assets that would further strengthen a candidate's profile.<>Must Have (Required Qualifications):
- Bachelor's or Master's degree in a relevant field (e.g., Life Sciences, Linguistics, Computer Science, or a related healthcare discipline).
- Proven experience in data annotation, specifically with clinical or biomedical text.
- Strong foundational understanding of NLP concepts and terminology.
- Exceptional attention to detail and ability to maintain high levels of accuracy in data validation.
- Excellent communication and collaboration skills.
- Ability to quickly adapt to various annotation tools (e.g., Inception, Label Studio, Prodigy). <>Good To Have (Preferred Qualifications/Skills):
- Prior experience working within the John Snow Labs (JSL) ecosystem for modeling and annotation, including familiarity with their Health AI Lab and GenAI tool.
- Experience with specific advanced annotation types such as named entity recognition (NER), relationship extraction, and assertion status annotation.
- Direct clinical background or extensive practical experience working with sensitive patient data in a healthcare context.
- Familiarity with incremental batch training processes in machine learning.
Apply Now
Apply Now