Job Description
Wisedocs is a fast-growing AI platform transforming how insurance companies analyze claims. They are seeking a Prompt Engineer to evaluate and test AI model outputs, ensuring they meet technical and clinical standards while collaborating with engineering and product teams.
Responsibilities
- Prompt and Model Testing: Test AI-generated outputs, focusing on medical prompts and machine learning models used in healthcare applications
- Feedback Loop: Analyze AI output for accuracy, relevance, and clinical validity. Provide detailed feedback to the engineering team to enhance prompt performance and model behavior
- Standard Operating Procedures (SOPs): Write, maintain, and update detailed SOPs for medical prompt engineering and testing, ensuring consistency and quality in prompt creation and validation
- Collaboration with Engineering: Work closely with the engineering team to troubleshoot, refine, and optimize machine learning models based on feedback from prompt testing and evaluation
- Quality Assurance: Ensure that the AI-generated responses align with clinical guidelines, best practices, and healthcare regulations
- Data Analysis: Assist in analyzing medical data sets and model outputs to evaluate the performance of AI models in clinical contexts
- Documentation: Document all testing procedures, outcomes, and improvements. Create user-friendly guides and best practice documents for prompt testing and optimization
- Continuous Improvement: Actively participate in the iterative process of improving prompt accuracy, functionality, and overall system performance in collaboration with cross-functional teams
Skills
- A degree in medicine, nursing, medical informatics, or a related field, or significant hands-on clinical experience
- Familiarity with AI, machine learning models, and prompt engineering, especially in healthcare or clinical settings
- Experience with testing and evaluating technology solutions, particularly in the healthcare or technology space
- Desire to learn and grow your skillset!
- Experience writing clear, concise, and detailed Standard Operating Procedures (SOPs) and documentation
- Excellent written and verbal communication skills, with the ability to deliver technical feedback in a clear and constructive manner
- Strong attention to detail, particularly when reviewing AI outputs for clinical accuracy and relevance
- Ability to identify issues, propose solutions, and collaborate with engineering teams to resolve challenges
- Knowledge of HIPAA, medical privacy laws, and healthcare compliance standards
- Experience with data analysis tools, basic coding (Python, R, etc.) and SQL
- Previous work with AI models in a medical context, such as AI-powered medical documentation systems, diagnostic tools, or virtual health assistants
Benefits
- Flexible hybrid environment with the option to collaborate in-person at our Toronto HQ.
- Modern employee benefits, including health and dental coverage
- Competitive compensation, with valuable stock options, as we’re still a young company growing very quickly.
- An opportunity to develop very rapidly in your career.
- Access to a learning and professional development fund to help you level up your career while you’re working with us.
- Company events
- Generous Paid Time Off
- Paid Sick Days
- Casual Dress code
- Employee Referral Bonus
- Tuition Assistance
- Plus, many other Recognition Programs!
Company Overview
Apply To This Job