Job Description
Note: The job is a remote job and is open to candidates in USA. Filevine is forging the future of legal work with cloud-based workflow tools, and they are seeking a forward-thinking QA Engineer to lead the testing strategy for their new AI-first Forms Platform. The role involves building evaluation harnesses for AI models, automating tests, and ensuring the quality of complex legal forms during migration from legacy systems. Responsibilities • AI Model Validation: Design and maintain evaluation harnesses (evals) to measure the accuracy of AI auto-mapping models, ensuring they meet strict quality thresholds before release • Test Automation: Build robust, automated test suites (end-to-end and integration) that run within our CI/CD pipelines to catch regressions immediately • Golden Datasets: Maintain and expand "golden datasets" of perfectly mapped forms to serve as the ground truth for regression testing • Defect Prevention: Collaborate with developers to identify "model quality variance" and implement gates that prevent low-confidence predictions from reaching production • Migration Quality: Verify data integrity during the migration of pilot customers from the legacy platform to the new architecture • Efficiency: Leverage LLMs and AI coding assistants to rapidly write test scripts and generate edge-case scenarios for complex legal forms Skills • Bachelor's degree in Computer Science, Engineering, or related field • 3+ years of experience in QA Automation, with proficiency in tools like Cypress, Selenium, or Playwright • Strong ability to write test scripts in JavaScript, TypeScript, or Python (Java/C++ proficiency also accepted) • Experience testing non-deterministic systems (AI/ML models) or complex data migration projects is highly preferred • 1 to 3 years experience working in agile environments with frequent release cycles • Demonstrated ability to use AI tools to solve coding roadblocks and optimize testing workflows • Excellent verbal and written communication skills to articulate quality risks to the product team • Experience testing non-deterministic systems (AI/ML models) or complex data migration projects is highly preferred Benefits • Medical, Dental, & Vision Insurance (for full-time employees) • Competitive & Fair Pay • Maternity & paternity leave (for full-time employees) • Short & long-term disability • Opportunity to learn from a dedicated leadership team • Centrally located open office building in Sugar House (onsite employees) • Top-of-the-line company swag Company Overview • Filevine provides a legal operating platform integrating AI for case, document, and lead management. It was founded in 2014, and is headquartered in Salt Lake City, Utah, USA, with a workforce of 501-1000 employees. Its website is Apply tot his job