Job Description
Note: The job is a remote job and is open to candidates in USA. Keeper Security, Inc. is a leading cybersecurity company transforming the industry with next-generation solutions. They are seeking a Senior QA Analyst to lead testing efforts for AI-driven features and machine learning models, ensuring quality and reliability across Keeper's platform. Responsibilities • Design and execute test plans for AI and ML-driven features, including classification, summarization, anomaly detection and autofill systems • Develop Python-based automation and evaluation scripts to validate model performance, reliability and output quality • Test LLM-based features for accuracy, guardrail effectiveness, hallucination risk and behavior under edge cases • Validate data pipelines, input/output transformations and model integration points across Keeper's platform • Conduct regression testing on ML models and AI workflows to ensure reliability across releases • Test enterprise-scale behavior analysis features driven by data from ARAM, PAM and session recordings • Validate security, privacy and zero-trust requirements for AI model interactions and data handling • Collaborate with ML engineers and cross-functional teams to refine model behavior, identify defects and improve output quality • Analyze logs, metrics and model outputs to detect anomalies, performance issues or silent failures Skills • 3+ years of experience in QA, test automation or evaluation of complex systems • Hands-on experience writing Python automation or evaluation scripts for testing APIs, model outputs or data pipelines • Understanding of machine learning concepts such as classification, summarization, embeddings, anomaly detection or behavior analysis • Experience testing LLM-integrated features or AI-driven workflows • Familiarity with ML tooling such as Pytest, Hugging Face libraries, Jupyter Notebooks or similar • Experience with API testing, JSON validation and test data generation • Ability to test nondeterministic systems and evaluate model quality with structured methodologies • Strong analytical, troubleshooting and communication skills • Experience testing cloud-integrated applications or systems deployed on AWS • Bachelor's degree in Computer Science, Information Systems or related field, or equivalent practical experience • Experience testing security-focused AI or threat analytics systems • Familiarity with ML observability tools, vector databases or data processing frameworks • Experience testing video, image or behavior analysis pipelines • Knowledge of zero-trust principles and secure data handling • Experience working with LLM providers or AI guardrail frameworks Benefits • Medical, Dental & Vision (inclusive of domestic partnerships) • Employer Paid Life Insurance & Employee/Spouse/Child Supplemental life • Voluntary Short/Long Term Disability Insurance • 401K (Roth/Traditional) • A generous PTO plan that celebrates your commitment and seniority (including paid Bereavement/Jury Duty, etc) • Above market annual bonuses Company Overview • Keeper Security is transforming cybersecurity for millions of individuals and thousands of organizations globally. It was founded in 2011, and is headquartered in Chicago, Illinois, USA, with a workforce of 501-1000 employees. Its website is Company H1B Sponsorship • Keeper Security, Inc. has a track record of offering H1B sponsorships, with 1 in 2021. Please note that this does not guarantee sponsorship for this specific role. Apply tot his job