Test Analyst - ETL QA Engineer

🌍 Remote, USA 🎯 Full-time 🕐 Posted Recently

Job Description

Title: Test Analyst - ETL QA Engineer
Location: Remote (Baltimore, MD)
Duration: Contract

Mandatory Skills: Pytest Framework exp
(build Pytest framework from the scratch)
Need current project experience with Pytest.

We are seeking a skilled and detail-oriented Databricks ETL QA Engineer to join our data engineering team. The ideal candidate will have a strong background in QA processes, data validation, and testing of ETL pipelines, with specific experience in Databricks and big data platforms.

Collaborate with Data Engineers and Analysts to understand business requirements and ETL processes.
Design, develop, and execute comprehensive test cases, test plans, and test scripts for Databricks ETL pipelines.
Perform data validation, data quality checks, and ensure data integrity across multiple systems.
Verify unity catalog implementation and build data governance scenarios and test them
Automate testing processes for large datasets using tools like PySpark, SQL, and Databricks notebooks.
Work on performance testing and optimization of ETL jobs.
Perform API testing by identifying the endpoints, request methods, and expected responses and build test scenarios and test data to cover all possible scenarios
Identify, document, and track defects and drive them to resolution.
Contribute to continuous improvement of QA processes and best practices.
Ensure compliance with data governance and security policies.
Build and maintain the end to end test management suite by leveraging the right test management tools

Strong experience in ETL testing, data validation, and QA methodologies.
Hands-on experience with Databricks, Spark (PySpark/Scala), and big data platforms.
Proficiency in SQL for data analysis and validation.
Familiarity with Delta Lake, Parquet, and other data formats.
Experience with API testing and automation
Experience with test automation frameworks and scripting.
Strong understanding of data warehousing concepts and ETL workflows.
Experience with cloud platforms like AWS
Familiarity with version control tools (e.g., Git).

Knowledge of CI/CD pipelines and DevOps practices.
Experience with Python or other scripting languages.
Familiarity with BI tools (e.g., Power BI, Tableau) for data validation reporting.
Experience with Agile/Scrum methodologies.