Data Engineer in Test

We are undergoing a significant data transformation to ensure that accurate and consistent data is available precisely when and where it's needed.

Responsibilities

• Write PySpark and SQL scripts to validate data pipelines, transformations, and integrations.
• Design and run tests for data validation, storage, and retrieval using Azure services like Data Lake, Synapse, and Data Factory, adhering to industry standards.
• Continuously enhance automated tests as new features are developed, ensuring Participate in data reconciliation and verify Data Quality frameworks to maintain data accuracy, completeness, and consistency across the platform.
• Formulate and maintain test strategies—including smoke, performance, functional, and regression testing—to ensure data processing and ETL jobs meet requirements.
• Collaborate with development teams to assess changes in data workflows and update test cases to preserve data integrity.