Data Quality & Test Automation Framework Engineer

Data Quality & Test Automation Framework Engineer

Join to apply for the Data Quality & Test Automation Framework Engineer role at Confiz

We are seeking an experienced engineer to design and implement a Quality Assurance (QA) framework for automated testing across our cloud-native, real‑time, and batch data pipelines. The ideal candidate will have deep expertise in Python‑based test frameworks, data validation at scale, and CI/CD integration, ensuring reliability, accuracy, and performance in our data platform.

Key Responsibilities

• Design and develop a scalable automated data quality and testing framework for real‑time (Kafka/Flink/Spark Streaming) and batch (PySpark) data pipelines.
• Integrate automated data validation (schema checks, statistical profiling, anomaly detection) into data workflows using tools like Great Expectations, Deequ, or custom‑built Python solutions.
• Build test harnesses and reusabl...