Senior Data Engineer

About Our Client

The employer is a large organization operating in the retail industry.

Job Description

• Design, develop, and maintain robust ETL pipelines using Python and orchestration tools like Airflow.
• Build and optimize data models, data lakes, and data warehouses using platforms such as Google BigQuery, Google Cloud Storage, and Amazon RedShift.
• Integrate data from diverse sources including SQL Server, PostgreSQL, MySQL, Excel, APIs, and cloud storage.
• Implement data governance and ensure compliance with data privacy regulations, including managing access controls and securing PII data.
• Conduct data validation and quality checks to ensure accuracy and consistency.
• Collaborate with Data Analysts and Data Scientists to deliver well-structured datasets and support analytical needs.
• Monitor and optimize query performance through indexing, partitioning, and caching strategies.