Azure Data Engineer

Job Description

KPI Partners is seeking a highly skilled and experienced Sr Data Engineer to join our dynamic team. The ideal candidate will have a strong background in data engineering, with expertise in Python Pandas, Databricks, Azure Data Factory (ADF), and DevOps practices.

Key Responsibilitie s:

- Design, develop, and implement robust data pipelines using Python Pandas and Databricks to ingest, process, and analyze large datasets.

- Collaborate with cross-functional teams to gather requirements and ensure data solutions meet business needs.

- Utilize Azure Data Factory to orchestrate data workflows and integrate data from various sources.

- Implement best practices for data governance, quality, and security throughout the data lifecycle.

- Develop and maintain automated deployment pipelines using DevOps tools to streamline the development process.

- Monitor and optimize data processes for perform...