Data Engineer (Real Time Streaming)

Only Immediate Joiner

Budget is 14-15 Lacs

JD -

• 3+ years hands-on Azure Databricks experience.
• Build real-time ingestion pipelines with Azure Event Hub and Databricks Structured Streaming.
• Design scalable streaming pipelines following best practices.
• Familiarity with Unity Catalog and data governance principles.
• Implement structured logging and metadata augmentation.
• Unit testing and DevOps practices for data workloads.
• Proficient in SQL and Python programming and advanced software development and DevOps concepts.
• Implement Azure Functions for XML/XSD validation.
• Optimize Spark performance in Databricks.
• Proficient in Python XML/XSD parsing/validation and modular packages.
• Hands-on with Azure DevOps/GitHub.
• Set up monitoring/alerting via Azure Monitor, Databricks Jobs, Log Analytics.
• Basic Terraform/IaC knowledge.
• Collaborate cross-functionally;...