Data Engineer

Responsibilities

• Design and implement data processing systems using distributed frameworks like Hadoop, Spark, Snowflake, Airflow, or other similar technologies.
• Build data pipelines to ingest data from various sources such as databases, APIs, or streaming platforms.
• Integrate and transform data to ensure its compatibility with the target data model or format.
• Design and optimize data storage architectures, including data lakes, data warehouses, or distributed file systems.
• Implement techniques like partitioning, compression, or indexing to optimize data storage and retrieval.
• Identify and resolve bottlenecks, tuning queries, and implementing caching strategies to enhance data retrieval speed and overall system efficiency.
• Design and implement data models that support efficient data storage, retrieval, and analysis.
• Collaborate with data scientists and analysts to understand their requiremen...