Assistant Manager

Job Description

We are seeking a seasoned Data Engineer with a robust background in building scalable data pipelines and managing large-scale data architectures. With 6–9 years of experience, you will play a pivotal role in designing and optimizing our data ecosystem, ensuring high performance and reliability. You are an expert in the Spark ecosystem and feel at home working within Databricks to solve complex data challenges.

Key Responsibilities

• Pipeline Development: Design, develop, and maintain high-performance ETL/ELT pipelines using PySpark and Databricks.
• Experience with Rust/Vector.
• Data Modeling: Architect sophisticated data models and schema designs using SQL to support complex analytical requirements.
• Optimization: Perform performance tuning on Spark jobs and SQL queries to ensure efficient processing of massive datasets.
• Cloud Infrastructure: Leverage AWS services (such as S3, Glue,...