PySpark QA

JOB DESCRIPTION

Skill: PySpark QA

Role / Tier: Lead Software Engineer/Tier 2

Experience: 6 – 9 years

Job Description:

Primary Skills

BIG Data technology mentioned below Hadoop / Big Data (HDFS, PYTHON, SPARK-SQL, MapReduce) with PYSpark.
build CI/CD pipelines
Spark APIs to cleanse, explore, aggregate, transform, store & analyse
installing, configuring, debugging and troubleshooting Hadoop clusters

Secondary Skills

Cloud-based infrastructure , AWS services EC2, EMR, S3, Lambda, EBS, IAM, Redshift, RDS
deploying and managing ETL pipelines, RDBMS technologies (PostgreSQL, MySQL, Oracle, etc.).
knowledge of data frames, Pandas , data visualization tools & data mining .
knowledge of JIRA, Bitbucket GitHub

JD
Seeking a developer to develop, enhance, in a Pyspark -QA role.
...