Data Engineer

Years of experience: 5- 10 years

Location: Bangalore

Job Description:

Senior Data Engineer with deep expertise in Scala-based Spark development and end-to-end deployment of data pipelines on Kubernetes cluster, orchestrated via Airflow.

The ideal candidate should have a strong software engineering foundation, excellent understanding of distributed systems, proficient in software design, modern project/code structuring skills, with good understanding on CI/CD processes and implementation which enables them to deliver reliable, scalable and robust data solutions.

Should have overall experience of minimum 6-8 years with minimum 5Years in Hadoop, Spark.

• Exp in Scala, Java, Airflow, Spark on Kubernetes, Yarn, Oozie, Hadoop, Kafka, Spark & Spark Structured Streaming.

• Advanced Scala experience (e.g. Functional Programming, using Case classes, Complex Data Structures & Algorithms)

• Proficient in developing automated frameworks for unit & integration testing.

• Experience with Docker and Helm and related container technologies.