Data Engineer PySpark

Sopra Steria logo

Sopra Steria

View Salaries, Reviews, and more  

Job Summary


Job Type
-

Seniority
Senior

Years of Experience
Information not provided

Tech Stacks
Hadoop Scala SQL kafka Spark Airflow pySpark EMR IAM AWS Snowflake Amazon S3 ETL

Job Description

Company Description

About Sopra Steria

Sopra Steria, major Tech player in Europe recognised for its consulting, digital services and software development, helps its clients drive their digital transformation and obtain tangible and sustainable benefits. It provides end-to-end solutions to make large companies and organisations more competitive by combining in-depth knowledge of a wide range of business sectors and innovative technologies with a fully collaborative approach. Sopra Steria places people at the heart of everything it does and is committed to putting digital to work for its clients in order to build a positive future for all. With 50,000 employees in nearly 30 countries, the Group generated revenue of โ‚ฌ5.1 billion in 2022.

Job Description

The world is how we shape it.

We are seeking a highly skilled and motivated Data Engineer to join our dynamic team. As a Data Engineer, you will collaborate closely with our Data Scientists to develop and deploy machine learning models. Proficiency in below listed skills will be crucial in building and maintaining pipelines for training and inference datasets.

Responsibilities:

  • Work in tandem with Data Scientists to design, develop, and implement machine learning pipelines.
  • Utilize PySpark for data processing, transformation, and preparation for model training.
  • Leverage AWS EMR and S3 for scalable and efficient data storage and processing.
  • Implement and manage ETL workflows using Stream sets for data ingestion and transformation.
  • Design and construct pipelines to deliver high-quality training and inference datasets.
  • Collaborate with cross-functional teams to ensure smooth deployment and real-time/near real-time inferencing capabilities.
  • Optimize and fine-tune pipelines for performance, scalability, and reliability.
  • Ensure IAM policies and permissions are appropriately configured for secure data access and management.
  • Implement Spark architecture and optimize Spark jobs for scalable data processing.

Requirements:

Mandatory

  • Proficiency in Advanced SQL (Window functions), Spark Architecture, Pyspark or Scala with Spark, Hadoop.
  • Proven expertise in designing and deploying data pipelines.
  • Strong problem-solving skills and ability to work effectively in a collaborative team environment.
  • Excellent communication skills and ability to translate technical concepts to non-technical stakeholder

Desirable

  • Hands-on experience with Airflow, S3, and Stream sets or similar ETL tools.[ can be trained locally ]
  • Understanding of real-time or near real-time inferencing architectures.
  • Basic Knowledge on Kafka ,AWS IAM, AWS EMR and Snowflake.

Total Experience Expected: 06-08 years

Qualifications

BE

Additional Information

At our organization, we are committed to fighting against all forms of discrimination. We foster a work environment that is inclusive and respectful of all differences.

All of our positions are open to people with disabilities.

Interview Questions of Data Engineer PySpark at Sopra Steria

Interview questions from Sopra Steria that are similar to Data Engineer PySpark
View more interview questions from Sopra Steria โ†’
Unlock Your Interview Potential
The only end-to-end front end interview preparation platform by FAANG ex-interviewers and Staff Engineers.
Get hired at FAANG
Users now work at:

Salary Insights of Data Engineer PySpark at Sopra Steria

Currently, there aren't any salaries for this role at Sopra Steria shared by other job seekers.

View more salaries from Sopra Steria โ†’

Achieve your dream job with our top-notch tools!

Resume Checker Illustration

Resume Checker

Our free resume checker analyzes the job description and identifies important keywords and skills missing from your resume in just a minute!

Check Now
Interview Preparation Illustration

AI InterviewPrep

Utilizing advanced AI, our tool generates tailored interview questions based on your industry, role, and experience. Practice and receive feedback on your answers in real time!

Check Now
Resume Builder Illustration

Resume Builder

Let us show you the differences between a bad, good, and great resume, and guide you in building a resume that helps you stand out to employers, ensuring you land your next position faster!

Check Now