Data Engineer

Project Role : Data Engineer

Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems.

Must have skills : Data Engineering

Good to have skills : Generative AI

Minimum 7.5 Year(s) Of Experience Is Required

Educational Qualification : 15 years full time education

Summary:

As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating robust data pipelines, ensuring the quality of data, and implementing efficient ETL processes to migrate and deploy data across various systems. You will collaborate with cross-functional teams to understand their data needs and provide innovative solutions that enhance data accessibility and usability. Your role will also require you to monitor data workflows, troubleshoot issues, and optimize processes to ensure seamless data operations, contributing to the overall success of data-driven initiatives within the organization.

Roles & Responsibilities:

• Expected to be an SME.
• Collaborate and manage the team to perform.
• Responsible for team decisions.
• Engage with multiple teams and contribute on key decisions.
• Provide solutions to problems for their immediate team and across multiple teams.
• Develop and implement data models that support business objectives.
• Monitor and optimize data pipelines for performance and reliability.
• Python, , Azure data services, Databricks, vector databases, embedding pipelines for GenAI/RAG, data governance, security, and compliance frameworks, containerization, CrewAI, Azure DevOps.
  
  
  

Professional & Technical Skills:

• Must To Have Skills: Proficiency in Data Engineering.
• Good To Have Skills: Experience with Python (Programming Language), Generative AI.
• Strong understanding of ETL processes and data warehousing concepts.
• Experience with cloud-based data solutions and services.
• Proficient in data quality assurance and data governance practices.
  
  
  

Additional Information:

• The candidate should have minimum 5 years of experience in Data Engineering.
• This position is based at our Pune office.
• A 15 years full time education is required.