Job Description

Role: Lead Data Engineer

Experience: 9+ years, with a minimum of 5 years Data Engineering experience on Azure

Required Skills:

Expert in using Azure Synapse, Azure Storage, and Azure Data Factory to build data solutions.
Strong understanding of metadata-driven approaches and data governance principles.
Excellent problem-solving skills and ability to work in a fast-paced, collaborative environment.
Experience with agile development methodologies and DevOps practices as the Lead Data Engineer is mandatory.
Extensive programming expertise with Python, Py Spark, SQL and shell scripting is essential.
Strong communication and interpersonal skills, with the ability to effectively interact with stakeholders at all levels.

Responsibilities:

Designing and implementing data engineering solutions using Azure services such as Synapse, Azure Storage, and Data Factory.
Providing technical direction to the team and assisting with the design and implementation of technical architecture for data solutions, ensuring scalability and performance.
Managing the ADO (Azure DevOps) board, including creating and tracking work items, ensuring timely updates, and facilitating smooth sprint planning and execution.
Engaging in problem solving and ad-hoc troubleshooting to resolve complex technical issues and acting as the primary contact for resolving issues.
Developing and maintaining data pipelines to efficiently extract, transform, and load data from various sources.
Implementing metadata-driven approaches to automate and optimize data workflows.
Collaborating with cross-functional teams to gather requirements and define data engineering architectures.
Ensuring data quality and reliability through rigorous testing and validation processes.
Implement monitoring frameworks for data pipelines and performance metrics to identify and address issues proactively.
Staying updated on emerging trends and best practices in data engineering and Azure technologies.
Leading teams to achieve project goals, providing guidance and mentorship to junior team members to foster growth.