Lead Azure Data Engineer

We are looking for a skilled and experienced Lead Azure Data Engineer to join our Data team in Hyderabad location.

About the job

Position: Lead Azure Data Engineer

Total Experience: Minimum 5+ years to 8 years.

Notice Period: Immediate

Location: Hyderabad

Work Mode: Work From Office (WFO) only

Must have skills: ADF, Synapse, Azure SQL, Snowflake, DW Concepts, SQL, ETL tool, Python

Job Description: Lead Azure Data Engineer

We are seeking a highly skilled and motivated Azure Data Engineer with over 5+ years of experience to join our dynamic data engineering team. In this role, you will play a crucial part in designing, building, and maintaining data pipelines, data warehouses, and data lakes using Azure Data Factory, Snowflake, Azure Synapse Analytics, and DataBricks. Your expertise in these technologies will drive the effective extraction, transformation, loading, and analysis of our organization's data assets.

Technical Skills:

• Experience should have 5+ years to 8 years as a Data engineer.
• Bachelor's degree in Computer Science, Engineering, or a related field.
• Proven experience as an Azure Data Engineer, with hands-on expertise in Azure Data Factory, Snowflake, Synapse Analytics, and DataBricks.
• Strong understanding of data warehousing concepts, data modeling, and ETL processes.
• Proficiency in SQL and T-SQL for data manipulation, query optimization, and performance tuning.
• Hands-on experience in data integration, data transformation, and data loading using Azure services.
• Experience with building and managing data lakes and data warehouses in Snowflake and Azure Synapse Analytics.
• Proficiency in PySpark and DataBricks for advanced data processing and analysis.
• Familiarity with cloud architecture, security, and best practices within the Azure ecosystem.
• Strong problem-solving skills and the ability to troubleshoot complex data engineering issues.
• Excellent communication skills and the ability to collaborate effectively within cross-functional teams.

Responsibilities:

• Collaborate with data architects, analysts, and business stakeholders to understand data requirements and translate them into scalable data engineering solutions.
• Design, develop, and manage end-to-end data pipelines using Azure Data Factory, ensuring the reliable extraction, transformation, and loading of data from diverse sources.
• Create and maintain data models, schemas, and data structures in Snowflake and Azure Synapse Analytics to support reporting, analytics, and business intelligence needs.
• Utilize DataBricks and PySpark to develop advanced data processing pipelines for batch and real-time data processing.
• Optimize and fine-tune data pipelines and ETL processes to ensure efficient data movement and transformation.
• Implement data quality checks, data validation, and data profiling to ensure data accuracy and reliability.
• Work closely with DevOps teams to deploy, monitor, and troubleshoot data pipelines and data engineering solutions in production environments.
• Collaborate with data scientists, machine learning engineers, and analysts to enable data-driven decision-making and predictive analytics.
• Stay updated with the latest advancements in Azure data technologies, industry best practices, and emerging trends in data engineering.
• Provide guidance and mentorship to junior team members, promoting technical excellence and continuous learning.