GCP Data Engineer - Big Data

Skill: GCP Data Engineer - Big Data

Experience: 5.5 to 10 years

Location: Pune/Hyderabad

Job Summary:

We are seeking an experienced Data Engineer with strong expertise in GCP and Big Data technologies. The ideal candidate will be responsible for building scalable data pipelines, handling large datasets, and enabling efficient data processing using modern cloud-based solutions.

Key Responsibilities:

• Design, develop, and maintain scalable and high-performance data pipelines.
• Work extensively with GCP services, especially BigQuery for data warehousing and analytics.
• Develop data processing frameworks using Python (PySpark).
• Handle large-scale distributed data processing using Big Data technologies.
• Optimize data pipelines for performance, reliability, and cost efficiency.
• Collaborate with Data Scientists, Analysts, and business stakeholders to deliver data solutions.
• Ensure data quality, governance, and security best practices.
• Troubleshoot and resolve data-related issues in production environments.
• Participate in architecture discussions and contribute to data platform design.

Mandatory Skills:

• Strong hands-on experience with Google Cloud Platform (GCP)
• Expertise in BigQuery
• Proficiency in Python (PySpark)
• Solid experience in Big Data technologies (Hadoop/Spark ecosystem)

Good to Have:

• Experience with data pipeline orchestration tools (e.g., Airflow)
• Knowledge of data modeling and data warehousing concepts
• Familiarity with streaming tools like Kafka
• Exposure to CI/CD and DevOps practices