Data Engineer

Senior Data Engineer (AWS & Snowflake Specialist)

Experience Level: 5–6 Years

Role Overview

This role is for a data powerhouse who can bridge the gap between high-performance big

data processing on AWS and elite-level data warehousing in Snowflake. You will be

responsible for the entire lifecycle of data, ensuring that every pipeline includes robust

validation and observability.

Key Responsibilities

● Big Data Processing: Leverage AWS EMR (Spark/Hive) for heavy-lift data

processing and Athena for serverless ad-hoc querying of S3 data lakes.

● Data Validation: Implement Great Expectations (GX) or equivalent Python-based

validation libraries to prevent "data silent failure" within EMR and Snowflake ingestion

layers.

● Snowflake Engineering: Architect end-to-end Snowflake environments, utilizing

SnowSQL for scripting and SnowPipe for continuous, automated data ingestion.

● System Integration: Build and maintain Snowflake Integrations (Storage

Integrations with S3, API Integrations, and Notification Services).

● Advanced ETL: Develop robust pipelines that transition data from raw AWS storage

into optimized Snowflake schemas.

Technical Requirements

● Core Languages (Strong Proficiency Required):

○ SQL: Expert mastery of Snowflake-specific SQL extensions, CTEs, and

performance tuning (Micro-partitioning).

○ Python: Advanced scripting for automation, Boto3 for AWS resource

management, and data manipulation.

● Data Quality: Proven experience implementing automated data testing and

validation frameworks within production pipelines.

● AWS Stack: Extensive experience with EMR, Redshift, Athena, S3, and IAM

security policies.

● Snowflake Mastery: * Proven experience with Snowflake Data Engineering

features (Streams, Tasks, and External Tables).

○ Hands-on usage of SnowPipe for real-time or near-real-time data loading.