Job Description

AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.

WHY JOIN US

If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you!

About The Role

WHAT YOU WILL DO

Build and maintain scalable, distributed, fault-tolerant data pipelines on GCP;
Develop and manage lakehouse layers and Delta Lake workflows using BigQuery and Dataproc;
Collaborate with stakeholders across data engineering, compliance, and business teams;
Design and implement pipelines to acquire, normalise, transform, and release large volumes of financial data;
Design and implement bitemporal data models on BigQuery for regulatory-grade time-series datasets;
Build and maintain testing frameworks for data pipelines and transformation logic;
Own end-to-end solutions including ingestion pipelines, QA workflows, correction management, and audit trails;
Contribute to shared platform services in a collaborative environment;
Support implementation of AI solutions including data ingestion, anomaly detection, and semantic search using Vertex AI.

MUST HAVES

6–8 years of experience in data engineering;
Proficiency in Python for data pipelines, transformation logic, and automation;
Proficiency in SQL with hands-on experience in BigQuery including partitioning, clustering, and time-series queries;
Experience with Cloud Composer (Apache Airflow) for pipeline orchestration;
Working knowledge of Dataproc (Apache Spark) for batch ingestion and incremental processing;
Experience with AI-assisted development tools such as GitHub Copilot or similar;
Experience with Git version control and collaboration workflows;
Familiarity with REST APIs for integrations;
Familiarity with GCP technologies (Cloud Storage, Pub/Sub, Datastream, Cloud Monitoring, IAM, VPC Service Controls);
Understanding of financial data concepts related to equities and& other asset classes;
Upper-intermediate English level.

NICE TO HAVES

Knowledge of data libraries such as pandas or PySpark;
Experience with columnar storage and time-series analytics tools such as ClickHouse;
Familiarity with Dataplex for data governance and lineage;
Understanding of Change Data Capture (CDC) using Datastream;
Understanding of bitemporal data modeling concepts;
Knowledge of financial reference data such as equities, fixed income, or corporate actions;
Experience with BigQuery cost management techniques;
Experience with CI/CD pipelines and Terraform for infrastructure as code;
Exposure to LLMs and& Agentic AI using Vertex AI for data-related use cases.

Perks And Benefits

Remote work & Local connection: Work where you feel most productive and connect with your team in periodic meet-ups to strengthen your network and connect with other top experts.
Legal presence in India: We ensure full local compliance with a structured, secure work environment tailored to Indian regulations.
Competitive Compensation in INR: Fair compensation in INR with dedicated budgets for your personal growth, education, and wellness.
Innovative Projects: Leverage the latest tech and create cutting-edge solutions for world-recognized clients and the hottest startups.