Job Description

About us

AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.

If you’re looking for a place to grow, make an impact, and work with people who care, we’d love to meet you!

About the Role

As a Data Engineer, you will build and scale robust, cloud-native data pipelines that power large-scale financial data processing and analytics. Working with Python, SQL, and Microsoft Fabric on Azure, you’ll design distributed, fault-tolerant systems, implement advanced data models, and ensure high data quality and compliance. This role offers strong ownership, cross-functional collaboration, and the opportunity to integrate AI-driven solutions into modern data platforms.

What you will do

Build and maintain scalable, distributed, fault-tolerant data pipelines using Microsoft Fabric;
Develop and manage lakehouse layers and Delta Lake workflows for data processing;
Collaborate with stakeholders across data engineering, compliance, and business teams;
Design and implement pipelines to acquire, normalize, transform, and release large volumes of financial data;
Design and implement bitemporal data models for regulatory-grade time-series datasets;
Build and maintain testing frameworks for data pipelines and transformation logic;
Own end-to-end solutions including ingestion pipelines, QA workflows, correction management, and audit trails;
Contribute to shared platform services in a collaborative environment;
Support implementation of AI solutions including data ingestion, anomaly detection, and semantic search.

Must haves

6–8 years of experience in data engineering;
Proficiency in Python for data pipelines, transformation logic, and automation;
Proficiency in SQL including window functions, partitioning, and time-series queries;
Hands-on experience with Microsoft Fabric (OneLake, Data Factory, Lakehouse, Warehouse);
Working knowledge of Delta Lake including incremental merges and Change Data Feed;
Experience with AI-assisted development tools such as GitHub Copilot or similar;
Experience with Git version control and collaboration workflows;
Familiarity with REST APIs for integrations;
Familiarity with Azure technologies (Azure Data Factory, Azure SQL, Azure Key Vault, RBAC);
Understanding of financial data concepts related to equities and other asset classes;
Upper-intermediate English level.

Nice to have

Knowledge of data libraries such as pandas or PySpark;
Experience with columnar storage and time-series analytics tools such as ClickHouse;
Familiarity with Microsoft Purview for data governance;
Understanding of bitemporal data modeling concepts;
Knowledge of financial reference data such as equities, fixed income, or corporate actions;
Experience with CI/CD pipelines and automated deployments;
Exposure to LLMs and Agentic AI for data-related use cases.

Perks and Benefits

Remote work & Local connection: Work where you feel most productive and connect with your team in periodic meet-ups to strengthen your network and connect with other top experts.
Legal presence in India: We ensure full local compliance with a structured, secure work environment tailored to Indian regulations.
Competitive Compensation in INR: Fair compensation in INR with dedicated budgets for your personal growth, education, and wellness.
Innovative Projects: Leverage the latest tech and create cutting-edge solutions for world-recognized clients and the hottest startups.