Job Description

We are looking for Senior Data Engineer role, and this role based here in Singapore.

Job Description

Key Responsibilities:

Developing and documenting technical solution designs
Test automation and tools such as Great expectations or Delta Live tables
Build and release management e.g. Azure Devops Pipelines, Azure Devops Release Manager
Source code management e.g. Azure Devops, Git
DevSecOps approach and tools for cloud (including automation of build, deployment, environment management and operational tooling for Azure PaaS and IaaS)
Agile delivery methodologies such as SCRUM or Kanban
Knowledge and work management tools (e.g. JIRA, Confluence)

Required Skills & Qualifications:

SQL Server database design and development (SQL, relational and dimensional modelling)
Analytical querying using technologies such as SSAS cubes or columnar data stores
ETL tools such as SSIS, Alteryx, Azure Data Factory
Building data pipelines using code e.g. Pyspark
Modern cloud-based data architectures and technologies such as Data Lake, Delta Lake, Data Lakehouse e.g. Azure Data Lake Gen 2, Azure Delta Lake
Lakehouse architecture using Databricks, Hudi or Iceberg
NoSQL using CosmosDB, MongoDB, DynamoDB etc.
Python scripting, including data engineering and machine learning libraries
Scripting languages such as powershell, bash, DAX, M-language etc. Note - We primarily use SQL, Pyspark and Python in our day-to-day work along with some coding around DAX, Bash, Powershell and other scripting languages.
Distributed computing mainly around Spark
Big data files formats e.g. Parquet, Avro, ORC etc.
Big data technologies such as Hadoop, Hive, Map-Reduce, Partition design, performance tuning in Spark etc.
Reporting and visualisation tools such as SSRS, Tableau, Power BI or Excel