Big Data Engineer

Platform Operations

●      Operate and support CDP Private Cloud environments across Dev/Test/Prod.

●      Perform BAU activities: monitoring, patching, upgrades, configuration tuning.

●      Administer Cloudera Manager, alerts, health checks, and diagnostics.

●      Ensure platform stability, availability, and SLA compliance.

Hadoop Ecosystem Support

●      Support core services: HDFS, YARN, Hive, Impala, Spark, Kafka, NiFi, HBase.

●      Manage HDFS quotas, snapshots, replication, and lifecycle policies.

●      Support ingestion pipelines and troubleshoot platform-level issues.

Performance & Capacity

●      Monitor and tune Hive/Impala queries and Spark workloads.

●      Manage YARN queues and resource allocation.

●      Perform capacity planning for compute and storage growth.

High Availability & Resilience

●      Operate HDFS HA, Resource Manager HA, and Zookeeper.

●      Support service failover, balancing, and resilience testing.

Security & Governance

●      Administer Apache Ranger (RBAC, data masking).

●      Integrate with LDAP/Active Directory.

●      Enforce encryption (TLS, HDFS TDE) and audit logging.

●      Support governance tooling (Atlas, lineage, audits).

●      Ensure compliance with enterprise security standards (e.g. IM8, PDPA).

Monitoring, Automation & Incident Management

●      Configure Cloudera Manager alerts and monitoring thresholds.

●      Develop automation scripts (Shell, Python).

●      Support log analysis (ELK, Splunk).

●      Participate in 24x7 on-call, incident response, and RCA.

●      Maintain runbooks and operational documentation.

Backup & Disaster Recovery

●      Manage HDFS backup and replication (snapshots, DistCp).

●      Support metadata backup and recovery.

●      Participate in DR drills and recovery validation.

Required Skills & Qualifications

Education : Degree/Diploma in IT, Computer Science, or Engineering.
Experience : 2–5+ years experience administering Cloudera / Hadoop platforms.

EA License # 14C6941