Job Description

Role Description

Collaborate with other teams to design and develop and deploy data tools that support both operations and product use cases

Perform analysis of large data sets using components from the Hadoop ecosystem
Own product features from the development, testing through to production deployment
Evaluate big data technologies and prototype solutions to improve our data processing architecture
Automate different pipelines
BS degree in computer science, computer engineering or equivalent
4-7 years of experience delivering enterprise software solutions
Familiar with Spark, Scala, Python, AWS Cloud technologies
2+ years of experience across multiple Hadoop / Spark technologies such as Hadoop, MapReduce, HDFS, HBase, Hive, Flume, Sqoop, Kafka, Scala
Flair for data, schema, data model, how to bring efficiency in big data related life cycle.
Experience with Agile Development methodologies.
Experience with data ingestion and transformation
Have understanding for secure application development methodologies.
Experience in with Airflow and Python will be preferred.
Understanding of automated QA needs related to Big data technology.
Strong object-oriented design and analysis skills
Excellent written and verbal communication skills Responsibilities
Utilize your software engineering skills including Spark, Python, Scala to Analyze disparate, complex systems and collaboratively design new products and services
Integrate new data sources and tools
Implement scalable and reliable distributed data replication strategies

Skills

big data,hadoop,spark,hbase,hive,

Interview Questions of Lead I - Software Engineering at UST

Interview questions from UST that are similar to Lead I - Software Engineering