What you'll do:

• Develop high quality, secure and scalable data pipelines using spark, Scala/ python on Hadoop or object storage.
• Leverage new technologies and approaches to innovate with increasingly large data sets.
• Drive automation and efficiency in Data ingestion, data movement and data access workflows by innovation and collaboration.
• Understand, implement and enforce Software development standards and engineering principles in the Big Data space.
• Work closely with business stakeholders and embedded engineering teams within business teams in a collaborative manner to help them build scalable products in quick time.
• Contribute ideas to help ensure that required standards and processes are in place and actively look for opportunities to enhance standards and improve process efficiency.
• Perform assigned tasks and production incident independently.

What makes you a great fit:

• Experience in building, and maintaining large-scale applications.
  
• Expertise in Python, spark, Hadoop/HDFS
• Experience of building data ingestion pipelines
• Understanding of RDBMS and NoSql databases including wide-column databases
• Understanding of messaging systems e.g. Kafka, Pulsar
• Exposure of working with Pandas, Nump
• Ability to understand Business challenges and define solutions aligned to the needs.