Big Data Engineer

What makes you a great fit:

• Expert-level proficiency in PySpark and python knowledge
  
• Strong understanding and experience in distributed computing frameworks, particularly Apache Hadoop (YARN, MR, HDFS) and associated technologies - one or more of Hive, Sqoop, Avro, Flume, Oozie, Zookeeper, Impala, etc
  
• Hands-on experience with Apache Spark and its components (Streaming, SQL, MLLib) is a strong advantage.
• Operating knowledge of cloud computing platforms (AWS/Azure/GCP)
• Experience working within a Linux computing environment, and use of command line tools including knowledge of Shell/Python scripting for automating common tasks
• Must be proficient in any Cloud computing platforms (AWS/Azure/GCP)
  
• Experience in GCP (Big Query/Bigtable, Pub sub, Data Flow, App engine )/ AWS, Azure would be preferred

• The role would involve big data pre-processing & reporting workflows including collecting, parsing, managing, analyzing and visualizing large sets of data to turn information into business insights
• Develop the software and systems needed for end-to-end execution on large projects
• Work across all phases of SDLC, and use Software Engineering principles to build scalable solutions
• Build the knowledge base required to deliver increasingly complex technology projects
• You would be responsible for evaluating, developing, maintaining and testing big data solutions for advanced analytics projects
• The role would also involve testing various machine learning models on Big Data, and deploying learned models for ongoing scoring and prediction.