RESPONSIBILITIES
· We are looking for a Big Data&AI Engineer with experience in designing, building and managing applications to process large amounts of data in a Hadoop/ Spark ecosystem. You’ll provide expert guidance to source and integrate structured and unstructured data from dozens of local data sources including streaming data feeds into a data lake.
· As a Big Data&AI Engineer, you will work closely with the team and our stakeholders to build and deliver our Hadoop based solutions for a next generation Big Data analytics platform. You will have a focus on data streaming, data management, data quality, data security and deliver the systems that process huge volumes of data. Working with our Data Scientists to help drive the company strategy and marketing campaigns out to our customers.
· At Vodafone, we don’t just produce creative products, we develop amazing people too. We are driven to empower people. We are committed to helping our people perform at their best and achieve their full potential.
KEY ACCOUNTABILITIES
· You’ll be designing and producing high performing and stable applications to perform complex processing of petabyte scale data in a Hadoop based environment.
· Working alongside Data Scientists to build end-to-end applications that make use of large volumes of source data from the operational systems and output insights back to business systems
· Building real-time data processing applications which are integrated with business systems to enable value from analytic models to drive rapid decision making
· Sourcing, ingesting, wrangling and validating data sets, building pipelines to transform data and produce analytical records for machine learning
· You’ll have extensive experience with performance tuning applications on Hadoop and configuring Hadoop based systems to maximise performance.
QUALIFICATIONS
· BS degree or higher in a technology related field (e.g. Computer Science, Math, Information Systems, Industrial Engineering or another quantitative field)
· Minimum 2 years experience in designing, building and managing applications to process large amounts of data in a (Cloudera) Hadoop ecosystem (Spark, Hive/Impala, HBase, Yarn, Oozie)
· Strong software development experience in Python, Java or Scala programming languages (other programming languages desirable);
· Experience in designing, building and managing applications to process large amounts of data in a Hadoop ecosystem
· Experience building systems to perform real-time data processing using Spark Streaming, Storm and Kafka, or similar technologies
· Experience with Linux systems, including bash programming
· Experience with other distributed technologies such as Cassandra, Solr/Elastic Search, Flink, Flume is a plus
· Building end-to-end applications that make use of large volumes of source data from the operational systems and output insights back to business systems
· Experience with data pipeline and workflow management tools (Apache Airflow, etc.) is a plus
· Building real-time data processing applications which are integrated with business systems to enable value from analytic models to drive rapid decision making
· Experience with ML and Deep Learning is a plus
