Responsibilities/Tasks

• Identify the most appropriate data sources to use for a given purpose and understand their structures and contents, in collaboration with SMEs when required
• Extract structured and unstructured data from the source systems (relational databases, data warehouses, document repositories, file systems, …), prepare such data (cleanse, re-structure, aggregate, …) and load them onto Hadoop.
• Actively support reporting teams in the data exploration and data preparation phases. Where data quality issues are detected, liaise with the data supplier to do root cause analysis
• Contribute to the design, build, and launch activities
• Ensure the maintenance and support of production applications
• Liaise with Technology Services teams to address infrastructure issues and to ensure that the components and software used on the platform are all consistent

Experience/Exposure

• Experience with understanding and creating data flows, with data architecture, with ETL/ELT development and with processing structured and unstructured data
• Proven experience with using data stored in RDBMSs and experience or good understanding of NoSQL databases
• Ability to write performant SQL statements
• Ability to analyze data, to identify issues like gaps and inconsistencies and to do root cause analysis
• Knowledge of Java Understanding of the Hadoop ecosystem including Hadoop file formats like Parquet and ORC
• Experience with open-source technologies used in Big Data analytics like Scala, Spark, Pig, Hive, HBase, Kafka etc.
• Ability to write MapReduce & Spark jobs
• Knowledge of Cloudera
• Experience delivering scripts
• Experience in working with customers to identify and clarify requirements
• Ability to design solutions that are fit for purpose whilst keeping options open for future needs
• Strong verbal and written communication skills, good customer relationship skills
• Have a true agile mindset, capable and willing to take on tasks outside of her/his core competencies to help the team
• Strong technical skills and a strong interest in the financial industry.

Please be aware we are not necessarily expecting each candidate to cover all the above listed technologies.

Education/Certification

• Bachelor’s degree