Data Engineer (EMEA)

Job Description

This role is for applicants on EMEA timezones. The data engineering team is responsible for designing and implementing scalable solutions that allow the company to make data-driven decisions fast and accurately on several terabytes of data.

The team maintains the company’s data warehouse and data-lake, and you will be responsible for creating various pipelines to move and process vast amounts of data into the different data products.

The team deals with both batch and streamed data, and split into different responsibilities and areas matching both the engineer’s and Kraken’s interest.

Responsibilities

1. Build scalable and reliable data pipeline that collects, transforms, loads and curates data from internal systems
   
2. Augment data platform with data pipelines from select external systems
   
3. Ensure high data quality for pipelines you build and make them auditable
   
4. Drive data systems to be as near real-time as possible
   
5. Support design and deployment of distributed data store that will be central source of truth across the organization
   
6. Build data connections to company’s internal IT systems
   
7. Develop, customize, configure self service tools that help our data consumers to extract and analyze data from our massive internal data store
   
8. Evaluate new technologies and build prototypes for continuous improvements in data engineering.

Qualification

1. 5+ years of work experience in relevant field (Data Engineer, DWH Engineer, Software Engineer, etc)
   
2. Experience with data warehouse technologies and relevant data modeling best practices (Presto, Druid, etc)
   
3. Experience building data pipelines/ETL and familiarity with design principles (Apache Airflow is a big plus!)
   
4. Excellent SQL and data manipulation skills using common frameworks like Spark/PySpark, Pandas, or similar
   
5. Proficiency in a major programming language (e.g. Scala, Python, Golang,..)
   
6. Experience with business requirements gathering for data sourcing.

Skills Needed

• Experience working with cloud services (e.g. AWS, GCP, ..) and/or Kubernetes.
  
• Experience in building and contributing to data lakes on the cloud.
  
• Designing and writing CI/CD pipelines.
  
• Working with petabytes of data.
  
• Enjoys Dockerizing services.