Job description
Data Engineer
A progressive technology company are looking for a Data Engineer to join their talented team of Data Scientists.
The Data Engineer will have the opportunity to help drive the architecture and development of data processing.
The responsibilities of the Data Engineer will include:
- Maintenance of the data science processing infrastructure.
- Building a scalable data processing pipeline.
- Developing tools to analyse large data-sets by using relevant concepts (i.e. MapReduce)
- Development of processes and procedures to manage data ETL pipelines.
- To work closely with the Dev Ops teams to assist in their product delivery.
- Identify and champion new approaches to big data.
The Data Engineer will have the following skills and experience:
- Strong Python skills for Data Management
- Understanding of multiple database technologies (Relational, nosql, Graph)
- Knowledge of Postgresql (advantageous)
- Understanding of distributed computational frameworks (Dask & PySpark)
- Experience with Linux, Docker and Kubernetes
- Strong Data Governence skills
- Knowledgeable on production knowledge of distributed databases.
- Experience of using big data technologies - (Hadoop, Cassandra, Spark, Dask)
- Good understanding of database design.
- Knowledge of data warehouse design.
- Able to create optimised SQL queries.
- Knowledge of AWS, GCP or Azure cloud.
This is a great opportunity to join a company who are committed to using Data and Software to improve a high profile industry. The organisation also offer an element of flexible working hours to give employees some balance if required.
To apply for the Data Engineer role please send your CV to CameronElloway@SpectrumIT.co.uk or call on 02380 765 301.
