Data Engineer/ Data Warehouse – ETL Developer (PROJ-4729)
Job Description
Remote is seeking a Data Engineer/ Data Warehouse – ETL Developer to join the team in one of our Commonwealth clients. The Data Engineer/ Data Warehouse – ETL Developer will be responsible for supporting the management, transformation of data for use by business areas. This includes the development and sustainment of data pipelines utilising Enterprise Data Platforms. They will also assist in the oversight of end-to-end data management processes, using appropriate tools and techniques that conform to agreed process standards and industry specific regulations. Working in a diverse team environment, this will involve sharing outcomes and experience with others to support development and growth. (RFI 55-2526)
Role Description
Key Responsibilities
- Support the design, implementation, and maintenance of data engineering solutions in SQL and Python.
- Build tooling and techniques which are well tested, documented and provide consistency in data engineering approaches.
- Fuse data sources using various data pre-processing techniques such as transformation, integration, normalisation, and feature extraction to create useful data structures.
- Carry out data quality checking and remediation
- Develop technical documentation and track work in collaboration tools.
Required Knowledge/Experience and tools
- 3+ years of relevant Data Engineering/ Data Warehousing experience
- Ability to develop, deploy and maintain scalable data engineering solutions in SQL and Python
- Ability to create and maintain standardised tooling and techniques, ensuring consistent ETL and data warehousing approaches, accompanied by thorough testing and documentation
- Ability to combine data sources using various data pre-processing techniques such as transformation, integration, normalisation, and feature extraction to create useful data structures
- Conduct data quality assurance and perform remediation as needed
- Demonstrated ability to craft technical documentation and effectively collaborate using a variety of tools to ensure seamless tracking and communication
Highly Desirable
- Proficiency in platform engineering, including containerisation
- Skills in designing and building APls to facilitate seamless integration
- Proficiency in technologies that control the flow of data
- Experience in a range of data storage platforms, including Online Analytical Processing (OLAP), Relational databases (RDBMS) and Document stores
- Tertiary qualification in a technology field