Senior Data Engineer (PROJ-4751)
Job Description
Remote is seeking a Data Engineer to join the team at ASD. The Data Engineer will design, develop and support data engineering solutions that make data accessible for analytical and operational use, enabling improved services, optimisation and enhanced analytics.
The role will involve analysing and resolving complex data engineering challenges, building and maintaining data pipelines, analytical environments, models and data stores, and transforming and integrating data from multiple sources. The Data Engineer will also provide advice on requirements, current and future state designs, data interoperability and storage, while supporting data quality, security and the effective management of risks associated with data use and transmission. (LH-07016)
Role Description
Key duties and responsibilities
- Identify and undertake analysis to define, structure and solve complex data engineering challenges.
- Understand, utilise and build tools and techniques to develop and solve complex data engineering solutions to achieve business outcomes.
- Prepare, publish and provide advice on requirements, current and future state models and plans, designs and solutions to improve data interoperability, storage, analysis and transmission.
- Build, maintain and deploy data pipeline analytical environments and models and data stores to store, transform, integrate, analyse and transport data and metadata from a variety of sources to enable others to use and analyse data.
- Contribute to the management of the risks associated with the use of data including ensuring the security of data storage and transmission mechanisms.
Technical skills
- Demonstrated experience in Data Engineering.
- Proficient in data tools and programming
- Demonstrated ability to implement data quality controls, perform data cleansing and transformation
Essential criteria
- Data modelling and design: Level 4 (SFIA)
Investigates enterprise data requirements where there is some complexity and ambiguity. Plans own data modelling and design activities, selecting appropriate techniques and the correct level of detail for meeting assigned objectives. Provides advice and guidance to others using the data structures and associated components.
- Data use and re-use: Level 2 (DCF)
Understands and can articulate the value of data in terms of use and re-use Implements a variety of techniques to ensure data is open and can be used beyond the specific purpose for which it was collected, in accordance with legislative requirements and organisational guidelines (includes data sharing legislation)
- Integrate data: Level 2 (DCF)
Can perform data integration using standard tools and can implement quality controls Knows where to obtain expert advice on data integration as needed Is able to understand and amend existing code (programming language) to integrate data Can assess datasets' suitability for linkage and extract appropriately prepared datasets
- Storage management: Level 4 (SFIA)
Prepares and maintains operational procedures for storage management. Monitors capacity, performance, availability and other operational metrics. Takes appropriate action to ensure corrective and proactive maintenance of storage and backup systems to protect and secure business information. Creates reports and proposals for improvement. Contributes to the planning and implementation of new installations and scheduled maintenance and changes of existing systems.