Expert Data Engineer (PROJ-4736)

Key duties and responsibilities

• Identify and undertake analysis to define, structure and solve complex data engineering challenges.
• Understand, use and build tools and techniques to develop and test data engineering solutions to achieve business outcomes.
• Provide advice on requirements, current and future state models and plans, designs and solutions to improve data interoperability, storage, analysis and transmission.  
• Build, maintain and deploy data pipelines, analytical environments and data stores to store, transform, integrate, analyse and transport data and metadata from a variety of sources to enable the use and analysis of data. 
• Conduct diagnostic procedures to identify system vulnerabilities and improvements. 
• Identify process improvements, apply metadata frameworks and assist with the governance of data engineering design patterns and practices. 

Technical skills

• Degree in Computer Science, IT or similar Demonstrated experience in analytics software

Essential criteria

• Data engineering: Level 6 (SFIA)

Leads the selection and development of data engineering methods, tools and techniques. Develops organisational policies, standards, and guidelines for the development and secure operation of data services and products. Ensures adherence to technical strategies and architectures. Plans and leads data engineering activities for strategic, large and complex programmes.

• Data modelling and design: Level 5 (SFIA)

Sets standards for data modelling and design tools and techniques, advises on their application and ensures compliance. Manages the investigation of enterprise data requirements based upon a detailed understanding of information requirements. Coordinates the application of analysis, design and modelling techniques to establish, modify or maintain data structures and their associated components. Manages the iteration, review and maintenance of data requirements and data models.

• Data use and re-use: Level 3 (DCF)

Is a leading strategic adviser on the use of organisational data assets Is an expert resource on the design and management of organisational data assets as open data. Promotes re-use and ongoing value of data across wider contexts, in accordance with legislative requirements and organisational guidelines (includes data sharing legislation)

• Improvement and Innovation - Data processes/systems and tools/products: Level 3 (DCF)

Thinks strategically to assess current processes/systems and tools/ products across a broad context Develops improvements where needed and encourages others to think critically about processes/systems and tools/ products relevant to them Advises those leading changes to processes / systems and tools/ products, measures resultant benefits, and makes recommendations Understands the impact of new trends and innovations on organisational data processes, systems and tools, and products Maintains understanding of new trends and innovations relating to data processes, systems and tools, and products within the organisation and externally. Provides technical expertise for systems testing, including validating the quality of the testing approach and results

• Integrate data: Level 3 (DCF)

Can perform and provide expert advice on data integration Can build capability of others in understanding and applying good data integration practice and principles Can write custom scripts and code (programming language) to integrate data Maintains understanding of new trends and innovations relating to integrating data (including Artificial Intelligence-based technologies) and develops skills in these where relevant