Senior Data and Research Data Engineer (PROJ-4536)

Job Duties and Responsibilities

• Identify and undertake analysis to define, structure and solve complex data engineering challenges. 
• Understand, use and build tools and techniques to develop and test data engineering solutions to achieve business outcomes. 
• Prepare, publish and provide advice on requirements, current and future state models and plans, designs and solutions to improve data interoperability, storage, analysis and transmission.
• Build maintain and deploy data pipelines analytical environments and models and data stores to store transform, integrate, analyse and transport data and meta-data from a variety of sources to enable others to use and analyse data.
• Responsible for the ethical use of data and interpreting and complying with legislation, Defence policy and regulatory frameworks including data management and engineering requirements.
• Responsible for contributing to the management of the risks associated with the use of data including ensuring the security of data storage and transmission mechanisms.
• Build and sustain strong relationships and negotiate with customers and key stakeholders to deliver tailored recommendations.
• Represent and advocate for the workgroup and Defence at conferences, seminars, briefings and meetings to support the Defence mission to technical and non-technical audiences.
• Resolve problems and evaluate lessons learned using expertise, professional judgement and initiative to identify and recommend alternative courses of action and opportunities for continuous improvement.
• Accountable for accurate and timely completion of work, sharing own expertise with others and guiding and mentoring less experienced employees.
• Develop and maintain business domain knowledge, data security and engineering skills, knowledge of the Defence data architecture and awareness of trends.
• Work collaboratively, operates as an effective team member and actively participates in organisational activities

Technical skills

• Holds a Bachelor Degree in information technology, computer science, computer engineering, software engineering, applied math, statistics, physics or a related field
• Holds a basic understanding of programming languages
• Knowledge of operating systems
• Strong understanding of analytics software
• Define, structure and solve complex data engineering challenges.

Essential criteria
Business intelligence: Level 3 (SFIA)

Sources and prepares data for analysis and performs standard business intelligence analysis activities. Creates and delivers standard reports in accordance with stakeholder needs and conforming to agreed standards. Investigates the need for new or revised business intelligence analysis. Contributes to the recommendation of improvements. Engages with stakeholders under direction.

Data engineering: Level 5 (SFIA)

Plans and drives the development of data engineering solutions ensuring that solutions balance functional and non-functional requirements. Monitors application of data standards and architectures including security and compliance. Contributes to organisational policies, standards, and guidelines for data engineering.

Data management: Level 5 (SFIA)

Devises and implements master data management processes. Derives data management structures and metadata to support consistency of information retrieval, combination, analysis, pattern recognition and interpretation, throughout the organisation. Plans effective data storage, sharing and publishing within the organisation. Independently validates external information from multiple sources. Assesses issues that might prevent the organisation from making maximum use of its information assets. Provides expert advice and guidance to enable the organisation to get maximum value from its data assets.

Data modelling and design: Level 4 (SFIA)

Investigates enterprise data requirements where there is some complexity and ambiguity. Plans own data modelling and design activities, selecting appropriate techniques and the correct level of detail for meeting assigned objectives. Provides advice and guidance to others using the data structures and associated components.

Measurement: Level 4 (SFIA)

Identifies and prioritises appropriate measures, scales, and targets. Supports projects, functions or teams in the development of measurement methods. Specifies base and derived measures which support agreed information needs. Specifies how to collect and store the data for each required measure. Provides guidance on collection of data. Designs reports and reporting formats.

Quality assurance: Level 4 (SFIA)

Plans, organises and conducts assessment activity and determines whether appropriate quality control has been applied. Conducts formal assessments or reviews for given domain areas, suppliers, or parts of the supply chain. Collates, collects and examines records, analyses the evidence and drafts all or part of formal compliance reports. Determines the risks associated with findings and non-compliance and proposes corrective actions. Provides advice and guidance in the use of organisational standards.