Expert NiFi Data Flow Data Engineer (PROJ-4741)
Job Description
Remote is seeking an Expert NiFi Data Flow Data Engineer to join the team in ASD. The Data Engineer will plan, develop and support data engineering solutions that prepare, integrate and make data accessible for analytical and operational use.
The role will involve integrating data from multiple sources, building scalable and reusable data ingestion and transformation components, and supporting efficient data storage, retrieval and analytics. The Expert NiFi Data Engineer will also contribute to data flow and integration strategies, monitor alignment with data standards, architectures and security requirements, and support the performance, availability and continuous improvement of data pipelines and data engineering practices. ( LH-06923)
Role Description
Key duties and responsibilities
- Plan and drive the development of data engineering solutions, balancing functional and non-functional requirements.
- Monitor application of data standards, architectures and security, ensuring compliance and scalability.
- Integrate data from multiple sources and formats into centralised systems.
- Build scalable and reusable NiFi templates and processors for data ingestion and transformation.
- Perform data modelling to support efficient data storage, retrieval and analytics
- Collaborate with cross functional teams to define data flow and integration strategies.
- Optimise performance and ensure high availability of data pipelines.
- Develops and promotes continuous integration, deployment and monitoring practices.
- Contributes to organisational policies, standards and guidelines for data engineering.
Technical skills
- Degree in ICT, Data Science or equivalent work experience.
Essential criteria
- Data engineering: Level 6 (SFIA)
Leads the selection and development of data engineering methods, tools and techniques. Develops organisational policies, standards, and guidelines for the development and secure operation of data services and products. Ensures adherence to technical strategies and architectures. Plans and leads data engineering activities for strategic, large and complex programmes.
- Data modelling and design: Level 5 (SFIA)
Sets standards for data modelling and design tools and techniques, advises on their application and ensures compliance. Manages the investigation of enterprise data requirements based upon a detailed understanding of information requirements. Coordinates the application of analysis, design and modelling techniques to establish, modify or maintain data structures and their associated components. Manages the iteration, review and maintenance of data requirements and data models.
- Integrate data: Level 3 (DCF)
Can perform and provide expert advice on data integration Can build capability of others in understanding and applying good data integration practice and principles Can write custom scripts and code (programming language) to integrate data Maintains understanding of new trends and innovations relating to integrating data (including Artificial Intelligence-based technologies) and develops skills in these where relevant
- Storage management: Level 5 (SFIA)
Develops standards and guidelines for implementing data protection and disaster recovery functionality for all business applications and business data. Provides expert advice and guidance to implement and improve storage management. Manages storage and backup systems to provide agreed service levels. Creates, improves and supports storage management services with optimal utilisation of storage resources, ensuring security, availability and integrity of data.