Data Flow Engineer

TL;DR

Data Flow Engineer (Apache NiFi/Cloudera DataFlow): Design, develop, and maintain scalable, reliable data flows and real-time/near real-time pipelines with an accent on Apache NiFi, Kafka, and CDC integrations. Focus on building governed, secure, and high-performance data movement with end-to-end traceability using Atlas/Ranger and on troubleshooting pipeline performance and reliability.

Company

hirify.global (part of Accenture) delivers complex public sector IT projects, including systems integration, informatics and analytics, and program management.

What you will do

• Design, develop, and maintain complex data flows in Cloudera DataFlow (Apache NiFi) to ensure scalable, reliable, high-performance data movement.
• Build and optimize real-time and near real-time pipelines using NiFi, Kafka, and CDC technologies (e.g., Debezium, SQL-based connectors).
• Implement secure integrations with internal/external systems via REST APIs, JDBC, Kafka, and other protocols.
• Design and manage data schemas, metadata, and lineage using Avro and Apache Atlas for governance and traceability.
• Define and enforce data security and access control policies using Apache Ranger.
• Monitor, troubleshoot, and optimize pipelines; maintain technical documentation, SOPs, and runbooks; support upgrades and migrations across CDP, NiFi, and Kafka.

Requirements

• Advanced university degree (Master’s or equivalent) in computer science, information systems, data engineering, or related field (or first-level degree with equivalent experience).
• 2–3+ years of hands-on experience with Apache NiFi, preferably in Cloudera Data Platform (CDP), including flow design, deployment, monitoring, and troubleshooting.
• Expert knowledge of designing, implementing, and maintaining complex data flows using Apache NiFi / Cloudera DataFlow.
• Advanced Python skills for data processing, automation, and custom flow development.
• Strong experience with REST APIs (including OAuth/JWT), plus CDC approaches using NiFi processors/connectors and SQL-based methods.
• Fluency in written and spoken English.

Culture & Benefits

• Work on complex public sector IT projects with integration, analytics, and governance requirements.
• Collaborate with data engineers, architects, and business stakeholders to deliver robust data flow solutions.
• Maintain operational readiness through documentation, SOPs, and runbooks for production support.
• Support platform lifecycle activities including upgrades, migrations, and enhancements across CDP, NiFi, and Kafka environments.