Lead Cloud Data Engineer (Data Mesh & AI)

TL;DR

Lead Cloud Data Engineer (Data Mesh & AI): Designing and implementing modern enterprise data platforms and AI-augmented solutions with an accent on distributed cloud architectures and federated data hubs. Focus on building end-to-end ELT pipelines, implementing data mesh governance, and developing agentic AI solutions using RAG pipelines.

Location: San Francisco, CA (USA)

Salary: $166,400 – $208,000

Company

hirify.global is a staffing and professional services firm providing technical talent for enterprise-scale projects.

What you will do

• Develop and maintain robust end-to-end ingestion, transformation, and consumption-layer pipelines using advanced cloud data services.
• Design and implement a scalable Data Mesh architecture, establishing domain boundaries and federated governance.
• Build reusable frameworks and self-serve tools to empower cross-functional domain teams to publish and consume data products.
• Integrate DevSecOps practices, including CI/CD automation and Infrastructure as Code via Terraform or CloudFormation.
• Design and build agentic AI solutions and RAG pipelines using modern LLMs to enhance platform capabilities.

Requirements

• Must be based in the USA (E-Verify program participant)
• 5+ years of experience designing and deploying distributed data architectures and modern cloud platforms.
• Deep hands-on experience with AWS services (S3, Glue, Lake Formation, EMR, Redshift/Snowflake).
• Proficiency with Databricks (Delta Lake, Unity Catalog) and federated query engines like Starburst/Trino.
• Strong background in CI/CD pipelines, containerization (Docker), and infrastructure automation.
• Practical experience leveraging generative AI tools for code acceleration and intelligent automation.

Nice to have

• Proven track record of leading technical design sessions and mentoring data engineering teams.
• Excellent communication skills for bridging technical concepts with executive stakeholders.
• Experience navigating hybrid cloud or secure enterprise environments.

Culture & Benefits

• Comprehensive medical, dental, and vision insurance for eligible assignments.
• 401k retirement plan.
• Inclusive work environment with reasonable accommodations for individuals with disabilities.