Senior Data Engineer (AI)

TL;DR

Senior Data Engineer (AWS/Databricks): Designing and building cloud-native data engineering pipelines for federal financial regulatory systems with an accent on semantic processing, knowledge graph construction, and AI-enabled analytics. Focus on implementing GraphRAG capabilities, optimizing ETL/ELT frameworks, and developing semantic retrieval services.

Location: Hybrid: Must be based in Washington, DC (on-site 2-3 days per week). Must reside in one of the 16 licensed US states.

Salary: $120,588 - $195,955

Company

A digital integration consultancy focused on solving meaningful problems by combining technology, people, and organizational development.

What you will do

• Design and develop enterprise data pipelines for structured and unstructured data ingestion, metadata extraction, and knowledge graph population.
• Build and maintain AWS-native ETL/ELT solutions using Databricks, Spark, and Delta Lake.
• Implement semantic processing architectures and graph generation pipelines using Neptune and OpenSearch.
• Develop APIs supporting natural language query (NLQ) and semantic retrieval services, including GraphRAG implementations.
• Build CI/CD pipelines, Infrastructure-as-Code deployments, and automated testing frameworks.
• Optimize pipeline performance and ensure compliance with federal security requirements.

Requirements

• Minimum 7 years of data engineering experience.
• Proven experience with Python, SQL, Spark, Databricks, and AWS services.
• Experience with graph databases (Neptune, OpenSearch) and vector databases for AI/ML workflows.
• Bachelor's degree in Computer Science, Data Engineering, or a related field.
• Ability to obtain and maintain Public Trust clearance.
• Must be based in the USA within specific licensed states.

Nice to have

• Experience implementing GraphRAG solutions and NLQ architectures using Databricks Genie.
• Experience building knowledge graphs with hundreds of millions of nodes and relationships.
• AWS Data Analytics Specialty or Databricks Data Engineer Professional certification.

Culture & Benefits

• Flexible, life-friendly schedules and generous PTO, including federal and birthday holidays.
• 100% coverage for medical HSA plan and full dental/vision coverage for dependents.
• 401(k) with 4% match and immediate vesting.
• Education, wellness, and tech allowances.
• Supportive "Purple Culture" emphasizing autonomy, coaching, and equity.
• Performance-based discretionary bonuses.

Hiring process

• Introductory interview with a recruiter to discuss experience and expectations.
• Role-specific technical assessment to evaluate skills.
• Skills and business fit interview focused on situational and values-based questions.