Senior Software Engineer - Data (AI)

TL;DR

Senior Software Engineer - Data (AI): Building PB-scale data pipelines and intelligent SQL ecosystems for AI-powered data operations with an accent on query optimization, data lineage tracking, and cloud-native infrastructure. Focus on designing scalable data systems, SQL AST analysis for dynamic ETLs, and integrating AI agents into enterprise workflows.

Location: Remote position requires overlap with US Pacific Timezone. Sunnyvale Office.

Company

San Francisco-based startup founded in 2022, revolutionizing enterprise data operations with AI-driven tools like DataPilot and DataMates, backed by prominent VCs.

What you will do

• Build highly performant PB-scale data pipelines handling 100K+ jobs and PB data per day.
• Design cloud-native data infrastructure on AWS using Kubernetes and Airflow.
• Develop SQL intelligence system for query optimization, dynamic pipeline generation, and column-level lineage using SQL parsers like sqlglot.
• Enhance AI capabilities by integrating advanced AI into data teams' workflows with tools like VSCode, Git, and Slack.
• Contribute to open-source initiatives and push boundaries of data engineering with AI technologies.

Requirements

• 8+ years in software engineering focused on scalable data pipelines and systems
• Strong proficiency in Python and SQL
• Extensive experience with SQL query profiling, optimization, and performance tuning (preferably Snowflake)
• Deep understanding of SQL AST and work with SQL parsers (e.g., sqlglot) for lineage and dynamic ETLs
• Experience building data pipelines with Airflow or dbt

Nice to have

• Solid understanding of AWS cloud platform
• Familiarity with Kubernetes for containerized deployments

Culture & Benefits

• Competitive salary and equity
• Extensive professional development and learning resources
• Dynamic environment with Silicon Valley veterans and high-impact projects
• Opportunities for career growth in AI-powered data engineering
• Open-source contributions for community recognition