Sr. Software Engineer - Ingestion Core Team (Data Ingestion)

Location: San Francisco, California, United States

Salary: $166,000–$225,000 USD per year

Company

Databricks is a data and AI company providing a unified platform for data warehousing, distributed processing, machine learning, analytics, and AI applications.

What you will do

• Build distributed infrastructure for incremental ingestion, streaming, processing, and replication from cloud storage, databases, and file sources.
• Ingest structured and unstructured data into Delta Lake with schema evolution and change data capture capabilities.
• Reduce end-to-end latency, increase throughput, and optimize infrastructure costs.
• Design and optimize distributed workloads for scale, reliability, latency, throughput, and cost.
• Build monitoring and observability capabilities for ingestion workflows and supporting systems.
• Collaborate with partner teams on RAG, AI agent, and agentic SDK use cases.

Requirements

• 5+ years of experience writing production code in Java, Scala, Go, C++, or Python.
• Experience architecting, developing, and deploying large-scale distributed and asynchronous systems.
• Experience with distributed systems, streaming, Spark, databases, data processing, or CDC.
• Experience operating systems where scale, throughput, latency, reliability, and cost are critical.
• Comfort using AI tools and defining effective evaluations.
• Ability to work in San Francisco, California

Culture & Benefits

• Comprehensive benefits and perks tailored to employees in each region.
• Annual performance bonus and equity may be included in the compensation package.
• Inclusive culture with equal employment opportunity standards.