Senior Software Engineer (Lakehouse)

TL;DR

Senior Software Engineer (Analytics Data Platform): Building and operating the foundations that power data engineers and applied AI with an accent on Apache Iceberg table management and open-source query engine integration. Focus on optimizing data layout, building observability for managed tables, and developing self-serve tooling for data engineering pipelines.

Location: Hybrid in New York, NY, USA

Salary: $130,000 — $300,000 USD

Company

Datadog is a monitoring and observability platform that provides visibility into infrastructure and application performance.

What you will do

• Design and operate lakehouse platform components, including Apache Iceberg table management and catalogs.
• Drive adoption of open table formats by integrating Trino, Spark, and DuckDB at petabyte scale.
• Build observability for managed Iceberg tables and contribute fixes back to open-source projects.
• Develop self-serve tooling and abstractions for data engineering teams to run thousands of pipelines daily.
• Collaborate with infrastructure teams to define the lakehouse architecture roadmap.

Requirements

• Production-grade experience with Apache Iceberg, Trino, or Apache Spark (open-source contributions preferred).
• Experience building and operating large-scale distributed data systems.
• Strong knowledge of query planning, columnar file formats (Parquet, ORC), and table format internals.
• Proficiency in Java, Scala, or Go, and comfortable with Python for tooling.
• Experience deploying data infrastructure on Kubernetes in cloud environments.
• Must be based in or be able to work hybrid in New York, NY, USA.

Culture & Benefits

• Hybrid workplace model to ensure work-life harmony.
• Comprehensive healthcare, dental, and mental health benefits.
• 401(k) plan and company match.
• Paid time off and fitness reimbursements.
• Discounted employee stock purchase plan.