Senior Software Engineer - Husky Storage

Location: Boston, Massachusetts, USA or New York, New York, USA

Salary: $192,000–$240,000 USD per year, plus equity and potential variable compensation

Company

Datadog provides an observability and security platform that unifies applications, infrastructure, data, models, and security for businesses operating at scale.

What you will do

• Drive migration of the Husky distributed columnar storage system from a custom format to open-source Parquet and a DataFusion-based, format-agnostic substrate.
• Rework compaction and deliver cost reductions through index compression, external object integration, storage tiering, and caching.
• Scale temporal-database capabilities so customers can manage event versioning at high volume.
• Operate and help define the future architecture of a petabyte-scale storage service.
• Collaborate with Ingestion, Metadata, Platform, and Lakehouse teams on write-path improvements and an Iceberg-compatible handoff for analytical workloads.
• Work with the open-source community to build reusable software that operates reliably at scale.

Requirements

• Experience with large-scale distributed systems and/or storage engines.
• Experience improving and operating distributed systems used for critical business functions.
• Strong fundamentals in columnar and file-storage formats such as Parquet, ORC, and Arrow, or comparable systems.
• Ability to work with large-scale storage architecture, reliability, and operational decisions.
• Ability to work from the Boston or New York office location.

Nice to have

• Experience with Go, Java, or Rust.
• Experience with compaction, compression, or encoding schemes such as RLE, dictionary, delta, or Gorilla.
• Experience with object storage such as S3, GCS, or Azure Blob at scale, including caching and cost optimization.
• Experience with Kafka, stream processing, or Kubernetes.

Culture & Benefits

• Engineering-driven, bottom-up culture with high agency and responsibility.
• Blameless environment focused on supporting colleagues through technical challenges.
• Remote-work flexibility and offices designed to support collaboration.
• New-hire stock equity, ESPP, professional development, product training, mentoring, and career pathing.
• Healthcare, dental, parental planning, mental health benefits, 401(k) matching, paid time off, fitness reimbursements, and employee resource groups.