Principal Software Engineer (Real Time Data Enrichment Platform)

TL;DR

Principal Software Engineer (Real Time Data Enrichment Platform): Design, build, and deliver a new self-service data enrichment platform for batch and streaming analytics with an accent on unified data cataloging, query analysis, and query performance optimization across large-scale structured data. Focus on hands-on distributed systems and big data internals using Spark and Flink runtimes to process petabytes of data and enable efficient customer threat-hunting workflows.

Company

CrowdStrike builds an AI-native cybersecurity platform that processes massive event streams to help stop breaches.

What you will do

• Shape the next phase of the Analytics Data Platform by building a Unified Data Catalog and Query Analysis for structured data stored in different formats (columnar vs graph).
• Design, develop, and maintain a data platform that processes petabytes of data.
• Build and optimize query performance using indexing and data partitioning techniques.
• Own hands-on delivery of a self-service data enrichment platform for customers.
• Contribute to query processing, scalable pipeline builds, materialized views, transformations, and data storage frameworks.
• Improve service efficiency and participate in technical reviews to enhance stability and add new features.

Requirements

• Location: Must be based in New York, NY, USA
• 17+ years of experience with a B.S., 15+ years with an M.S., or 12+ years with a PhD in a related field.
• Experience building and supporting very high-scale data platform and data storage systems (minimum: 100s of TB/day).
• Significant performance-tuning experience within Spark, Flink, Iceberg, or Pinot (or equivalent real-time structured streaming/OLAP/Open Table systems).
• Production experience with Spark- or Flink-based self-service data platforms (or equivalent frameworks such as Ray or Scala/Akka-based Spark/Flink-like frameworks).
• 3+ years coding in Java, Scala, Kotlin, or another JVM language; plus production experience with relational SQL and NoSQL databases.

Culture & Benefits

• Market-leading compensation and equity awards.
• Comprehensive physical and mental wellness programs.
• Competitive vacation and holidays, plus paid parental and adoption leaves.
• Professional development opportunities regardless of level or role.
• Vibrant office culture with world-class amenities.