Principal Software Engineer (AI)

TL;DR

Principal Software Engineer (AI): Building and optimizing real-time data pipelines and feature/embedding materialization systems for ML models with an accent on robustness, operational maturity, and performance optimization. Focus on designing and implementing streaming ETL, owning data contracts, and integrating with ML inference serving.

Location: Must be based in India

Company

hirify.global builds and operates large-scale, latency-sensitive systems for modern ads platforms.

What you will do

• Design and implement real-time streaming ETL and feature pipelines (e.g., Flink or Spark Structured Streaming).
• Build and operate reliable messaging and ingestion systems with Kafka/Pulsar.
• Own data contracts between producers, pipelines, and consumers, managing schema evolution and versioning.
• Define and meet SLOs using observability stacks like OpenTelemetry, Prometheus, and Grafana.
• Integrate pipelines with online stores/caches and ML consumers (feature stores, embedding pipelines).
• Optimize end-to-end performance and efficiency across compute, storage, and serving integrations.

Requirements

• Bachelor’s or Master’s degree in Computer Science or a related field, with 8+ years of related experience.
• Strong programming skills in C++, C#, or Python (at least one required).
• Hands-on experience in building and operating streaming data pipelines (Flink or Spark Structured Streaming) or distributed systems engineering.
• Experience with messaging systems such as Kafka/Pulsar.
• Experience operating services with Kubernetes/containers and production readiness practices.
• Experience with observability stacks such as OpenTelemetry, Prometheus, and Grafana.

Nice to have

• Experience with feature stores, embedding pipelines, and online/offline consistency.
• Experience with data lakehouse/table formats and optimizations.
• Experience with GPU inference serving (Triton, ONNX Runtime/TensorRT).

Culture & Benefits

• Microsoft is an equal opportunity employer.
• Consideration for employment without regard to age, ancestry, citizenship, or other protected characteristics.
• Assistance with religious accommodations and/or reasonable accommodation due to a disability is available.