2 часа назад
Senior Data Infra Engineer
175 000 - 221 000$
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
Текст:
TL;DR
Senior Data Infra Engineer (Spark/Kafka): Building and scaling batch and streaming data pipelines that power feed ranking, recommendations, A/B testing, and machine learning models with an accent on high-throughput event processing, reliability, and data quality. Focus on optimizing Spark jobs processing billions of rows daily, defining freshness and completeness SLOs, and improving query performance, resource scheduling, and storage efficiency.
Location: Mountain View, California, United States
Annual base pay: $175,000–$221,000 USD
Company
is a content intelligence platform serving over 40 million monthly active users through personalized local news and information powered by AI, recommendation systems, and adtech.
What you will do
- Design and operate high-throughput Kafka pipelines carrying user events to the data warehouse and real-time analytics systems.
- Build and optimize Spark batch jobs processing billions of rows daily for content ingestion, engagement aggregation, and user-level feature computation.
- Own pipeline reliability, incremental backfills, data freshness and completeness SLOs, and cost per terabyte processed.
- Implement data quality checks, freshness monitoring, anomaly alerts, and observability for production pipelines.
- Design structured and unstructured data models and drive data infrastructure decisions across data science, application, and platform teams.
- Improve efficiency through query optimization, partitioning, resource scheduling, tiered storage, Airflow orchestration patterns, and reusable Spark libraries; mentor junior engineers.
Requirements
- 5+ years of data engineering experience, including at least 3 years owning production pipelines in distributed data environments.
- Strong hands-on experience with Spark and Kafka at scale, including production incident debugging.
- Experience with Hadoop, Presto/Trino, and Flink.
- Proficiency in Python and SQL.
- Cloud experience, preferably with AWS services including S3, EMR, Glue, and Redshift.
- BS or MS in Computer Science or equivalent, with strong communication skills and experience driving infrastructure decisions across teams.
Nice to have
- Experience with Scala.
Culture & Benefits
- Health, dental, and vision coverage for employees and families, with 100% employee coverage.
- 401(k) plan with company matching.
- Paid time off and paid holidays.
- FSA, HSA, and commuter benefits programs.
- Team activity budget.
- Potential eligibility for a discretionary bonus and stock options.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →
Похожие вакансии
6 часов назад
Software Engineer - Data (Python/AWS)
135 000 - 180 000$
3 часа назад
Staff Software Engineer (Data), TVision
160 000 - 190 000$
5 часов назад
Senior Software Engineer, Platform - Data + AI (Back-End)
145 000 - 187 000$
1 час назад
Senior Software Engineer I (Data Engineering)
200 000 - 230 000$
4 часа назад
Data Engineer (AWS/PySpark)
192 000 - 281 000$
55 минут назад
Senior Backend Engineer (Data)
180 000 - 220 000$