Назад
Company hidden
1 день назад

Software Engineer II (DataProc)

Формат работы
onsite
Тип работы
fulltime
Грейд
middle
Английский
b2
Страна
UK
Вакансия из списка Hirify.GlobalВакансия из Hirify Global, списка международных tech-компаний
Для мэтча и отклика нужен Plus

Мэтч & Сопровод

Для мэтча с этой вакансией нужен Plus

Описание вакансии

Текст:
/
TL;DR
Software Engineer II (DataProc) (Spark/Airflow/Databricks): Building and operating a data orchestration and distributed processing platform for large-scale advertising systems with an accent on workflow scheduling, Spark workloads, and platform reliability. Focus on migrating big data jobs across EMR, Kubernetes, and Databricks, improving observability, and debugging complex distributed production systems.

Location: London, United Kingdom

Company

hirify.global is a global technology company building an independent digital advertising platform that processes large-scale data and serves traffic worldwide.

What you will do

  • Design, build, and operate services and libraries for the Airflow and TaskService data orchestration platform.
  • Develop, scale, and optimize distributed Spark processing jobs across Amazon EMR, Kubernetes, and Databricks.
  • Partner with engineering teams to migrate and modernize big data workloads, including migrations to Databricks and Spark on Kubernetes.
  • Improve platform reliability and observability through metrics, alerting, tracing with OpenTelemetry, and on-call tooling.
  • Build internal tools and lightweight UIs for monitoring, debugging, and operating data pipelines.
  • Own projects from design and code review through deployment, rollout, production support, and roadmap contributions.

Requirements

  • Strong software engineering fundamentals and production experience with Python, Scala, or Java.
  • Experience building or operating distributed data processing systems such as Spark, Databricks, or Hadoop and/or workflow orchestration tools such as Airflow.
  • Experience with cloud infrastructure, including AWS and Kubernetes, and CI/CD pipelines.
  • Knowledge of data structures, algorithms, and distributed systems design.
  • Strong debugging and troubleshooting skills in large-scale production systems.
  • Some experience with or interest in internal tools, dashboards, or lightweight UIs.
  • Clear communication, cross-team collaboration, and an ownership-driven approach to software development and operations.
  • At least 2 years of software development or engineering experience.
  • Bachelor’s or Master’s degree in computer science or a relevant engineering field, or equivalent experience; consideration is not limited by education level.

Nice to have

  • SQL and data warehousing experience, including Snowflake or Vertica.
  • Kubernetes operator and cluster administration experience.
  • Observability tooling experience with OpenTelemetry, Grafana, or Prometheus.
  • Production on-call experience for critical systems.
  • Exposure to machine learning pipelines or ad-tech and real-time bidding systems.

Culture & Benefits

  • Work in a collaborative global engineering environment with diverse perspectives.
  • Engineers are expected to be trust-builders, generous collaborators, pragmatic problem solvers, and owners of their work.
  • Technical opportunities span data infrastructure, internal tools, model pipelines, and client-facing systems.
  • The company supports an inclusive hiring experience and provides accessibility accommodations on request.

Hiring process

  • Unsolicited resumes from search firm recruiters are not accepted unless an agreement is in place.
  • Accessibility accommodation requests can be sent to accommodations@thetradedesk.com with preferred contact details and the nature of the request.

Будьте осторожны

Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →