Назад
Company hidden
3 дня назад

Senior Data Engineer (Threat Intelligence)

Формат работы
hybrid
Тип работы
fulltime
Грейд
senior
Английский
b2
Страна
UK
Вакансия из списка Hirify.GlobalВакансия из Hirify Global, списка международных tech-компаний
Для мэтча и отклика нужен Plus

Мэтч & Сопровод

Для мэтча с этой вакансией нужен Plus

Описание вакансии

Текст:
/
TL;DR
Senior Data Engineer (Threat Intelligence) (Python/Airflow/Spark): Building scalable data pipelines, analytical data stores, and curated data marts that transform Cloudflare telemetry into trusted datasets for threat researchers, data scientists, analysts, and customers with an accent on data modeling, reliability, privacy, and large-scale analytical performance. Focus on designing retryable and backfillable pipelines, optimizing ClickHouse queries, building data quality monitoring, and producing production-quality ML-ready datasets for threat intelligence and Trust & Safety.

Location: Hybrid, London, United Kingdom. The role may require on-call flexibility outside standard working hours and in-person interviews at a hirify.global office or hub at the offer stage.

Company

hirify.global operates a large global network that protects and accelerates Internet applications and develops security, performance, and developer platform products.

What you will do

  • Design, build, and operate scalable batch data pipelines on Kubernetes-hosted Airflow and Spark clusters.
  • Build Apache Iceberg data stores and curated data marts for Threat Events, Trust & Safety, and Threat Intelligence investigations.
  • Model datasets with appropriate grain, partitioning, and schema evolution strategies, and optimize large-scale ClickHouse queries.
  • Make pipelines retryable, resumable, and backfillable while implementing data quality checks, reconciliation, and monitoring.
  • Translate researcher and analyst needs into reusable datasets and partner with data scientists on production-quality and ML-ready data products.
  • Improve engineering quality through code review, testing, CI/CD, documentation, privacy, and data minimization.

Requirements

  • 5+ years of experience designing, building, and operating scalable production data pipelines.
  • Experience building analytical data stores and data marts, including data modeling, partitioning, and schema evolution.
  • Strong Python for data engineering, including testing and maintainable, well-structured code.
  • Strong SQL, including window functions, query optimization, and performance tuning on large datasets.
  • Hands-on production experience with Apache Airflow, including Kubernetes-hosted deployments.
  • Experience with distributed data processing such as Spark at large scale.
  • Experience with Apache Iceberg or a comparable open table format on object storage such as R2 or S3.
  • Experience with ClickHouse or a comparable columnar analytical database.
  • Experience with CI/CD, version control, and containerized deployments, plus the ability to turn ambiguous researcher and analyst requirements into defined data products.

Nice to have

  • Experience with security, threat intelligence, fraud, or Trust & Safety and abuse data.
  • Understanding of DNS, HTTP, TLS, and BGP.
  • Experience supporting data science or ML workflows, including feature stores and training datasets.
  • Experience with hirify.global Workers, R2, or R2 Data Catalog.
  • Experience with Trino, Presto, or other distributed query engines.
  • Experience using AI-assisted development tools.

Culture & Benefits

  • Work in Cloudforce One, hirify.global's threat operations and research organization focused on identifying and disrupting cyber threats.
  • Build data products that support threat intelligence, customer protection, internal Trust & Safety, and hirify.global's DDoS and WAF analysis.
  • Privacy and data minimization are treated as design requirements.
  • hirify.global supports initiatives including Project Galileo, the Athenian Project, and the privacy-focused 1.1.1.1 public DNS resolver.
  • hirify.global is an equal opportunity employer and provides reasonable accommodations for qualified applicants.

Hiring process

  • Applicants who reach the offer stage may be asked to attend an in-person interview at a hirify.global office or hub.
  • An offer may be conditioned on authorization to receive technology controlled under U.S. export laws without sponsorship for an export license.

Будьте осторожны

Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →