обновлено 1 день назад
Data Engineer (Fintech)
5 000 - 7 000$
RUВакансия из Hirify RU, списка российских tech-компаний
Мэтч & Сопровод
Покажет вашу совместимость и напишет письмо
Описание вакансии
Текст:
TL;DR
Data Engineer (Fintech/Apache Iceberg): Building and operating a market data lakehouse from raw exchange feeds to petabyte-scale datasets for research, backtesting, and real-time trading with an accent on ingestion, Iceberg storage modeling, and data reliability. Focus on reconstructing order books, optimizing partitioning and query performance, and designing observable pipelines across Spark, Kubernetes, and AWS S3.
Location: Fully remote; daily overlap from 12:00 to 18:00 GMT+3
Salary: $5,000–$7,000 gross per month
What you will do
- Design, build, and maintain a market data lakehouse from raw exchange feeds and vendor sources to reliable, petabyte-scale datasets.
- Decode and normalize PCAP, multicast UDP, ITCH, FIX, and vendor data into a unified canonical model with nanosecond timestamps.
- Build batch and streaming pipelines with Airflow, Spark, and dbt, including L2/L3 order-book reconstruction and gap handling.
- Own the Apache Iceberg over S3 lakehouse, including partitioning, schema evolution, snapshots, compaction, TTL, and storage cost optimization.
- Develop data contracts, validation, lineage, shared Spark and Polars access services, and Python and Rust producer SDKs.
- Implement monitoring, alerting, SLAs/SLOs, CI/CD, data-quality dashboards, and incident runbooks on Kubernetes/EKS.
Requirements
- 5+ years of experience building production-grade data systems and operating data lakes or lakehouses at scale.
- Hands-on experience with Apache Iceberg or comparable table formats, including partitioning, schema evolution, snapshots, compaction, and catalog operations.
- Expert Python skills, including Polars and/or PySpark, plus advanced SQL for complex aggregations, window functions, query optimization, and partition pruning.
- Experience with Airflow, Apache Spark, StarRocks, Linux, Docker, Kubernetes/EKS, and AWS S3.
- Experience with CI/CD, Terraform, ArgoCD, Grafana, Prometheus, and data-quality or observability practices.
- English fluency at B2+ level for documentation and collaboration; availability for the 12:00–18:00 GMT+3 overlap window.
Nice to have
- Market-data or network-packet experience, including PCAP, ITCH, FIX/FAST, multicast UDP, order-book reconstruction, and time-series data at scale.
- Experience normalizing OneTick, Refinitiv/Reuters, Bloomberg, or ICE data into a unified schema and symbology.
- Rust experience for high-performance capture and decoding.
Culture & Benefits
- Fully remote work with a daily GMT+3 collaboration window.
- Reimbursement for health insurance, sports, and personal development.
- Flat structure with substantial autonomy over working methods.
- Opportunity to expand the role as the team grows.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →