Senior Core Data Engineer (Real-Time Data)

TL;DR

Senior Core Data Engineer (Real-Time Data): Building and optimizing high-load data products for low-latency analytical capabilities with an accent on automated pipelines, quality checks, and end-to-end data product lifecycle management. Focus on leveraging real-time OLAP databases and cloud platforms to provide immediate access to reliable data across the organization.

Location: Onsite in Amsterdam

Company

hirify.global is a leading proprietary trading firm with an entrepreneurial and innovative culture at the heart of its business.

What you will do

• Deliver high-quality core data products with a specific focus on low-latency analytics.
• Develop and own the end-to-end lifecycle of data products, including building and maintaining pipelines, developing quality checks, and automating manual processing steps.
• Define and manage project plans to assume ownership of these data products.
• Create and refine a common processing model across data products.
• Investigate available data sources and ensure maximum information and value extraction.
• Explore, evaluate, and onboard additional data sources.
• Work closely with other teams to automate existing workflows, develop real-time platform capabilities, and provide guidance on data-related architecture/design topics.

Requirements

• Organized with strong attention to detail and quality, innately driven to generalize, systematize, and automate.
• Strong interpersonal skills, working effectively with both technical and non-technical stakeholders.
• Ability to balance individual stakeholder requirements to preserve data quality and wide applicability.
• Ability to define, manage, and execute multiple projects in parallel while effectively triaging operational concerns.
• Primarily technical, but domain-literate (e.g., exchange market data, network traffic, application logs).
• Excellent programming skills (Python preferred).
• Working proficiency with Linux, SQL and relational databases, and version control systems.
• Proven experience with real-time OLAP databases like ClickHouse, Apache Pinot, StarRocks required.

Nice to have

• Domain expertise (e.g., market data, security reference/pricing data, ETF compositions).
• Experience with Airflow, Kafka, and GCP (especially GKE, Dataproc, and Big Query).
• Experience with Pandas, Spark, Apache Beam, Apache NiFi.

Culture & Benefits

• Join a leading proprietary trading firm with an entrepreneurial and innovative culture.
• Opportunity to work with quick-witted, creative minds.
• Be uniquely positioned to influence hirify.global’ operations.
• Drive future growth and provide substantial, real-time benefits across all divisions daily.