Core Data Engineer (Big Data)

TL;DR

Core Data Engineer (Big Data): Owning the end-to-end lifecycle of fundamental data products, including security master and market data, with an accent on building and monitoring automated pipelines to ensure reliable data access. Focus on optimizing data usage, exploring new data sources, and collaborating with teams on data-related architecture and design.

Location: Hong Kong (Onsite)

Company

hirify.global is a global trading firm specializing in providing liquidity across financial markets.

What you will do

• Be accountable for timely and high-quality delivery of core data products such as Security Master, ETF Compositions, Market Data, Trades, Orders, Quotes, and Inquiries.
• Develop and own the end-to-end lifecycle of these data products, including building and maintaining pipelines, developing quality checks, and automating manual processing steps.
• Define and manage project plans to assume full ownership of key data products.
• Create and refine a common data model, focusing on identifiers, normalized schemas, and master data usage across data products.
• Investigate available data sources and ensure maximum information and value extraction.
• Explore, evaluate, and onboard additional data sources, both independently and at stakeholder request.
• Work closely with other teams to automate workflows, leverage standard Flow technologies, and provide guidance on data-related architecture and design.

Requirements

• Excellent programming skills, with Python preferred.
• Working proficiency with Linux, SQL, and both relational and non-relational databases.
• Demonstrated ability to work with large datasets and familiarity with big data solutions.
• Domain-literate with exposure to symbology, security reference/pricing data, ETF compositions, or exchange market data.
• Experience with Airflow, Kafka, and GCP (especially GKE, Dataproc, and BigQuery) is highly desired.
• Must work onsite in Hong Kong.

Nice to have

• Experience with Pandas, Spark, Apache Beam, or Apache NiFi.

Culture & Benefits

• Unique opportunity to make a significant mark on hirify.global and drive future strategic growth.
• Accountable for delivering data products at an exceptional level of quality.
• Collaborative environment, working effectively with both technical and non-technical stakeholders.
• Strong focus on generalization, systematization, and automation in daily work.