Data Engineer, Forward Deployed (AI)

Location: Houston, United States; work format: Hybrid

Company

hirify.global builds Orbital, a physics-informed foundation model for energy operations across oil and gas, refineries, and petrochemicals.

What you will do

• Architect and maintain scalable Lakehouse pipelines for high-frequency time-series, laboratory, historian, IoT, and industrial data.
• Ingest and contextualise data from OPC UA servers, process historians, LIMS systems, sensors, alarms, events, and P&IDs.
• Build real-time streaming and batch ingestion pipelines across Databricks, Lakeflow, Lakebase, AWS, and hybrid cloud/on-premise environments.
• Synchronise historian archives, unstructured files, and AWS S3/EBS storage while managing secure, high-throughput transfers.
• Implement schema-change detection, signal-drift handling, lineage, and audit trails across Spark, Databricks, and AWS pipelines.
• Build parallel training and inference pipelines supporting time-series forecasting, anomaly detection, optimisation, and retrieval-augmented LLM workloads.

Requirements

• Deep expertise in PostgreSQL, including partitioning, indexing, query optimisation, and storage design.
• Strong Python skills for data processing, scripting, and pipeline orchestration.
• Hands-on AWS experience, including EKS, S3, EBS, IAM, KMS, and CloudWatch.
• Proven experience with Databricks and PySpark for large-scale distributed data processing.
• Experience with industrial time-series data, control systems, DCS/SCADA logs, or process historians.
• Experience managing unstructured data synchronisation across hybrid cloud and on-premise environments.

Nice to have

• Data engineering experience in oil and gas or energy environments.
• Knowledge of Kafka, Flink, Spark Streaming, or MLOps stacks for data versioning and lineage.