Senior Data Engineer (ClickHouse)

Senior Data Engineer Infra

Company

Solidus Labs

Conditions

5 hours agoSenior New York, USA Onsite Full Time Engineering Jobs by Solidus Labs

Skills

Query Optimization Airflow Snowflake Schema Versioning Monitoring Cloud Observability Clickhouse Spark Sql Python Rust Kubernetes Java Communication Data-Pipelines Redis Kafka Data Modeling Governance

About the Role

You will design and optimize data pipelines and build scalable data infrastructure on cloud environments. You will ensure data reliability and integrity across real time ingestion and batch processing. You will collaborate with analytics and product teams to shape data models and deliver timely insights. You will monitor performance and contribute to governance and best practices. You will work in a friendly environment with a self starting attitude.

Requirements

• BSc. in Computer Sciences.
• Strong background as a software engineer with at least 5+ years of hands-on experience with Java, Rust, or Python.
• 8+ years in data engineering and data pipeline development on high-volume, low-latency production environments.
• Experience working in low-latency, real-time systems processing billions of events a day.
• Deep, hands-on ClickHouse expertise - including cluster architecture, table engine selection, replication, sharding, and query optimization. Experience engaging with the ClickHouse vendor team or community is a strong plus.
• Proficiency across the broader data engineering stack: Apache Kafka, Spark, Airflow, Kubernetes, Redis, Snowflake, and caching technologies.
• Expert-level SQL and query optimization skills, with a strong emphasis on ClickHouse-specific patterns - materialized views, projections, TTLs, and merge tree tuning.
• Experience with monitoring and observability tools (Prometheus, Grafana, or similar), with the ability to define and own operational health metrics for a ClickHouse deployment.
• Curiosity, ability to work independently, and a track record of proactively identifying and driving solutions.
• Excellent verbal and written communication skills, including the ability to coach and influence engineers across teams in a remote environment.

Responsibilities

• Design and optimize the ClickHouse data layer - including table engines, partition strategies, materialized views, and storage policies - to ensure high performance at billions-of-events scale.
• Own ClickHouse clusters sizing, topology decisions, and capacity planning across both real-time ingestion and T+1 batch workloads, balancing cost, latency, and throughput.
• Drive data reliability and deduplication strategies within ClickHouse, leveraging engine-level features (ReplacingMergeTree, CollapsingMergeTree, etc.) and pipeline-level controls to guarantee data completeness and consistency.
• Establish and continuously improve monitoring, alerting, and observability for the ClickHouse layer — covering replication health, merge performance, query latency, and resource utilization.
• Serve as the internal ClickHouse authority, coaching engineering teams across the organization on query optimization, data modeling best practices, and efficient use of ClickHouse-specific constructs.
• Act as the primary liaison with the ClickHouse vendor team - triaging issues, incorporating product feedback, evaluating new features, and translating vendor guidance into actionable improvements for our deployment.
• Collaborate with downstream consumers (analytics, ML, product) to understand access patterns and continuously refine how data is stored and served — improving query performance, schema design, and data formats for diverse client needs.
• Define and enforce schema versioning and governance standards within the ClickHouse environment, ensuring schema evolution does not compromise pipeline reliability or consumer compatibility.

Показать контакты