Data Lakehouse and AI Data Platform Engineer (AI)

TL;DR

Data Lakehouse and AI Data Platform Engineer (AI): Design, build and support batch/streaming data pipelines and curated datasets on modern Lakehouse platform with an accent on ingestion, transformation, modeling and optimization. Focus on ensuring data quality, scalability for analytics/AI use cases, and production reliability using Spark, SQL and distributed processing.

Location: New York, United States (office-based)

Salary: USD 115,000 - 180,000

Company

Leading global investment bank developing data foundations for AI, analytics and business decisions.

What you will do

• Build, enhance and support batch and streaming data pipelines on Lakehouse and AI data platform.
• Develop raw, refined and curated datasets supporting analytics, reporting and AI use cases with sound data modeling.
• Implement data quality controls, reconciliation and monitoring for production reliability.
• Refactor existing data flows for improved performance, maintainability and scalability.
• Collaborate with engineers, platform teams and data consumers to deliver high-quality outcomes.
• Contribute to technical design, standards and leadership for senior candidates.

Requirements

• 0-2+ years experience in data engineering with Bachelor’s/Master’s or equivalent.
• Strong programming in Python or Java and solid SQL skills including optimization.
• Experience with distributed processing like Apache Spark, data formats (JSON, Avro, Parquet).
• Knowledge of temporal data modeling, schema design, partitioning and data quality practices.
• Familiarity with software engineering best practices: version control, CI/CD, testing.
• Ability to learn new tools quickly and work in collaborative, fast-paced environment.

Culture & Benefits

• Best-in-class benefits package.
• Positioned at center of business, solving complex problems with scalable systems.
• Opportunities for long-term growth and expertise development within the firm.
• Modern data stack including Snowflake, Databricks, Kafka and more.