3 дня назад
Data Engineer (AI)
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
Текст:
TL;DR
Data Engineer (AI) (Azure/Data Platforms): Building scalable data lakes, ingestion frameworks, and ETL/ELT pipelines for internal automation, analytics, reporting, and AI platforms with an accent on Azure, data integration, governance, and reliability. Focus on designing batch and real-time ingestion, implementing CDC and data quality controls, and solving complex scalability and monitoring challenges across the data estate.
Location: UK office, hybrid
Company
is a fast-growing technology startup providing cybersecurity and AI solutions that help managed service providers manage and secure Microsoft 365 at scale.
What you will do
- Design and maintain scalable data lakes and modern data platform architecture.
- Develop storage solutions for structured and unstructured data and optimize data models and storage performance.
- Build ingestion frameworks, ETL/ELT pipelines, and integrations across SaaS platforms, databases, APIs, and third-party applications.
- Implement batch and real-time ingestion, data synchronization, and change data capture processes.
- Transform raw data into trusted datasets and implement validation, monitoring, alerting, and quality controls.
- Support analytics, reporting, automation, and AI initiatives while contributing to architecture decisions, roadmaps, and engineering best practices.
Requirements
- Proven experience as a Data Engineer or in a similar role.
- Strong SQL and Python skills, with experience working with large datasets.
- Commercial experience building ETL/ELT pipelines, data integrations, and ingestion frameworks.
- Experience with cloud data environments, Microsoft Azure, and Azure Data Lake Storage.
- Experience with Azure Data Factory, Databricks, Apache Spark, dbt, or Airflow, and platforms such as Azure Synapse Analytics or Snowflake.
- Knowledge of Git, REST APIs, event-driven architectures, Kafka or other streaming technologies, CDC, data governance, and master data management.
Culture & Benefits
- Continuous learning and strong enablement in an environment where experimentation and mistakes are supported.
- High-trust, autonomous working environment with opportunities to make a significant impact.
- Transparent and collaborative communication across technical and non-technical stakeholders.
- Global team environment with regular monthly team socials.
- Employee recognition programs for outstanding contributions.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →