Lead Data Engineer

TL;DR

Lead/Senior Data Engineer: Building data systems and pipelines, handling structured, semi-structured, unstructured, and image-based data with an accent on preparing data for prescriptive and predictive modeling. Focus on designing end-to-end pipelines within a hybrid big data architecture, leveraging AWS Cloud services for building data pipelines and ensuring effective monitoring.

Location: Remote

Company

hirify.global is an international team.

What you will do

• Organize and analyze data, handling structured, semi-structured, unstructured, and image-based data.
• Build data systems and pipelines.
• Prepare data for prescriptive and predictive modeling.
• Design end-to-end pipelines within a hybrid big data architecture, using Cloud Native AWS, Python, SQL, etc.
• Leverage AWS Cloud services for building data pipelines and ensuring effective monitoring.
• Collaborate with data scientists and architects across various projects.

Requirements

• 7+ years of Data Engineering experience utilizing designated technologies and tools.
• Profound knowledge and hands-on experience with Python (Pandas, boto3), SQL, Spark, Databricks, DWH
• Proficient use of orchestrators such as Airflow.
• Expertise with AWS services essential for Data Engineering: AWS Glue, AWS Lambda, S3, RDS, Redshift, Athena, SQS, SNS, etc.
• Ability to write robust code with Python.
• Strong upper-intermediate or higher (B2+) proficiency in English.

Nice to have

• Experience with AWS Dynamo DB, Aurora (including serverless nuances), Elasticache, Kinesis services, Neptune.
• Familiarity with AWS Networking.
• Familiarity with Azure.
• Experience with Snowflake.

Culture & Benefits

• Competitive compensation.
• Remote or office work.
• Flexible working hours.
• Healthcare benefits: medical insurance and paid sick leave.
• Continuous education, mentoring, and professional development programs.
• Certifications paid by the company.