Python Data & AI Engineer — Web Scraping / Document Extraction / Databricks

🔥 Python Data & AI Engineer — Web Scraping / Document Extraction / Databricks at Elinext

We are currently looking for a Python Data & AI Engineer to join a project focused on automated search, extraction, structuring, validation, and delivery of technical asset data from publicly available Internet sources.

📍 Location: EU
🇬🇧 English: Fluent
🇩🇪 German: Nice to have

About the Project
The goal is to design and implement a solution that automatically collects and structures technical information from sources such as:
— manufacturer websites and digital product catalogs
— online directories, portals, and databases
— engineering and professional publications
— technical datasheets and product-related documents
— APIs, HTML pages, tables, PDFs, and downloadable files
The solution will create structured product and model catalogs containing manufacturer information, model names, and technical specifications.
The resulting data will be normalized, validated, and provided in standardized CSV files for further technical review and integration into an existing EAM structure.

What You Will Do
— Analyze and evaluate relevant public data sources
— Design the target architecture for data acquisition and extraction
— Develop components for data collection, parsing, extraction, mapping, and normalization
— Process structured and unstructured information from websites, APIs, PDFs, tables, and documents
— Apply AI / LLM-based approaches to extract and normalize technical characteristics
— Develop rules for data validation, duplicate detection, and quality control
— Transform collected information into structured asset, manufacturer, product, and type objects
— Prepare standardized CSV outputs with full source traceability
— Build validation workflows and supporting functionality in Databricks
— Implement completeness, plausibility, and reliability checks
— Support pilot implementation for selected asset classes, manufacturers, and data sources
— Measure extraction quality and identify opportunities for optimization and scaling
— Prepare technical, operational, architecture, and handover documentation

MUST HAVE
— Strong Python development experience
— Experience with Pandas and PySpark
— Hands-on experience with Databricks and Delta Lake
— Experience building data pipelines and data transformation workflows
— Strong understanding of data normalization, schema mapping, and validation
— Experience working with REST APIs and file-based data sources
— Experience processing structured and unstructured documents
— PDF / datasheet extraction
— Table extraction and schema mapping
— Source and metadata management
— Error handling and monitoring of data processing runs
AI / Information Extraction
— Experience with Large Language Models / Generative AI
— NLP-based information extraction
— Prompt engineering
— Structured output generation
— Extraction and normalization of technical attributes
— Classification and mapping
— Duplicate detection / entity matching
— Experience integrating AI components into data processing workflows
⭐️ Strong Advantage — Web Crawling & Scraping
Hands-on experience in Web Crawling & Web Scraping would be a significant advantage for this role, especially:
— Scrapy, Beautiful Soup, or comparable frameworks
— HTML / DOM / table analysis
— Working with dynamic web pages
— Download structures and document discovery
— Building robust and extensible crawlers and parsers
— Selenium, Playwright, or similar browser automation tools
— Handling different website structures and source formats

Nice to Have
— Databricks App development
— Azure services and Databricks deployment
— MLOps and CI/CD for Data / AI components
— Experience designing crawling solutions with data protection and legal requirements in mind
— Product, asset, or technical type catalogs
— Ontologies and semantic models
— Technical feature catalogs
— EAM / SAP attribute models
— Electrical engineering domain knowledge