Data Engineer

Summary

Join our growing global company as a remote Data Engineer! You will maintain and manage website scraping configurations using Python, monitor for errors, oversee retrieved data, and coordinate with stakeholders. Responsibilities include developing data pipelines, ingesting data into the Datalake, and performing transformations. You'll prepare periodic reports and collaborate with data scientists and analysts. This role requires proven data engineering experience, expertise in ETL processes, database technologies, and cloud platforms (Azure preferred). Strong Python skills, data security knowledge, and experience with version control are essential.

Requirements

• Proven experience in data engineering with expertise in designing and implementing scalable data architectures
• Strong experience with ETL processes, data modeling, and data warehousing
• Expertise in database technologies, both relational (SQL) and NoSQL
• Knowledge of cloud platforms, particularly Azure
• Solid understanding of data security measures and compliance standards
• Excellent Python experience for data engineering and automation
• Strong collaboration skills to work closely with data scientists and analysts
• Ability to optimize data pipelines for performance and efficiency
• Ability to build, test, and maintain tasks and projects
• Experience with version control systems, such as Git
• Minimum 2 years of experience in a similar role
• Strong academic background in a relevant field
• Fluent in English

Responsibilities

• Maintain and manage website scraping configurations using Python
• Monitor scraping configurations for errors and potential crashes
• Oversee retrieved data to detect potential issues and blockages
• Coordinate with stakeholders to understand scraping task requirements and report issues
• Prepare and share periodic reports on scraping activities with stakeholders
• Develop necessary pipelines to ingest data into the Datalake and perform required transformations

Preferred Qualifications

• Strong experience with ETL processes, data modeling, and data warehousing (Airflow & DBT preferred)
• Hands-on experience with Airflow and/or DBT
• Experience with Terraform for infrastructure management
• French is a plus