Lead Data Engineer

Summary

Join our multinational team as a Lead Data Engineer and leverage your expertise in PySpark, SQL, Python, and Azure cloud services to design, build, and maintain scalable data pipelines. Collaborate with cross-functional teams, ensuring data quality and integrity. This is a 6-month contract position (with possible extension) for nearshore candidates located in Central or South America, working remotely on a full-time basis. The role requires proven leadership experience and advanced English fluency. The contract is independent and does not include PTO, tax deductions, or insurance. We offer opportunities for professional development and a collaborative work environment.

Requirements

• 8+ years of overall experience working with cross-functional teams (machine learning engineers, developers, product managers, analytics teams)
• 3+ years of hands-on experience developing and managing data pipelines using PySpark
• 3 to 5 years of experience with Azure-native services, including Azure Data Lake Storage (ADLS), Azure Data Factory (ADF), Databricks, Azure Synapse Analytics / Azure SQL DB / Fabric
• Strong programming skills in Python and SQL
• Solid experience doing ETL processes and data modeling/data warehousing end to end solutions
• Self-driven, resourceful, and comfortable working in dynamic, fast-paced environments
• Advanced written and spoken English is a must have for this position (B2, C1 or C2 only)
• Strong communication skills is a must
• 3+ years of experience with PySpark/Python, ETL and data warehousing processes, Azure data factory, Synapse, Databricks, Azure Data Lake Storage, Fabric, Azure SQL DB etc
• Proven leadership experience in a current project or previous projects/work experiences
• Advanced written and spoken English fluency is a MUST HAVE (from B2 level to C1/C2)
• MUST BE located in Central or South america, as this is a nearshore position (Please note that we are not able to consider candidates requiring relocation or those located offshore)

Responsibilities

• Design and develop scalable data pipelines using PySpark to support analytics and reporting needs
• Write efficient SQL and Python code to transform, cleanse, and optimize large datasets
• Collaborate with machine learning engineers, product managers, and developers to understand data requirements and deliver solutions
• Implement and maintain robust ETL processes to integrate structured and semi-structured data from various sources
• Ensure data quality, integrity, and reliability across pipelines and systems
• Participate in code reviews, troubleshooting, and performance tuning
• Work independently and proactively to identify and resolve data-related issues
• Contribute to Azure-based data solutions, including ADF, Synapse, ADLS, and other services
• Support cloud migration initiatives and DevOps practices
• Provide guidance on best practices and mentor junior team members when needed

Preferred Qualifications

• Databricks certification
• Knowledge of DevOps, CI/CD pipelines, and cloud migration best practices
• Familiarity with Event Hub, IoT Hub, Azure Stream Analytics, Azure Analysis Services, and Cosmos DB
• Basic understanding of SAP HANA
• Intermediate-level experience with Power BI

Benefits

Opportunities for professional development and career growth