Senior DataBricks Python Developer

Summary

Join Encora as a Senior DataBricks, Python Engineer and be responsible for designing, developing, and maintaining high-quality software solutions. You will collaborate with cross-functional teams, lead technical projects, mentor junior engineers, and improve software development practices. This remote position, based in Peru, Colombia, Costa Rica, or Bolivia, requires extensive software development experience with a focus on scalable applications. You will work with Databricks, Python, Apache Spark, and Delta Lake. The role involves ETL pipeline optimization, real-time and batch data processing, and CI/CD implementation.

Requirements

• Bachelor’s degree in computer science, software engineering, or a related field
• Extensive experience in software development with a focus on designing and building scalable applications
• Professional/Advanced English skills
• +5 years of experience
• Experienced automation engineer writing end-to-end tests
• Proficient in SQL. Should understand complex SQL queries about transformations data undergo to effectively write expected behaviors
• Basic knowledge of Databricks. Should know the basics about catalogs/schemas/tables and how to run queries against an existing warehouse
• Basic knowledge on DBT. Must at the very least know the structure of a DBT project and how to write seeds

Responsibilities

• Design and optimization of ETL pipelines in Databricks and Python
• Development of real-time and batch data processing
• Performance optimization in Apache Spark and Delta Lake
• Monitoring and troubleshooting of Databricks clusters
• Collaboration with data and business teams for modeling and architecture
• Implementation of CI/CD for Databricks notebooks and scripts
• Management of workloads in Azure, AWS, or GCP