Senior Data Engineer

Summary

Join Encora as a Senior Data Engineer and be responsible for designing, developing, and maintaining high-quality software solutions. You will collaborate with cross-functional teams, lead technical projects, mentor junior engineers, and improve software development practices. The role requires extensive experience in software development, particularly with data processing in large cloud-based infrastructure (AWS). You will work remotely from Colombia or Costa Rica. Strong skills in Python, PySpark, SQL, and data pipeline orchestration tools are essential. The position offers opportunities for professional growth within a global company.

Requirements

• Hold a Bachelor’s degree in computer science, software engineering, or a related field
• Possess extensive experience in software development with a focus on designing and building scalable applications
• Have professional/advanced English skills
• Have 5+ years of data processing experience in large cloud-based infrastructure (AWS is a must)
• Have hands-on software development experience in Python, with strong proficiency in Py Spark for data engineering tasks and data pipeline development
• Possess expert understanding of SQL, dimensional modeling, and analytical data warehouses, such as Snowflake
• Understand Data Engineering best practices for medium to large scale production workloads
• Have expertise with data pipeline orchestration tools, such as Airflow
• Be familiar with processing semi-structured file formats such as json or parquet
• Be a team player with good communication skills
• Be a problem solver with excellent written and interpersonal skills
• Possess strong problem-solving skills
• Have the ability to learn new technical skills

Responsibilities

• Deploy and configure Apache Spark clusters on AWS (particularly EKS), optimizing for performance, resource allocation, and scalability
• Develop, automate, and support robust and reliable Spark data pipelines, focusing on high performance and low latency
• Design and implement highly optimized queries to improve data processing efficiency, streamline analysis, minimize latency, and enhance overall system performance
• Collaborate to make the Data Platform an effective, scalable, and resilient system
• Gather requirements from stakeholders, prioritize work, and document technical solutions clearly and effectively
• Engage with data engineers, data scientists, product managers, and internal stakeholders to align project goals and implementations
• Contribute to a team culture that values quality, robustness, and scalability while fostering initiatives and innovation

Preferred Qualifications

• Have experience with Jinja, Shell scripting, DBT
• Have experience developing in a Cloud platform using serverless technologies such as glue, lambda, EMR
• Have experience with remote development using AWS SDK
• Have experience with ELT pipelines - DBT
• Have experience with REST API design and implementation
• Be familiar with containers and infrastructure-as-code principles
• Have experience with automation frameworks - Git, Jenkins, and Terraform

Benefits

Remote work