Staff Data Engineer (Spark and Databricks Advanced)

Summary

Join our team at Wizeline as we build, scale, and deliver high-quality digital products and services. We're looking for a data engineer with experience in large-scale data engineering, analytics, and reporting to lead projects and develop cloud-scalable solutions.

Requirements

• 10+ years of experience with large-scale data engineering with an emphasis on analytics and reporting
• Spark and Databricks Advanced
• SQL
• Data Analysis
• Project Leadership skills
• Strong Programming Skills with Python
• Solid experience with Spark
• Airflow
• Solid engineering foundations (good coding practices, good architectural design skills)
• Experience working with SQL in advanced scenarios that require heavy optimization
• 3+ years of experience developing on Hadoop-like Ecosystem
• Experience building cloud-scalable, real-time and high-performance Data Lake solutions
• Proficiency in designing and implementing ETL (Extract, Transform, load) processes, dealing with big volumes of data (terabytes of data which required distributed processing)
• Experience developing solutions within Cloud Services (AWS, GCP, or Azure)
• Experience with NoSQL databases such as Apache HBase, MongoDB, or Cassandra
• Experience in data streams processing technologies including Kafka, Spark Streaming, etc
• Advanced English level

Responsibilities

• Lead projects
• Develop cloud-scalable solutions