Senior Data Pipeline Engineer

Summary

Join YipitData's dynamic Data Engineering team as a Senior Data Engineer and play a crucial role in building out the team's presence in India. You will build and maintain end-to-end data pipelines, set best practices for data modeling, create documentation, and solve complex data pipeline issues using PySpark and SQL. This hybrid role requires collaboration with stakeholders to incorporate business logic into central pipelines and deep learning of Databricks, Spark, and internal ETL toolings. The ideal candidate possesses 6-8 years of data engineering experience, a solid understanding of Spark and SQL, and data pipeline experience. This is a unique opportunity to be a first hire with potential to build and lead the team.

Requirements

• You hold a Bachelor’s or Master’s degree in Computer Science, STEM, or a related technical discipline
• You have 6+ years of experience as a Data Engineer or in other technical functions
• You are excited about solving data challenges and learning new skills
• You have a great understanding of working with data or building data pipelines
• You are comfortable working with large-scale datasets using PySpark, Delta, and Databricks
• You understand business needs and the rationale behind data transformations to ensure alignment with organizational goals and data strategy
• You are eager to constantly learn new technologies
• You are a self-starter who enjoys working collaboratively with stakeholders
• You have exceptional verbal and written communication skills

Responsibilities

• Report directly to the Senior Manager of Data Engineering, who will provide significant, hands-on training on cutting-edge data tools and techniques
• Build and maintain end-to-end data pipelines
• Help with setting best practices for our data modeling and pipeline builds
• Create documentation, architecture diagrams, and other training materials
• Become an expert at solving complex data pipeline issues using PySpark and SQL
• Collaborate with stakeholders to incorporate business logic into our central pipelines
• Deeply learn Databricks, Spark, and other ETL toolings developed internally

Preferred Qualifications

Experience with Airflow, dbt, Snowflake, or equivalent

Benefits

We offer vacation time, parental leave, team events, learning reimbursement, and more!