Algolia is hiring a
Senior Data Engineer

Summary

Join our Data team as a Senior Data Engineer to help us make Algolia a data-driven organization. As a member of the Data Engineering team, you will develop data pipelines and ETL workflows mainly with Python, Airflow (AWS MWAA), Spark (AWS Glue, EMR), and other AWS services managed with Terraform.

Requirements

• Strong experience designing and building data pipelines
• Experience orchestrating pipelines with Airflow
• Professional knowledge of Python
• Strong experience working with cloud platforms and architecting them
• Strong experience with data warehouses
• Experience with infrastructure topics specific to data engineering
• Interest in Big Data challenges (~600Tb of data, +14Tb per week, ~700M files, +1.2M per week, ~300 dbt models, ~70 Airflow DAGs)
• Interest in understanding the data and business requirements
• Excellent spoken and written English skills
• A humble, curious, proactive mindset, and a balance between creativity, resourcefulness, and pragmatism

Responsibilities

• Developing data pipelines and ETL workflows mainly with Python, Airflow (AWS MWAA), Spark (AWS Glue, EMR), and other AWS services managed with Terraform
• Improving and maintaining our data warehouse (AWS Redshift)
• Supporting the Analytics Engineering team to build data models with dbt to be used by Analysts on business reports
• Interacting with Engineering and Business teams to understand requirements
• Design, build, enrich, and scale up our data pipelines
• Work with engineers, data analysts, and business analysts to capture and model data
• Monitor data integrity and growth
• Help improve our data lake and data warehouse architecture to increase performance, simplicity, and autonomy of the users
• Help analysts industrialize reports and dashboards to improve company productivity
• Ensure every product is released with a data-driven approach

Preferred Qualifications

• Experience using and managing AWS Redshift
• Experience with a parallel data processing framework such as Apache Spark
• Experience analyzing data quality using dbt
• Experience at our current stage and beyond ($50-200M ARR range, high growth, lots of change, and building internal infrastructure)