Senior ETL Developer

Summary

Join StackAdapt's Enterprise Data Office (EDO) as a Senior ETL Developer and take ownership of the data architecture, pipelines, operations, and platform management for our data lake and enterprise data warehouse. You will assess business requirements, build ingestion pipelines from various sources, architect data models, and construct data transformation pipelines. This role involves orchestrating pipeline execution, collaborating with BI Engineers, and ensuring seamless data consumption by stakeholders. StackAdapt is a Remote-First company, and this position is open to candidates located anywhere in Canada. The ideal candidate will have extensive experience in data engineering within an enterprise data warehouse environment and a strong understanding of cloud-based data warehouses and big data technologies.

Requirements

• Minimum 5 years of experience leading data engineering within an Enterprise Data Warehouse environment
• Hands-on experience with cloud-based data warehouses (e.g., Snowflake, BigQuery, Redshift) and bigdata mediums (e.g., Delta Lake, Parquet, Avro)
• Knowledgeable in data warehousing architecture fundamentals (e.g., Kimball/Inmon methodology, dimensional modeling, conformed dimensions, SCDs, etc)
• Proven experience managing cloud-based infrastructure on AWS/AZURE/GCP with knowledge of container technologies such as Kubernetes and networking fundamentals
• Hands-on experience building ETL/ELT data pipelines via custom-coded scripts (e.g., Spark, Python, JAVA, SQL stored procedures) and via integration platforms (e.g., PowerCenter, DataStage, Talend)
• Highly experienced in orchestrating data operations via tools such as Apache Airflow, Cron, Astronomer etc. and administering the data platform via Infrastructure-as-Code (e.g., Terraform)

Responsibilities

• Build reliable data ingestion pipelines to extract data from a variety of data sources including databases (e.g., RDBMS/NOSQL/file stores), applications (via API), flatfiles, etc into the Data Lake with appropriate metadata tagging
• Orchestrate data pipelines via batch, near-real-time, or real-time operations depending on requirements to ensure a seamless and predictable execution
• Advise the team on all data architecture, data modeling, data operations, and data platform decisions
• Support the day to day operation of the EDO pipelines by monitoring alerts and investigating, troubleshooting, and remediating production issues

Benefits

Remote work, flexible hours