Senior Data Engineer

Summary

Join Wiser Solutions' dynamic team as a Senior Data Engineer and contribute to the development of a next-generation matching platform processing over 200 million transactions daily. You will design, build, and optimize high-performance, scalable solutions for data integration and matching. This role involves developing advanced matching algorithms, optimizing data processing pipelines, and applying cutting-edge technologies. As a technical leader, you will mentor junior developers and collaborate with cross-functional teams. The ideal candidate possesses extensive experience in software and data engineering, proficiency in various programming languages and technologies, and a strong understanding of distributed systems and cloud-native technologies. The position offers a competitive salary and potential for performance-based bonuses.

Requirements

• Bachelors/Master's degree in Computer Science or relevant technical degree
• Over 7 years of professional experience in software and data engineering
• Strong proficiency in Python and Node.js, with a deep understanding of distributed systems and cloud-native technologies
• Strong proficiency working of RDMS/NoSQL/Big Data solutions (Postgres, MongoDB, Trino, etc.)
• Proficiency working with data processing technologies such as Spark, Flink, and Airflow
• Solid understanding of Data Lakes, Data Warehouses, and modeling practices (Data Vault, etc.)
• Understanding of streaming solutions such as Kafka, Pulsar, Kinesis/Firehose, etc
• Excellent understanding of Docker and Kubernetes
• Solid understanding of ETL/ELT and OLTP/OLAP concepts
• Solid understanding of columnar/row-oriented data structures (e.g. Parquet, ORC, Avro, etc.)
• Mastery of AWS, Linux and infrastructure as code
• Experience leveraging data lake solutions (e.g. AWS Glue, DBT, Trino, Iceberg, etc.)
• Understanding of Domain Driven Design concepts and Microservices Architecture
• Experience working in an agile team environment

Responsibilities

• Design and implement robust data integration pipelines to collect, clean, and standardize product data from various sources
• Develop advanced matching algorithms to identify and correlate products across different retailers
• Research, evaluate and synthesize users’ requirements and other technical information to design and develop our distributed software platform, ensuring its scalability, reliability, and performance
• Collaborate with other engineers to architect and implement microservices-based architectures
• Continuously optimize matching algorithms and data processing pipelines to improve efficiency and accuracy
• Leverage your expertise in data structures, algorithms, and performance tuning to enhance system responsiveness
• Apply the latest technologies to enhance software performance and security
• Troubleshoot and resolve complex technical issues
• Translate business needs into efficient technical solutions
• Drive improvements in software development processes and promote continuous learning
• Foster a collaborative team environment
• Mentor and guide junior developers, fostering a culture of innovation and excellence
• Share your knowledge and experience to drive best practices and ensure high-quality code
• Coordinate between technical teams and stakeholders in the development of software and integrated information systems
• Collaborate with cross-functional teams to identify and solve complex technical challenges
• Apply your analytical skills and problem-solving abilities to deliver innovative solutions

Preferred Qualifications

• Experience working within a retail or ecommerce industry
• Full stack engineering experience, including front end frameworks (React, Redux)
• Experience with CQRS
• Experience with C4 modeling

Benefits

• Base pay is estimated to be CAD$120,000 to $140,000
• Performance-based discretionary bonuses and variable pay plans are available for some positions