Lead Data Engineer

Summary

Join Wiser Solutions, a company providing in-store and eCommerce intelligence and execution tools, as a Lead Software Engineer. You will lead a team in developing the data architecture for data collection and analytics. Responsibilities include designing, developing, and extending data pipeline services, fostering team growth, improving solution quality, and owning projects from conception to production. The ideal candidate possesses extensive software engineering experience, strong proficiency in data languages and technologies, and a solid understanding of data warehousing and lake solutions. The role requires a Bachelor's or Master's degree in Computer Science or a related field and 10+ years of experience. Bonus points are awarded for experience with vector databases and retail/eCommerce environments.

Requirements

• Bachelors/Master’s degree in Computer Science or relevant technical degree
• 10+ years of professional software engineering experience
• Strong proficiency with data languages such as Python and SQL
• Strong proficiency working with data processing technologies such as Spark, Flink, and Airflow
• Strong proficiency working of RDMS/NoSQL/Big Data solutions (Postgres, MongoDB, Snowflake, etc.)
• Solid understanding of streaming solutions such as Kafka, Pulsar, Kinesis/Firehose, etc
• Solid understanding of Docker and Kubernetes
• Solid understanding of ETL/ELT and OLTP/OLAP concepts
• Solid understanding of columnar/row-oriented data structures (e.g. Parquet, ORC, Avro, etc.)
• Solid understanding of Apache Iceberg, or other open table formats
• Proven ability to transform raw unstructured/semi-structured data into structured data in accordance to business requirements
• Solid understanding of AWS, Linux and infrastructure concepts
• Proven ability to diagnose and address data abnormalities in systems
• Proven ability to learn quickly, make pragmatic decisions, and adapt to changing business needs
• Experience building data warehouses using conformed dimensional models
• Experience building data lakes and/or leveraging data lake solutions (e.g. Trino, Dremio, Druid, etc.)
• Experience working with business intelligence solutions (e.g. Tableau, etc.)
• Experience working with ML/Agentic AI pipelines (e.g. , Langchain, LlamaIndex, etc.)
• Understands Domain Driven Design concepts and accompanying Microservice Architecture
• Passion for data, analytics, or machine learning
• Focus on value: shipping software that matters to the company and the customer

Responsibilities

• Think like our customers – you will work with product and engineering leaders to define data solutions that support customers’ business practices
• Design/develop/extend our data pipeline services and architecture to implement your solutions – you will be collaborating on some of the most important and complex parts of our system that form the foundation for the business value our organization provides
• Foster team growth – provide mentorship to both junior team members and evangelizing expertise to those on others
• Improve the quality of our solutions – help to build enduring trust within our organization and amongst our customers by ensuring high quality standards of the data we manage
• Own your work – you will take responsibility to shepherd your projects from idea through delivery into production
• Bring new ideas to the table – some of our best innovations originate within the team
• Provide mentorship to team members on adopted patterns and best practices
• Organize and lead agile ceremonies such as daily stand-ups, planning, etc

Preferred Qualifications

• Experience working with vector databases
• Experience working within a retail or ecommerce environment
• Proficiency in other programming languages such as Scala, Java, Golang, etc
• Experience working with Apache Arrow and/or other in-memory columnar data technologies