Principal Software Engineer, Catalog & Real-Time Serving Systems

Summary

Join Instacart's Customers organization as a Sr. Staff or Principal Engineer to lead the evolution and scalability of core Catalog and data-intensive systems. This crucial role involves advancing Machine Learning serving and infrastructure capabilities, impacting core business functions and driving significant revenue. You will design, build, and scale reliable solutions, optimize ML serving endpoints, centralize ML serving logic, and contribute to company-wide initiatives. As a subject matter expert, you will provide guidance and mentorship, identify innovative solutions, and collaborate with cross-functional teams. This is a remote position offering a unique opportunity to make a significant impact on Instacart's platform.

Requirements

• Extensive experience in software engineering, with a focus on distributed systems, streaming processing (e.g., Flink), data intensive applications, and particularly, Machine Learning serving and deployment
• Proven track record of designing, implementing, and scaling large-scale, high-performance systems, including ML serving infrastructure
• Deep understanding of database technologies, data modeling, data pipelines, and ML model deployment patterns
• Strong architectural skills and the ability to design and evaluate complex technical solutions across diverse technology domains, including Catalog, Streaming, and Machine Learning
• Excellent problem-solving and debugging skills, with specific experience in addressing issues related to ML model serving, data quality, and infrastructure stability
• Strong communication and collaboration skills, with the ability to effectively work across teams, influence stakeholders, and mentor junior engineers
• Experience with cloud platforms and related technologies, including ML serving platforms (e.g., Sagemaker)
• Ability to quantify and demonstrate the impact of technical contributions on business results (e.g., revenue, efficiency, cost savings, and ML model performance)
• Familiarity with challenges related to ML lifecycle, data flow, and best practices

Responsibilities

• Provide architectural leadership for Catalog, streaming, and data-intensive systems, emphasizing ML serving infrastructure and best practices, and drive the technical roadmap
• Design, build, and scale reliable, efficient, and adaptable solutions to address changing business and ML needs
• Lead the development and optimization of ML serving endpoints, ensuring high availability, low latency, robust performance, and implement fail-fast input validations and track metrics using Datadog
• Centralize ML serving logic and decouple it from product applications to improve debugging, manageability, and system performance
• Drive and contribute to company-wide transformational initiatives, impacting key business metrics like revenue, personalization, and operational efficiency, and influence the direction of ML infrastructure including real-time inferencing
• Serve as a subject matter expert for Catalog, streaming, data-intensive, and ML serving technologies, providing guidance and mentorship to engineering and data science teams
• Identify and implement innovative solutions to optimize system performance, reduce costs, and improve data processing and ML serving latency
• Collaborate with cross-functional teams, including Product, Retailer, IC App, Ads, ML Infrastructure, and Data Science, to deliver integrated ML-driven solutions, and lead incident response and resolution for high-severity issues

Preferred Qualifications

• Experience working with large-scale catalog systems or similar data-intensive platforms
• Significant experience in designing and implementing high-throughput, low-latency ML serving systems
• Contributions to open-source projects or technical publications related to distributed systems, data engineering, or Machine Learning serving
• Experience in a high-growth, fast-paced environment, particularly in the context of scaling ML initiatives

Benefits

• Instacart provides highly market-competitive compensation and benefits in each location where our employees work
• This role is remote
• Additionally, this role is eligible for a new hire equity grant as well as annual refresh grants