Senior Machine Learning Engineer - Search and Recommendations

Summary

Join Instacart's Search & Recommendations ML team as a Senior Machine Learning Engineer and help build the LLM backbone powering the entire shopping journey. You will architect and scale foundational LLM systems, develop agentic workflows to support shoppers, fine-tune large language models, and design intelligent discovery pipelines. Collaborate with cross-functional teams to translate high-level goals into scalable LLM-powered systems. This role offers the flexibility to work remotely and includes a competitive compensation and benefits package, including equity grants. The specific salary range varies based on location. The team is focused on unifying search, discovery, merchandising, and recommendations under a single AI platform.

Requirements

• Have 5+ years of industry experience using machine learning to solve real-world problems with large datasets with 3+ years in a technical leadership role
• Proven track record designing and deploying sophisticated ML/AI systems in production environments that drive measurable business impact through improved recommendations, search relevance, and user engagement metrics
• Have strong programming skills in Python and fluency in data manipulation (SQL, Pandas) and Machine Learning (scikit-learn, XGBoost, Keras/Tensorflow) tools
• Have strong analytical skills and problem-solving ability
• Are a strong communicator who can collaborate with diverse stakeholders across all levels

Responsibilities

• Architect and scale foundational LLM systems that unify query understanding, personalization, ranking, ads, and merchandising—replacing dozens of siloed models with a single, adaptive backbone
• Develop agentic workflows that proactively support shoppers by generating personalized cart starters, thematic bundles, and dynamic storefronts—driven by LLMs reasoning over historical behavior, preferences, and context
• Fine-tune large language models using SFT, DPO, and GRPO on rich behavioral feedback signals to align system outputs with evolving customer needs and business objectives
• Design intelligent, end-to-end discovery pipelines that handle everything from long-tail semantic retrieval to real-time multi-objective ranking and merchandising optimization
• Collaborate cross-functionally with product, design, infra, and ads teams to translate high-level discovery goals into scalable LLM-powered systems that drive measurable impact
• Coach and mentor a team of ML engineers, fostering their technical and professional growth

Preferred Qualifications

• Extensive expertise with modern deep learning frameworks (PyTorch, TensorFlow, JAX) and advanced LLM architectures including transformer models, attention mechanisms, and multimodal AI systems
• Demonstrated experience implementing and fine-tuning large language models, including prompt engineering, embedding techniques, and efficient inference optimization for production environments
• Strong foundation in AI fundamentals including neural network architectures, generative models, and foundation model adaptation methodologies like PEFT, LoRA, and RLHF
• Experience optimizing AI model performance across the full stack, from model architecture and training workflows to distributed inference and serving infrastructure
• Self-motivated innovator with a strong sense of ownership who can navigate the rapidly evolving AI landscape, evaluate emerging techniques, and implement novel approaches to solve complex business challenges
• Passion for applying cutting-edge AI research to real-world applications and a keen understanding of the practical considerations in developing responsible, efficient AI systems at scale

Benefits

• Instacart provides highly market-competitive compensation and benefits in each location where our employees work
• This role is remote
• The base pay range for a successful candidate is dependent on their permanent work location
• This role is eligible for a new hire equity grant as well as annual refresh grants