Senior Product Manager AI

Summary

Join our client's dynamic team as a Senior Product Manager AI. This role requires experience in American startups, a proven track record in e-commerce, and a solid understanding of AI model evaluation. You will transform complex processes into tangible product improvements, designing and implementing evaluation frameworks for AI models. The ideal candidate will have strong communication skills and experience working with cross-functional leaders. This position offers opportunities for growth and development within a remote and distributed team.

Requirements

• +7 years of Product Management experience, preferably within the e-commerce sector
• Experience working and establishing credibility with cross-functional leaders
• Strong communication skills, both written and verbal
• Previous experience in AI/LLM model evaluation
• Deep knowledge of tracking real model performance metrics rather than UX-only metrics
• Ability to dig into the details without losing sight of the bigger picture
• Product management experience with a strong process orientation

Responsibilities

• Design and implement evaluation frameworks for AI models, focusing on hallucination rate, latency, accuracy, and model consistency
• Develop evaluation tooling, dashboards, and automated benchmarks to measure AI model performance
• Own A/B testing for model selection and performance validation
• Collaborate closely with machine learning engineers on real-time model monitoring
• Embrace a process-focused approach, using clear workflows in Jira, Gantt charts, and effective engineering story management
• Ability to demonstrate how model evaluations have directly improved product outcomes

Preferred Qualifications

Engineering degree