Lead, Product Experimentation

Summary

Join NBCUniversal's Media Group as a Lead for Statistical Inference & Research, where you will be the principal architect of the experimentation platform's statistical engine. Design, implement, and optimize advanced statistical methodologies for measuring and innovating through experimentation. Conduct frequent analysis of experiment data to uncover insights and improve causal inference. Collaborate with data engineering teams to build and optimize statistical methodologies in production data pipelines. Mentor analytics and data science teams on experimental design and advanced inference methods. Contribute to roadmap planning and technical strategy. This fully remote position offers competitive compensation and company-sponsored benefits.

Requirements

• 6+ years in experimentation science, applied statistics, data science, or data engineering, with significant experience in large-scale digital experimentation or analytics platforms
• Deep expertise in statistical inference for experimentation (randomization, regression, variance reduction, sequential/adaptive methods, multiple hypothesis correction, causal inference, etc.)
• Hands-on experience implementing statistical models and methodologies in production data systems (SQL, Python, R, Spark, Databricks)
• Demonstrated ability to build and optimize data pipelines and workflows in partnership with data engineering
• Proven skill in analyzing large and complex experimental datasets to inform and optimize inference methodologies
• Excellent communication and documentation skills, with a track record of mentoring technical colleagues and cross-functional partners
• Individual contributor technical leadership, ideally within experimentation, analytics, or data engineering teams
• Fully Remote: This position has been designated as fully remote, meaning that the position is expected to contribute from a non-NBCUniversal worksite, most commonly an employee’s residence

Responsibilities

• Design, maintain, and implement core inference capabilities (A/B, CUPED, sequential/adaptive methods, variance reduction, diagnostics) within our experimentation platform
• Ensure all methodologies are robust, scalable, and production-ready
• Conduct frequent statistical analysis on live and historical experiment data to optimize detection of real effects, reduce variance, and enhance signal purity
• Research, prototype, and validate new statistical techniques, including handling zero-inflated data, covariate adjustment, multiple hypothesis correction, and causal inference
• Lead the design and execution of simulations, backtests, and validation studies for inference engine upgrades
• Build, optimize, and deploy statistical methodologies into production data pipelines and analytics workflows
• Work directly with data engineering teams to define requirements, data models, and ensure successful scaling and integration of new methods
• Mentor and guide analytics and data science teams on experimental design, statistical troubleshooting, and the use of advanced inference methods
• Translate complex statistical innovations into clear technical documentation, best practices, and internal training resources
• Advise on design and analysis of high-impact experiments across the company
• Identify platform opportunities and challenges; contribute to roadmap planning and technical strategy

Preferred Qualifications

• Deep intellectual curiosity and drive to improve experimentation at scale
• Technical rigor and a strong sense of ownership from research to production delivery
• Collaborative, proactive, and skilled at working across engineering, data science, and analytics functions
• Able to distill and communicate complex statistical concepts to technical audiences
• Thrives in a high-velocity, innovative product organization

Benefits

• This position is eligible for company sponsored benefits, including medical, dental and vision insurance, 401(k), paid leave, tuition reimbursement, and a variety of other discounts and perks
• Salary range: $115,000 - $145,000 (bonus and long-term incentive eligible)