Senior Data Scientist

Summary

Join Udemy's Data organization as a Psychometrician and partner with product, design, and engineering teams to build algorithmic and model-driven solutions for skill assessments. You will leverage your expertise in psychometrics and data science to maintain and grow our skill assessment platform, communicate findings to both technical and non-technical audiences, and implement cutting-edge research in educational assessment. This role demands strong analytical and communication skills, proficiency in SQL and at least one programming language (R or Python), and experience with large datasets. You will be responsible for developing psychometric models, defining metrics, and collaborating with content developers. Udemy offers a collaborative environment and opportunities for professional development.

Requirements

• 3+ years of hands-on industry experience in a psychometric or data science-oriented role
• Domain experience in educational assessment and psychometrics (e.g., Classical Test Theory (CTT), Item Response Theory (IRT), Computerized Adaptive Testing (CAT), equating and scaling)
• Experience creating or applying novel measurement models (Diagnostic Classification, Knowledge Tracing, MIRT) and their associated estimation algorithms
• Strong SQL ability and proficiency with 1+ programming languages (e.g., R or Python)
• End-to-end experience organizing and analyzing large data sets for operational purposes
• Ability to communicate effectively with non-technical stakeholders
• Strong knowledge of applied statistics and experimental design
• Ability to work autonomously, relentlessly prioritize, and collaborate with team members across time zones

Responsibilities

• Collaborate with other psychometricians, data scientists, PMs, designers, and engineers to maintain and grow our skill assessment platform
• Effectively communicate and present to both technical and non-technical audiences
• Critically evaluate and implement the state-of-the-art research literature in educational assessment
• Develop production-grade data pipelines and continuously improve the quality and accessibility of data
• Develop psychometric models for skill assessment through all phases, including prototyping, productionization, and experimental evaluation
• Define metrics and build dashboards to track the adoption and health of skill assessments
• Collaborate with content developers and instructional designers to develop and refine item writing and review processes

Preferred Qualifications

• 5+ years of relevant industry experience
• Experience with machine learning and causal inference
• Familiarity with standard software engineering practices (e.g., Scrum)
• Experience using big data technologies (e.g., Hadoop, Spark)

Benefits

• Comprehensive medical, dental, vision, life/AD&D, and disability coverage
• Pre-tax savings/spending plans, including HSA employer contributions
• Expanded medical coverage for gender-affirming care
• Financial assistance for fertility (including egg-freezing), adoption, and surrogacy expenses; twelve weeks of paid family bonding for birth, surrogacy, adoption, and foster placement (plus up to six to ten weeks of paid disability leave for birthing parents) as well as a two-week “ease-back” program that enables parents to return part-time at full pay
• Fully paid memberships for meditation/mindfulness, therapy/coaching visits, financial planning, fertility and family building, maternity and newborn care, parenting and pediatrics, menopause care, breastmilk shipping, and caregiving support
• Free access to Udemy and Udemy Business plus a yearly stipend to spend on professional development
• Pre-tax and Roth 401(k) plan with an employer contribution
• Flexible vacation for salaried Udemates and fifteen days per year for hourly
• Twelve paid holidays throughout the year