Machine Learning Quality Engineer

Summary

Join ServiceNow's Conversational AI team as a Quality Engineering professional, focusing on machine learning technologies. You will be responsible for testing large-scale production systems, building and maintaining automation test frameworks, and evaluating AI/ML model quality. The role requires strong test automation skills in Java or Python, experience with various testing techniques, and familiarity with agile methodologies. A bachelor's degree and 2+ years of relevant experience are required, along with experience in leveraging AI in work processes. The position offers a competitive salary, equity, benefits, and flexible work arrangements.

Requirements

• Experience in quality assurance and/or application development
• Strong test automation skills in Java / Python
• Hands-on experience testing large-scale, production systems in Python or Java
• Excellent communication, reporting and problem-solving skills
• Ability to take a project from scoping the requirements and building the test cases
• Maintain existing automation test frameworks
• Familiarity with testing AI/ML models and evaluation of the model quality
• Ability to review results from the test set and create defects based on result patterns
• Work with developers to design specific testing strategies for features being developed and automate them
• Create comprehensive test plans; execute and automate them
• Partner with the engineering organizations in troubleshooting or addressing issues with applications and dev/test environments
• Experience in leveraging or critically thinking about how to integrate AI into work processes, decision-making, or problem-solving. This may include using AI-powered tools, automating workflows, analyzing AI-driven insights, or exploring AI's potential impact on the function or industry
• 2+ years of experience with technologies relevant to SN, Quality engineering and coding skills with high-quality results
• 2 years of experience as a Machine Learning Quality Engineer
• Bachelor's degree and a minimum of 2 years of related experience; or an advanced degree without experience; or equivalent work experience
• Experience working within different automated testing frameworks, including Java, JUnit, Python, Selenium, TestNG and other open-source projects
• Experience with the agile methodology for software development teams
• Ability to know several testing techniques (e.g. performance, unit, integration, automated), their strengths and weakness, and ability to use them to best effect - including tracking and addressing of any discovered issues
• Ability to use tools (such as IDE, debugger, build tools, source control, ServiceNow instances, profilers, system administration/Unix tools) to assist with daily tasks

Responsibilities

• Develop and maintain automation test frameworks
• Test large-scale, production systems in Python or Java
• Evaluate the quality of AI/ML models and review test results to create defects
• Collaborate with developers to design and automate testing strategies for new features
• Create and execute comprehensive test plans
• Troubleshoot and address application and environment issues with engineering teams

Benefits

• Health plans, including flexible spending accounts
• A 401(k) Plan with company match
• ESPP
• Matching donations
• A flexible time away plan
• Family leave programs