Staff Machine Learning Engineer

Summary

Join ServiceNow's AI team and build next-generation Large Language Models (LLMs) for enterprise language generation. You will leverage your AI/ML expertise to solve real-world challenges, research and implement models for LLM benchmarking, participate in data collection and synthetic data generation, and collaborate with a team of developers and researchers. Responsibilities include contributing to LLM evaluation tools, developing innovative ideas, working with product owners, and potentially publishing research papers. This role requires experience in integrating AI into work processes and solid expertise in Python and machine learning. A Bachelor's degree with 6+ years of experience, a Master's degree with 4+ years of experience, or a PhD is required. Experience with LLM benchmarking and evaluation is essential.

Requirements

• Experience in leveraging or critically thinking about how to integrate AI into work processes, decision-making, or problem-solving. This may include using AI-powered tools, automating workflows, analyzing AI-driven insights, or exploring AI’s potential impact on the function or industry
• 6+ years of relevant experience with a Bachelor's degree; or 4 years with a Master's degree; or a PhD with no experience; or equivalent work experience
• Solid expertise in Python
• Solid Machine Learning / Deep Learning theoretical knowledge and hands-on experience
• Experience with LLM benchmarking and evaluation is required; experience generating synthetic data is a plus
• Ability to read and experiment with ideas from recent research papers
• High level of creativity, quick problem-solving abilities, and adaptability
• Effective communication skills to convey research findings to both technical and non-technical stakeholders, ensuring a clear understanding of the benefits and limitations of LLMs across the organization

Responsibilities

• Research, propose and implement appropriate models/techniques for LLM benchmarking
• Participate in data collection and synthetic data generation, and generate custom benchmarks to evaluate diverse LLMs capabilities
• Collaborate daily with a team of like-minded developers, applied research scientists, product managers and quality engineers to produce quality software
• Contribute to the implementation of tools to facilitate LLMs evaluation and perform model / error analysis
• Develop innovative patentable ideas that ensure the competitiveness of this product within the domain of similar work being done in the industry
• Potentially write research papers in collaboration with our Research Group, to be published to the most notorious AI Conferences
• Work with product owners to understand detailed requirements and own your code from design, implementation, testing and delivery of high-quality & high-impact solutions to our users

Benefits

• Health plans, including flexible spending accounts
• A 401(k) Plan with company match
• ESPP
• Matching donations
• A flexible time away plan
• Family leave programs