Staff Machine Learning Engineer

Summary

Join ServiceNow's CoreLLM team and contribute to the development of next-generation Large Language Models (LLMs) for the NOW platform. You will be involved in all phases of LLM development, from data curation and training to evaluation and improvement. Collaborate with a team of developers, researchers, and product managers to build AI-powered solutions for our enterprise customers. Your work will directly impact the AI experiences within our platform, enabling our customers to solve complex problems and accelerate innovation. This role requires expertise in prompt engineering, fine-tuning LLMs, and using AI productivity tools. You will also need strong communication skills to convey research findings to both technical and non-technical stakeholders.

Requirements

• Prompt Engineering: proficient in prompt engineering and developing LLM based features
• Fine tuning: experience with methods of training and fine tuning LLMs, such as distillation, supervised fine-tuning and policy optimization
• AI productivity tools: experience in using AI productivity tools such as Cursor, Windsurf, etc
• Experience in leveraging or critically thinking about how to integrate AI into work processes, decision-making, or problem-solving. This may include using AI-powered tools, automating workflows, analyzing AI-driven insights, or exploring AI’s potential impact on the function or industry
• 6+ years of relevant experience with a Bachelor's degree; or 4 years with a Master's degree; or a PhD with no experience; or equivalent work experience
• Solid expertise in Python
• Solid Machine Learning / Deep Learning theoretical knowledge and hands-on experience
• Experience with LLM benchmarking and evaluation is required; experience generating synthetic data is a plus
• Ability to read and experiment with ideas from recent research papers
• High level of creativity, quick problem-solving abilities, and adaptability
• Effective communication skills to convey research findings to both technical and non-technical stakeholders, ensuring a clear understanding of the benefits and limitations of LLMs across the organization

Responsibilities

• Research, propose and implement appropriate models/techniques for LLM benchmarking
• Participate in data collection and synthetic data generation, and generate custom benchmarks to evaluate diverse LLMs capabilities
• Collaborate daily with a team of like-minded developers, applied research scientists, product managers, quality engineers and linguists, to produce quality software
• Contribute to the implementation of tools to facilitate LLMs evaluation and perform model / error analysis
• Develop innovative patentable ideas that ensure the competitiveness of this product within the domain of similar work being done in the industry
• Potentially write research papers in collaboration with our Research Group, to be published to the most notorious AI Conferences
• Work with product owners to understand detailed requirements and own your code from design, implementation, testing and delivery of high-quality & high-impact solutions to our users

Benefits

• Health plans, including flexible spending accounts
• A 401(k) Plan with company match
• ESPP
• Matching donations
• A flexible time away plan
• Family leave programs