Staff Machine Learning Engineer

Summary

Join ServiceNow's CoreLLM team and contribute to the development of next-generation Large Language Models (LLMs) for the NOW platform. You will be involved in all phases of LLM development, from data curation and training to evaluation and improvement. The models you help build will power a wide range of applications across various business units, solving complex problems and accelerating innovation. You will collaborate with a team of developers, researchers, and product managers to deliver high-quality solutions. This role requires expertise in prompt engineering, fine-tuning LLMs, and LLM benchmarking. The ideal candidate will have a strong background in machine learning, Python, and experience with AI productivity tools.

Requirements

• Prompt Engineering: proficient in prompt engineering and developing LLM based features
• Fine tuning: experience with methods of training and fine tuning LLMs, such as distillation, supervised fine-tuning and policy optimization
• 6+ years of relevant experience with a Bachelor's degree; or 4 years with a Master's degree; or a PhD with no experience; or equivalent work experience
• Solid expertise in Python
• Solid Machine Learning / Deep Learning theoretical knowledge and hands-on experience
• Experience with LLM benchmarking and evaluation is required; experience generating synthetic data is a plus
• Ability to read and experiment with ideas from recent research papers
• High level of creativity, quick problem-solving abilities, and adaptability
• Effective communication skills to convey research findings to both technical and non-technical stakeholders, ensuring a clear understanding of the benefits and limitations of LLMs across the organization

Responsibilities

• Research, propose and implement appropriate models/techniques for LLM benchmarking
• Participate in data collection and synthetic data generation, and generate custom benchmarks to evaluate diverse LLMs capabilities
• Collaborate daily with a team of like-minded developers, applied research scientists, product managers, quality engineers and linguists, to produce quality software
• Contribute to the implementation of tools to facilitate LLMs evaluation and perform model / error analysis
• Develop innovative patentable ideas that ensure the competitiveness of this product within the domain of similar work being done in the industry
• Potentially write research papers in collaboration with our Research Group, to be published to the most notorious AI Conferences
• Work with product owners to understand detailed requirements and own your code from design, implementation, testing and delivery of high-quality & high-impact solutions to our users

Preferred Qualifications

• Experience in using Al productivity tools such as Cursor, Windsurf, etc. is a plus or nice to have
• Experience in leveraging or critically thinking about how to integrate AI into work processes, decision-making, or problem-solving. This may include using AI-powered tools, automating workflows, analyzing AI-driven insights, or exploring AI’s potential impact on the function or industry

Benefits

• Health plans, including flexible spending accounts
• A 401(k) Plan with company match
• ESPP
• Matching donations
• A flexible time away plan
• Family leave programs