Cohere is hiring a
Member of Technical Staff, Inference & Model Serving in United States, Canada

Summary

The job is for a Member of Technical Staff in the Model Serving team at Cohere. The role involves developing, deploying, and operating the AI platform to deliver large language models through API endpoints. The candidate should have relevant experience with ML models, deep learning, distributed systems, performance optimization, cloud infrastructure, and Golang.

Requirements

• Experience with serving ML models
• Experience designing, implementing, and maintaining a production service at scale
• Familiarity with inference characteristics of deep learning models, specifically, Transformer based architectures
• Familiarity with computational characteristics of accelerators (GPUs, TPUs, and/or Inferentia), especially how they influence latency and throughput of inference
• Strong understanding or working experience with distributed systems
• Experience in performance benchmarking, profiling, and optimization
• Experience with cloud infrastructure (e.g. AWS, GCP)
• Experience in Golang (or, other languages designed for high-performance scalable servers)

Responsibilities

Developing, deploying, and operating the AI platform to deliver large language models through API endpoints

Benefits

• An open and inclusive culture and work environment
• Work closely with a team on the cutting edge of AI research
• Weekly lunch stipend, in-office lunches & snacks
• Full health and dental benefits, including a separate budget to take care of your mental health
• 100% Parental Leave top-up for 6 months for employees based in Canada, the US, and the UK
• Personal enrichment benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement
• Remote-flexible, offices in Toronto, New York, San Francisco and London and co-working stipend
• 6 weeks of vacation