Senior MLOps/LLMOps Engineer

Summary

Join Zeals, a rapidly growing tech company in Japan, and become a Senior MLOps/LLMOps Engineer. You will play a crucial role in deploying, optimizing, and monitoring LLMs in production environments. This involves building scalable pipelines, ensuring low-latency inference, and implementing best practices in monitoring and observability. You will collaborate with a team of AI engineers and data scientists, utilizing state-of-the-art tools like Hugging Face and MLFlow. Zeals offers a highly flexible, remote-first work environment with competitive salary and various benefits. The company is experiencing significant growth and has recently secured substantial funding.

Requirements

• 5+ years of experience in MLOps, DevOps, or related fields, with a focus on deploying and managing LLMs or other large-scale machine learning models
• Proven experience with tools like Hugging Face, MLFlow, and containerization technologies (Docker, Kubernetes)
• Strong experience with cloud platforms (AWS, Azure, GCP) and infrastructure as code (Terraform)
• Hands-on experience in reducing inference latency and optimizing AI infrastructure
• Proficiency in Python, with experience in ML libraries such as TensorFlow, PyTorch, and related frameworks
• Expertise in CI/CD pipelines, version control (Git), and orchestration tools
• Familiarity with Generative AI, prompt engineering, and deploying models at scale
• Excellent problem-solving skills with the ability to tackle complex challenges independently
• Strong communication skills, with the ability to translate technical concepts for non-technical stakeholders
• A proactive mindset with a focus on continuous learning and staying updated with industry trends

Responsibilities

• Develop and maintain scalable pipelines for deploying LLMs, focusing on efficient, low-latency inference
• Utilize tools like Hugging Face and MLFlow for seamless model integration and version control
• Automate deployment processes, including model validation and continuous integration
• Implement comprehensive monitoring frameworks to track performance and reliability of models in production
• Use advanced observability tools to proactively detect and address performance issues
• Deploy alerting systems to ensure rapid response to anomalies in model behavior
• Architect and optimize cloud and on-premise infrastructure to support large-scale LLM operations
• Collaborate with cloud providers like AWS, Azure, and GCP to optimize costs and performance
• Work with backend engineers to ensure smooth integration of AI models into conversational platforms
• Partner with AI engineers and data scientists to align on project objectives and deployment strategies
• Document MLOps processes, best practices, and tools to maintain operational excellence
• Provide training and support to team members on MLOps methodologies and tools

Benefits

• Competitive salary (performance review every 6 months)
• Performance review: twice a year
• 10 days paid holidays during the first year, weekends off, national holidays, summer and winter break
• Visa support: We sponsor visas for the right candidates. You can expect full visa support from our professional HR team
• Highly flexible, remote-first international organization
• For Japanese residence: Work from anywhere, interim work from overseas, full flex time
• Housing allowance (within 1.5KM away from office)
• Club activity allowance
• Shuffle Lunch allowance (Cross department lunch)
• Zeals Bar (bi-monthly free flow beer party)