Member Of Engineering Inference

Summary

Join poolside, a company building AI to drive economically valuable work and scientific progress, focusing on software development. We're a remote-first team building multi-device inference for Large Language Models (LLMs). This role centers on developing and improving LLM inference for source code generation, optimizing for low latency, high throughput, and hardware utilization. You will work with cutting-edge inference techniques, various programming languages, and hardware accelerators. We offer a remote-first work environment with flexible hours and a comprehensive benefits package.

Requirements

• Experience with Large Language Models (LLM)
• Confident knowledge of the computational properties of transformers
• Knowledge/Experience with cutting-edge inference tricks
• Knowledge/Experience of distributed and lower precision inference
• Knowledge of deep learning fundamentals
• Strong engineering background
• Theoretical computer science knowledge is a must
• Experience with programming for hardware accelerators
• SIMD algorithms
• Expert in matrix multiplication bottlenecks
• Know hardware operation latencies by heart
• Research experience

Responsibilities

• Follow the latest research on LLMs, inference and source code generation
• Propose and evaluate innovations, both in the quality and the efficiency of the inference
• Monitor and implement LLM inference metrics in production
• Write high-quality high-performance Python, Cython, C/C++, Triton, ThunderKittens, native CUDA, Amazon Neuron code
• Work in the team: plan future steps, discuss, and always stay in touch

Preferred Qualifications

• Author of scientific papers on any of the topics: applied deep learning, LLMs, source code generation, etc
• Can freely discuss the latest papers and descend to fine details
• You have strong opinions, weakly held
• Programming experience
• Linux
• Git
• Python with PyTorch or Jax
• C/C++, CUDA, Triton, ThunderKittens
• Use modern tools and are always looking to improve
• Opinionated but reasonable, practical, and not afraid to ignore best practices
• Strong critical thinking and ability to question code quality policies when applicable
• Prior experience in non-ML programming is a nice to have

Benefits

• Fully remote work & flexible hours
• 37 days/year of vacation & holidays
• Health insurance allowance for you and dependents
• Company-provided equipment
• Wellbeing, always-be-learning and home office allowances
• Frequent team get togethers
• Great diverse & inclusive people-first culture