Senior Staff Machine Learning Scientist

Summary

Join Tempus and contribute to the design and architecture of large-scale multimodal machine learning (LMM) models. You will implement, optimize, and deploy these models using deep learning frameworks like PyTorch or TensorFlow, handling large datasets and distributed training workflows. Responsibilities include designing training pipelines, fusing knowledge into multimodal representations, and collaborating with knowledge integration engineers. The ideal candidate possesses a deep understanding of deep learning, multimodal machine learning, and knowledge representation, along with strong Python and deep learning framework proficiency. Experience with distributed training frameworks and cloud computing platforms is essential. Competitive salary and benefits are offered.

Requirements

• Deep understanding of deep learning principles and architectures (especially transformers)
• Extensive experience with multimodal machine learning concepts and techniques (for example, different fusion methods for text and images)
• Solid understanding of optimization techniques for large-scale models
• Strong proficiency in Python and deep learning frameworks (PyTorch/TensorFlow) and model management libraries like HF Transformers
• Experience with training large multimodal models with distributed training frameworks (for example, Horovod, MosaicML) and GPU fleet management
• Strong understanding of knowledge representation concepts (for example, knowledge graphs, ontologies)
• Experience with distributed training frameworks and cloud computing platforms (for example, GCP, Azure)

Responsibilities

• Design and definition of the architecture of the LMMs, considering different fusion strategies and modality-specific processing
• Implement, refine, benchmark and optimize model architectures using deep learning frameworks such as PyTorch or TensorFlow
• Develop and manage the end-to-end training pipelines, including data loading, preprocessing, and model training. Architect and deploy distributed training workflows, optimizing for performance across cloud GPU fleets
• Implement distributed training strategies to handle large-scale datasets and models
• Design and implement methods to fuse knowledge with the multimodal representations within the LMM
• Experiment with different approaches to enhance the model's understanding and reasoning abilities through knowledge integration
• Monitor and debug training processes, identifying and resolving performance bottlenecks
• Collaborate with the knowledge integration engineer to ensure the architecture can accommodate knowledge injection mechanisms

Benefits

• Incentive compensation
• Restricted stock units
• Medical and other benefits