Senior Data Scientist

Summary

Join PandaDoc's AI team as a Senior Data Scientist to build and optimize AI models for our platform, empowering small and medium-sized businesses. You will design, evaluate, and deploy AI models, focusing on LLMs and GenAI applications. This role involves creating high-quality datasets, fine-tuning models, and implementing robust evaluation frameworks. You will collaborate with engineers, designers, and product managers to deliver impactful AI solutions. The ideal candidate possesses extensive experience with Python, LLMs, and dataset creation, along with a strong understanding of AI model evaluation and deployment. This is a high-impact role offering the opportunity to work on cutting-edge AI technology and contribute to a rapidly growing company.

Requirements

• 5+ years of experience in Python using in DS
• Hands-on experience with LLMs, RAG, knowledge graphs and fine-tuning
• Proven ability to measure and improve model performance
• Expertise in building and curating datasets for AI training
• Experience deploying LLMs and implementing guardrails for responsible AI usage
• Stays updated on the latest AI research, tools, and frameworks, identifying ways to apply them effectively
• Passion for building AI solutions that provide real value to end users
• Thrives in a high-speed environment with autonomy to solve problems
• Focuses on building AI solutions that drive tangible business impact

Responsibilities

• Create evaluation frameworks to assess LLMs and GenAI applications, measuring recall, effectiveness, and accuracy
• Develop high-quality datasets for model training, fine-tuning, and inference
• Establish evaluation pipelines for prompt performance, ensuring reliability and efficiency
• Optimize LLMs for specific business use cases to improve their effectiveness
• Deploy LLMs in production and implement systems to maintain model constraints and safe usage
• Work with various LLMs (e.g., GPT-4, Claude, fine-tuned models) for efficient routing and task handling
• Contribute to AI-powered automation systems that enhance customer workflows and streamline routine tasks
• Work closely with PMs, designers, and engineers to define AI requirements and bring solutions to life