Principal Data Scientist

Summary

Join Elastic's Search Data Science team as a Data Scientist to lead innovation in AI/ML-driven search experiences. You will focus on improving semantic search with proprietary and open-source models, developing query and document understanding techniques, implementing RAG and LLM-driven search, and creating tools for end-to-end RAG systems. This role involves exploring and benchmarking new models, staying current with NLP and information retrieval advancements, and collaborating with cross-functional teams. The ideal candidate possesses extensive experience in NLP, software development (Python), generative AI, and information retrieval. A strong academic background and experience with Elasticsearch are beneficial. Compensation includes a base salary, stock program, and a comprehensive benefits package.

Requirements

• 8+ years of confirmed experience building and applying NLP to production use cases
• 8+ years of professional software development experience in Python
• Experience in Generative AI, Retrieval Augmented Generation, and information retrieval
• Experience with libraries and frameworks such as PyTorch, transformers, and Pandas
• Experience using collaborative notebook-based workflows (e.g. Jupyter) for prototyping and knowledge sharing
• Expertise in AI/ML quality evaluation and improvement, including balancing tuning techniques with cost/benefit tradeoffs
• Self motivated, collaborative style, open communicator, experience in a distributed team
• Good attention to detail and highly organized
• Real passion for data, analysis and achieving excellence

Responsibilities

• Explore, select and benchmark open source and Elastic proprietary models
• Implementing RAG and other LLM-based search experiences
• Designing evaluation protocols for semantic search, tool selection, and generation in LLM-based search experiences
• Keeping up-to-date with the most significant recent developments in the field of NLP and information retrieval
• Engage with the NLP and information retrieval communities (blogs, documentation, Python examples, conference talks, academic papers, etc.)
• Collaborate with cross-functional teams of data scientists, engineers, and product managers
• Promote knowledge sharing and collaboration in a distributed team

Preferred Qualifications

• Experience with Elasticsearch is useful
• An academic background in the domain is also a plus

Benefits

• Competitive pay based on the work you do here and not your previous salary
• Health coverage for you and your family in many locations
• Ability to craft your calendar with flexible locations and schedules for many roles
• Generous number of vacation days each year
• Increase your impact - We match up to $2000 (or local currency equivalent) for financial donations and service
• Up to 40 hours each year to use toward volunteer projects you love
• Embracing parenthood with minimum of 16 weeks of parental leave
• Elastic's stock program
• Company-matched Registered Retirement Savings Plan (RRSP) with dollar-for-dollar matching up to 6% of eligible earnings