Senior LLM Engineer, Data Scientist

Summary

Join Docplanner Tech as a Senior LLM Engineer and contribute to the development of Noa.ai, a product revolutionizing healthcare access. You will focus on Noa Notes, automating medical documentation, and improving transcription and note quality using LLMs. Responsibilities include prompt engineering, optimizing workflows, identifying failure points, and ensuring system personalization. You will also collaborate with cross-functional teams and maintain quality assurance mechanisms. The ideal candidate possesses proven experience with LLMs, prompt engineering, and AI model improvement in domain-specific applications. Docplanner offers a competitive salary, share options, flexible work arrangements, comprehensive health benefits, professional development opportunities, and relocation support.

Requirements

• Proven experience working with large language models (e.g., OpenAI, Anthropic, Llama)
• Track record in prompt engineering and fine-tuning language models
• Experience in improving AI models for accuracy and customization in domain-specific applications
• Proficiency in programming languages (eg Python, .Net)
• Ability to analyze and interpret user feedback for continuous system improvement
• Problem-solving mindset and adaptability in dynamic environments with a focus on delivering business value to end customers
• Proven ability to manage timelines, prioritize tasks, and deliver results under tight deadlines
• Experience collaborating with cross-functional teams (e.g., product, marketing, engineering)

Responsibilities

• Prompt engineering
• Optimize prompts to improve recall and veracity, especially for medical terminology and concepts
• Continuously refine input-output workflows to enhance LLM accuracy in complex medical scenarios
• Identify common failure points in LLM outputs, particularly in medical contexts, and implement strategies to mitigate inaccuracies
• Enable system-level personalization for healthcare providers by tailoring AI outputs to align with individual doctors' preferences and needs
• Design methods for customized output formatting, ensuring compatibility with various medical systems (e.g., EHRs, diagnostic tools)
• QA & validation
• Integrate quality assurance mechanisms to reduce misinformation in generated content
• Build and maintain validation datasets to validate the improvements and run regression tests
• Review reported cases to monitor potential errors and implement improvements to mitigate them
• Collaboration
• Collaborate with a Product Manager and Software Engineers to integrate structured AI outputs seamlessly into existing healthcare workflows
• Work closely with other members of the ML/DS team to improve the overall performance of Noa (not only your silo)
• Collaborate with legal and security teams to ensure compliance with data processing policies

Preferred Qualifications

• Experience working with Speech Recognition / Transcription systems
• Understanding of medical terminology and healthcare processes

Benefits

• A salary adequate to your experience and skills
• Share options plan after 6 months of working with us
• Remote or hybrid work model with or hub in Warsaw
• Flexible working hours (fully flexible, as in most cases you only have to be on a couple of meetings weekly)
• 20/26 days of paid time off (depending on your contract)
• Additional paid day off on your birthday or work anniversary (you choose what you want to celebrate)
• Private healthcare plan with Signal Iduna for you and subsidized for your family
• Multisport card co-financing for you to have access to sports facilities across Poland
• Access to iFeel , a technological platform for mental wellness offering online psychological support and counseling
• Free English classes
• Relocation support to Poland (we handle the bureaucracy, so you don’t have to)