Data Scientist

Summary

Join Wellhub, a corporate wellness platform, as a Data Scientist specializing in Generative AI. You will be part of the CARE team in Brazil, focusing on evaluating and measuring Agentic systems to improve user wellbeing. Collaborate with a multidisciplinary team to create innovative AI applications. Your responsibilities include metric design and analysis, data preparation, evaluation and testing of AI performance, prompt engineering, LLM observability, A/B testing, and dashboard development. This role requires a Master's degree in a related field, proficiency in Python, advanced NoSQL and SQL skills, strong problem-solving abilities, and excellent communication skills. Fluency in English is mandatory. Wellhub offers a comprehensive benefits package including health insurance, flexible work options, flexible scheduling, access to wellness resources, paid time off, paid parental leave, and opportunities for career growth.

Requirements

• Master’s degree in Computer Science, Data Science, Machine Learning, Statistics, or a related field
• Proficiency in Python for data handling and analysis
• Advanced NoSQL and SQL skills for querying large datasets
• Strong problem-solving abilities, with a focus on experimental design and data analysis
• Ability to work collaboratively in a team environment and communicate effectively across different departments
• Excellent verbal and written communication skills, with the ability to explain complex technical concepts to non-technical stakeholders
• Fluent in English

Responsibilities

• Metric Design & Analysis: Collaborate with product managers and engineering teams to define and measure metrics for user engagement, satisfaction, and chatbot effectiveness
• Data Preparation: Perform data cleansing, transformation, and quality analysis on product data
• Evaluation and Testing: Assist and design robust evaluation metrics to assess agentic AI performance, combining automated evaluation (LLM-as-judge), adversarial testing, human-in-the-loop evaluations, and custom behavioral metrics
• Prompt Engineering: Craft reusable templates and clear instructions to measure system behavior, optimizing for challenging edge cases
• LLM Observability: Design monitoring systems that track LLM behavior in production, capturing key metrics around information retrieval, hallucinations, and latency
• A/B testing : Design, run, and analyze A/B tests to optimize chatbot interactions and user engagement
• Dashboard Development: Create and maintain interactive dashboards using BI tools (e.g., Tableau, Power BI, Looker, Metabase) for real-time visualization of performance metrics and insights

Preferred Qualifications

• Familiarity with evaluation methodologies for generative AI, including human-in-the-loop assessments, LLM-as-a-judge approaches, and adversarial robustness testing
• Proven track record of tracking and evaluating generative AI models in production
• Experience in multi-agent systems and LLMs

Benefits

• Health, dental, and life insurance
• At Wellhub, flexibility fosters a happier, healthier, and more productive work environment for everyone. As a Flexible First company, we offer two work model options: flexible hybrid and full remote, and make the office a place for collaboration, community, and team building. The model for this role can be discussed with your recruiter and hiring manager. We offer all employees a home office stipend and a monthly flexible work allowance to help cover the costs of working from home
• Wellhubbers and their leaders can make the best decisions for their scope. This includes flexibility to adjust their working hours based on their personal schedule, time zone, and business needs
• We believe in our mission and encourage our employees and their families to take care of their wellbeing too. Access onsite gyms and fitness studios, digital fitness programs, and online wellness resources for meditation, nutrition, mental health support, and more. You will receive the Gold plan at no cost, and other premium plans will be significantly discounted
• Vacations after 6 months and 3 days off per year + 1 day off for each year of tenure (up to 5 additional days) + extra day off for your birthday
• Welcoming a new child is one of the most special moments in your life and we want our employees to take the time to be present and enjoy their growing family. We offer 100% paid parental leave to all new parents and extended maternity leave
• Outstanding opportunities for personal and career growth. That means we maintain a growth mindset in everything we do and invest deeply in employee development