Data Scientist

Summary

Join Wellhub's GenAI team in Portugal as a Data Scientist specializing in Generative AI. You will evaluate, measure, and assess Agentic systems focused on improving user wellbeing. Collaborate with a multidisciplinary team to create innovative AI applications. Design and analyze metrics for user engagement and chatbot effectiveness. Perform data cleansing and transformation. Develop robust evaluation metrics for AI performance, including A/B testing and dashboard development. This role requires a Master's degree in a related field, proficiency in Python, advanced NoSQL and SQL skills, and strong problem-solving abilities. Prior experience in data science and Generative AI or LLMs is mandatory.

Requirements

• Master’s degree in Computer Science, Data Science, Machine Learning, Statistics, or a related field
• Proficiency in Python for data handling and analysis
• Advanced NoSQL and SQL skills for querying large datasets
• Strong problem-solving abilities, with a focus on experimental design and data analysis
• Ability to work collaboratively in a team environment and communicate effectively across different departments
• Excellent verbal and written communication skills, with the ability to explain complex technical concepts to non-technical stakeholders
• Fluent in English

Responsibilities

• Metric Design & Analysis: Collaborate with product managers and engineering teams to define and measure metrics for user engagement, satisfaction, and chatbot effectiveness
• Data Preparation: Perform data cleansing, transformation, and quality analysis on product data
• Evaluation and Testing: Assist and design robust evaluation metrics to assess agentic AI performance, combining automated evaluation (LLM-as-judge), adversarial testing, human-in-the-loop evaluations, and custom behavioral metrics
• Prompt Engineering: Craft reusable templates and clear instructions to measure system behavior, optimizing for challenging edge cases
• LLM Observability: Design monitoring systems that track LLM behavior in production, capturing key metrics around information retrieval, hallucinations, and latency
• A/B testing : Design, run, and analyze A/B tests to optimize chatbot interactions and user engagement
• Dashboard Development: Create and maintain interactive dashboards using BI tools (e.g., Tableau, Power BI, Looker, Metabase) for real-time visualization of performance metrics and insights

Preferred Qualifications

• Familiarity with evaluation methodologies for generative AI, including human-in-the-loop assessments, LLM-as-a-judge approaches, and adversarial robustness testing
• Proven track record of tracking and evaluating generative AI models in production
• Experience in multi-agent systems and LLMs

Benefits

• WELLHUB: We believe in our mission and encourage our employees and their families to take care of their wellbeing too. Access digital fitness programs, and online wellness resources for meditation, nutrition, mental health support, and more. You will receive the Gold plan at no cost, and other premium plans will be significantly discounted
• FITNESS: Additional fitness subsidy to access onsite gyms and fitness studios
• FLEXIBLE WORK: At Wellhub, flexibility fosters a happier, healthier, and more productive work environment for everyone. As a Flexible First company, we offer two work model options: flexible hybrid and full remote, and make the office a place for collaboration, community, and team building. The model for this role can be discussed with your recruiter and hiring manager. We offer all employees a home office stipend and a monthly flexible work allowance to help cover the costs of working from home
• FLEXIBLE SCHEDULE: We understand that together, Wellhubbers and their leaders can make the best decisions for their own individual scopes. This includes flexibility to adjust their working hours based on their personal schedule, time zone, and business needs
• PAID TIME OFF: We know how important it is to take time away from work to recharge. Employees receive a minimum of 25 days paid holiday per year with an additional day for each year of tenure (up to 5) in addition to annual holidays (including an extra holiday on your birthday!)
• PARENTAL LEAVE: Welcoming a new child is one of the most special moments in your life and we want our employees to take the time to be present and enjoy their growing family. We offer 100% paid parental leave to all new parents and extended maternity leave
• CAREER GROWTH: Outstanding opportunities for personal and career growth. That means we maintain a growth mindset in everything we do and invest deeply in employee development
• CULTURE: An exciting and supportive environment filled with passionate individuals from all over the world! You’ll partner with global colleagues and share in the success of a high-growth technology company disrupting the health and wellness space. Our value-based culture of trust, flexibility, and integrity makes this possible every day. Find more info on our careers page !