Middle QA Engineer

Summary

Join SupportYourApp's product team as a QA Engineer to ensure the stability, security, and quality of AI responses for B2B AI solutions. You will create and maintain test cases, conduct manual testing of new features and AI responses, validate integrations with LLMs and APIs, analyze edge cases, and write automated tests. Collaboration with developers, ML engineers, and product owners is essential. The role involves validating responses in production and proposing solutions to improve code quality. This position offers a competitive salary in USD, flexible remote work options, and professional growth opportunities.

Requirements

• 3+ years of experience as a QA Manual or QA Full-Stack engineer
• Understanding of REST APIs, ability to test via Postman
• Experience with automated testing tools (Cypress/Playwright/Selenium)
• Ability to read logs, analyze errors, and work with CI/CD
• Basic knowledge of Git, experience with Jira or a similar task tracker
• Knowledge of the basics of AI/ML systems - how LLMs, prompts, and vector databases work
• Responsibility, independence, attention to detail

Responsibilities

• Create and maintain test cases, checklists, and test plans
• Conduct manual testing of new features, AI responses, and the interface
• Check the correctness of integrations with LLMs, vector databases, and APIs
• Analyze edge cases, validate the tone, logic, and accuracy of model responses
• Write automated tests (smoke/regression) - preferably Cypress or Playwright
• Interact with developers, ML engineers, and Product Owners
• Validate the quality of responses in production (human feedback loop)
• Propose and implement solutions to improve code quality

Preferred Qualifications

• Experience testing AI/ML products
• Knowledge of prompt engineering
• Ability to work with MongoDB
• Experience writing scripts for automation (Python or JS)
• Experience with observability tools (Sentry, Datadog, LogRocket, etc.)

Benefits

• Convenient, fixed schedule during business hours
• Competitive salary in USD
• Ability to work fully remotely, or in our spacious and well-equipped art office (https://officesnapshots.com/2021/04/28/supportyourapp-offices-kyiv)
• Financial and professional growth
• Paid training and probationary period
• Creative team of professionals and lack of bureaucracy
• Corporate library