Senior Automation Engineer

Summary

Join Skyward, a people-centered business with a civic tech mission, as a Senior Automation Engineer. You will be part of a team developing an AI/ML platform for the Centers for Medicare and Medicaid Services (CMS), designing and maintaining automated test frameworks, developing automated pipelines for model validation, and defining and executing test plans. Collaborate with AI/ML engineers, implement CI/CD-integrated quality gates, mentor junior engineers, and advocate for ethical AI standards. This role requires 5+ years of experience in software QA/automation engineering, with at least 2 years focused on AI/ML product testing, proficiency in automation frameworks and scripting languages, and experience with AI model testing and Agile DevSecOps environments. Skyward offers competitive compensation, benefits including fully paid medical, dental, and vision insurance, paid time off, 401k, parental leave, professional development funds, and opportunities for remote work and flexible hours.

Requirements

• 5+ years of experience in software QA/automation engineering, with at least 2 years focused on AI/ML product testing
• Strong proficiency in at least one automation framework (e.g., Selenium, Cypress, Playwright) and scripting language (Python preferred)
• Hands-on experience with testing AI models, including validation metrics, dataset versioning, and model explainability
• Familiarity with tools like MLflow, TensorBoard, or similar for AI observability
• Understanding of model bias, fairness audits, and compliance considerations in government or regulated environments
• Experience working in Agile DevSecOps environments with modern CI/CD tooling (Jenkins, GitHub Actions, GitLab, etc.)
• Strong collaboration skills with a knack for clear communication across engineering, data science, and product teams

Responsibilities

• Be part of a team supporting the development of an AI/ML platform that will leverage generative AI to relieve user burden across the Centers for Medicare and Medicaid Services (CMS)
• Design and maintain robust, scalable automated test frameworks for web apps, APIs, and AI-driven features
• Develop automated pipelines for model validation, bias detection, and drift monitoring in ML-based systems
• Define and execute test plans that include functional, regression, performance, and AI-specific testing strategies
• Partner with AI/ML engineers to create test datasets and simulate real-world user behavior and edge cases
• Implement CI/CD-integrated quality gates to ensure AI releases meet defined reliability and fairness metrics
• Mentor junior engineers and help establish best practices in intelligent automation and AI testing
• Advocate for ethical AI standards and contribute to cross-functional discussions about responsible AI

Preferred Qualifications

You’ve previously supported the Centers for Medicare and Medicaid Services (CMS)

Benefits

• Medical, dental, vision insurance (fully paid for employees)
• 15 days of paid leave
• 7 days of sick leave
• 2 days bereavement leave
• 11 paid Federal holidays
• Up to 40 hours for jury duty
• 401K with 4% employer contribution (and no vesting period)
• Up to 4 weeks of paid paternity and maternity leave
• Company provided laptop
• $5,000 per year for professional development
• $600 per year for technical supplies and equipment
• $2,000 referral bonus
• Life and disability insurance
• HSA and FSA
• Legal Shield and ID Shield Voluntary Benefits
• Opportunity to work in a collaborative, motivated team focused on modernizing government services with cutting-edge technology and innovative solutions