Automation Test Engineer

Quick Facts

• Work Policy: Remote
• Job Type: Full Time
• Salary Range: $110,000 – $130,000
• Experience Level: 5+ years
• Core Tech Stack: Selenium, Cypress, Playwright, Python, MLflow, TensorBoard, CI/CD
• Clearance: Public Trust required

About Skyward IT Solutions, LLC

Skyward IT Solutions is a people-centered civic technology business driven by a desire to serve others and advance human capability through information technology. Operating with a core set of uncompromising values—Integrity, Compassion, Inquisitiveness, Unconventionality, and Authenticity—Skyward builds collaborative and creative teams dedicated to modernizing essential government services. They emphasize a culture where work fits organically inside of life, empowering their teams to fail openly, learn continuously, and engineer future-looking solutions.

About the Role

Skyward is seeking an analytical and forward-thinking Automation Test Engineer who is passionate about responsible AI and machine learning validation. If you naturally question the bias in training data and thrive on building scalable automation for complex neural networks, this role offers the chance to make a profound impact. You will join a mission-driven team developing an AI/ML platform that leverages generative AI to reduce user burden across the Centers for Medicare and Medicaid Services (CMS).

What You’ll Do

You will play a pivotal role in ensuring that bleeding-edge AI implementations are highly reliable, ethically sound, and fully compliant with government standards.

• Design, engineer, and maintain scalable automated testing frameworks encompassing web applications, APIs, and AI-powered features.
• Construct automated pipelines specifically for model validation, data drift monitoring, and bias detection within ML-based systems.
• Establish and execute comprehensive test plans that cover functional requirements, regression, performance, and specialized AI testing strategies.
• Collaborate directly with AI/ML engineers to generate test datasets and accurately simulate edge cases and real-world user interactions.
• Integrate rigorous quality gates into CI/CD pipelines to guarantee that all AI releases meet strict fairness and reliability metrics.
• Serve as an advocate for ethical AI standards, actively contributing to cross-functional conversations regarding responsible AI deployment.
• Provide mentorship and guidance to junior engineers, helping to solidify best practices in intelligent automation across the department.

What You’ll Need

• At least 5 years of professional experience in software QA and automation engineering.
• A minimum of 1 year of dedicated experience testing AI/ML products.
• High-level proficiency in scripting languages, preferably Python.
• Strong hands-on experience with at least one major automation framework such as Selenium, Cypress, or Playwright.
• Practical expertise in evaluating AI models, including tracking validation metrics, dataset versioning, and ensuring model explainability.
• Familiarity with AI observability and tracking tools like MLflow or TensorBoard.
• A solid understanding of model bias, fairness auditing, and the unique compliance considerations required in government or regulated sectors.
• Proven experience navigating Agile DevSecOps environments using modern CI/CD tooling (e.g., Jenkins, GitHub Actions, GitLab).
• Excellent cross-functional communication and collaboration skills to work seamlessly with engineering, data science, and product teams.
• The ability to obtain and maintain a Public Trust security clearance (U.S. citizenship or lawful residency typically required).

Nice to Have

• Previous experience supporting or working alongside the Centers for Medicare and Medicaid Services (CMS).

Benefits

• Comprehensive medical, dental, and vision insurance coverage.
• 15 days of paid leave, 7 days of sick leave, and 11 paid Federal holidays.
• 2 days of bereavement leave and up to 40 hours of paid jury duty.
• Up to 4 weeks of paid maternity and paternity leave.
• 401K plan featuring a 4% employer contribution with zero vesting period.
• $5,000 annual allowance for professional development and a $600 annual stipend for technical supplies.
• $2,000 employee referral bonus.
• Company-provided laptop, Life and Disability insurance, HSA/FSA options, plus Legal Shield and ID Shield voluntary benefits.