Manual QA Engineer

Description

This is a full-time Manual QA Engineer role at Alivo, a fast-growing AI startup based in Portland, ME that builds AI agents for roofing contractors and home services businesses. The role owns manual testing, release validation, and evaluation of AI-driven phone and chat conversation quality, and is a good fit for someone with 2-5+ years of QA, software testing, or product operations experience. To apply, candidates email their resume directly to Alivo rather than through a standard application form.

About Alivo

Alivo helps businesses connect with customers using AI-powered phone and chat agents, aiming to make every customer interaction fast, intelligent, and personalized.

What You’ll Do

• Manually test new features and workflows before release, including exploratory testing to catch bugs, usability issues, and edge cases.
• Run structured test plans and release checklists, and verify bug fixes through regression testing.
• Review release candidates with Engineering and file clear, reproducible bug reports.
• Evaluate AI-powered phone and chat conversations for accuracy, naturalness, and effectiveness, including objection handling and conversational flow.
• Identify AI failure modes, hallucinations, and off-script behavior, and help define quality standards for AI-powered interactions.
• Build and maintain evaluation datasets, scoring rubrics, and quality frameworks to track AI performance over time.
• Work cross-functionally with Engineering, Product, Customer Success, and Leadership to prioritize quality improvements.

What You’ll Need

• 2-5+ years of experience in QA, software testing, product operations, or a similar role.
• Strong attention to detail and a methodical, structured approach to testing.
• Excellent written and verbal communication skills.
• Ability to spot edge cases, failure modes, and customer experience issues.
• Comfort working in a fast-paced, evolving startup environment.

Nice to Have

• Experience evaluating AI, LLM, or conversational AI systems.
• Familiarity with bug tracking tools such as Linear, Jira, or TestRail, or API testing tools like Postman.
• Basic SQL, scripting, or technical troubleshooting skills.
• Background in customer-facing, sales, or support roles, or experience at a SaaS or high-growth startup.