AI Quality Engineer

About the role

Populosoft EOOD is a Bulgarian software and quality assurance company. We build and run our own products, including mathsolver.eu and textrunway.com, and we test software for client companies.

As an AI Quality Engineer, you'll test web applications, APIs and AI/LLM-powered features for our clients. You'll report to Populosoft's QA Lead, Rohit Singh, founder of Populosoft and a former Senior Member of Technical Staff at VMware/Broadcom.

This is a freelance (contractor) role at $10/hour. Work is project-based: hours depend on client projects, and there is no guaranteed minimum. Start date: depends on the first suitable project.

Location and hours

• 100% remote. Open to applicants worldwide; we especially welcome applicants based in Bulgaria and the EU.
• Project-based hours with no guaranteed minimum. The schedule is agreed per project.
• You must be able to work as an independent contractor (freelancer) where you live.

What you'll do

• Automate end-to-end tests for web applications with Playwright.
• Test REST APIs, both manually and in code: functionality, error handling and data validation.
• Test AI and LLM-powered features: check answers for accuracy, consistency and safety, and find the edge cases where they fail.
• Write tests faster with AI assistance, and review every generated test critically before it ships.
• Write clear, complete bug reports that developers can act on without back-and-forth.
• Help us adopt new AI-driven ways of testing as the tools evolve.

What we're looking for

• Genuine enthusiasm for AI: you enjoy working with powerful technologies like AI, and you're eager to keep learning new AI skills as they evolve.
• AI literacy: you already use AI tools in your daily work, and you're critical of their output.
• Ambition to grow: we want people who can expand into bigger roles over time.
• Hands-on Playwright experience (TypeScript, JavaScript or Python).
• API testing experience, with Postman or in code.
• Strong written English. All work and reporting is in English.
• Reliability in a remote, mostly asynchronous setup, and the curiosity to try to break things.
• No minimum years of experience. Show us what you can do.

Nice to have

• Experience testing AI/LLM features, or with LLM evaluation approaches (prompt test sets, automated output checks).
• Running test suites in CI (for example GitHub Actions).
• Previous freelance or client-facing QA work.
• ISTQB Foundation Level.

What we offer

• $10/hour, fully remote, from anywhere in the world.
• Project-based work you can combine with your other commitments.
• Real client projects, including testing AI-powered features.
• Direct mentorship from a senior QA lead. Your work is reviewed, so you improve fast.
• Room to learn and apply new AI skills. We build AI-powered products of our own.
• Room to grow into bigger roles as the team grows.

How we hire

1. Application. Fill in the form below. We read every application.
2. Screening call. A short video call about you, your skills and your availability.
3. Practical exercise. A small, realistic testing task, such as a few Playwright or API tests, or reviewing an AI feature. It's an exercise, not unpaid work on a real product.
4. Final call. A short conversation with the QA Lead.

Strong candidates who aren't matched to a project right away can choose to join our talent pool, so we can contact them about future QA work. You can opt in on the form below.

Equal opportunity

We welcome applicants of every background. We hire on skills and potential. If you need any adjustments during the hiring process, tell us at [email protected].