QA & Test Automation#

And, as the person to first implement any type of QA at TOL, I thought it would be great to combine QA and manual creation, so I used Playwright as a foundation for that. I developed a Playwright framework that was robust, reused functions, and was much less flaky than previous attempts (see Playwright Testing Documentation). I then abstracted a lot of the functionality behind bigger functions that made the test suites easier to read, and developed a system of rules and skills that allowed me to go through a site once with Playwright’s codegen, feed it to the agent, and have it develop the test suite with very readable code. Another workflow was then designed to take those functions and create a manual from them.

This could be combined with some of Claude’s later abilities, which allowed it to accept a simple URL, gather the codegen, then develop both simultaneously. Of course, nothing is perfect, and everything still needed to be reviewed, but this saved a lot of time in e2e testing and documentation. The Playwright system itself was developed with exploratory testing in mind, where the user could easily change the tests on the fly, either through the agent or manually, to see if any bugs occurred.