From reusable helpers to an agent-assisted workflow

AI-SAI-Generated Summary H-RHuman Reviewed H-EHuman Edited

The helper library became context for agent-assisted test development. The central point: the helper library gave the agent an existing vocabulary for describing and implementing a workflow. Recordings supplied observations; stories organized those observations; tests called the established helpers.

Much of the Claude and agent work took place between early 2025 and December 2025, as described in the project timeline.

Give the agent a vocabulary it can reuse

Once common actions were functions, I could ask an agent to compose tests using the same building blocks I used myself. The email-template and address walkthroughs show the level of operation available to the agent, including real signatures and configuration examples.

Three kinds of reference material support that lookup:

Reference	What it gives the agent
Helper API catalog	A list of established operations and parameters to consult
Codegen helper mappings	Suggested translations from recorded interactions to helpers
Existing feature specs and the style guide	Examples of the expected test structure

The conversion dictionary explicitly connects repeated dropdown selections to the helper and shared configuration used in the final example.

    "multiple option selections for same dropdown": {
      "helper": "testDropdownOptions(page, label, statusOptions.Feature.Status, finalSelection)",
      "configKey": "statusOptions",
      "import": "components/dropdowns.js",
      "notes": "Use config array instead of hardcoded options"
    },

The division of responsibility follows the earlier helper examples: the spec names the action; the helper contains the browser interaction; configuration supplies expected values. This made the existing test architecture useful as context for agent-generated drafts. The dictionary is guidance that needs checking against the actual helper signatures.

The translation is recorded interaction → helper lookup → configuration lookup → test step.

Raw codegen as input

Recording a workflow was a practical way to capture an unfamiliar application. It supplied a starting point even when changing the frontend was outside the scope of the work.

This archived codegen conversion example already includes conversion instructions and examples around its recorded actions.

await page.getByRole('combobox', { name: 'Status All' }).locator('path').click();
await page.getByRole('option', { name: 'All' }).locator('span').click();
await page.getByRole('combobox', { name: 'Status All' }).locator('path').click();
await page.locator('.cdk-overlay-backdrop').click();

The recording contains: a particular control label, its current value, inner SVG or span elements, and an overlay dismissal. These details record how that session was operated. The useful behavior is “inspect the Status options and restore the filter.”

The same archived document gives a concrete early translation into existing helpers.

await page.getByRole('textbox', { name: 'Search' }).fill('A');
await page.getByRole('textbox', { name: 'Search' }).press('Enter');

await searchForUser(page, searchTerm, 'Search');
// Verify result
await safeClick(page, searchTerm, { exact: true });

This shows the conversion approach being documented. The resulting helper calls express the search and selection actions; clicking a result alone does not assert its correctness.

CIT Corporations across the artifacts

CIT Corporations supplies a continuing example. It appears in the raw account recording, the account summary, the refined account story, and the final feature specs.

await page.getByText('account_balanceAccountexpand_more').click();
await page.getByRole('link', { name: 'CIT Corporations' }).click();
await page.getByRole('columnheader', { name: 'ID' }).click();
await page.getByRole('columnheader', { name: 'CIT Corporation' }).click();
await page.getByRole('columnheader', { name: 'Sponsor Organization' }).click();
await page.getByRole('columnheader', { name: 'Website' }).click();
await page.getByRole('columnheader', { name: 'Email' }).click();
await page.getByRole('columnheader', { name: 'Phone' }).click();
await page.getByRole('columnheader', { name: 'Status' }).click();
await page.getByRole('columnheader', { name: 'Action' }).click();
await page.getByRole('combobox', { name: 'Status All' }).locator('svg').click();
await page.getByRole('option', { name: 'All' }).click();
await page.getByRole('combobox', { name: 'Status All' }).locator('svg').click();
await page.getByRole('option', { name: 'Active', exact: true }).click();
await page.getByRole('combobox', { name: 'Status Active' }).locator('svg').click();
await page.getByRole('option', { name: 'Inactive' }).click();
await page.locator('div').filter({ hasText: 'StatusInactive' }).nth(3).click();
await page.getByText('Suspended').click();
await page.getByRole('combobox', { name: 'Status Suspended' }).locator('svg').click();
await page.getByText('Terminated').click();
await page.getByRole('combobox', { name: 'Status Terminated' }).locator('svg').click();
await page.getByRole('option', { name: 'Pending' }).click();
await page.getByRole('combobox', { name: 'Status Pending' }).locator('svg').click();
await page.getByRole('option', { name: 'All' }).click();

The accessible name changes from Status All to Status Active, and a later action uses the positional fallback nth(3). The recording preserves these mechanics as observations. The shared helper layer gives recurring mechanics an implementation point as tests mature.

The recording also contains heading and column-header clicks, form corrections, and generated Material IDs. These are observations to interpret: a header click might be sorting, inspecting text, or incidental activity. Story extraction needs that distinction.

From recording to readable stories

The documented extraction procedure states the intended intermediate output directly.

Transform raw Playwright codegen recordings into sanitized user stories that describe what actions the user performs. This creates a human-readable summary of the test coverage before actual test implementation.

The account summary turns those actions into a readable workflow:

### CIT Corporations:

- User navigates to Account → CIT Corporations
- User verifies column headers: ID, CIT Corporation, Sponsor Organization, Website, Email, Phone, Status, Action
- User opens Filter section
- User selects a Status from dropdown (All, Active, Inactive, Suspended, Terminated, Pending)
- User selects a Sponsor Organization from dropdown (All or specific organization)
- User searches for CIT Corporation using Search CIT Corporation field
- User clicks View More Details button on CIT Corporation row

The first pass gave us a workflow we could read and review. It separated the actions we cared about from extra clicks and edits captured during recording.

Review the stories for correct feature names, meaningful expected outcomes, and actions that require data or permissions. A story describing verification still needs an actual assertion or a helper that fails when the expectation is not met.

A refined story connects tests and documentation

The refined Markdown stories connect workflow steps to configuration, screenshot labels, documentation sections, and named specs. They also record coverage observations; they are not executable test files.

## CIT Corporations List
<!-- page-type: table+detail | locked-test: cit-corporation.spec.ts -->
<!-- config: columnNames.CITCorporation, navigationText.Account -->
<!-- config-status: ok -->
<!-- doc-section: admin-manual/Account -->

### Stories:

- User navigates to Account → CIT Corporations
  [Screenshot: admin_account_citcorp_page_loaded]
- User verifies column headers: ID, CIT Corporation, Sponsor Organization, Website, Email, Phone, Status, Action
  [Screenshot: admin_account_citcorp_columns]
  [Config: columnNames.CITCorporation]
- User opens Filter section
  [Screenshot: admin_account_citcorp_filter]
- User selects a Status from dropdown (All, Active, Inactive, Suspended, Terminated, Pending)

The annotations connect the story to implementation and documentation:

Annotation	How I used it to guide the agent
page-type	Choose a table/detail or form test pattern
locked-test	Locate the associated existing spec
config	Find shared values instead of duplicating arrays
doc-section	Connect the workflow to its manual section
Screenshot	Plan the visual evidence for a step

The account story also connects the creation workflow to cit-corporation-create.spec.ts. Its list and creation sections are a useful bridge to the final example below, which includes a list-view group in that creation spec.

The relationship is recording → summary → refined story ↔ spec. The two-way arrow matters because the stories can be reviewed against existing tests as the workflow evolves.

The final test vocabulary

This source excerpt includes the imports, suite declaration, session variables, setup, cleanup, and three list/detail tests. Its test bodies use helper calls; the control interactions live inside the functions explained earlier.

/**
 * Account -> CIT Corporations
 *
 * Tests CIT Corporation list view and creation workflow.
 */

import { test, BrowserContext, Page } from '@playwright/test';
import { columnNames, navigationText, statusOptions } from '../../HelperFiles/cit-app-config.js';
import { fillCITCorpLegalInfo, createUserWithPermissions } from '../../HelperFiles/cit-app-helpers.js';
import { clickNextButton, clickFinishButton, clickDropdownFromSideBar } from '../../HelperFiles/components/buttons.js';
import { verifyHeadingPresence } from '../../HelperFiles/components/verification_checks.js';
import { addNewButton } from '../../HelperFiles/components/UserInfo/common-user.js';
import { testDropdownOptions } from '../../HelperFiles/components/dropdowns.js';
import { takeNamedScreenshot, takeDocScreenshot, buildCITCorpUser } from '../../HelperFiles/components/helperFunctions/utilities.js';
import {
    setupCITCorpPage,
    fillCITCorpCreationForm,
    waitForCITCorpForm,
    viewFirstCITCorpDetails,
    navigateBackToCITCorpList,
    assertCITCorpCreated
} from '../../HelperFiles/components/cit-corp-form.js';
import { getLatestCITCorp } from '../../HelperFiles/components/helperFunctions/entity-naming.js';

test.setTimeout(180_000);

test.describe.serial('Account -> CIT Corporations (List View)', () => {
    let context: BrowserContext;
    let page: Page;

    test.beforeAll(async ({ browser }) => {
        ({ context, page } = await setupCITCorpPage(browser));
    });

    test.afterAll(async () => {
        await page?.close();
        await context?.close();
    });

    test('Verify list page loads', async () => {
        await verifyHeadingPresence(page, 'CIT Corporations');
        await takeNamedScreenshot(page, 'cit_corp', 'list_page');
        await takeDocScreenshot(page, 'CITCorp', 'cit-corp-create-list-page');
    });

    test('Test Status dropdown filter', async () => {
        await testDropdownOptions(page, 'Status', statusOptions.CITCorporation.Status, 'All');
        await takeNamedScreenshot(page, 'cit_corp', 'status_filter');
        await takeDocScreenshot(page, 'CITCorp', 'cit-corp-create-status-filter');
    });

    test('View CIT Corporation details', async () => {
        await viewFirstCITCorpDetails(page);
        await verifyHeadingPresence(page, 'CIT Corporation');
        await takeNamedScreenshot(page, 'cit_corp', 'detail_view');
        await takeDocScreenshot(page, 'CITCorp', 'cit-corp-create-detail-view');
        await navigateBackToCITCorpList(page);
    });
});

In the status-filter test, testDropdownOptions receives the control label, the shared expected option list, and All as the final selection. The screenshot helpers then produce a consistently named test artifact and a documentation artifact.

The named-screenshot walkthrough explains both capture functions in full. Here, the documentation call maps CITCorp to Account/CITCorporations. By default it writes under the screenshot root’s docs/ folder; DOC_SCREENSHOT_DIR can direct output to a documentation folder.

I could use the stories as an outline and the test screenshots as illustrations for a rough user manual, then review the draft and add context, prerequisites, and explanations. User manual drafts covers that part of the workflow.

Continue with the complete Proof of Address walkthrough to explain imports, the login/navigation setup, shared serial lifecycle, verification helpers, and cleanup.

Add an agent repair loop

The next step was to use failures as input. We ran a spec, collected its errors and story context, asked the appropriate agent to investigate, and reran the spec after the change. The selector-diagnostics chapter shows the diagnostic functions, hook integration, and example output that could support that investigation.

The per-file orchestrator runs this loop. It reads a maximum iteration count, runs a spec, parses failures, routes them by their dominant type, invokes the agent, and reruns the spec.

  for ((attempt=1; attempt<=MAX; attempt++)); do
    FILE_ATTEMPTS=$attempt

    # Run just this file
    node src/scripts/test/test-runner.js "$SPEC" --output="$SCRATCH/single-run.json" || true

    # Parse failures
    python3 src/scripts/test/parse_failures.py "$SCRATCH/single-run.json" "$SCRATCH/single-fail.json" 2>/dev/null || true

    FAIL_COUNT=$(count_failures "$SCRATCH/single-fail.json")

    # ── Invoke Claude with ALL failures from this file ────────────────
    DOM_TYPE=$(dominant_type "$SCRATCH/single-fail.json")
    PROMPT=$(route_prompt "$DOM_TYPE")
    [[ -z "$PROMPT" ]] && PROMPT=".claude/prompts/qa_reviewer.md"

    echo "  🤖 Invoking Claude ($DOM_TYPE → $PROMPT)"

    FAILURE_CONTEXT=$(build_failure_context "$SCRATCH/single-fail.json" "$SPEC")

The routing table assigns each dominant failure type to a focused prompt:

Failure type	Assigned prompt
Selector	locator_fixer.md
Assertion	logic_specialist.md
Timeout	perf_optimizer.md
Other	qa_reviewer.md

The configuration sets a limit of three iterations. The orchestrator includes error messages, URLs, artifact paths, and resolved story content in its failure context, and searches the available story material so those artifacts remain useful during repair.

After the agent invocation, the orchestrator reruns the test. Files that exhaust the retry limit are recorded and sent to bug-report generation, creating a bounded investigation loop with a place for unresolved failures.

The validation addendum checks that a repair preserves the desired spec style.

- No page.waitForTimeout() in test body
- No for loops in test body
- No try/catch in test body
- No .catch(() => {}) or .catch(() => false)
- No raw page.getByText().click() — use helpers
- No hardcoded arrays — use config
- No page.goto() in individual tests

The locator workflow can also use MCP-assisted investigation: inspect the failing page, compare the accessibility snapshot with expected selectors, and rerun the test. I used Playwright MCP selectively and generally preferred diagnostic output from logAllBoxes and its enhanced counterpart as a first step. The selector-diagnostics walkthrough explains their calls, output, and limitations.

What I wanted to improve

I wanted to reduce repeated selector and timing failures by repairing shared helpers. Measuring the change in failure rate would require comparable results from repeated runs.

The helper calls made tests easier to read, but waits and fallbacks inside those helpers still needed review. I also needed to check that a missing element or failed action would actually fail the test.