Case Study

ERPNext QA Automation & Engineering

Independent QA automation portfolio project demonstrating Playwright-based UI testing across selected ERPNext workflows using Page Object Model architecture and GitHub Actions CI.

ERPNext QA Automation: project coverCover for ERPNext QA Automation: 6 test definitions (5 UI + 1 auth setup). Workflow covered by the suite: reuse the saved login, open a new document, select an item through the autocomplete, set quantity and rate, save and verify. Built with Playwright, Page Object Model, GitHub Actions.UI TEST AUTOMATION · ERPERPNextQA AutomationPlaywright · Page Object Model · GitHub Actions6test definitions5 UI + 1 auth setup1Authsaved login2Formnew document3Itemautocomplete4Qtyand rate5Savethen verifyMaterial Request · Quotation · Sales Ordersayedasif.comERPNext QA Automation: project coverCover for ERPNext QA Automation: 6 test definitions (5 UI + 1 auth setup). Workflow covered by the suite: reuse the saved login, open a new document, select an item through the autocomplete, set quantity and rate, save and verify. Built with Playwright, Page Object Model, GitHub Actions.ERPNextQA Automation6test definitions5 UI + 1 auth setupPlaywright · Page Object Model · GitHub ActionsAuthFormItemQtySave

01 Overview

Project Overview

This is an independent QA automation portfolio project — not client work. It demonstrates practical QA automation engineering against a real ERP application.

The project focuses on validating critical business workflows through automated UI testing, using industry-standard practices: Page Object Model architecture, repeatable test scenarios, and CI/CD integration.

02 System under test

Application Tested

ERPNext — an open-source Enterprise Resource Planning (ERP) system. The application provides business management workflows including materials management, quotations, sales orders, and financial processes.

Testing ERPNext provides realistic scenarios: complex workflows, data dependencies, business rule validation, and multi-step user interactions.

03 Scope

Test Scope

The automation suite covers three critical workflow areas:

Workflow Scenarios Coverage
Material Request Creation, item selection, quantity, amount calculation, save Business rule validation
Quotation Creation, customer, items, rates, validation, save Required field validation
Sales Order Creation, customer, items, rates, save Order workflow validation

04 Suite facts

Automation Suite Facts

Current Scope:

  • 6 Playwright tests: 5 UI workflow tests plus 1 authentication setup test
  • 3 application workflows tested
  • Playwright UI automation framework
  • Page Object Model architecture
  • GitHub Actions workflow (runs on pushes and pull requests to main)
  • Last successful GitHub Actions run: 4 September 2026 (commit 8990014), in which all 6 tests, including the setup test, passed.

Current status

  • Latest run: #7, 22 October 2026 (opens in a new tab) (commit bd3846a, which only added a manual workflow trigger). It failed. By my reading of the job log, the authentication setup test failed at tests/auth/auth.setup.js line 54 because ${baseURL}/app redirected to cloud.frappe.io/dashboard/login, so 1 test failed and the other 5 did not run. Job logs need a GitHub sign-in.
  • Environment status (checked 22 October 2026): the hosted ERPNext instance the suite ran against is no longer available, so the suite cannot currently be run. Run #7 is the evidence; the only commit after 8990014 changed the workflow trigger, not the tests.

Important: The recorded green result refers to this automation suite only, as of 4 September 2026. It does not represent full ERPNext coverage, production validation, or zero-defect status. A flaky-test count is not published here: CI is configured with retries, so a passing run does not by itself show that no test needed one.

05 Architecture

Automation Architecture

The project uses industry-standard QA automation practices to ensure maintainability, scalability, and reliability.

Technology Stack

Playwright (UI Automation) Page Object Model JavaScript / Node.js GitHub Actions (CI/CD) Environment Configuration

Pattern: Page Object Model

Each application page/workflow is represented as a reusable Page Object class. This separates test logic from UI element selectors, making tests:

  • Maintainable: UI changes require updates in one place (the Page Object)
  • Readable: Tests describe business actions, not technical element interactions
  • Scalable: New tests reuse existing Page Objects without duplication

CI/CD Integration

The GitHub Actions workflow is configured to run the tests on every push and pull request to the main branch. Its last successful run was on 4 September 2026; its latest run, on 22 October 2026, failed. The hosted ERPNext environment the tests ran against is no longer available, so the workflow cannot currently pass until an environment is restored. The design provides:

  • Consistency: The workflow ran tests in a fixed CI environment while the hosted test instance existed
  • Visibility: Run history is listed in the repository’s Actions tab
  • Early Detection: Failures would surface on each push while an environment is available

06 Investigation

Automation Reliability Investigation

While building the suite, selecting an item from the Frappe autocomplete caused timeouts and unreliable results in the Material Request, Quotation and Sales Order workflows. The repository's root-cause analysis documents the investigation; this section summarizes it.

The Problem: Timeouts and Masked Failures

The item-selection step waited for the autocomplete options to become visible and timed out. Playwright reported the option elements as present in the DOM but hidden, even though the correct text had been entered. A silent fallback also pressed Enter whenever that wait failed, so some runs carried on without proving that the intended item had been selected.

Investigation Process

  • Step 1 — Read the failure: The timeout output showed the option element resolving repeatedly as hidden
  • Step 2 — Inspect the page: The options were in the DOM with aria-selected set, but hidden by CSS
  • Step 3 — Compare interaction methods: fill() sets the value without firing key events; type() and key presses do
  • Step 4 — Check the framework behavior: Frappe's autocomplete (Awesomplete) reveals its dropdown in response to keyboard input events
  • Step 5 — Review the test code: A try/catch fallback pressed Enter when the visibility wait failed, hiding the failure
  • Step 6 — Root cause: The test's interaction did not match how the autocomplete works, and error suppression hid the consequences

Solution: Targeted Fix in the Item-Selection Step

The fix changed how the test interacts with the autocomplete when selecting an item:

  • Type the search text key by key with a 50 ms delay (the repository calls locator.type()), so the autocomplete receives real key events
  • Wait for the listbox with a Locator (locator.waitFor({ state: 'attached' })) rather than for it to be visible, because the options can be hidden by CSS
  • Select the option with ArrowDown and Enter, as a user would
  • Remove the silent fallback, so a failed selection surfaces as a failure

What I would change next: locator.type() is deprecated in current Playwright in favour of locator.pressSequentially(), and the step still ends with a fixed 500 ms pause (waitForTimeout) instead of a web-first assertion such as expect(cell).toHaveText(...) on the selected item. I have not changed the repository code: the hosted instance is offline, so I cannot re-run the suite to confirm a change is safe.

Outcome and Limits

After the change, the suite passed in GitHub Actions (five successful runs on 4 September 2026), and the repository's execution report records repeated passing local and CI runs.

Limits: This fix covers the item-selection step. Customer-field selection in the Quotation and Sales Order page objects uses a different pattern and was not part of this investigation. GitHub Actions logs are not viewable without a GitHub sign-in, and CI is configured with retries, so a passing run does not by itself show that no test needed one. The hosted ERPNext test instance the suite runs against is no longer available (checked 22 October 2026; the latest run, #7, failed for that reason), so the suite cannot be re-run today and the last green result is historical.

Lesson:

A passing test only counts if it exercises the intended path. Interact with the application the way users do, wait for the state the application actually exposes, and never let error handling turn a failure into a pass.

Key Takeaway:

Automation reliability is not just about writing test code. It requires understanding the application's behavior, recognizing when the standard approach fails, investigating the root cause, and adapting the test strategy accordingly.

07 Evidence

Evidence & Verification

Source code, tests and the CI workflow are public on GitHub. CI run logs require a GitHub sign-in, and the report artifacts (30-day retention) have expired.

What You Can Inspect

  • Source Code

    Complete Playwright automation framework with Page Objects, fixtures, and test specifications.

  • CI/CD Execution

    GitHub Actions run history (last successful run 4 Sep 2026; latest run, 22 Oct 2026, failed). Job logs require a GitHub sign-in.

  • Architecture Documentation

    README with test structure, workflow descriptions, and setup instructions.

View the Project

Explore the complete source code, test suite, and CI/CD configuration on GitHub.