Skip to content

Fix JUnit XML test case name inconsistency when using scenario retries #5082

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Aug 20, 2025

Conversation

Copilot
Copy link
Contributor

@Copilot Copilot AI commented Aug 20, 2025

When using CodeceptJS with JUnit XML reporters (like mocha-junit-reporter), test case names were inconsistent depending on whether scenario retries were used:

  • Without retries or retries that pass on first attempt: "JUnit reporting: Test 1" (with colon)
  • With retries that fail and retry: "JUnit reporting Test 1" (missing colon)

This inconsistency made automated processing of JUnit XML files unreliable.

Root Cause

The issue occurred because during test retries, CodeceptJS clones tests using the cloneTest() function, which calls deserializeTest(). The deserializeTest() function was not restoring the custom fullTitle() method that CodeceptJS sets up to format test names as "${suite.title}: ${test.title}". This caused retried tests to fall back to Mocha's default fullTitle() implementation, which doesn't include the colon separator.

Solution

Updated the deserializeTest() function in lib/mocha/test.js to restore the custom fullTitle() method that maintains the consistent format. The fix ensures that cloned tests preserve the same title formatting as their originals.

Testing

  • Added comprehensive unit tests to verify the fix works across different scenarios
  • Verified all existing tests still pass
  • Tested edge cases including empty suite titles and special characters
  • Confirmed retry functionality remains unaffected

This change ensures JUnit XML output has consistent test case names regardless of retry behavior, making test result processing more reliable.

Fixes #4983.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • google.com
    • Triggering command: node /home/REDACTED/work/CodeceptJS/CodeceptJS/bin/codecept.js run --config /home/REDACTED/work/CodeceptJS/CodeceptJS/test/data/sandbox/codecept.bdd.js --grep @fail --steps (dns block)
  • googlechromelabs.github.io
  • https://storage.googleapis.com/chrome-for-testing-public/138.0.7204.168/linux64/chrome-linux64.zip
    • Triggering command: node install.mjs (http block)

If you need me to access, download, or install something from one of these locations, you can either:


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 2 commits August 20, 2025 14:04
Co-authored-by: kobenguyent <7845001+kobenguyent@users.noreply.github.com>
Co-authored-by: kobenguyent <7845001+kobenguyent@users.noreply.github.com>
@Copilot Copilot AI changed the title [WIP] Test case name in JUnit XML not consistent when using scenario retries Fix JUnit XML test case name inconsistency when using scenario retries Aug 20, 2025
@Copilot Copilot AI requested a review from kobenguyent August 20, 2025 14:08
Copilot finished work on behalf of kobenguyent August 20, 2025 14:08
@kobenguyent kobenguyent marked this pull request as ready for review August 20, 2025 15:31
@kobenguyent kobenguyent merged commit 58f83a1 into 3.x Aug 20, 2025
21 of 24 checks passed
@kobenguyent kobenguyent deleted the copilot/fix-4983 branch August 20, 2025 15:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Test case name in JUnit XML not consistent when using scenario retries
2 participants