Skip to content

gh-106368: Argument clinic tests: improve failure message when tests in ClinicExternalTests fail #107364

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Jul 27, 2023
Merged

gh-106368: Argument clinic tests: improve failure message when tests in ClinicExternalTests fail #107364

merged 2 commits into from
Jul 27, 2023

Conversation

AlexWaygood
Copy link
Member

@AlexWaygood AlexWaygood commented Jul 27, 2023 •
edited by bedevere-bot
Loading

Currently, if I apply this diff to Tools/clinic/clinic.py...

--- a/Tools/clinic/clinic.py
+++ b/Tools/clinic/clinic.py
@@ -4702,6 +4702,7 @@ def state_modulename_name(self, line: str | None) -> None:
                 fields = [x.strip() for x in existing.split('.')]
                 function_name = fields.pop()
                 module, cls = self.clinic._module_and_class(fields)
+                1/0

                 for existing_function in (cls or module).functions:
                     if existing_function.name == function_name:

...then the argument clinic tests fail for me locally (good!), but the failure message is terrible (bad!):

======================================================================
FAIL: test_external (test.test_clinic.ClinicExternalTest.test_external)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\alexw\coding\cpython\Lib\test\test_clinic.py", line 1410, in test_external
    out = self.expect_success("-f", "-o", TESTFN, source)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\test\test_clinic.py", line 1397, in expect_success
    return self._do_test(*args)
           ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\test\test_clinic.py", line 1389, in _do_test
    self.fail("".join(proc.stderr))
AssertionError

----------------------------------------------------------------------

This change makes the error message much more helpful!

======================================================================
FAIL: test_external (test.test_clinic.ClinicExternalTest.test_external)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\alexw\coding\cpython\Lib\test\test_clinic.py", line 1411, in test_external
    out = self.expect_success("-f", "-o", TESTFN, source)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\test\test_clinic.py", line 1398, in expect_success
    return self._do_test(*args)
           ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\test\test_clinic.py", line 1390, in _do_test
    self.fail("".join(itertools.chain(proc.stdout, proc.stderr)))
AssertionError: Error in file "C:\Users\alexw\coding\cpython\Lib\test\clinic.test.c" on line 117:
Exception raised during parsing:
Traceback (most recent call last):
  File "C:\Users\alexw\coding\cpython\Tools\clinic\clinic.py", line 2244, in parse
    parser.parse(block)
  File "C:\Users\alexw\coding\cpython\Tools\clinic\clinic.py", line 4608, in parse
    self.state(line)
  File "C:\Users\alexw\coding\cpython\Tools\clinic\clinic.py", line 4666, in state_dsl_start
    self.next(self.state_modulename_name, line)
  File "C:\Users\alexw\coding\cpython\Tools\clinic\clinic.py", line 4648, in next
    self.state(line)
  File "C:\Users\alexw\coding\cpython\Tools\clinic\clinic.py", line 4705, in state_modulename_name
    1/0
    ~^~
ZeroDivisionError: division by zero


----------------------------------------------------------------------

(@erlend-aasland do you see the same thing locally? Am I seeing something different because I'm using a Windows box?)

@bedevere-bot bedevere-bot added awaiting core review tests Tests in the Lib/test dir labels Jul 27, 2023
@AlexWaygood AlexWaygood changed the title Argument clinic tests: improve failure message when tests in ClinicExternalTests fail gh-106368: Argument clinic tests: improve failure message when tests in ClinicExternalTests fail Jul 27, 2023
@bedevere-bot bedevere-bot mentioned this pull request Jul 26, 2023
Closed
erlend-aasland
erlend-aasland reviewed Jul 27, 2023
View reviewed changes
AlexWaygood
AlexWaygood commented Jul 27, 2023
View reviewed changes
erlend-aasland
erlend-aasland approved these changes Jul 27, 2023
View reviewed changes
Copy link
Contributor

@erlend-aasland erlend-aasland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! (BTW, I love itertools!)

@AlexWaygood AlexWaygood enabled auto-merge (squash) July 27, 2023 23:20
@AlexWaygood AlexWaygood merged commit 76c26ea into python:main Jul 27, 2023
@miss-islington
Copy link
Contributor

Thanks @AlexWaygood for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖

@bedevere-bot
Copy link

GH-107365 is a backport of this pull request to the 3.12 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.12 only security fixes label Jul 27, 2023
@bedevere-bot
Copy link

GH-107366 is a backport of this pull request to the 3.11 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.11 only security fixes label Jul 27, 2023
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jul 27, 2023
@AlexWaygood @miss-islington
pythongh-106368: Argument clinic tests: improve failure message when …
08fe261 
…tests in `ClinicExternalTests` fail (pythonGH-107364)

(cherry picked from commit 76c26ea)

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jul 27, 2023
@AlexWaygood @miss-islington
pythongh-106368: Argument clinic tests: improve failure message when …
39e202f 
…tests in `ClinicExternalTests` fail (pythonGH-107364)

(cherry picked from commit 76c26ea)

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
@AlexWaygood AlexWaygood deleted the clinic-test-message branch July 27, 2023 23:58
AlexWaygood added a commit that referenced this pull request Jul 28, 2023
@miss-islington @AlexWaygood
[3.11] gh-106368: Argument clinic tests: improve failure message when…
51b302e 
… tests in `ClinicExternalTests` fail (GH-107364) (#107366)

gh-106368: Argument clinic tests: improve failure message when tests in `ClinicExternalTests` fail (GH-107364)
(cherry picked from commit 76c26ea)

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
AlexWaygood added a commit that referenced this pull request Jul 28, 2023
@miss-islington @AlexWaygood
[3.12] gh-106368: Argument clinic tests: improve failure message when…
2827ad2 
… tests in `ClinicExternalTests` fail (GH-107364) (#107365)

gh-106368: Argument clinic tests: improve failure message when tests in `ClinicExternalTests` fail (GH-107364)
(cherry picked from commit 76c26ea)

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
@AlexWaygood AlexWaygood mentioned this pull request Aug 3, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@erlend-aasland erlend-aasland erlend-aasland approved these changes

Assignees
No one assigned
Labels
skip news tests Tests in the Lib/test dir
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@AlexWaygood @miss-islington @bedevere-bot @erlend-aasland