Skip to content

bpo-39337: Add a test case for normalizing of codec names #19069

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Oct 8, 2020
Merged

bpo-39337: Add a test case for normalizing of codec names #19069

Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
e9ecac0
add a test case to against nomralizeing() of codecs
shihai1991 Mar 19, 2020
cf46290
remove redundant operation
shihai1991 Mar 19, 2020
59911e1
not clear all codec_search_paths
shihai1991 Mar 20, 2020
aa3c4d0
remove codecs_unregister()
shihai1991 Mar 21, 2020
aa4bf51
remove useless head file
shihai1991 Mar 21, 2020
8ce5956
remove useless import
shihai1991 Mar 21, 2020
41a1672
retrigger checks
shihai1991 Apr 2, 2020
00b300c
Merge remote-tracking branch 'origin/master' into bpo_1635741_1
shihai1991 Sep 8, 2020
f48e5d9
add codecs_unregister() in testinternalcapi
shihai1991 Sep 10, 2020
3795420
update code
shihai1991 Sep 10, 2020
e3b9365
Merge remote-tracking branch 'origin/master' into bpo_1635741_1
shihai1991 Sep 29, 2020
a2887f3
Using codecs.unregister() in test case
shihai1991 Sep 29, 2020
632a448
apply Tal's comment
shihai1991 Sep 29, 2020
bb6e78c
apply victor's comment
shihai1991 Oct 3, 2020
72e2435
apply tal's comment
shihai1991 Oct 3, 2020
26fcbe4
apply victor's comment
shihai1991 Oct 5, 2020
37b80e0
apply victor's comment
shihai1991 Oct 8, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions Lib/test/test_codecs.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3415,5 +3415,30 @@ def test_rot13_func(self):
'To be, or not to be, that is the question')


class CodecNameNormalizationTest(unittest.TestCase):
"""Test codec name normalization"""
def test_normalized_encoding(self):
FOUND = (1, 2, 3, 4)
NOT_FOUND = (None, None, None, None)
def search_function(encoding):
if encoding == "aaa_8":
return FOUND
else:
return NOT_FOUND

codecs.register(search_function)
self.addCleanup(codecs.unregister, search_function)
self.assertEqual(FOUND, codecs.lookup('aaa_8'))
self.assertEqual(FOUND, codecs.lookup('AAA-8'))
self.assertEqual(FOUND, codecs.lookup('AAA---8'))
self.assertEqual(FOUND, codecs.lookup('AAA 8'))
self.assertEqual(FOUND, codecs.lookup('aaa\xe9\u20ac-8'))
self.assertEqual(NOT_FOUND, codecs.lookup('AAA.8'))
self.assertEqual(NOT_FOUND, codecs.lookup('AAA...8'))
self.assertEqual(NOT_FOUND, codecs.lookup('BBB-8'))
self.assertEqual(NOT_FOUND, codecs.lookup('BBB.8'))
self.assertEqual(NOT_FOUND, codecs.lookup('a\xe9\u20ac-8'))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The whole point of https://bugs.python.org/issue39337 is to test that non-ASCII characters are ignored. Your test doesn't check that.

You should also test that codecs.lookup('aaa\xe9\u20ac-8') returns FOUND, no?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, tal have mentioned this detail too.
My doubt: positive and negative testcases are both needed. I create another PR #22219. But there have no clear conclusion about the right behavior of normalize_encoding().



if __name__ == "__main__":
unittest.main()