Skip to content

gh-104240: return code unit metadata from codegen #104300

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
May 9, 2023
Merged

gh-104240: return code unit metadata from codegen #104300

merged 2 commits into from
May 9, 2023

Conversation

iritkatriel
Copy link
Member

@iritkatriel iritkatriel commented May 8, 2023 •
edited by bedevere-bot
Loading

With this we can do:

from _testinternalcapi import compiler_codegen, optimize_cfg, assemble_code_object
import ast
import opcode
import textwrap
import types

a = ast.parse("(x+y)/2", mode='eval')
filename = "myfile.py"
eval_mode = 1  # 0=exec, 1=eval, 2=single

insts, metadata = compiler_codegen(a, filename, 0, eval_mode)

consts = [v[1] for v in sorted([(v, k) for k, v in metadata['consts'].items()])]
insts = optimize_cfg(insts, consts)

from test.test_compiler_assemble import IsolatedAssembleTests
IsolatedAssembleTests().complete_metadata(metadata)

co = assemble_code_object(filename, insts, metadata)

for x,y in [(3, 4), (-100, 200), (10, 18)]:
    f = types.FunctionType(co, {'x': x, 'y': y})
    print(f'avg of {x} and {y} is {f()}')

@iritkatriel iritkatriel requested a review from markshannon as a code owner May 8, 2023 19:06
@bedevere-bot bedevere-bot mentioned this pull request May 8, 2023
Closed
@iritkatriel iritkatriel requested a review from carljm May 8, 2023 19:06
@iritkatriel iritkatriel added tests Tests in the Lib/test dir interpreter-core (Objects, Python, Grammar, and Parser dirs) 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section labels May 8, 2023
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @iritkatriel for commit e3c83dd 🤖

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label May 8, 2023
carljm
carljm approved these changes May 8, 2023
View reviewed changes
Copy link
Member

@carljm carljm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix the MSVC warning, but otherwise LGTM!

Python/compile.c
if (res < 0) goto finally; \
} while (0);

SET_MATADATA_INT("argcount", umd->u_argcount);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like MSVC wants an explicit cast here.

@carljm
Copy link
Member

carljm commented May 8, 2023

Oh, I had the files tab open for a while and didn't notice the buildbots coming back unhappy. Guess that should be fixed, too -- if it's actually related to this PR? Seems surprising to me for this PR to cause those tests to fail.

@iritkatriel
Copy link
Member Author

Oh, I had the files tab open for a while and didn't notice the buildbots coming back unhappy. Guess that should be fixed, too -- if it's actually related to this PR? Seems surprising to me for this PR to cause those tests to fail.

These buildbots seem broken for a while actually.

@AlexWaygood
Copy link
Member

Oh, I had the files tab open for a while and didn't notice the buildbots coming back unhappy. Guess that should be fixed, too -- if it's actually related to this PR? Seems surprising to me for this PR to cause those tests to fail.

These buildbots seem broken for a while actually.

Looks like I'm the guilty culprit. I can repro the refleaks locally with 1b19bd1 but not with the previous commit.

@AlexWaygood
Copy link
Member

Oh, I had the files tab open for a while and didn't notice the buildbots coming back unhappy. Guess that should be fixed, too -- if it's actually related to this PR? Seems surprising to me for this PR to cause those tests to fail.

These buildbots seem broken for a while actually.

Looks like I'm the guilty culprit. I can repro the refleaks locally with 1b19bd1 but not with the previous commit.

Should be fixed by 9196da4.

@iritkatriel iritkatriel merged commit ca95edf into python:main May 9, 2023
carljm added a commit to carljm/cpython that referenced this pull request May 9, 2023
@carljm
Merge branch 'main' into fixsendfamily
e40071d 
* main:
  pythongh-97696 Add documentation for get_coro() behavior with eager tasks (python#104304)
  pythongh-97933: (PEP 709) inline list/dict/set comprehensions (python#101441)
  pythongh-99889: Fix directory traversal security flaw in uu.decode() (python#104096)
  pythongh-104184: fix building --with-pydebug --enable-pystats (python#104217)
  pythongh-104139: Add itms-services to uses_netloc urllib.parse. (python#104312)
  pythongh-104240: return code unit metadata from codegen (python#104300)
carljm added a commit to carljm/cpython that referenced this pull request May 9, 2023
@carljm
Merge branch 'main' into supermon
c48eee3 
* main: (156 commits)
  pythongh-97696 Add documentation for get_coro() behavior with eager tasks (python#104304)
  pythongh-97933: (PEP 709) inline list/dict/set comprehensions (python#101441)
  pythongh-99889: Fix directory traversal security flaw in uu.decode() (python#104096)
  pythongh-104184: fix building --with-pydebug --enable-pystats (python#104217)
  pythongh-104139: Add itms-services to uses_netloc urllib.parse. (python#104312)
  pythongh-104240: return code unit metadata from codegen (python#104300)
  pythongh-104276: Make `_struct.unpack_iterator` type use type flag instead of custom constructor (python#104277)
  pythongh-97696: Move around and update the whatsnew entry for asyncio eager task factory (python#104298)
  pythongh-103193: Fix refleaks in `test_inspect` and `test_typing` (python#104320)
  require-pr-label.yml: Add missing "permissions:" (python#104309)
  pythongh-90656: Add platform triplets for 64-bit LoongArch (LA64) (python#30939)
  pythongh-104180: Read SOCKS proxies from macOS System Configuration (python#104181)
  pythongh-97696 Remove unnecessary check for eager_start kwarg (python#104188)
  pythonGH-104308: socket.getnameinfo should release the GIL (python#104307)
  pythongh-104310: Add importlib.util.allowing_all_extensions() (pythongh-104311)
  pythongh-99113: A Per-Interpreter GIL! (pythongh-104210)
  pythonGH-104284: Fix documentation gettext build (python#104296)
  pythongh-89550: Buffer GzipFile.write to reduce execution time by ~15% (python#101251)
  pythongh-104223: Fix issues with inheriting from buffer classes (python#104227)
  pythongh-99108: fix typo in Modules/Setup (python#104293)
  ...
carljm added a commit to carljm/cpython that referenced this pull request May 9, 2023
@carljm
Merge branch 'main' into fixloadsuperspec
cf11908 
* main: (35 commits)
  pythongh-97696 Add documentation for get_coro() behavior with eager tasks (python#104304)
  pythongh-97933: (PEP 709) inline list/dict/set comprehensions (python#101441)
  pythongh-99889: Fix directory traversal security flaw in uu.decode() (python#104096)
  pythongh-104184: fix building --with-pydebug --enable-pystats (python#104217)
  pythongh-104139: Add itms-services to uses_netloc urllib.parse. (python#104312)
  pythongh-104240: return code unit metadata from codegen (python#104300)
  pythongh-104276: Make `_struct.unpack_iterator` type use type flag instead of custom constructor (python#104277)
  pythongh-97696: Move around and update the whatsnew entry for asyncio eager task factory (python#104298)
  pythongh-103193: Fix refleaks in `test_inspect` and `test_typing` (python#104320)
  require-pr-label.yml: Add missing "permissions:" (python#104309)
  pythongh-90656: Add platform triplets for 64-bit LoongArch (LA64) (python#30939)
  pythongh-104180: Read SOCKS proxies from macOS System Configuration (python#104181)
  pythongh-97696 Remove unnecessary check for eager_start kwarg (python#104188)
  pythonGH-104308: socket.getnameinfo should release the GIL (python#104307)
  pythongh-104310: Add importlib.util.allowing_all_extensions() (pythongh-104311)
  pythongh-99113: A Per-Interpreter GIL! (pythongh-104210)
  pythonGH-104284: Fix documentation gettext build (python#104296)
  pythongh-89550: Buffer GzipFile.write to reduce execution time by ~15% (python#101251)
  pythongh-104223: Fix issues with inheriting from buffer classes (python#104227)
  pythongh-99108: fix typo in Modules/Setup (python#104293)
  ...
@iritkatriel iritkatriel deleted the compilation_pipeline branch July 25, 2023 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@carljm carljm carljm approved these changes

@markshannon markshannon Awaiting requested review from markshannon markshannon is a code owner

Assignees
No one assigned
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) skip news tests Tests in the Lib/test dir
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@iritkatriel @bedevere-bot @carljm @AlexWaygood