Skip to content

Port metadata from the linear node onto the reference custom op for int4 #2860

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
Aug 29, 2025
Merged

Port metadata from the linear node onto the reference custom op for int4 #2860

merged 6 commits into from
Aug 29, 2025

Conversation

kimishpatel
Copy link
Contributor

@kimishpatel kimishpatel commented Aug 22, 2025 •
edited
Loading

Stack from ghstack (oldest at bottom):

Summary:
Allow for numerical debugger in ExecuTorch use the from_node info for
correlation

Test Plan:
CI

Reviewers:

Subscribers:

Tasks:

Tags:

@kimishpatel
Port metadata from the linear node onto the reference custom op for int4
6f49929 
Summary:
Allow for numerical debugger in ExecuTorch use the from_node info for
correlation

Test Plan:
CI

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Aug 22, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/2860

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit aa44ccb with merge base 8669213 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

kimishpatel added a commit that referenced this pull request Aug 22, 2025
@kimishpatel
Port metadata from the linear node onto the reference custom op for int4
0467181 
Summary:
Allow for numerical debugger in ExecuTorch use the from_node info for
correlation

Test Plan:
CI

Reviewers:

Subscribers:

Tasks:

Tags:

ghstack-source-id: fd3775e
Pull Request resolved: #2860
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 22, 2025
@kimishpatel kimishpatel requested a review from jerryzh168 August 22, 2025 23:12
@kimishpatel kimishpatel added the pt2e_quant pt2 export quantization (prepare_pt2e, convert_pt2e) label Aug 22, 2025
@kimishpatel
Update on "Port metadata from the linear node onto the reference cust…
27def63 
…om op for int4"

Summary:
Allow for numerical debugger in ExecuTorch use the from_node info for
correlation

Test Plan:
CI

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
kimishpatel added a commit that referenced this pull request Aug 22, 2025
@kimishpatel
Port metadata from the linear node onto the reference custom op for int4
9d91022 
Summary:
Allow for numerical debugger in ExecuTorch use the from_node info for
correlation

Test Plan:
CI

Reviewers:

Subscribers:

Tasks:

Tags:

ghstack-source-id: f190917
Pull Request resolved: #2860
jerryzh168
jerryzh168 reviewed Aug 23, 2025
View reviewed changes
torchao/quantization/pt2e/reference_representation_rewrite.py
model,
pattern,
replacement,
match_filters=rewrite_info.filter_fn,
ignore_literals=rewrite_info.ignore_literals,
) # type: ignore[arg-type]
if rewrite_info.port_metadata_fn:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this not added to https://github.com/pytorch/ao/blob/main/torchao/quantization/pt2e/quantizer/port_metadata_pass.py?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reference rewrite is an option pass. Or at least thats how I was using it in my debug. In reference rewrite the rules for porting metadata is not as generic, so you have to let each pattern dictate what they want to do

jerryzh168
jerryzh168 approved these changes Aug 26, 2025
View reviewed changes
Copy link
Contributor

@jerryzh168 jerryzh168 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I'm also wondering if we should move reference representation rewrite and port metadata etc. outside of torchao, since it seems that it's not used by most of torchao users, would it make more sense for this to be in ET?

@kimishpatel
Copy link
Contributor Author

OK, I'm also wondering if we should move reference representation rewrite and port metadata etc. outside of torchao, since it seems that it's not used by most of torchao users, would it make more sense for this to be in ET?

Yeah it could. Not opposed to it. I do think it is useful for debugging especially when doing lowbit quant. Even when doing say fp4 perhaps? But if this doesnt seem of interest in forseeable future than I am find moving it out

@jerryzh168
Copy link
Contributor

we could add some logging and decide in a few months, my guess is we don't have people using these right now, at least the reference rewrite

@kimishpatel
Update base for Update on "Port metadata from the linear node onto th…
d7aafcc 
…e reference custom op for int4"

Summary:
Allow for numerical debugger in ExecuTorch use the from_node info for
correlation

Test Plan:
CI

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
@kimishpatel
Update on "Port metadata from the linear node onto the reference cust…
5d2f23d 
…om op for int4"

Summary:
Allow for numerical debugger in ExecuTorch use the from_node info for
correlation

Test Plan:
CI

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
kimishpatel added a commit that referenced this pull request Aug 27, 2025
@kimishpatel
Port metadata from the linear node onto the reference custom op for int4
58fb9f0 
Summary:
Allow for numerical debugger in ExecuTorch use the from_node info for
correlation

Test Plan:
CI

Reviewers:

Subscribers:

Tasks:

Tags:

ghstack-source-id: cd8b708
Pull Request resolved: #2860
@kimishpatel kimishpatel added the topic: improvement Use this tag if this PR is an improvement (doesn't fit into any of the other categories) label Aug 28, 2025
@kimishpatel
Update base for Update on "Port metadata from the linear node onto th…
e45e0b3 
…e reference custom op for int4"

Summary:
Allow for numerical debugger in ExecuTorch use the from_node info for
correlation

Test Plan:
CI

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
@kimishpatel
Update on "Port metadata from the linear node onto the reference cust…
aa44ccb 
…om op for int4"

Summary:
Allow for numerical debugger in ExecuTorch use the from_node info for
correlation

Test Plan:
CI

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
kimishpatel added a commit that referenced this pull request Aug 28, 2025
@kimishpatel
Port metadata from the linear node onto the reference custom op for int4
4f57444 
Summary:
Allow for numerical debugger in ExecuTorch use the from_node info for
correlation

Test Plan:
CI

Reviewers:

Subscribers:

Tasks:

Tags:

ghstack-source-id: 858a6f4
Pull Request resolved: #2860
@kimishpatel kimishpatel changed the base branch from gh/kimishpatel/8/base to main August 29, 2025 15:16
@kimishpatel kimishpatel merged commit 3a9b8d1 into main Aug 29, 2025
34 checks passed
@kimishpatel kimishpatel deleted the gh/kimishpatel/8/head branch August 29, 2025 15:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jerryzh168 jerryzh168 jerryzh168 approved these changes

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. pt2e_quant pt2 export quantization (prepare_pt2e, convert_pt2e) topic: improvement Use this tag if this PR is an improvement (doesn't fit into any of the other categories)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@kimishpatel @jerryzh168