Onnx slim transform

[tchawada]

Performance comparison with onnx-slim transformed model and original model for GPT2 model.

========================= Performance Stats with Onnx-Slim=========================
Average Prefill time a.k.a TTFT is= 0.01 sec        
Decode is= 408.75 tokens/sec        
Total is= 399.23 tokens/sec        
Total (E2E) inference time is= 0.31 sec
onnx file size=420K
qpc file size=391M
GPT2LMHeadModel_0.onnx.data=622.94M
time taken to slim 19 seconds
compile time=39.90 seconds

========================= Performance Stats without Onnx-Slim =========================
Average Prefill time a.k.a TTFT is= 0.01 sec        
Decode is= 384.3 tokens/sec        
Total is= 374.99 tokens/sec        
Total (E2E) inference time is= 0.33 sec
onnx file size=516K
qpc file size= 391M
GPT2LMHeadModel_0.onnx.data=622.94M
compile time =36.44 seconds

[tchawada]

Performance comparison with onnx-slim transformed model and original model for GPT2 model.
========================= Performance Stats with Onnx-Slim=========================
Average Prefill time a.k.a TTFT is= 0.01 sec
Decode is= 408.75 tokens/sec
Total is= 399.23 tokens/sec
Total (E2E) inference time is= 0.31 sec
onnx file size=420K
qpc file size=391M
GPT2LMHeadModel_0.onnx.data=622.94M
time taken to slim 19 seconds
compile time=39.90 seconds
========================= Performance Stats without Onnx-Slim =========================
Average Prefill time a.k.a TTFT is= 0.01 sec
Decode is= 384.3 tokens/sec
Total is= 374.99 tokens/sec
Total (E2E) inference time is= 0.33 sec
onnx file size=516K
qpc file size= 391M
GPT2LMHeadModel_0.onnx.data=622.94M
compile time =36.44 seconds

[quic-amitraj]

Please once test with the full model https://huggingface.co/meta-llama/Llama-2-7b-chat-hf.

[quic-amitraj]

Apply ruff check and format @tchawada

[tchawada]

I have applied ruff check and format. Regards Tanisha

…

________________________________ From: Amit Raj ***@***.***> Sent: Tuesday, August 12, 2025 3:17 PM To: quic/efficient-transformers ***@***.***> Cc: Tanisha Chawada ***@***.***>; Mention ***@***.***> Subject: Re: [quic/efficient-transformers] Onnx slim transform (PR #536) WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros. [https://avatars.githubusercontent.com/u/168538872?s=20&v=4]quic-amitraj left a comment (quic/efficient-transformers#536)<#536 (comment)> Apply ruff check and format @tchawada<https://github.com/tchawada> — Reply to this email directly, view it on GitHub<#536 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BVI2PQ66QHJDI2VQQ2HXNUL3NGZ4LAVCNFSM6AAAAACDVIPW6CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTCNZYGYYDKNBRGE>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[quic-hemagnih]

Please resolve the lint warnings

[quic-hemagnih]

I would suggest to use logger to print any messages.

[quic-hemagnih]

remove print

[quic-hemagnih]

Add Type Checking or Validation Ensure temp_onnx_path is not None before saving

[tchawada]

I will do the appropriate changes.

________________________________ From: Hem Agnihotri ***@***.***> Sent: Wednesday, August 13, 2025 11:51 AM To: quic/efficient-transformers ***@***.***> Cc: Tanisha Chawada ***@***.***>; Mention ***@***.***> Subject: Re: [quic/efficient-transformers] Onnx slim transform (PR #536) WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros. @quic-hemagnih commented on this pull request.

________________________________ In QEfficient/base/onnx_transforms.py<#536 (comment)>:

+ *,

+ onnx_base_dir: Optional[str] = None, + **kwargs, + ) -> Tuple[ModelProto, bool]: + """ + :param enable_onnx_slim_transform: If True, applies onnx-slim transformations. + """ + # print(kwargs) + transformed = False + onnx_slim_transform = kwargs.get("enable_onnx_slim_transform", False) + temp_onnx_path = kwargs.get("temp_onnx_path", None) + if onnx_slim_transform: + print("onnx slim transform done") + transformed = True + slimmed_model = onnxslim.slim(model) + onnx.save(slimmed_model, temp_onnx_path) Add Type Checking or Validation Ensure temp_onnx_path is not None before saving — Reply to this email directly, view it on GitHub<#536 (review)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BVI2PQYNFOARIKTG4PFZAND3NLKNNAVCNFSM6AAAAACDVIPW6CVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZTCMJUGA4TEMBTGE>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[quic-rishinr]

Instead of adding onnx_slim_transform to every AutoModel class can we consider creating a transform configuration module that returns enabled/disabled transforms as dict. Apply transforms in the base class based on this config? this can be applicable for both pytorch and onnx transforms.

[inisis]

Hi, I'm the author of onnxslim, thanks for using it, and onnxslim applies to evevy single onnx model, feel free to message me if you have any problem and looking forward to more cooperation and intergration for your porjects.