Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Concept Entry] Created the export entry #6127

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
---
title: Model Export with TorchScript
description: Convert PyTorch models to TorchScript for production deployment in non-Python environments.
subject:
- 'Computer Science'
- 'Data Science'
- 'Machine Learning'
tags:
- 'Deployment'
- 'TorchScript'
- 'Model Export'
- 'PyTorch'
catalog_content:
- 'learn-pytorch'
- 'paths/data-science'
Comment on lines +2 to +15
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The headers Title, Description, Subject, and Tags should be in title case, and the sentences should be inside ''.

Suggested change
title: Model Export with TorchScript
description: Convert PyTorch models to TorchScript for production deployment in non-Python environments.
subject:
- 'Computer Science'
- 'Data Science'
- 'Machine Learning'
tags:
- 'Deployment'
- 'TorchScript'
- 'Model Export'
- 'PyTorch'
catalog_content:
- 'learn-pytorch'
- 'paths/data-science'
Title: 'Model Export with TorchScript'
Description: 'Convert PyTorch models to TorchScript for optimized deployment in non-Python environments like mobile and embedded systems.'
Subject:
- 'Computer Science'
- 'Data Science'
- 'Machine Learning'
Tags:
- 'Deployment'
- 'TorchScript'
- 'Model Export'
- 'PyTorch'
CatalogContent:
- 'learn-pytorch'
- 'paths/data-science'

---

**TorchScript** is a PyTorch intermediate representation that allows models to be serialized and optimized for execution in resource-constrained environments like mobile devices, embedded systems, or C++ applications. It decouples models from Python dependencies while preserving their logic.

## Syntax

- **Scripting** (for models with dynamic control flow):

```pseudo
scripted_model = torch.jit.script(model) # Converts model to TorchScript
scripted_model.save("model.pt")
```

- **Tracing** (for static models):

```pseudo
traced_model = torch.jit.trace(model, example_input) # Records operations via example input
traced_model.save("model.pt")
```

- **Hybrid Approach** (script/trace specific submodules):

```pseudo
@torch.jit.script
def custom_function(x: torch.Tensor) -> torch.Tensor:
return x * 2
```

## Example

### Scripting a Model with Conditional Logic

Export a model that uses `if` statements (unsupported by tracing):

```python
import torch

class DynamicModel(torch.nn.Module):
def forward(self, x: torch.Tensor) -> torch.Tensor:
if x.sum() > 0:
return x * 2
else:
return x - 1

model = DynamicModel()
scripted_model = torch.jit.script(model) # Handles dynamic control flow
scripted_model.save("dynamic_model.pt")
Comment on lines +51 to +62
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The example should give some tangible output that helps the reader understand what's happening -

Suggested change
import torch
class DynamicModel(torch.nn.Module):
def forward(self, x: torch.Tensor) -> torch.Tensor:
if x.sum() > 0:
return x * 2
else:
return x - 1
model = DynamicModel()
scripted_model = torch.jit.script(model) # Handles dynamic control flow
scripted_model.save("dynamic_model.pt")
import torch
class DynamicModel(torch.nn.Module):
def forward(self, x: torch.Tensor) -> torch.Tensor:
if x.sum() > 0:
return x * 2
else:
return x - 1
model = DynamicModel()
scripted_model = torch.jit.script(model) # Handles dynamic control flow
scripted_model.save("dynamic_model.pt")
# Testing the scripted model
x1 = torch.tensor([1.0, -0.5, 3.0])
x2 = torch.tensor([-2.0, -1.5, -0.5])
print(scripted_model(x1))
print(scripted_model(x2))

Add output below this -

tensor([ 2., -1.,  6.])
tensor([-3.0000, -2.5000, -1.5000])

```

### Tracing a ResNet for Mobile Deployment

Convert a pretrained **ResNet** using tracing:

```python
import torch
import torchvision

model = torchvision.models.resnet18(weights="IMAGENET1K_V1").eval()

# Trace with example input
dummy_input = torch.rand(1, 3, 224, 224)
traced_model = torch.jit.trace(model, dummy_input)
traced_model.save("resnet18_traced.pt")
Comment on lines +70 to +78
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here. The example should have an output -

Suggested change
import torch
import torchvision
model = torchvision.models.resnet18(weights="IMAGENET1K_V1").eval()
# Trace with example input
dummy_input = torch.rand(1, 3, 224, 224)
traced_model = torch.jit.trace(model, dummy_input)
traced_model.save("resnet18_traced.pt")
import torch
import torchvision
model = torchvision.models.resnet18(weights="IMAGENET1K_V1").eval()
# Trace with example input
dummy_input = torch.rand(1, 3, 224, 224)
traced_model = torch.jit.trace(model, dummy_input)
traced_model.save("resnet18_traced.pt")
# Running inference with traced model
output = traced_model(dummy_input)
print(output.shape)

Add output below that:

torch.Size([1, 1000])

This confirms that the traced ResNet model processes an image and produces 1000 output logits (corresponding to ImageNet classes).

```

## Key Considerations to Make

- **Scripting vs Tracing**:

- Use `torch.jit.script` for models with:
- Loops/conditionals
- Variable-length inputs
- Non-tensor data dependencies
- Use `torch.jit.trace` for static models (faster execution).

- **Limitations**:

- TorchScript supports a subset of Python/PyTorch operations.
- Avoid `**kwargs` or dynamic tensor shapes in traced models.
2 changes: 2 additions & 0 deletions documentation/tags.md
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,7 @@ Matrices
Media Queries
Memory
Metadata
Model Export
Methods
Middleware
Models
Expand Down Expand Up @@ -358,6 +359,7 @@ Trigonometry
Try
Tuples
Types
TorchScript
Type Guard
Type Narrowing
Typography
Expand Down
Loading