Skip to content

Move to a faster base64 implementation #19984

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 25, 2025
+6 −5
Merged

Move to a faster base64 implementation #19984

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
635c559
moving to a faster base64 implementation
h-avsha Jun 23, 2025
ce59e33
import fix
h-avsha Jun 23, 2025
0811163
Merge branch 'vllm-project:main' into main
h-avsha Jun 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions requirements/common.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,4 @@ watchfiles # required for http server to monitor the updates of TLS files
python-json-logger # Used by logging as per examples/others/logging_configuration.md
scipy # Required for phi-4-multimodal-instruct
ninja # Required for xgrammar, rocm, tpu, xpu
pybase64 # fast base64 implementation
10 changes: 5 additions & 5 deletions vllm/multimodal/image.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project

import base64
from io import BytesIO
from pathlib import Path

import pybase64
import torch
from PIL import Image

Expand Down Expand Up @@ -55,7 +55,7 @@ def load_bytes(self, data: bytes) -> Image.Image:
return convert_image_mode(image, self.image_mode)

def load_base64(self, media_type: str, data: str) -> Image.Image:
return self.load_bytes(base64.b64decode(data))
return self.load_bytes(pybase64.b64decode(data, validate=True))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

By switching to pybase64.b64decode and setting validate=True, you've made the base64 decoding stricter. The previous implementation using base64.b64decode with its default settings would silently ignore non-base64 characters in the input string. The new implementation will raise an error.

This is a positive change for input validation and data integrity. I just want to confirm this change in behavior is intended. If so, this is a great improvement!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

validate=True is actually faster, since with validate=False, pybase64 will proactively filter the input for illegitimate characters, which entails some performance overhead


def load_file(self, filepath: Path) -> Image.Image:
image = Image.open(filepath)
Expand All @@ -75,7 +75,7 @@ def encode_base64(
image.save(buffer, image_format)
data = buffer.getvalue()

return base64.b64encode(data).decode('utf-8')
return pybase64.b64encode(data).decode('utf-8')


class ImageEmbeddingMediaIO(MediaIO[torch.Tensor]):
Expand All @@ -88,10 +88,10 @@ def load_bytes(self, data: bytes) -> torch.Tensor:
return torch.load(buffer, weights_only=True)

def load_base64(self, media_type: str, data: str) -> torch.Tensor:
return self.load_bytes(base64.b64decode(data))
return self.load_bytes(pybase64.b64decode(data, validate=True))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Similar to the change in ImagePixelMediaIO.load_base64, this change introduces stricter validation for base64-encoded image embeddings. The previous implementation would ignore invalid characters, while this one will raise an error.

This is a good enhancement for robustness. Confirming this is the intended behavior.


def load_file(self, filepath: Path) -> torch.Tensor:
return torch.load(filepath, weights_only=True)

def encode_base64(self, media: torch.Tensor) -> str:
return base64.b64encode(media.numpy()).decode('utf-8')
return pybase64.b64encode(media.numpy()).decode('utf-8')