Add append operation for efficient text streaming #170
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds a new
append
operation to python-json-patch for efficient text streaming in chat applications. When streaming text character-by-character or word-by-word, the standardreplace
operation requires sending the entire string with each update. The newappend
operation only sends the incremental text, significantly reducing payload size and improving performance.This method is used by official ChatGPT and is an implementation optimized for chat.
Key Features
append
operation: Appends text to existing string valuesmake_patch()
automatically detects append scenariosImplementation Details
operation:
Performance
The demo script shows append operations are ~2.8x faster than replace operations for streaming scenarios.