Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hardware JPEG decoder #4576

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Hardware JPEG decoder #4576

wants to merge 4 commits into from

Conversation

cepetr
Copy link
Contributor

@cepetr cepetr commented Feb 4, 2025

This PR implements a hardware JPEG decoder for T3W1.

  • Implemented a new gfx/jpegdec module that wraps the STM32 hardware JPEG decoder.
  • Implemented a new trezorhal/jpegdec.rs module that provides the JpegDecoder API for Rust.
  • Modified shape::JpegImage to support the hardware jpeg decoder.
  • Additionally, two new features have been introduced - hw_jpeg_decoder and ui_jpeg - which replace jpeg and ui_jpeg_decoder.
  • Implemented missing blurring for RGBA8888 canvas

JpegImage shape limitations on T3W1:

  1. Scaling is not supported. If enabled, the resulting image is simply cropped. A simple (but slow) scaling method can be easily implemented in the future if needed.

The simulator is still using the original JPEG decoder implemented in Rust.
All other models (except T3W1) should not be affected by this PR.

@cepetr cepetr added the T3W1 label Feb 4, 2025
@cepetr cepetr requested a review from TychoVrahe February 4, 2025 13:12
@cepetr cepetr self-assigned this Feb 4, 2025
Copy link

github-actions bot commented Feb 4, 2025

core UI changes device test click test persistence test
T2T1 Model T test(screens) main(screens) test(screens) main(screens) test(screens) main(screens)
T3B1 Safe 3 test(screens) main(screens) test(screens) main(screens) test(screens) main(screens)
T3T1 Safe 5 test(screens) main(screens) test(screens) main(screens) test(screens) main(screens)
All main(screens)

@prusnak
Copy link
Member

prusnak commented Feb 4, 2025

Can you post the results of the benchmark? I.e. how much faster is it than SW decoding on the same chip?

Also are there any limitations of the HW decoder when compared to SW encoding?

@cepetr
Copy link
Contributor Author

cepetr commented Feb 4, 2025

Can you post the results of the benchmark? I.e. how much faster is it than SW decoding on the same chip?

Also are there any limitations of the HW decoder when compared to SW encoding?

For 380×520 color images, software decoding takes approximately 600 ms, while hardware decoding takes about 10 ms (both include YCbCr to RGB conversion and display presentation).

According to the STM datasheet, the hardware decoder should comply with the JPEG standard and parse all standard JPEG headers. Unfortunately, it cannot decode progressive JPEGs (though the same applies to the software decoder we currently use).

One current limitation of the hardware decoder, compared to the software decoder we use, is that it cannot scale JPEG images. As a result, we cannot display a small preview of the uploaded image. However, this is not a problem if we slightly change the UI.

@cepetr cepetr force-pushed the cepetr/hw-jpeg-decoding branch 2 times, most recently from 852b6d6 to b565601 Compare February 4, 2025 17:35
@cepetr cepetr marked this pull request as ready for review February 4, 2025 17:35
@cepetr cepetr requested a review from prusnak as a code owner February 4, 2025 17:35
@matejcik
Copy link
Contributor

matejcik commented Feb 5, 2025

One current limitation of the hardware decoder, compared to the software decoder we use, is that it cannot scale JPEG images. As a result, we cannot display a small preview of the uploaded image. However, this is not a problem if we slightly change the UI.

we should be able to scale the image while streaming it -- unless the jpeg decoder is barfing pixels directly into the framebuffer?

@cepetr
Copy link
Contributor Author

cepetr commented Feb 5, 2025

One current limitation of the hardware decoder, compared to the software decoder we use, is that it cannot scale JPEG images. As a result, we cannot display a small preview of the uploaded image. However, this is not a problem if we slightly change the UI.

we should be able to scale the image while streaming it -- unless the jpeg decoder is barfing pixels directly into the framebuffer?

Yes, it would definitely be possible - and it wouldn’t be too slow if we simply skipped pixels and used (1/2, 1/4, …) scaling as before. However, it would still be much slower than just copying with DMA2D. You also have to consider that the image quality wouldn’t be perfect for some images due to aliasing.

@cepetr cepetr force-pushed the cepetr/hw-jpeg-decoding branch from 4019044 to 86c2993 Compare February 7, 2025 10:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

3 participants