Upgrade mbedtls to 3.5.0-alpha.1+0b3de6f

.config/nextest.toml

@@ -19,7 +19,7 @@ serial-integration = { max-threads = 1 }
 # * retries = 3
 # * retries = { backoff = "fixed", count = 2, delay = "1s" }
 # * retries = { backoff = "exponential", count = 10, delay = "1s", jitter = true, max-delay = "10s" }
-retries = 0
+retries = 2
 
 # The number of threads to run tests with. Supported values are either an integer or
 # the string "num-cpus". Can be overridden through the `--test-threads` option.

3.X-migration-guide.md

@@ -1,41 +1,47 @@
 # Overview of changes in 3.X mbedtls
 
-- Update vendor `mbedtls` code from version `2.28.3` to a `3.4.0`
-	- Please checkout C `mbedtls` migration guide for 3.X here [3.0-migration-guide.md](https://github.com/fortanix/rust-mbedtls/blob/yx/upgrade-mbedtls/mbedtls-sys/vendor/docs/3.0-migration-guide.md) to
+- Update vendor `mbedtls` code from version `2.28.3` to a commit at development version: [0b3de6fce](https://github.com/Mbed-TLS/mbedtls/commit/0b3de6fce)
+  - Please checkout C `mbedtls` [3.0-migration-guide.md](https://github.com/fortanix/rust-mbedtls/blob/yx/upgrade-mbedtls/mbedtls-sys/vendor/docs/3.0-migration-guide.md) for more info
+  - The vendor code at this commit includes the **TLS server side version negotiation** which implements [rfc844 4.2.1](https://www.rfc-editor.org/rfc/rfc8446#section-4.2.1)
+  - This is a temporary approach at current time point since upstream has not released major version ( `3.5.0` ) that contains the **Server Side Version Negotiation**, we need to upgrade to `3.5.0` once it's released, its estimated release date is 2023-09-30, Ref: https://github.com/orgs/Mbed-TLS/
+    projects/1#column-19402885
 - Update the build code for `mbedtls-sys-auto` crate to sync up with vendor code change:
-	- Changes in C DEFINE's for configuration
-	- Changes in header files
-	- Add binding code generation for `static inline` code in C side
-	- Removing deprecated cargo features or dependencies
-	- Add a cargo feature for TLS 1.3
-	- Put pkcs12 behind a cargo feature
+  - Changes in C DEFINE's for configuration
+  - Changes in header files
+  - Add binding code generation for `static inline` code in C side
+  - Removing deprecated cargo features or dependencies
+  - Add a cargo feature for TLS 1.3
+  - Put pkcs12 behind a cargo feature
+  - Bump crate version to `3.5.0-alpha.1+0b3de6f` to represent we are using a deveoplent version of upstream code
 - Update rust wrapper code in `./mbedtls` to sync up the changes in  `mbedtls-sys-auto`
-	- Removing deprecated cargo features or dependencies
-	- Update rust code to sync up API changes
-	- Add types and functions for TLS 1.3
-	- Add a cargo feature for TLS 1.3
-	- Update integration tests for new API and TLS 1.3
-	- Update dependencies
+  - Removing deprecated cargo features or dependencies
+  - Update rust code to sync up API changes
+  - Add types and functions for TLS 1.3
+  - Add a cargo feature for TLS 1.3
+  - Update integration tests for new API and TLS 1.3
+  - Update dependencies
 
 ## Changes:
 
 **Note**: entries with :boom: are ensured that they will break the downstream.
 
-
 ### Changes in `mbedtls-sys-auto`
 
 #### Vendor code changes
 
 **Upstream changes:**
 
-- :boom: Upgrade vendor `mbeldtls` code to `3.4.0`
+- :boom: Upgrade vendor `mbeldtls` code to version `3.5.0-alpha.1+0b3de6f`, it means we are using upstream code at commit [0b3de6fce](https://github.com/Mbed-TLS/mbedtls/commit/0b3de6fce): this is needed because it contains the server side version negotiation features which is necessary in a functional complete TLS 1.3 stack.
+
+Note: Here we use a pre-release version since upstream has not release `3.5.0` yet. Ref: https://doc.rust-lang.org/cargo/reference/semver.html
 
 **Changes on our side:**
 
 - Cherry picked previous changes in old versions:
-	- commit: [vendor change: Adding mpi_force_c_code feature](https://github.com/fortanix/rust-mbedtls/pull/213/commits/c8cd4067ff51f2c74cf623e0a42028215a77f883)
+  - commit: [vendor change: Adding mpi_force_c_code feature](https://github.com/fortanix/rust-mbedtls/pull/213/commits/c8cd4067ff51f2c74cf623e0a42028215a77f883)
 - New changes
-    - [vendor change: fix time call in tls13 client&server](https://github.com/fortanix/rust-mbedtls/pull/213/commits/bafc52d97bda9963f36b8eefd19a31110e8e5dac) : This has been merged into upstream, see: https://github.com/Mbed-TLS/mbedtls/pull/7639 .
+  - [vendor change: fix time call in tls13 client&server](https://github.com/fortanix/rust-mbedtls/pull/213/commits/bafc52d97bda9963f36b8eefd19a31110e8e5dac) : This has been merged into upstream, see: https://github.com/Mbed-TLS/mbedtls/pull/7639 .
+  - [vendor change: add accessors](https://github.com/fortanix/rust-mbedtls/pull/213/commits/96961cce495dd6c6643a4d8f7b60dc4fc67f4170) : these are added to following new upstream requirements that we should not access to fields with `private_` prefix, and these changes will be sent to upstream soon.
 
 #### rust code changes
 
@@ -64,20 +70,20 @@ Following changes are made according to [3.0-migration-guide.md](https://github.
 - Remove `MBEDTLS_CONFIG_H` in `mbedtls_config.h`
 - Remove `#include <mbedtls/check_config.h>`
 - `mbedtls-sys/build/bindgen.rs`
-	-  Allow `bindgen` to generate bindings for functions, types and variables start with `psa_`, and put them in a sub `mod psa` because they are needed by TLS 1.3
-	- Use  `bindgen` experiment feature to generate C function wrapper for C `static inline` functions
+  - Allow `bindgen` to generate bindings for functions, types and variables start with `psa_`, and put them in a sub `mod psa` because they are needed by TLS 1.3
+  - Use  `bindgen` experiment feature to generate C function wrapper for C `static inline` functions
 - `mbedtls-sys/build/headers.rs` : Update header files
 - `mbedtls-sys/build/config.rs` : Remove/add C defines
-	- :boom: Added `MBEDTLS_PSA_CRYPTO_EXTERNAL_RNG` : this is added because TLS 1.3 need to use PSA library which need this when feature `std` is off, user need to provide their implementation. In `rus-mbedtls`, one implementation is proveded, see: `mbedtls/src/rng/mod.rs`.
+  - :boom: Added `MBEDTLS_PSA_CRYPTO_EXTERNAL_RNG` : this is added because TLS 1.3 need to use PSA library which need this when feature `std` is off, user need to provide their implementation. In `rus-mbedtls`, one implementation is proveded, see: `mbedtls/src/rng/mod.rs`.
 
 ### Changes in `mbedtls-platform-support`
 
 - Added `once_cell` for initializing PSA only one times: see `fn psa_crypto_init()` in `mbedtls-platform-support/src/lib.rs`, this is needed because:
-	- > MBEDTLS_USE_PSA_CRYPTO means that X.509 and TLS will use PSA Crypto as much as possible (that is, everywhere except for features that are not supported by PSA Crypto, see "Internal Changes" below for a complete list of exceptions). When it is enabled, you need to call psa_crypto_init() before calling any function from PK, X.509 or TLS;
-	- Ref: https://github.com/Mbed-TLS/mbedtls/blob/0b3de6fcec4aa4b23a9ee1e076714cbc796f3ac4/docs/use-psa-crypto.md#general-considerations
+  - > MBEDTLS_USE_PSA_CRYPTO means that X.509 and TLS will use PSA Crypto as much as possible (that is, everywhere except for features that are not supported by PSA Crypto, see "Internal Changes" below for a complete list of exceptions). When it is enabled, you need to call psa_crypto_init() before calling any function from PK, X.509 or TLS;
+  - Ref: https://github.com/Mbed-TLS/mbedtls/blob/0b3de6fcec4aa4b23a9ee1e076714cbc796f3ac4/docs/use-psa-crypto.md#general-considerations
 - Add function pointer `mbedtls_psa_external_get_random`  which is needed when C DEFINE option `MBEDTLS_PSA_CRYPTO_EXTERNAL_RNG` is turned on. This will be used in case when system default RNG or entropy is no available (for example in SGX)
-    - Using function pointer here ensure there is no link time conflict in future when mutiple `rust-mbedtls` is using this crate.
-    - User need to call function `set_psa_external_rng_callback` before using any PSA functions or TLS 1.3
+  - Using function pointer here ensure there is no link time conflict in future when mutiple `rust-mbedtls` is using this crate.
+  - User need to call function `set_psa_external_rng_callback` before using any PSA functions or TLS 1.3
 - Add rust implementation of `explicit_bzero`, which is needed in SGX. Because in SGX, our [rs-libc](https://github.com/fortanix/rust-sgx/tree/master/rs-libc) does not support this function.
 - Update self tests:
   - following are removed:
@@ -104,7 +110,7 @@ Following changes are made according to [3.0-migration-guide.md](https://github.
 **Dependencies:**
 
 - :boom: Bump `rust-mbedtls` version to `0.11.0`
-- :boom: Bump dependency `mbedtls-sys-auto` version to `3.4.0`
+- :boom: Bump dependency `mbedtls-sys-auto` version to `3.5.0-alpha.1+0b3de6f`
 - Added `rstest` `rstest_reuse` `lazy_static` `env_logger`: used for improving code of tests
 
 
@@ -115,11 +121,11 @@ Following changes are made according to [3.0-migration-guide.md](https://github.
 - :boom: Removed and added some options in `Error`, `CipherType`, `CipherMode` and `CipherSuite`  to sync with changes in `mbedtls` 3.X
 - :boom: `mbedtls/src/pk/ec.rs` : User need to provide a RNG callback for function `EcPoint::mul`, this originally is not a hard requirement, but in C `mbedtls` 3.X this become a hard requirement for blinding to defend side channel attack.
 - :boom: `mbedtls/hash` :
-	- `Md2` and `Md4` are removed since they are no longer supported in `mbedtls` 3.X
-	- fn `pbkdf_pkcs12` is removed since `pkcs12` is removed
+  - `Md2` and `Md4` are removed since they are no longer supported in `mbedtls` 3.X
+  - fn `pbkdf_pkcs12` is removed since `pkcs12` is removed
 - :boom: `mbedtls/pk/mod.rs` :
-	- Remove `CustomPk`
-	- User need to provide a RNG callback for `Pk::from_private_key`, this originally is not a hard requirement, but in C `mbedtls` 3.X this become a hard requirement for blinding to defend side channel attack.
+  - Remove `CustomPk`
+  - User need to provide a RNG callback for `Pk::from_private_key`, this originally is not a hard requirement, but in C `mbedtls` 3.X this become a hard requirement for blinding to defend side channel attack.
 - :boom: `mbedtls/src/ssl/ciphersuites.rs`: Rename `TlsCipherSuite` to `Tls12CipherSuite`, and add enum: `Tls13CipherSuite`, `IanaTlsNamedGroup`, `TLS13SignatureAlgorithms`: these are introduced by TLS 1.3
 - `mbedtls/src/ssl/ssl_states.rs`: Add `SslStates` to represent the state of SSL handshake
 - Update tests accordingly
@@ -133,7 +139,7 @@ This case is found when connecting to `goolge.com`, where Google's server send t
 ## CI changes
 
 - Use [cargo-nextest](https://nexte.st/#cargo-nextest) to run tests
-	- Reduce time to run tests
-	- Have ability to run some tests in serial
-		- tests under `hyper.rs` need to access to `google.com` which has QPS limit
-		- some tests function use some system resource, see https://github.com/fortanix/rust-mbedtls/blob/yx/upgrade-mbedtls/mbedtls/tests/support/net.rs
+  - Reduce time to run tests
+  - Have ability to run some tests in serial
+    - tests under `hyper.rs` need to access to `google.com` which has QPS limit
+    - some tests function use some system resource, see https://github.com/fortanix/rust-mbedtls/blob/yx/upgrade-mbedtls/mbedtls/tests/support/net.rs

mbedtls-platform-support/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "mbedtls-platform-support"
-version = "0.2.0"
+version = "0.3.0"
 authors = ["Yuxiang Cao <[email protected]>"]
 build = "build.rs"
 edition = "2018"
@@ -29,7 +29,7 @@ chrono = "0.4"
 zeroize = { version = "1.6.0", default-features = false }
 
 [dependencies.mbedtls-sys-auto]
-version = "3.4.0"
+version = "3.5.0-alpha.1+0b3de6f"
 default-features = false
 features = ["threading", "custom_printf"]
 path = "../mbedtls-sys"

mbedtls-platform-support/src/lib.rs

@@ -107,6 +107,14 @@ pub unsafe extern "C" fn mbedtls_time(tp: *mut mbedtls_sys::types::time_t) -> mb
     timestamp
 }
 
+#[cfg(any(all(feature = "time", feature = "custom_time"), sys_time_component = "custom"))]
+#[doc(hidden)]
+#[no_mangle]
+// needs to be pub for global visibility
+pub unsafe extern "C" fn mbedtls_ms_time() -> mbedtls_sys::ms_time_t {
+    chrono::Utc::now().timestamp_millis() as mbedtls_sys::ms_time_t
+}
+
 /// You need to call `psa_crypto_init()` before calling any function from the SSL/TLS, X.509 or PK modules.
 /// This function is fine to be called mutiple times while ensure underlying initilization function is only
 /// been called only once.

mbedtls-sys/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "mbedtls-sys-auto"
-version = "3.4.0"
+version = "3.5.0-alpha.1+0b3de6f"
 authors = ["Jethro Beekman <[email protected]>"]
 build = "build/build.rs"
 license = "Apache-2.0 OR GPL-2.0-or-later"

mbedtls-sys/build/bindgen.rs

@@ -100,12 +100,14 @@ impl super::BuildConfig {
             };
         }
 
-        
+
         // generate static function wrappers without any other rust related parameters to ensure
         // correctness of result C code
         bindgen::builder()
             .clang_args(cc.get_compiler().args().iter().map(|arg| arg.to_str().unwrap()))
             .header_contents("bindgen-input.h", &header)
+            .allowlist_function("^(?i)mbedtls_.*")
+            .allowlist_function("^(?i)psa_.*")
             .wrap_static_fns(true)
             .wrap_static_fns_path(&self.static_wrappers_c)
             .generate().expect("bindgen error");

mbedtls-sys/vendor/.readthedocs.yaml

@@ -0,0 +1,26 @@
+# .readthedocs.yaml
+# Read the Docs configuration file
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+# Required
+version: 2
+
+# Set the version of Python and other tools you might need
+build:
+  os: ubuntu-20.04
+  tools:
+    python: "3.9"
+  jobs:
+    pre_build:
+      - make apidoc
+      - breathe-apidoc -o docs/api apidoc/xml
+
+# Build documentation in the docs/ directory with Sphinx
+sphinx:
+  builder: dirhtml
+  configuration: docs/conf.py
+
+# Optionally declare the Python requirements required to build your docs
+python:
+  install:
+  - requirements: docs/requirements.txt

mbedtls-sys/vendor/CMakeLists.txt

@@ -59,7 +59,7 @@ option(MBEDTLS_FATAL_WARNINGS "Compiler warnings treated as errors" ON)
 if(CMAKE_HOST_WIN32)
     option(GEN_FILES "Generate the auto-generated files as needed" OFF)
 else()
-    option(GEN_FILES "Generate the auto-generated files as needed" OFF)
+    option(GEN_FILES "Generate the auto-generated files as needed" ON)
 endif()
 
 option(DISABLE_PACKAGE_CONFIG_AND_INSTALL "Disable package configuration, target export and installation" ${MBEDTLS_AS_SUBPROJECT})

mbedtls-sys/vendor/ChangeLog.d/add-directoryname-san.txt

@@ -0,0 +1,3 @@
+Features
+   * Add parsing of directoryName subtype for subjectAltName extension in
+     x509 certificates.

mbedtls-sys/vendor/ChangeLog.d/add-milliseconds-time-api.txt

@@ -0,0 +1,5 @@
+API changes
+   * Add new millisecond time type `mbedtls_ms_time_t` and `mbedtls_ms_time()`
+     function, needed for TLS 1.3 ticket lifetimes. Alternative implementations
+     can be created using an ALT interface.
+

mbedtls-sys/vendor/ChangeLog.d/driver-only-ecdh.txt

@@ -0,0 +1,7 @@
+Features
+   * When a PSA driver for ECDH is present, it is now possible to disable
+     MBEDTLS_ECDH_C in the build in order to save code size. For TLS 1.2
+     key exchanges based on ECDH(E) to work, this requires
+     MBEDTLS_USE_PSA_CRYPTO. Restartable/interruptible ECDHE operations in
+     TLS 1.2 (ECDHE-ECDSA key exchange) are not supported in those builds yet,
+     as PSA does not have an API for restartable ECDH yet.

mbedtls-sys/vendor/ChangeLog.d/driver-only-hashes.txt

@@ -0,0 +1,10 @@
+Features
+   * All modules that use hashes or HMAC can now take advantage of PSA Crypto
+     drivers when MBEDTLS_PSA_CRYPTO_C is enabled and psa_crypto_init() has
+     been called. Previously (in 3.3), this was restricted to a few modules,
+     and only in builds where MBEDTLS_MD_C was disabled; in particular the
+     entropy module was not covered which meant an external RNG had to be
+     provided - these limitations are lifted in this version. A new set of
+     feature macros, MBEDTLS_MD_CAN_xxx, has been introduced that can be used
+     to check for availability of hash algorithms, regardless of whether
+     they're provided by a built-in implementation, a driver or both.

mbedtls-sys/vendor/ChangeLog.d/some-max-size-macro-are-too-small-when-psa-ecc-is-accelerated.txt

@@ -0,0 +1,5 @@
+Bugfix
+   * Fix proper sizing for PSA_EXPORT_[KEY_PAIR/PUBLIC_KEY]_MAX_SIZE and
+     PSA_SIGNATURE_MAX_SIZE buffers when at least one accelerated EC is bigger
+     than all built-in ones and RSA is disabled.
+     Resolves #6622.

mbedtls-sys/vendor/ChangeLog.d/tls13-server-version-negotiation.txt

@@ -0,0 +1,5 @@
+Features
+   * Add support for server-side TLS version negotiation. If both TLS 1.2 and
+     TLS 1.3 protocols are enabled, the TLS server now selects TLS 1.2 or
+     TLS 1.3 depending on the capabilities and preferences of TLS clients.
+     Fixes #6867.

mbedtls-sys/vendor/docs/.gitignore

@@ -1,2 +1,4 @@
 *.html
 *.pdf
+_build/
+api/

mbedtls-sys/vendor/docs/Makefile

@@ -0,0 +1,40 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = .
+BUILDDIR      = _build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help clean apidoc breathe_apidoc Makefile
+
+# Intercept the 'clean' target so we can do the right thing for apidoc as well
+clean:
+	@# Clean the apidoc
+	$(MAKE) -C .. apidoc_clean
+	@# Clean the breathe-apidoc generated files
+	rm -rf ./api
+	@# Clean the sphinx docs
+	@$(SPHINXBUILD) -M clean "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+apidoc:
+	@# Generate doxygen from source using the main Makefile
+	$(MAKE) -C .. apidoc
+
+breathe_apidoc: apidoc
+	@# Remove existing files - breathe-apidoc skips them if they're present
+	rm -rf ./api
+	@# Generate RST file structure with breathe-apidoc
+	breathe-apidoc -o ./api ../apidoc/xml
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile breathe_apidoc
+	@# Build the relevant target with sphinx
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

mbedtls-sys/vendor/docs/architecture/mbed-crypto-storage-specification.md

@@ -354,7 +354,7 @@ Supported features:
 
 * [Persistent keys](#key-file-format-for-mbed-tls-2.25.0) designated by a [key identifier and owner](#key-names-for-mbed-tls-2.25.0). Keys can be:
     * Transparent, stored in the export format.
-    * Opaque, using the unified driver interface with statically registered drivers (`MBEDTLS_PSA_CRYPTO_DRIVERS`). The driver determines the content of the opaque key blob.
+    * Opaque, using the PSA driver interface with statically registered drivers. The driver determines the content of the opaque key blob.
     * Opaque, using the deprecated secure element interface with dynamically registered drivers (`MBEDTLS_PSA_CRYPTO_SE_C`). The driver picks a slot number which is stored in the place of the key material.
 * [Nonvolatile random seed](#nonvolatile-random-seed-file-format-for-mbed-tls-2.25.0) on ITS only.