Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve CI #296

Merged
merged 5 commits into from
Jul 12, 2023
Merged

Improve CI #296

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
b3ae629
test: better nextest config
Taowyoo Jul 12, 2023
68e8244
ci: correct and improve publish-crates
Taowyoo Jul 12, 2023
c12f54f
ci: remove CI triiger on push master
Taowyoo Jul 12, 2023
b053769
ci: add cargo cache for test ci
Taowyoo Jul 12, 2023
f639448
ci: remove creation of symbolic of libclang
Taowyoo Jul 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions .config/nextest.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ serial-integration = { max-threads = 1 }
# * retries = 3
# * retries = { backoff = "fixed", count = 2, delay = "1s" }
# * retries = { backoff = "exponential", count = 10, delay = "1s", jitter = true, max-delay = "10s" }
retries = 2
retries = 0

# The number of threads to run tests with. Supported values are either an integer or
# the string "num-cpus". Can be overridden through the `--test-threads` option.
Expand Down Expand Up @@ -111,8 +111,10 @@ store-success-output = false
store-failure-output = true

[[profile.default.overrides]]
filter = 'binary(=hyper)'
retries = { backoff = "fixed", count = 6, delay = "1s" }
# `hyper` test targeting external Google's service, it would sometomes failed because of rate limit
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unit tests really shouldn't rely on external services, especially ones that might rate limit us. Can this test be changed so it runs its own server locally that the test can make requests to? Ideally, the server should use a different network implementation, so something like python.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sense.
Created an issue for this: #297

# `is_probably_prime` test heavily use rdrand, sometimes the VM where test is running has shortage on it
filter = 'binary(=hyper) or test(=is_probably_prime)'
retries = { backoff = "exponential", count = 4, delay = "1s", jitter = true, max-delay = "10s" }
test-group = 'serial-integration'

[[profile.default.overrides]]
Expand Down
48 changes: 35 additions & 13 deletions .github/workflows/publish-crates.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,11 @@ on:
release:
types: [created]
workflow_dispatch:

inputs:
crate_name:
description: 'Name of crate to be published'
required: true
type: string
permissions:
contents: read

Expand All @@ -13,24 +17,42 @@ jobs:
environment: "publish to crates.io"
runs-on: ubuntu-latest
steps:
- name: Install dependencies
- name: Checkout repository
uses: actions/checkout@v3
- name: Install Rust toolchain
uses: actions-rs/toolchain@v1
with:
toolchain: stable
override: true
profile: minimal
- name: Install build dependencies
run: |
wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add -
echo "deb http://apt.llvm.org/focal/ llvm-toolchain-focal-11 main" | sudo tee -a /etc/apt/sources.list
sudo apt-get update
sudo apt-get install -y clang-11 cmake
- name: Symlink libclang.so
run: sudo ln -s /lib/x86_64-linux-gnu/libclang-11.so.1 /lib/x86_64-linux-gnu/libclang.so
- uses: actions/checkout@v2
- uses: actions-rs/toolchain@v1
with:
toolchain: stable
override: true
- run: |
export crate_name=$(python3 -c "print('$GITHUB_REF'.split('/')[2].rsplit('_v', 1)[0])")
echo "Publishing crate: $crate_name"
cargo publish --locked --token ${CARGO_REGISTRY_TOKEN} --package "$crate_name"
# Create a symbolic link to ensure correct usage of libclang
sudo ln -s /lib/x86_64-linux-gnu/libclang-11.so.1 /lib/x86_64-linux-gnu/libclang.so
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the .so needs to be there, just possibly not in the location where it's being looked for, can this be fixed by adding the appropriate directory to the search path for the linker and/or setting LD_LIBRARY_PATH when whatever needs this library is being run? Adding symlinks to standard system locations seems like not a good idea for a build/test system to be doing.

Copy link
Collaborator Author

@Taowyoo Taowyoo Jul 12, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed
I see, after recheck. These lines seems not necessary.

if [ -f mbedtls-sys/vendor/scripts/basic.requirements.txt ]; then
sudo apt-get install -y python3-pip
python3 -m pip install -r mbedtls-sys/vendor/scripts/basic.requirements.txt
fi
- name: Get name of crate to be published
run: |
if [[ -z "${{ inputs.crate_name }}" ]]; then
# Extract the crate name from the GITHUB_REF environment variable
# GITHUB_REF contains the GitHub reference (e.g., refs/tags/mbedtls-sys-auto_v3.5.0) associated with the event
export CRATE_NAME=$(python3 -c "print('$GITHUB_REF'.split('/')[2].rsplit('_v', 1)[0])")
else
export CRATE_NAME="${{ inputs.crate_name }}"
fi
echo "CRATE_NAME=$CRATE_NAME" >> $GITHUB_ENV
- name: Publish crate to crates.io
run: |
echo "Publishing crate: $CRATE_NAME"
cargo publish --locked --token ${CARGO_REGISTRY_TOKEN} --package "$CRATE_NAME"
env:
CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
RUSTFLAGS: "-A ambiguous_glob_reexports"
RUST_BACKTRACE: "1"
PYTHONDONTWRITEBYTECODE: "1"
8 changes: 7 additions & 1 deletion .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ name: Test
on:
push:
branches:
- master
- 'v0.*'
- staging
- trying
Expand All @@ -16,6 +15,8 @@ on:
env:
RUST_BACKTRACE: 1
CARGO_TERM_COLOR: always
CARGO_INCREMENTAL: 0
CARGO_NET_RETRY: 10

jobs:
build:
Expand Down Expand Up @@ -51,6 +52,11 @@ jobs:
target: ${{ matrix.target }}
override: true

- name: Cache Dependencies
uses: Swatinem/rust-cache@988c164c3d0e93c4dbab36aaf5bbeb77425b2894
with:
key: ${{ matrix.rust }}

- name: Run tests
run: |
./ct.sh
Expand Down