Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

issue with efs-utils after recent update #213

Open
vtotare opened this issue May 4, 2024 · 7 comments
Open

issue with efs-utils after recent update #213

vtotare opened this issue May 4, 2024 · 7 comments

Comments

@vtotare
Copy link

vtotare commented May 4, 2024

Hello Team,
I am using Alpine (3.15) and we are using this utility to mount efs file system on docker container but I noticed that recent update in efs-utils started breaking docker build image process. Previously, was getting below error message so thought it may because "cargo" package but not it started with different error message.

Error message without "Cargo" package:

cd src/proxy
cargo build --release --manifest-path /tmp/efs-utils/src/proxy/Cargo.toml
./build-deb.sh: 33: cargo: not found_


Error message with "Cargo" package:

=> ERROR [ 9/30] RUN cd /tmp && git clone https://github.com/aws/efs-utils && cd /tmp/efs-utils/ && rm -rf ~/.cargo 95.1s

[ 9/30] RUN cd /tmp && git clone https://github.com/aws/efs-utils && cd /tmp/efs-utils/ && rm -rf ~/.cargo/registry/cache && export CARGO_LOG=trace && ./build-deb.sh && dpkg --force-all -i ./build/amazon-efs-utils*.deb && sed -i '/#region = us-east-1/s/^#//' /etc/amazon/efs/efs-utils.conf && sed -i 's/us-east-1/ap-east-1/g' /etc/amazon/efs/efs-utils.conf:
#0 0.249 Cloning into 'efs-utils'...
#0 2.431 + pwd
#0 2.434 + BASE_DIR=/tmp/efs-utils
#0 2.434 + BUILD_ROOT=/tmp/efs-utils/build/debbuild
#0 2.434 + VERSION=2.0.1
#0 2.434 + RELEASE=1
#0 2.434 + DEB_SYSTEM_RELEASE_PATH=/etc/os-release
#0 2.435 + echo 'Cleaning deb build workspace'
#0 2.435 Cleaning deb build workspace
#0 2.435 + rm -rf /tmp/efs-utils/build/debbuild
#0 2.446 + mkdir -p /tmp/efs-utils/build/debbuild
#0 2.457 Creating application directories
#0 2.457 + echo 'Creating application directories'
#0 2.457 + mkdir -p /tmp/efs-utils/build/debbuild/etc/amazon/efs
#0 2.467 + mkdir -p /tmp/efs-utils/build/debbuild/etc/init/
#0 2.477 + mkdir -p /tmp/efs-utils/build/debbuild/etc/systemd/system
#0 2.487 + mkdir -p /tmp/efs-utils/build/debbuild/sbin
#0 2.497 + mkdir -p /tmp/efs-utils/build/debbuild/usr/bin
#0 2.507 + mkdir -p /tmp/efs-utils/build/debbuild/var/log/amazon/efs
#0 2.517 + mkdir -p /tmp/efs-utils/build/debbuild/usr/share/man/man8
#0 2.528 Building efs-proxy
#0 2.528 + echo 'Building efs-proxy'
#0 2.528 + cd src/proxy
#0 2.528 + cargo build --release --manifest-path /tmp/efs-utils/src/proxy/Cargo.toml
#0 2.574 [2024-05-04T14:25:39Z TRACE cargo::util::config] get cv ConfigKey { env: "CARGO_ALIAS_BUILD", parts: [("alias", 5), ("build", 11)] }
#0 2.576 [2024-05-04T14:25:39Z TRACE cargo::util::config] get cv ConfigKey { env: "CARGO_TERM", parts: [("term", 5)] }
#0 2.577 [2024-05-04T14:25:39Z TRACE cargo::util::config] get cv ConfigKey { env: "CARGO_NET", parts: [("net", 5)] }
#0 2.578 [2024-05-04T14:25:39Z TRACE cargo::util::config] get cv ConfigKey { env: "CARGO_HTTP", parts: [("http", 5)] }
#0 2.645 [2024-05-04T14:25:39Z TRACE cargo::util::config] get cv ConfigKey { env: "CARGO_BUILD", parts: [("build", 5)] }
#0 2.648 [2024-05-04T14:25:39Z TRACE cargo::util::toml] read_manifest; path=/tmp/efs-utils/src/proxy/Cargo.toml; source-id=/tmp/efs-utils/src/proxy
#0 2.662 [2024-05-04T14:25:39Z TRACE cargo::util::config] get cv ConfigKey { env: "CARGO_REGISTRY_INDEX", parts: [("registry", 5), ("index", 14)] }
#0 2.669 [2024-05-04T14:25:39Z DEBUG cargo::core::workspace] find_root - trying /tmp/efs-utils/src/Cargo.toml
#0 2.669 [2024-05-04T14:25:39Z DEBUG cargo::core::workspace] find_root - trying /tmp/efs-utils/Cargo.toml
#0 2.670 [2024-05-04T14:25:39Z DEBUG cargo::core::workspace] find_root - trying /tmp/Cargo.toml
#0 2.670 [2024-05-04T14:25:39Z DEBUG cargo::core::workspace] find_root - trying /Cargo.toml
#0 2.670 [2024-05-04T14:25:39Z DEBUG cargo::core::workspace] find_members - only me as a member
#0 2.673 [2024-05-04T14:25:39Z TRACE cargo::util::config] get cv ConfigKey { env: "CARGO_TERM", parts: [("term", 5)] }
#0 2.675 [2024-05-04T14:25:39Z DEBUG cargo::util::rustc] failed to read rustc info cache: failed to read /tmp/efs-utils/src/proxy/target/.rustc_info.json
#0 2.675 [2024-05-04T14:25:39Z DEBUG cargo::util::rustc] rustc info cache miss
#0 2.676 [2024-05-04T14:25:39Z DEBUG cargo::util::rustc] running rustc -vV
#0 2.873 [2024-05-04T14:25:40Z TRACE cargo::util::config] get cv ConfigKey { env: "CARGO_TARGET_X86_64_ALPINE_LINUX_MUSL_RUSTFLAGS", parts: [("target", 5), ("x86_64-alpine-linux-musl", 12), ("rustflags", 37)] }
#0 2.874 [2024-05-04T14:25:40Z DEBUG cargo::util::rustc] rustc info cache miss
#0 2.875 [2024-05-04T14:25:40Z DEBUG cargo::util::rustc] running rustc - --crate-name ___ --print=file-names --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro -Csplit-debuginfo=packed
#0 3.566 [2024-05-04T14:25:40Z DEBUG cargo::util::rustc] rustc info cache miss
#0 3.566 [2024-05-04T14:25:40Z DEBUG cargo::util::rustc] running rustc - --crate-name ___ --print=file-names --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --force-warn=rust-2021-compatibility
#0 3.823 [2024-05-04T14:25:41Z DEBUG cargo::util::rustc] rustc info cache miss
#0 3.823 [2024-05-04T14:25:41Z DEBUG cargo::util::rustc] running rustc - --crate-name ___ --print=file-names --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=cfg
#0 4.072 [2024-05-04T14:25:41Z TRACE cargo::util::config] get cv ConfigKey { env: "CARGO_TARGET_X86_64_ALPINE_LINUX_MUSL_RUSTFLAGS", parts: [("target", 5), ("x86_64-alpine-linux-musl", 12), ("rustflags", 37)] }
#0 4.072 [2024-05-04T14:25:41Z TRACE cargo::util::config] get cv ConfigKey { env: "CARGO_TARGET", parts: [("target", 5)] }
#0 4.072 [2024-05-04T14:25:41Z DEBUG cargo::util::config::target] Got all targets {}
#0 4.073 [2024-05-04T14:25:41Z TRACE cargo::util::config] get cv ConfigKey { env: "CARGO_TARGET_X86_64_ALPINE_LINUX_MUSL_RUSTDOCFLAGS", parts: [("target", 5), ("x86_64-alpine-linux-musl", 12), ("rustdocflags", 37)] }
#0 4.073 [2024-05-04T14:25:41Z TRACE cargo::util::config] get cv ConfigKey { env: "CARGO_TARGET_X86_64_ALPINE_LINUX_MUSL_RUNNER", parts: [("target", 5), ("x86_64-alpine-linux-musl", 12), ("runner", 37)] }
#0 4.074 [2024-05-04T14:25:41Z TRACE cargo::util::config] get cv ConfigKey { env: "CARGO_TARGET_X86_64_ALPINE_LINUX_MUSL_RUSTFLAGS", parts: [("target", 5), ("x86_64-alpine-linux-musl", 12), ("rustflags", 37)] }
#0 4.074 [2024-05-04T14:25:41Z TRACE cargo::util::config] get cv ConfigKey { env: "CARGO_TARGET_X86_64_ALPINE_LINUX_MUSL_LINKER", parts: [("target", 5), ("x86_64-alpine-linux-musl", 12), ("linker", 37)] }
#0 4.074 [2024-05-04T14:25:41Z TRACE cargo::util::config] get cv ConfigKey { env: "CARGO_TARGET_X86_64_ALPINE_LINUX_MUSL", parts: [("target", 5), ("x86_64-alpine-linux-musl", 12)] }
#0 4.076 [2024-05-04T14:25:41Z TRACE cargo::util::config] get cv ConfigKey { env: "CARGO_TARGET_X86_64_ALPINE_LINUX_MUSL_RUNNER", parts: [("target", 5), ("x86_64-alpine-linux-musl", 12), ("runner", 37)] }
#0 4.076 [2024-05-04T14:25:41Z TRACE cargo::util::config] get cv ConfigKey { env: "CARGO_TARGET_X86_64_ALPINE_LINUX_MUSL_RUSTFLAGS", parts: [("target", 5), ("x86_64-alpine-linux-musl", 12), ("rustflags", 37)] }
#0 4.076 [2024-05-04T14:25:41Z TRACE cargo::util::config] get cv ConfigKey { env: "CARGO_TARGET_X86_64_ALPINE_LINUX_MUSL_LINKER", parts: [("target", 5), ("x86_64-alpine-linux-musl", 12), ("linker", 37)] }
#0 4.076 [2024-05-04T14:25:41Z TRACE cargo::util::config] get cv ConfigKey { env: "CARGO_TARGET_X86_64_ALPINE_LINUX_MUSL", parts: [("target", 5), ("x86_64-alpine-linux-musl", 12)] }
#0 4.078 [2024-05-04T14:25:41Z TRACE cargo::util::config] get cv ConfigKey { env: "CARGO_SOURCE", parts: [("source", 5)] }
#0 4.079 [2024-05-04T14:25:41Z TRACE cargo::util::config] get cv ConfigKey { env: "CARGO_PATCH", parts: [("patch", 5)] }
#0 4.080 [2024-05-04T14:25:41Z DEBUG cargo::ops::resolve] avoid_patch_ids={}
#0 4.080 [2024-05-04T14:25:41Z DEBUG cargo::core::registry] load/missing /tmp/efs-utils/src/proxy
#0 4.080 [2024-05-04T14:25:41Z DEBUG cargo::core::registry] loading source /tmp/efs-utils/src/proxy
#0 4.080 [2024-05-04T14:25:41Z DEBUG cargo::sources::config] loading: /tmp/efs-utils/src/proxy
#0 4.081 [2024-05-04T14:25:41Z TRACE cargo::core::source::source_id] loading SourceId; /tmp/efs-utils/src/proxy
#0 4.082 [2024-05-04T14:25:41Z TRACE cargo::ops::cargo_read_manifest] read_package; path=/tmp/efs-utils/src/proxy/Cargo.toml; source-id=/tmp/efs-utils/src/proxy
#0 4.082 [2024-05-04T14:25:41Z TRACE cargo::util::toml] read_manifest; path=/tmp/efs-utils/src/proxy/Cargo.toml; source-id=/tmp/efs-utils/src/proxy
#0 4.085 [2024-05-04T14:25:41Z TRACE cargo::core::registry] locking summary of efs-proxy v2.0.1 (/tmp/efs-utils/src/proxy)
#0 4.085 [2024-05-04T14:25:41Z TRACE cargo::core::registry] anyhow/^1.0.72/registry crates-io
#0 4.085 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.086 [2024-05-04T14:25:41Z TRACE cargo::core::registry] async-trait/^0.1/registry crates-io
#0 4.086 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.086 [2024-05-04T14:25:41Z TRACE cargo::core::registry] bytes/^1.4.0/registry crates-io
#0 4.086 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.086 [2024-05-04T14:25:41Z TRACE cargo::core::registry] chrono/^0.4/registry crates-io
#0 4.086 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.086 [2024-05-04T14:25:41Z TRACE cargo::core::registry] clap/=4.0.0/registry crates-io
#0 4.086 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.086 [2024-05-04T14:25:41Z TRACE cargo::core::registry] fern/^0.6/registry crates-io
#0 4.086 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.086 [2024-05-04T14:25:41Z TRACE cargo::core::registry] futures/^0.3/registry crates-io
#0 4.086 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.086 [2024-05-04T14:25:41Z TRACE cargo::core::registry] log/^0.4/registry crates-io
#0 4.086 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.086 [2024-05-04T14:25:41Z TRACE cargo::core::registry] log4rs/^0/registry crates-io
#0 4.086 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.086 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nix/^0.26.2/registry crates-io
#0 4.086 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.086 [2024-05-04T14:25:41Z TRACE cargo::core::registry] onc-rpc/^0.2.3/registry crates-io
#0 4.086 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] rand/^0.8.5/registry crates-io
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] s2n-tls/^0.0/registry crates-io
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] s2n-tls-sys/^0.0/registry crates-io
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] s2n-tls-tokio/^0.0/registry crates-io
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] serde/^1.0.175/registry crates-io
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] serde_ini/^0.2.0/registry crates-io
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] thiserror/^1.0.44/registry crates-io
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] tokio/^1.29.0/registry crates-io
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] tokio-util/^0.7.8/registry crates-io
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] uuid/^1.4.1/registry crates-io
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] xdr-codec/^0.4.4/registry crates-io
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] test-case/*/registry crates-io
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] tokio/^1.29.0/registry crates-io
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] xdrgen/^0.4.4/registry crates-io
#0 4.087 [2024-05-04T14:25:41Z TRACE cargo::core::registry] nope, unlocked
#0 4.090 [2024-05-04T14:25:41Z DEBUG cargo::core::resolver] initial activation: efs-proxy v2.0.1 (/tmp/efs-utils/src/proxy)
#0 4.091 [2024-05-04T14:25:41Z TRACE cargo::core::resolver] activating efs-proxy v2.0.1 (/tmp/efs-utils/src/proxy)
#0 4.093 [2024-05-04T14:25:41Z DEBUG cargo::core::registry] load/missing registry crates-io
#0 4.093 [2024-05-04T14:25:41Z DEBUG cargo::core::registry] loading source registry crates-io
#0 4.093 [2024-05-04T14:25:41Z DEBUG cargo::sources::config] loading: registry crates-io
#0 4.093 [2024-05-04T14:25:41Z TRACE cargo::core::source::source_id] loading SourceId; registry crates-io
#0 4.094 [2024-05-04T14:25:41Z DEBUG cargo::sources::registry::remote] updating the index
#0 4.097 [2024-05-04T14:25:41Z TRACE cargo::sources::registry::remote] acquiring registry index lock
#0 4.108 Updating crates.io index
#0 4.109 [2024-05-04T14:25:41Z DEBUG cargo::sources::git::utils] attempting GitHub fast path for https://api.github.com/repos/rust-lang/crates.io-index/commits/HEAD
#0 4.111 [2024-05-04T14:25:41Z DEBUG cargo::sources::git::utils] failed to check github reference 'refs/remotes/origin/HEAD' not found; class=Reference (4); code=NotFound (-3)
#0 4.112 [2024-05-04T14:25:41Z DEBUG cargo::sources::git::utils] skipping gc as there's only 0 pack files
#0 4.112 [2024-05-04T14:25:41Z DEBUG cargo::sources::git::utils] doing a fetch for https://github.com/rust-lang/crates.io-index
#0 4.113 [2024-05-04T14:25:41Z DEBUG cargo::sources::git::utils] initiating fetch of ["HEAD:refs/remotes/origin/HEAD"] from https://github.com/rust-lang/crates.io-index
#0 95.00 Killed


error: failed to solve: executor failed running [/bin/sh -c cd /tmp && git clone https://github.com/aws/efs-utils && cd /tmp/efs-utils/ && rm -rf ~/.cargo/registry/cache && export CARGO_LOG=trace && ./build-deb.sh && dpkg --force-all -i ./build/amazon-efs-utils*.deb && sed -i '/#region = us-east-1/s/^#//' /etc/amazon/efs/efs-utils.conf && sed -i 's/us-east-1/ap-east-1/g' /etc/amazon/efs/efs-utils.conf]: exit code: 137

@vtotare
Copy link
Author

vtotare commented May 6, 2024

Team,
Any update on this issue ?

@vtotare
Copy link
Author

vtotare commented May 8, 2024

Is this issue resolved ? How can I use this utility if its currently breaking

@vtotare
Copy link
Author

vtotare commented May 9, 2024

I think, this issue is appearing because of this message

#0 4.020 [2024-05-09T05:51:52Z DEBUG cargo::sources::git::utils] attempting GitHub fast path for https://api.github.com/repos/rust-lang/crates.io-index/commits/HEAD
#0 4.022 [2024-05-09T05:51:52Z DEBUG cargo::sources::git::utils] failed to check github reference 'refs/remotes/origin/HEAD' not found; class=Reference (4); code=NotFound (-3)

@saimucharla
Copy link

saimucharla commented Jun 4, 2024

was able to get around the issue by adding this before cloning the repo:

yum -y install openssl openssl-devel

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
cp -r /root/.cargo /home/<user>/
. "$HOME/.cargo/env"
yum install -y gcc gcc-c++ make autoconf automake libtool

@seanzatzdev-amazon
Copy link
Contributor

efs-utils does not officially support alpine https://github.com/aws/efs-utils?tab=readme-ov-file#efs-utils

Are you seeing this issue on any supported distributions?

@KatharinaSt
Copy link

I can confirm that this issue also happens on Ubuntu 20.04 (since at least April 15th) which actually is in the supported list.

It can be fixed by installing the following additional libraries (although it would be nice to be working "out of the box" again):
apt -y install cargo
apt -y install pkg-config
apt -y install libssl-dev

@tomalok
Copy link

tomalok commented Aug 6, 2024

@vtotare I'm the Alpine Linux package manager for efs-proxy - I suspect that we didn't yet have the #136 patch (or its predecessor) which we apply for building the APK, way back in Alpine 3.15, but it has been available for the more recent Alpine releases.

There is an issue with efs-proxy 1.35.x (I believe) in Alpine 3.20 that tends to fill up /var/log with Python deprecation warnings, and am such working on 2.0.3 in Alpine edge and have it successfully building. Once that's proven itself, I may backport it to 3.20.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants