Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(env): cargo:rerun-if-env-changed doesn't work with env configuration #14027

Closed
wants to merge 4 commits into from
Closed
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions src/cargo/core/compiler/build_context/target_info.rs
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ use cargo_util::{paths, ProcessBuilder};
use serde::{Deserialize, Serialize};
use std::cell::RefCell;
use std::collections::hash_map::{Entry, HashMap};
use std::collections::BTreeMap;
use std::ffi::OsString;
use std::path::{Path, PathBuf};
use std::str::{self, FromStr};

Expand Down Expand Up @@ -588,6 +590,10 @@ impl TargetInfo {
.iter()
.any(|sup| sup.as_str() == split.as_str())
}

pub fn get_target_envs(&self) -> CargoResult<&BTreeMap<String, Option<OsString>>> {
return Ok(self.crate_type_process.get_envs());
}
}
Comment on lines +594 to 597
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels very brittle because any change to how we pass in environment variables automatically gets used as fingerprinting, rather than intentionally choosing what we fingerprint. Just to check to see if this is correct is requiring checking several other places.

In fact, I think this will cause every build to rebuild everything when using jobserver because I think we are capturing the env after that is configured and that env includes file descriptors.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was overlooking that we aren't fingerprinting all of it, only what the user requests. So maybe its not all that of a problem?


/// Takes rustc output (using specialized command line args), and calculates the file prefix and
Expand Down
34 changes: 30 additions & 4 deletions src/cargo/core/compiler/fingerprint/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -357,7 +357,9 @@ mod dirty_reason;

use std::collections::hash_map::{Entry, HashMap};

use std::collections::BTreeMap;
use std::env;
use std::ffi::OsString;
use std::hash::{self, Hash, Hasher};
use std::io;
use std::path::{Path, PathBuf};
Expand Down Expand Up @@ -772,10 +774,21 @@ impl LocalFingerprint {
// TODO: This is allowed at this moment. Should figure out if it makes
// sense if permitting to read env from the config system.
#[allow(clippy::disallowed_methods)]
fn from_env<K: AsRef<str>>(key: K) -> LocalFingerprint {
fn from_env<K: AsRef<str>>(
key: K,
envs: &BTreeMap<String, Option<OsString>>,
) -> LocalFingerprint {
let key = key.as_ref();
let var = key.to_owned();
let val = env::var(key).ok();

let val = envs
.get(key)
.and_then(|opt| {
opt.as_ref()
.and_then(|os_str| os_str.clone().into_string().ok())
})
.or_else(|| env::var(key).ok());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find it strange to fallback to env::var if the envs[key] is non-UTF8. Maybe we should call env::var_os and deal with into_string at the end.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest, I didn't consider using env::var_os instead of env::var because I wanted to minimize the impact of the change and avoid affecting other behaviors. If there is an up-to-date value in the current environment variable, then use envs.get(key); otherwise the previous env::var(key) is still used.

It looks like the only difference between env::var and env::var_os is whether the environment variable is a valid Unicode.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

iiuc on Windows, env::var calls GetEnvironmentVariableW which is case insensitive. Since we're recreating our own environment look up, should this matter?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a more in-depth study of windows, I'm very sorry.

Do you mean that you shouldn't use env::var or env::var_os to query environment variables? That doesn't seem to work : see: rerun_if_env_changes and rerun_if_env_or_file_changes


LocalFingerprint::RerunIfEnvChanged { var, val }
}

Expand Down Expand Up @@ -1608,6 +1621,13 @@ fn build_script_local_fingerprints(
bool,
) {
assert!(unit.mode.is_run_custom_build());
let envs = build_runner
.bcx
.target_data
.info(unit.kind)
.get_target_envs()
.unwrap()
.clone();
// First up, if this build script is entirely overridden, then we just
// return the hash of what we overrode it with. This is the easy case!
if let Some(fingerprint) = build_script_override_fingerprint(build_runner, unit) {
Expand Down Expand Up @@ -1660,7 +1680,12 @@ fn build_script_local_fingerprints(
// Ok so now we're in "new mode" where we can have files listed as
// dependencies as well as env vars listed as dependencies. Process
// them all here.
Ok(Some(local_fingerprints_deps(deps, &target_dir, &pkg_root)))
Ok(Some(local_fingerprints_deps(
deps,
&target_dir,
&pkg_root,
&envs,
)))
};

// Note that `false` == "not overridden"
Expand Down Expand Up @@ -1695,6 +1720,7 @@ fn local_fingerprints_deps(
deps: &BuildDeps,
target_root: &Path,
pkg_root: &Path,
envs: &BTreeMap<String, Option<OsString>>,
) -> Vec<LocalFingerprint> {
debug!("new local fingerprints deps {:?}", pkg_root);
let mut local = Vec::new();
Expand All @@ -1719,7 +1745,7 @@ fn local_fingerprints_deps(
local.extend(
deps.rerun_if_env_changed
.iter()
.map(LocalFingerprint::from_env),
.map(|v| LocalFingerprint::from_env(v, &envs)),
);

local
Expand Down
47 changes: 47 additions & 0 deletions tests/testsuite/build_script_env.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,53 @@ use cargo_test_support::basic_manifest;
use cargo_test_support::project;
use cargo_test_support::sleep_ms;

#[cargo_test]
fn rerun_if_env_changes_config() {
let p = project()
.file("Cargo.toml", &basic_manifest("foo", "0.1.0"))
.file("src/main.rs", "fn main() {}")
.file(
".cargo/config.toml",
r#"
[env]
FOO = "good"
"#,
)
.file(
"build.rs",
r#"
fn main() {
println!("cargo:rerun-if-env-changed=FOO");
if let Ok(foo) = std::env::var("FOO") {
assert!(&foo != "bad");
}
}
"#,
)
.build();

p.cargo("check")
.with_stderr(
"\
[COMPILING] foo v0.1.0 ([..])
[FINISHED] [..]",
)
.run();

p.change_file(
".cargo/config.toml",
r#"
[env]
FOO = "bad"
"#,
);

p.cargo("check")
.with_status(101)
.with_stderr_contains("[ERROR] failed to run custom build command for `foo v0.1.0 ([..])`")
.run();
}

#[cargo_test]
fn rerun_if_env_changes() {
let p = project()
Expand Down