Skip to content

"Cannot drop a runtime in a context where blocking is not allowed" panic in the blocking Client #1017

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
yandexx opened this issue Aug 25, 2020 · 10 comments

Comments

@yandexx
Copy link

yandexx commented Aug 25, 2020

Hi!

I was working in async context with the blocking reqwest Client with tokio runtime, and it panics every time. Now, I don't know tokio internals so I can't tell whether this is a reqwest or a tokio issue.

This is not critical and I will simply switch to the async client, however I want to report this.

This happens on Windows 10 and on WSL 2. Here's a minified example.

[dependencies]
reqwest = { version = "0.10.7", features = ["blocking"] }
tokio = { version = "0.2.22", features = ["macros"] }
use reqwest::blocking::Client;

#[tokio::main]
async fn main() {
    let client: Client = Client::builder().build().unwrap();
    client.get("http://rust-lang.org/").send().unwrap();
}
c:\rust\test-reqwest>cargo run
   Compiling test-reqwest v0.1.0 (C:\rust\test-reqwest)
    Finished dev [unoptimized + debuginfo] target(s) in 1.74s
     Running `target\debug\test-reqwest.exe`
thread 'main' panicked at 'Cannot drop a runtime in a context where blocking is not allowed. This happens when a runtime is dropped from within an asynchronous context.', C:\Users\vzubarev\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-0.2.22\src\runtime\blocking\shutdown.rs:49:21
stack backtrace:
   0: backtrace::backtrace::trace_unsynchronized
             at C:\Users\VssAdministrator\.cargo\registry\src\github.com-1ecc6299db9ec823\backtrace-0.3.46\src\backtrace\mod.rs:66
   1: std::sys_common::backtrace::_print_fmt
             at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54\/src\libstd\sys_common\backtrace.rs:78
   2: std::sys_common::backtrace::_print::{{impl}}::fmt
             at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54\/src\libstd\sys_common\backtrace.rs:59
   3: core::fmt::write
             at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54\/src\libcore\fmt\mod.rs:1076
   4: std::io::Write::write_fmt<std::sys::windows::stdio::Stderr>
             at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54\/src\libstd\io\mod.rs:1537
   5: std::sys_common::backtrace::_print
             at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54\/src\libstd\sys_common\backtrace.rs:62
   6: std::sys_common::backtrace::print
             at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54\/src\libstd\sys_common\backtrace.rs:49
   7: std::panicking::default_hook::{{closure}}
             at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54\/src\libstd\panicking.rs:198
   8: std::panicking::default_hook
             at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54\/src\libstd\panicking.rs:218
   9: std::panicking::rust_panic_with_hook
             at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54\/src\libstd\panicking.rs:486
  10: std::panicking::begin_panic<str*>
             at C:\Users\vzubarev\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\src\libstd\panicking.rs:410
  11: tokio::runtime::blocking::shutdown::Receiver::wait
             at C:\Users\vzubarev\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-0.2.22\src\runtime\blocking\shutdown.rs:49
  12: tokio::runtime::blocking::pool::BlockingPool::shutdown
             at C:\Users\vzubarev\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-0.2.22\src\runtime\blocking\pool.rs:132
  13: tokio::runtime::blocking::pool::{{impl}}::drop
             at C:\Users\vzubarev\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-0.2.22\src\runtime\blocking\pool.rs:138
  14: core::ptr::drop_in_place<tokio::runtime::blocking::pool::BlockingPool>
             at C:\Users\vzubarev\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\src\libcore\ptr\mod.rs:184
  15: core::ptr::drop_in_place<tokio::runtime::Runtime>
             at C:\Users\vzubarev\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\src\libcore\ptr\mod.rs:184
  16: reqwest::blocking::wait::enter
             at C:\Users\vzubarev\.cargo\registry\src\github.com-1ecc6299db9ec823\reqwest-0.10.7\src\blocking\wait.rs:74  17: reqwest::blocking::wait::timeout<tokio::sync::oneshot::Receiver<core::result::Result<(), reqwest::error::Error>>,core::result::Result<(), reqwest::error::Error>,tokio::sync::oneshot::error::RecvError>
             at C:\Users\vzubarev\.cargo\registry\src\github.com-1ecc6299db9ec823\reqwest-0.10.7\src\blocking\wait.rs:14  18: reqwest::blocking::client::ClientHandle::new
             at C:\Users\vzubarev\.cargo\registry\src\github.com-1ecc6299db9ec823\reqwest-0.10.7\src\blocking\client.rs:767
  19: reqwest::blocking::client::ClientBuilder::build
             at C:\Users\vzubarev\.cargo\registry\src\github.com-1ecc6299db9ec823\reqwest-0.10.7\src\blocking\client.rs:95
  20: test_reqwest::main::{{closure}}
             at .\src\main.rs:5
  21: core::future::from_generator::{{impl}}::poll<generator-0>
             at C:\Users\vzubarev\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\src\libcore\future\mod.rs:73
  22: tokio::runtime::enter::{{impl}}::block_on::{{closure}}<core::future::from_generator::GenFuture<generator-0>>
             at C:\Users\vzubarev\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-0.2.22\src\runtime\enter.rs:160
  23: tokio::coop::with_budget::{{closure}}<core::task::poll::Poll<()>,closure-0>
             at C:\Users\vzubarev\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-0.2.22\src\coop.rs:127
  24: std::thread::local::LocalKey<core::cell::Cell<tokio::coop::Budget>>::try_with<core::cell::Cell<tokio::coop::Budget>,closure-0,core::task::poll::Poll<()>>
             at C:\Users\vzubarev\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\src\libstd\thread\local.rs:263
  25: std::thread::local::LocalKey<core::cell::Cell<tokio::coop::Budget>>::with<core::cell::Cell<tokio::coop::Budget>,closure-0,core::task::poll::Poll<()>>
             at C:\Users\vzubarev\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\src\libstd\thread\local.rs:239
  26: tokio::coop::with_budget
             at C:\Users\vzubarev\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-0.2.22\src\coop.rs:120
  27: tokio::coop::budget
             at C:\Users\vzubarev\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-0.2.22\src\coop.rs:96
  28: tokio::runtime::enter::Enter::block_on<core::future::from_generator::GenFuture<generator-0>>
             at C:\Users\vzubarev\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-0.2.22\src\runtime\enter.rs:160
  29: tokio::runtime::thread_pool::ThreadPool::block_on<core::future::from_generator::GenFuture<generator-0>>
             at C:\Users\vzubarev\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-0.2.22\src\runtime\thread_pool\mod.rs:82
  30: tokio::runtime::{{impl}}::block_on::{{closure}}<core::future::from_generator::GenFuture<generator-0>>
             at C:\Users\vzubarev\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-0.2.22\src\runtime\mod.rs:446
  31: tokio::runtime::context::enter<closure-0,()>
             at C:\Users\vzubarev\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-0.2.22\src\runtime\context.rs:72  32: tokio::runtime::handle::Handle::enter<closure-0,()>
             at C:\Users\vzubarev\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-0.2.22\src\runtime\handle.rs:76
  33: tokio::runtime::Runtime::block_on<core::future::from_generator::GenFuture<generator-0>>
             at C:\Users\vzubarev\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-0.2.22\src\runtime\mod.rs:441
  34: test_reqwest::main
             at .\src\main.rs:3
  35: std::rt::lang_start::{{closure}}<()>
             at C:\Users\vzubarev\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\src\libstd\rt.rs:67
  36: std::rt::lang_start_internal::{{closure}}
             at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54\/src\libstd\rt.rs:52
  37: std::panicking::try::do_call
             at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54\/src\libstd\panicking.rs:297
  38: std::panicking::try
             at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54\/src\libstd\panicking.rs:274
  39: std::panic::catch_unwind
             at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54\/src\libstd\panic.rs:394
  40: std::rt::lang_start_internal
             at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54\/src\libstd\rt.rs:51
  41: std::rt::lang_start<()>
             at C:\Users\vzubarev\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\src\libstd\rt.rs:67
  42: main
  43: invoke_main
             at d:\A01\_work\6\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
  44: __scrt_common_main_seh
             at d:\A01\_work\6\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
  45: BaseThreadInitThunk
  46: RtlUserThreadStart
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
error: process didn't exit successfully: `target\debug\test-reqwest.exe` (exit code: 101)

Thank you.

@seanmonstar
Copy link
Owner

This is expected (though the panic message used to be a little different and perhaps more helpful...).

@wchargin
Copy link
Contributor

wchargin commented Feb 5, 2021

Hi! I’m also hitting this. Is there a recommended workaround?

Of course, if you know that you’re in an async context, it makes sense
to use the async client instead. But if you’re writing synchronous code
that implements a non-async trait, shouldn’t it be valid to use the
blocking API, even if a downstream consumer eventually happens to use
that in an async context? For a contrived example, suppose that your
impl Display for SomeType needs to make a network request…

It seems like a particularly surprising failure of compositionality for
the API to simply fail depending on properties of transitive callers.
Adding to the confusion, this appears to only fail in debug mode, and
I can’t find the behavior documented anywhere.

wchargin added a commit to tensorflow/tensorboard that referenced this issue Feb 5, 2021
wchargin-branch: rust-gcs-logdir
wchargin-source: f8b766436143e6faa603ca551c0ceb8813ea829c
@seanmonstar
Copy link
Owner

If the user is in async function, and they are going to be doing something blocking, they should probably do something like spawn_blocking(do_the_thing).await, so that it happens in a dedicated thread that is allowed to block.

Note that it being in debug mode is mostly because this is meant to help you when developing your app, hopefully local or unit testing will alert you to fix it. We hope to eventually be able to add a lint to the compiler to help detect this, see this goal: rust-lang/wg-async#19

@wchargin
Copy link
Contributor

wchargin commented Feb 5, 2021

Thanks for the quick response! This is helpful. It still seems strange
to me that the async function should need to know the implementation
details of all its transitive dependencies to know if they need to be
“allowed to block”. But in my case I am indeed offloading to another
thread anyway, so it suffices to just move the client initialization.
And the linked issue is helpful for general context.

If you don’t mind, I might send a PR to reqwest::blocking docs to
mention this (if you do mind, you can reject the PR :-) ). Thanks for
your work on reqwest, et al.—other than this issue, it’s worked great
for our purposes.

wchargin added a commit to wchargin/reqwest that referenced this issue Feb 5, 2021
See discussion on seanmonstar#1017. This patch adds documentation to `blocking` at
module level and to its `Client::new` and `ClientBuilder::build`, noting
that you can’t create or use a blocking client from within an async
runtime, and suggesting potential alternatives.

Presumably, all the other methods on `Client` also have this property,
but hitting the failure mode would require building a `blocking::Client`
outside an async runtime and then moving it into a runtime to send
requests; seems potentially not worth polluting all the docs.

Test Plan:
Ran `cargo doc --features blocking` and verified that the links work.

wchargin-branch: docs-blocking-no-async
wchargin-source: 0eb36352959cd2ca0b19df5836e75230dc619b9d
seanmonstar pushed a commit that referenced this issue Feb 5, 2021
See discussion on #1017. This patch adds documentation to `blocking` at
module level and to its `Client::new` and `ClientBuilder::build`, noting
that you can’t create or use a blocking client from within an async
runtime, and suggesting potential alternatives.

Presumably, all the other methods on `Client` also have this property,
but hitting the failure mode would require building a `blocking::Client`
outside an async runtime and then moving it into a runtime to send
requests; seems potentially not worth polluting all the docs.

Test Plan:
Ran `cargo doc --features blocking` and verified that the links work.

wchargin-branch: docs-blocking-no-async
wchargin-source: 0eb36352959cd2ca0b19df5836e75230dc619b9d
@tobz1000
Copy link

This is a problem when used with lazy_static. I have a static whose initialisation uses reqwest::blocking::get. I may want to use this static value in both sync and async code. The one-time thread delay is acceptable if it happens to initialise within async code.

Is is possible to use the current reqwest::blocking implementation to initialise a static in this way?

@nordken
Copy link

nordken commented Sep 1, 2021

Hi @tobz1000, I also have the same problem when using reqwest::blocking inside lazy_static. Do you have the workaround for that, or you still got the issue?

@MathisPeyronne
Copy link

I solved my problem by using tokio::task::spawn_blocking. My case is just downloading an image so the code looks like: (cf. second line)

let mut file = std::fs::File::create("image.png").unwrap();
let result = spawn_blocking(move || {reqwest::blocking::get("your_image_address.com/image.jpg").unwrap().copy_to(&mut file).unwrap()}).await;

Hope it is smth similar to your problem, or to anyone googling this :)

@ta32
Copy link

ta32 commented Jun 16, 2022

I am using Rocket with blocking routes ( non async functions )

When I use the blocking client inside one of these endpoints it panics. ( release and debug ) . This panic started after upgrading from reqwest 0.10.X to 0.11.X
work arounds:

#[macro_use] extern crate rocket;

use rocket::{Build, Rocket, tokio};

#[get("/async")]
async fn index_async() -> &'static str {
    let handle = tokio::task::spawn_blocking(|| {
        let client = reqwest::blocking::Client::new();
        "Hello, world! Async"
    });
    let res = handle.await.unwrap();
    res
}


#[get("/")]
fn index() -> &'static str {
    let res = tokio::task::block_in_place(|| {
        let client = reqwest::blocking::Client::new();
        "Hello, world!"
    });
    // using the blocking client here will panic
    // reqwest::blocking::Client::new() 
    res
}

#[launch]
fn rocket() -> Rocket<Build> {
    rocket::build().mount("/", routes![index, index_async])
}

These work arounds seem to work with rocket for async and blocking endpoints.

Using the blocking client in a blocking endpoint directly will cause a panic in reqwest version 0.11.X ( not sure why though...)

@szymek156
Copy link

szymek156 commented Jun 21, 2022

Using the blocking client in a blocking endpoint directly will cause a panic in reqwest version 0.11.X ( not sure why though...)
I face the same issue as @ta32, but in warp.
Since I am using blocking client in non-async function, I cannot do any await there.

@SpaceMonkeyForever
Copy link

@ta32 thanks for the tip. This still happens in "0.11.14", I had to downgrade to 0.10.x to get rid of this error when calling from within sync code

SpaceMonkeyForever added a commit to SpaceMonkeyForever/binance-rs that referenced this issue Mar 14, 2023
Allstreamer added a commit to Allstreamer/alltrader that referenced this issue May 19, 2023
- parse_json now async
- download_file now async
- fixed crash in download_file where tokio runtime is killed (seanmonstar/reqwest#1017)
- make backend use one large async block instead of many small ones, allowing for actual async instead of just blocking

- Fix issue where GUI would block waiting for backend
- (Related to above) now using tokio::sync::mutex instead of std::sync::mutex
- New ContinueLock Macro
Allstreamer added a commit to Allstreamer/alltrader that referenced this issue May 19, 2023
- parse_json now async
- download_file now async
- fixed crash in download_file where tokio runtime is killed (seanmonstar/reqwest#1017)
- make backend use one large async block instead of many small ones, allowing for actual async instead of just blocking
- Remove old dependencies

- Fix issue where GUI would block waiting for backend
- (Related to above) now using tokio::sync::mutex instead of std::sync::mutex
- New ContinueLock Macro
Nutomic pushed a commit to Nutomic/reqwest that referenced this issue Nov 7, 2024
See discussion on seanmonstar#1017. This patch adds documentation to `blocking` at
module level and to its `Client::new` and `ClientBuilder::build`, noting
that you can’t create or use a blocking client from within an async
runtime, and suggesting potential alternatives.

Presumably, all the other methods on `Client` also have this property,
but hitting the failure mode would require building a `blocking::Client`
outside an async runtime and then moving it into a runtime to send
requests; seems potentially not worth polluting all the docs.

Test Plan:
Ran `cargo doc --features blocking` and verified that the links work.

wchargin-branch: docs-blocking-no-async
wchargin-source: 0eb36352959cd2ca0b19df5836e75230dc619b9d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

9 participants