Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Infinite output of ETCD logs, causing service crashes #3823

Open
bCoder778 opened this issue Dec 29, 2023 · 5 comments
Open

Infinite output of ETCD logs, causing service crashes #3823

bCoder778 opened this issue Dec 29, 2023 · 5 comments

Comments

@bCoder778
Copy link

Describe the bug
A large number of identical error logs without any pauses result in 100% CPU usage. The log files exceeded a few gigabytes in a short amount of time, causing the server to crash.

To Reproduce

The program started working normally, but after running for a period of time, it began to output the same logs without restrictions, causing the server to crash

  1. The code is

Etcd:
Hosts:
- ...:379
Key: *.rpc
User: user
Pass: *

  1. The error is

"etcd monitor chan has been canceled, error: rpc error: code = Unauthenticated desc = etcdserver: invalid auth token","level":"error"}

Environments (please complete the following information):

  • etcd:docker
  • go-zero version v1.6.1
@codeErrorSleep
Copy link
Contributor

We have also encountered this problem. You can try to downgrade the version of gRPC a bit; that's how we resolved it.

@bCoder778
Copy link
Author

We have also encountered this problem. You can try to downgrade the version of gRPC a bit; that's how we resolved it.

Thank you very much. Can you tell me which specific version of GRPC you are using

@arllen133
Copy link

Has it been resolved yet?

@iyyzh
Copy link
Contributor

iyyzh commented Sep 26, 2024

mark

@dingdinglz
Copy link

We have also encountered this problem. You can try to downgrade the version of gRPC a bit; that's how we resolved it.

Hello, I met this problem as well , you mean the grpc version is the version in go mod ? Thank you .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants