Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]: The Quick Launch Kafka Cluster section in contributing.md does not work when creating a Kafka topic #2152

Closed
2 tasks done
yczhang-nv opened this issue Jan 29, 2025 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@yczhang-nv
Copy link
Contributor

Version

25.02

Which installation method(s) does this occur on?

No response

Describe the bug.

When setting up Kafka cluster for Morpheus examples by following Quick Launch Kafka Cluster section in contributing.md, the Kafka cluster is not able to create a topic because broker-list.sh does not return any bootstrap-server.

Minimum reproducible example

# follow Quick Launch Kafka Cluster from step 1-7, then in step 8.Create the topic
kafka-topics.sh --bootstrap-server ${BOOTSTRAP_SERVER} --create --topic ${MY_TOPIC}
# echo ${BOOTSTRAP_SERVER} is empty here, which should be setup by `broker-list.sh`

Relevant log output

Click here to see error details

[Paste the error here, it will be hidden by default]

Full env printout

Click here to see environment details

[Paste the results of print_env.sh here, it will be hidden by default]

Other/Misc.

No response

Code of Conduct

  • I agree to follow Morpheus' Code of Conduct
  • I have searched the open bugs and have found no duplicates for this bug report
@yczhang-nv yczhang-nv added the bug Something isn't working label Jan 29, 2025
@yczhang-nv yczhang-nv assigned yczhang-nv and unassigned yczhang-nv Jan 29, 2025
@dagardner-nv dagardner-nv added this to the 25.02 - Release milestone Jan 30, 2025
@yczhang-nv yczhang-nv self-assigned this Jan 30, 2025
rapids-bot bot pushed a commit that referenced this issue Jan 30, 2025
…2158)

This issue is caused by multiple version incompatibilities.

- The `DOCKER_API_VERSION` IN `docker-compose.yml` of `kafka-docker` is out of date. This prevents the kafka server from launching. Below is the error message from docker:
`Error response from daemon: {"message":"client version 1.22 is too old. Minimum supported API version is 1.24, please upgrade your client to a newer version"}`
 Thus the `DOCKER_API_VERSION` needs to be updated manually.

- After launching the `kafka` service, there are two exceptions being thrown when the LLM agent pipeline connecting to `kafka` service. According to https://stackoverflow.com/questions/78518146/docker-compose-fails-with-error-urllib3-exceptions-urlschemeunknown-not-suppor and google-deepmind/alphafold#867 (comment), those are caused by conda package version incompatibility of `requests` and `urllib3`. Those need to be pinned to a lower version to avoid the issue.

Closes #2152 

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Yuchen Zhang (https://github.com/yczhang-nv)

Approvers:
  - David Gardner (https://github.com/dagardner-nv)

URL: #2158
@github-project-automation github-project-automation bot moved this from Todo to Done in Morpheus Boards Jan 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: Done
Development

No branches or pull requests

2 participants