-
-
Notifications
You must be signed in to change notification settings - Fork 1
feat: Hive listener integration #605
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
I might have run into stackabletech/hdfs-operator#686 during development. I recognized that I can have an empty string in my discovery configMap from time to time. Behaviour appears to be flaky, but yet more often then not the emtpy string appears: Expected
Flaky faulty one
|
…tackabletech/hive-operator into feat/hive-listener-integration
🟢
|
chroot: Option<&str>, | ||
listener_refs: BTreeMap<&String, Listener>, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should explicitly mention this in the release notes / breaking changes that the discovery key changed. Or we keep the HIVE
key for backwards compatibility?
Trino wont work anymore etc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are pro and cons about it:
Each rolegroup has it's service and thus it's own entry in the configmap.
like
apiVersion: v1
data:
cluster-internal: thrift://hive-postgres-s3-metastore-cluster-internal.default.svc.cluster.local:9083
default: thrift://172.18.0.2:32658
external-unstable: thrift://172.18.0.2:30607
kind: ConfigMap
metadata:
creationTimestamp: "2025-06-04T13:39:09Z"
labels:
app.kubernetes.io/component: metastore
I like it better as you can decide which hive rolegroup you want to talk to.
However I understand the argument being backwards compatible. I could offer to make it upper case. Keeping compatibility would then mean to change the rg from default to hive to have the same picture as before?
I just don't like
apiVersion: v1
data:
HIVE: thrift://hive-postgres-s3-metastore-cluster-internal.default.svc.cluster.local:9083
thrift://172.18.0.2:32658
thrift://172.18.0.2:30607
kind: ConfigMap
metadata:
creationTimestamp: "2025-06-04T13:39:09Z"
labels:
app.kubernetes.io/component: metastore
as it can't be distinguished anymore if every rolegroup uses NodePort
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The problem is that "HIVE" is hardcoded/expected by the trino operator, using different rolegroups will cause problems (or at least not completely utilize the cluster) as its implemented currently. Will hit that up on Slack.
Co-authored-by: Malte Sander <[email protected]>
…tackabletech/hive-operator into feat/hive-listener-integration
Co-authored-by: Malte Sander <[email protected]>
…tackabletech/hive-operator into feat/hive-listener-integration
Description
Adds listener Support
Definition of Done Checklist
Author
Reviewer
Acceptance