Encoder interface #194
Encoder interface #194
Conversation
fabiojmendes
force-pushed
the
encoder_interface
branch
from
f82ce28 to
db491d8
Compare
| } | ||
|
|
||
| func keyEncoder() codec.Encoder { | ||
| if keyProtoType != "" { |
There was a problem hiding this comment.
this might be the time for a new flag edit: (it's actually not a breaking change)
We could add:
--key-codec auto|proto|avro-json|avro
auto would be default, and behave as-is. it would try its best to decide what do do: if proto key/value flags are given, it will try proto. if avro flags are given, it will try avro-json. but it can be set to specifically pick avro (see #195)
There was a problem hiding this comment.
I'm not sure about this one, for proto I think it would be a little redundant because you have to specify --key-proto-type anyways. I guess we don't have the same problem we have with avro here.
Maybe a flag specific for avro would be enough. It also doesn't need to be specific to keys, It can apply for both keys and values at the same time. Something like:
value | kaf produce --avro-key-schema-id x -- avro-schema-id y --key 'key' --avro-input [json|avro-json]
What do you think?
There's a separate issue with avro and proto though. The current behaviour is a little misleading imo. If you specify both proto and avro, proto takes precedence. Maybe we should have some validation and throw an error in this case.
There was a problem hiding this comment.
it's ok for me. do you know if "avro-json" is the right term? is there any "better" term in the avro world for that?
There was a problem hiding this comment.
Probably not, I'll check their docs see if I find anything. Another option would be just a boolean flag like --avro-strict then the docs can say something regarding being strict about unions.
A wiki page with examples of what we've been talking would be nice too.
|
Speaking of tests, I faced a very weird issue when trying to add another produce/consume test. Take a look on this commit, pretty straight forward. But the tests fails. If I remove the timeout context it will fix it, but it is still weird. It looks like the cleanup between tests runs is not working properly. Have you faced this as well? |
can't say too much about this, sorry |
fabiojmendes
force-pushed
the
encoder_interface
branch
from
db491d8 to
6b57272
Compare
|
Hi Team, Kindly resolve this conflict as this is causing production level issues in benthos plugin. @fabiojmendes Thanks |
This is pretty much WIP but I think we can start the discussion.
On my TODO I have: