Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert io.debezium.time.MicroTimestamp to DATETIME #305

Open
LarsKlingen opened this issue Apr 13, 2022 · 2 comments
Open

Convert io.debezium.time.MicroTimestamp to DATETIME #305

LarsKlingen opened this issue Apr 13, 2022 · 2 comments

Comments

@LarsKlingen
Copy link

For legacy reasons I would like to have DATETIME formau for columns that come as MicroTimestamp (We use Debezium for postgressql that converts timestamps to MicroTimestamp). The Sink automatically converts to TIMESTAMP format.
I tried creating the table first with type DATETIME for the columns to circumentvent this, but then I got the error:

Field updated_at has changed type from DATETIME to TIMESTAMP; See logs for more detail
	at com.wepay.kafka.connect.bigquery.write.batch.KCBQThreadPoolExecutor.maybeThrowEncounteredErrors(KCBQThreadPoolExecutor.java:108)
	at com.wepay.kafka.connect.bigquery.BigQuerySinkTask.put(BigQuerySinkTask.java:233)
	at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:582)
	... 10 more

Is there a way to save columns as DATETIME on BQ? I also considered using a view to convert to DATETIME, but then we lose the parittioning property on these columns. Thanks!

@saumyasuhagiya
Copy link

same question. @OneCricketeer

@LarsKlingen
Copy link
Author

This repo is deprecated, please take a look here: https://github.com/confluentinc/kafka-connect-bigquery
Opened the same issue there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants