Skip to content

Upgrade

Jump to bottom
Jachen Duschletta edited this page Apr 8, 2022 · 9 revisions


Codefresh build status CodeQL

cLoki

Major Upgrades

2.x to 3.x

Version 3 introduces performance improvements via a new data schema. cLoki will create new sample tables for fresh data. Previous data will remain accessible read-only through the older tables. We recommend (if you can) to migrate your data using below process. You need to start your cloki with the new version installed to create the necessary tables for the migration to succeed.

It is not recommended to update directly from 1.X to 3.X, but rather update data incrementally.

Index all the fingerprints to time_series_gin 

INSERT INTO time_series_gin (date, key, val, fingerprint) 
SELECT date, pairs.1 as key, pairs.2 as val, fingerprint FROM time_series ARRAY JOIN JSONExtractKeysAndValues(time_series.labels, 'String') as pairs

Copy the previous data to samples_v4

INSERT INTO samples_v4 (fingerprint, labels, timestamp_ns, value, string) SELECT
    samples_read_v2_2.fingerprint AS fingerprint,
    JSONExtractKeysAndValues(time_series.labels, 'String') AS labels,
    timestamp_ns,
    value,
    string
FROM samples_read_v2_2
LEFT ANY JOIN time_series ON time_series.fingerprint = samples_read_v2_2.fingerprint

Drop table samples_v3 

DROP TABLE samples_v3

1.x to 2.x

Version 2 introduces breaking changes to the samples table. CLoki will create a new samples table to support the new schema, however data that has previously been written will not be available via the API until the data is updated.

To migrate the data after cloki v2.x has been installed and run, please follow below steps:

  • Log into clickhouse via it's client cli
clickhouse-client --host 127.0.0.1 --user=default --password=yourClickhousePassword
  • Navigate to your cloki database, via the USE command (default database name is cloki)
USE cloki;
  • Migrate past data from samples_v2 and samples tables with the below commands
INSERT INTO samples_v3 (fingerprint, timestamp_ns, value, string) SELECT fingerprint, timestamp_ms * 1000000 as timestamp_ms, value, string from samples_v2
INSERT INTO samples_v3 (fingerprint, timestamp_ns, value, string) SELECT fingerprint, timestamp_ms * 1000000 as timestamp_ms, value, string from samples
  • After successful completion of the INSERT into samples_v3, you delete the old tables via
DROP TABLE samples
DROP TABLE samples_v2
Clone this wiki locally