You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Reading this file saved with snappy 1.1.2.6, and writing it with higher version results in compression ratio dropping from 2.05 to 1.26
Any snappy-version higher than 1.1.2.6 reproduced this issue.
The text was updated successfully, but these errors were encountered:
Does anyone know what changed to cause such a measurable change in compression ratio? I just tested 1.1.2.6 on one of my datasets and saw an immediate 20% savings.
When testing upgrade to spark 3.1.1 I've noticed the compression of repeated INT64 columns compression got worse.
https://stackoverflow.com/questions/67413589/parquet-compression-degradation-when-upgrading-spark/67455721#67455721
Reading this file saved with snappy 1.1.2.6, and writing it with higher version results in compression ratio dropping from 2.05 to 1.26
Any snappy-version higher than 1.1.2.6 reproduced this issue.
The text was updated successfully, but these errors were encountered: