fix(gcp sink): Provide an option to override content encoding for GCP Cloud Storage Sink

[hieu-db]

Overview

This PR provides an option to override content encoding for objects uploaded to GCP Cloud Storage.
This breaks piping the uploaded objects to downstream systems such as Hadoop Filesystems, which don't work well with ".gzip" file with "content-encoding" header enabled.

Refer to this bug for further details

Test Plan

Added unit tests

$ rustfmt src/sinks/gcp/cloud_storage.rs
$ cargo test --color=always  --package vector --lib sinks::gcp::cloud_storage
$ cargo build

Canaried the change (w/o content_encoding == "") to Vector Aggregator and made sure files can still be uploaded to GCS with content_encoding header populated.

Canaried the change (with content_encoding == "") to Vector Aggregator and verified that files no longer have content_encoding header populated.

Canaried the change (with set_content_encoding == "") to staging Vector Aggregator in GCP and verified that Shadow Service Request Logs are now available on Lumberjack:

[flaviofcruz]

Have you looked at how S3 allows configuring content_encoding? They have a content_encoding option [1] which can override the content encoding derived from the compression method. I would guess this would probably have much higher changes of being accepted upstream

[1] https://github.com/vectordotdev/vector/blob/master/src/sinks/s3_common/config.rs#L118
[2] https://github.com/vectordotdev/vector/blob/master/src/sinks/s3_common/service.rs#L106

[hieu-db]

I thought about that. However, we want to avoid setting content_encoding altogether, not overriding it with a different value than the one determined by the compression method (gzip).

I can try providing a different content_encoding value ("" or something invalid) and see if Lumberjack can still process these logs.

[hieu-db]

@flaviofcruz I fixed the PR to provide a content_encoding option similar to the S3 sink.
Things worked on Vector Aggregator when content_encoding is set to an empty string.