feature-request

Add compression option to external log drain

Please consider adding a compress option to log drain form in environment settings panel.

Reference documentation here.

From my initial observation, compressed request bodies are unusual in HTTP traffic, but not impossible. When sending the request with compressed body, client must trust that the server is able to decompress the request body data. Server can decompress request body data based on Content-Encoding header sent by client, i.e.: Content-Encoding: gzip

Cycle agent pushes logs in a format, that is highly compressible (NDJSON). Client or, in Cycle case, Agent side compression may reduce network traffic for logs by 10x and more.

Example curl for compressed request body:

# compress json data
gzip -9 -c body.json > body.gz
# send compressed data
curl -v --location 'http://my.endpoint.example.com' --header 'Content-Type: text/plain' --header 'Content-Encoding: gzip' --data-binary @body.gz

If destination server does not support request decompression, apache httpd can do it with the following directives:

LoadModule deflate_module modules/mod_deflate.so
SetInputFilter DEFLATE
ProxyPass "/" "http://upstream/"
avatar
1
  • Thanks Jure!

    Interesting concept here. I'll discuss with the team and see if this is something we can do. I've added the request officially on our internal system and will post updates here :)

    avatar
    platform
v2025.08.04.01 © 2024 Petrichor Holdings, Inc.

We use cookies to enhance your experience. You can manage your preferences below.