Add compression option to external log drain
Please consider adding a compress option to log drain form in environment settings panel.
Reference documentation here.
From my initial observation, compressed request bodies are unusual in HTTP traffic, but not impossible. When sending the request with compressed body, client must trust that the server is able to decompress the request body data. Server can decompress request body data based on Content-Encoding header sent by client, i.e.: Content-Encoding: gzip
Cycle agent pushes logs in a format, that is highly compressible (NDJSON). Client or, in Cycle case, Agent side compression may reduce network traffic for logs by 10x and more.
Example curl for compressed request body:
# compress json datagzip -9 -c body.json > body.gz# send compressed datacurl -v --location 'http://my.endpoint.example.com' --header 'Content-Type: text/plain' --header 'Content-Encoding: gzip' --data-binary @body.gzIf destination server does not support request decompression, apache httpd can do it with the following directives:
LoadModule deflate_module modules/mod_deflate.soSetInputFilter DEFLATEProxyPass "/" "http://upstream/"Thanks Jure!
Interesting concept here. I'll discuss with the team and see if this is something we can do. I've added the request officially on our internal system and will post updates here :)
Join the conversation
Sign in with your Cycle account to reply to this thread.