Answer by jwitt · Oct 25, 2015 at 05:23 PM
Site-to-Site uses deflate at level 1 and compresses data in blocks/buffers. With site-to-site a series of 1..N flowfiles are sent at once and ack'd as a group. It is not configurable at this time. Keep in mind of course you can of course compress before sending to s2s and decompress after receiving from s2s using the CompressContent processor.
Do you feel there would be a good bit of value in letting the compression of s2s be configurable? If so would that be for cases like where snappy makes sense because it is certain types of text data?
Thanks
Joe
Yes, Joe, I had something like snappy in my mind as a good middle ground between size and performance.
As a minimum, a compression level property should be exposed to the operator to balance an existing compression protocol between speeed/cpu load and network traffic volume.
This website uses cookies for analytics, personalisation and advertising. To learn more or change your cookie settings, please read our Cookie Policy. By continuing to browse, you agree to our use of cookies.
HCC Guidelines | HCC FAQs | HCC Privacy Policy | Privacy Policy | Terms of Service
© 2011-2019 Hortonworks Inc. All Rights Reserved.
Hadoop, Falcon, Atlas, Sqoop, Flume, Kafka, Pig, Hive, HBase, Accumulo, Storm, Solr, Spark, Ranger, Knox, Ambari, ZooKeeper, Oozie and the Hadoop elephant logo are trademarks of the Apache Software Foundation.