HDP-2.3.6 Release Notes
Also available as:
PDF

Flume

HDP 2.3.6 provides Flume 1.5.2 with no additional Apache patches beyond any ported over from the 2.4.x core.

HDP 2.3.6 provides Flume 1.5.2 and ports the following Apache patches from the 2.4.x core:

HDP 2.3.4.7 provided Flume 1.5.2 with no additional Apache patches.

HDP 2.3.4 provided Flume 1.5.2 and the following Apache patches:

HDP 2.3.2 provided Flume 1.5.2, with no additional Apache patches.

HDP 2.3.0 provided Flume 1.5.2 and the following Apache patches:

NEW FEATURES

  • FLUME-1734 Hive Sink based on the new Hive Streaming support

  • FLUME-2442 Need an alternative to providing clear text passwords in Flume config

Kafka Sink (preview)

  • FLUME-2251 Add support for Kafka Sink

  • FLUME-2454 Support batchSize to allow multiple events per transaction to the Kafka Sink

  • FLUME-2455 Documentation update for Kafka Sink

  • FLUME-2470 Kafka Sink and Source must use camel case for all configs.

  • FLUME-2499 Include Kafka Message Key in Event Header, Updated Comments

Kafka Source

IMPROVEMENTS

  • FLUME-2095 JMS source with TIBCO (patch-1)

  • FLUME-2226 Refactor BlobHandler out of morphline sink and into HTTP source

  • FLUME-2227 Move BlobDeserializer from Morphline Sink to flume-ng-core

  • FLUME-2337 export JAVA_HOME in flume-env.sh.template and increase heap size

  • FLUME-2450 Improve replay index insertion speed

  • FLUME-2511 Allow configuration of enabled protocols in Avro source and Rpc client

  • FLUME-2586 HDFS Sink should have an option to try rename even if close fails

  • FLUME-2595 Add option to checkpoint on file channel shutdown

  • FLUME-2624 Streaming ingest performance improvement

  • FLUME-2662 Upgrade to Commons-IO 2.4

  • FLUME-2663 Address Build warnings of duplicate dependencies listed

  • FLUME-2665 Update documentation for hdfs.closeTries based on FLUME-2586

BUG FIXES

  • FLUME-2122 Minor cleanups of User guide

  • FLUME-2123 Morphline Solr sink missing short type name

  • FLUME-2162 TestHDFSEventSinkOnMiniCluster.maxUnderReplicationTest fails on hadoop2

  • FLUME-2175 Update Developer Guide with notes on how to upgrade Protocol Buffer version

  • FLUME-2358 File Channel needs to close BackingStore and EventQueue before deleting files in checkpoint directory

  • FLUME-2402 Warning seen when overflow is disabled for Spillable Channel

  • FLUME-2407 Spillable Channel sometimes fails on reconfigure

  • FLUME-2412 Improve Logging in Spillable Channel

  • FLUME-2441 Unit test TestHTTPSource.java failed with IBM JDK 1.7

  • FLUME-2451 HDFS Sink Cannot Reconnect After NameNode Restart

  • FLUME-2501 Updating HttpClient lib version to ensure compatibility with Solr

  • FLUME-2520 HTTP Source should be able to block a prefixed set of protocols.

  • FLUME-2530 Resource leaks found by Coverity tool

  • FLUME-2533 HTTPS tests fail on Java 6

  • FLUME-2541 Bug in TestBucketWriter.testSequenceFileCloseRetries