Cloudera Docs » 3.1.2 » Release Notes
Release Notes
Also available as:
PDF

​Chapter 1. Hortonworks DataFlow 3.1.2 Release Notes

This document provides you with the latest information about the HDF 3.1.2 release and its product documentation.

​Component Support

HDF 3.1.2 includes the following components:

  • Apache Ambari 2.6.2

  • Apache Kafka 1.0.0

  • Apache NiFi 1.5.0

  • NiFi Registry 0.1.0

  • Apache Ranger 0.7.0

  • Apache Storm 1.1.1

  • Apache ZooKeeper 3.4.6

  • Apache MiNiFi Java Agent 0.4.0

  • Apache MiNiFi C++ 0.4.0

  • Hortonworks Schema Registry 0.5.0

  • Hortonworks Streaming Analytics Manager 0.6.0

​Component Availability in HDF

Previous HDF releases shipped with the following components versions.

 NiFiStormKafkaZooKeeperAmbariRangerMiNiFi Java AgentMiNiFi C++Streaming Analytics ManagerSchema RegistryNiFi Registry
HDF 3.1.21.5.01.1.11.0.03.4.62.6.10.7.00.4.00.4.00.6.0.5.00.1.0
HDF 3.1.11.5.01.1.11.0.03.4.62.6.10.7.00.4.00.4.00.6.0.5.00.1.0
HDF 3.1.01.5.01.1.11.0.03.4.62.6.10.7.00.4.00.4.00.6.00.5.00.1.0
HDF 3.0.3 (IBM Power System only)1.2.01.1.00.10.2.13.4.62.6.00.7.00.2.0TP0.5.00.3.0N/A
HDF 3.0.21.2.01.1.0`0.10.2.13.4.62.6.00.7.00.2.0TP0.5.00.3.0N/A
HDF 3.0.11.2.01.1.00.10.2.1 3.4.62.5.10.7.00.2.0TP0.5.00.3.0N/A
HDF 3.0.01.2.01.1.00.10.2.1 3.4.62.5.10.7.00.2.0TP0.5.00.3.0N/A
HDF 2.1.41.1.01.0.21.10.13.4.62.4.2.00.6.20.1.0TPN/AN/AN/A
HDF 2.1.21.1.01.0.20.10.13.4.62.4.2.00.6.20.1.0TPN/AN/AN/A
HDF 2.1.11.1.01.0.20.10.13.4.62.4.2.00.6.20.1.0TPN/AN/AN/A
HDF 2.1.01.1.01.0.20.10.13.4.62.4.2.00.6.20.1.0TPN/AN/AN/A
HDF 2.0.21.0.01.0.10.10.0.13.4.62.4.1.00.6.00.0.1TPN/AN/AN/A
HDF 2.0.11.0.01.0.10.10.0.13.4.62.4.1.00.6.00.0.1TPN/AN/AN/A
HDF 2.0.01.0.01.0.10.10.0.13.4.62.4.0.10.6.00.0.1TPN/AN/AN/A
HDF 1.2.10.6.10.10.00.9.0.13.4.6N/AN/AN/AN/AN/AN/AN/A
HDF 1.2.0.10.6.10.10.00.9.0.13.4.6N/AN/AN/AN/AN/AN/AN/A
HDF 1.2.00.6.00.10.00.9.0.13.4.6N/AN/AN/AN/AN/AN/AN/A
HDF 1.1.00.4.0N/AN/AN/AN/AN/AN/AN/AN/AN/AN/A
HDF 1.00.3.0N/AN/AN/AN/AN/AN/AN/AN/AN/AN/A

​What's New in HDF 3.1.2

HDF 3.1.2 is a maintenance release that includes bug fixes. HDF 3.1.2 includes the following bug fixes:

Hortonworks Bug IDApache Bug IDDescription
BUG-103676NIFI-5228Allow user to choose whether or not to add File Attributes as FlowFile Attributes when using ListFile.
BUG-103666NIFI-5109AbstractList processor stops calling performListing after primary node reelection.
BUG-103284NIFI-5204When node joins cluster, if a processor is stopping but cluster says the state is disabled, node ends up in inconsistent state.
BUG-102998NIFI-5194ConsumeKafka processor can occasionally hang.
BUG-102932NIFI-5099Changing the destination of a relation is not identified as "Local change" by NiFi Registry.
BUG-102776NIFI-4862Copy original FlowFile attributes to output FlowFiles at SelectHiveQL processor.
BUG-102766NIFI-5124Upgrade commons-fileupload.
BUG-102765NIFI-5108Update to Commons Compress to 1.16.1.
BUG-102764NIFI-4870Upgrade ActiveMQ dependencies.
BUG-102761NIFI-4945In NiFi 1.5, START_TLS in combination with LDAP will allow any password during auth.
BUG-102753NIFI-4788Expose whitelisting configuration for docker instantiation.
BUG-102746N/ASAM and Schema Registry fail due to not found DBConnectionVerification.jar on Ambari 2.6.2.
BUG-102016NIFI-5134NiFi can encounter "no TGT" after Hive service outage.
BUG-101099N/AHDF 2.0..x/ 2.1.x -> HDF 3.1.x upgrade overrides manual changes to toolkit memory settings causing OutOfMemoryException.
BUG-100013NIFI-5043TailFile opens and never closes readers in Multiple Files mode.
BUG-100012NIFI-5045PutHiveQL routes parse errors to retry instead of failure.
BUG-99842N/AHDF 3.1 Kafka configuration parameter log.message.format.version is not set during upgrade as expected.
BUG-99695NIFI-5012Controller Services are not automatically enabled/disabled when joining cluster.
BUG-99694MINIFI-435MiNiFi - Cannot perform secured S2S with NiFi 1.5.0.
BUG-99629NIFI-4395GenerateTableFetch can't fetch column type by state after instance reboot.
BUG-99627NIFI-5154Out of Scope processors can block Controller Services.
BUG-99426N/ANiFi Certificate Authority fails to stop on restart.
BUG-99164NIFI-4944PutHiveStreaming multiple instances with Snappy fail intermittently.
BUG-98548NIFI-4959HandleHttpRequest processor doesn't close/release incomplete message error.
BUG-98442NIFI-4969Failed to import versioned flow when processor uses "Event Driven" scheduling.
BUG-98189NIFI-4884Failed to import flow from registry when processor has CRON schedule.
BUG-97870NIFI-4920Change Version and Revert Local Changes incorrectly reset sensitive properties.
BUG-97868NIFI-4925Ranger Authorizer - Memory Leak.
BUG-96648NIFI-4773QueryDatabaseTable does not update processor State with ID and its value consistently.
BUG-94944NIFI-4836Allow QueryDatabaseTables to send out batches of flow files while result set is being processed.
BUG-94556NIFI-4772If several processors do not return from their @OnScheduled method, NiFi will stop scheduling any Processors.

For more information about the HDF 3.1.x releases, see:

​Unsupported Features

Some features exist within HDF 3.1.2, but Hortonworks does not currently support these capabilities.

​Technical Preview Features

The following features are available within HDF 3.1.2 but are not ready for production deployment. Hortonworks encourages you to explore these technical preview features in non-production environments and provide feedback on your experiences through the Hortonworks Community Forums.

​

Table 1.1. Technical Previews

ComponentFeature

MiNiFi

Android/iOS MiNiFi libraries for mobile integration


​Community Driven Features

The following features are developed and tested by the Hortonworks community but are not officially supported by Hortonworks. These features are excluded for a variety of reasons, including insufficient reliability or incomplete test case coverage, declaration of non-production readiness by the community at large, and feature deviation from Hortonworks best practices. Do not use these features in your production environments.

Community Driven Kafka features

  • Kafka Connect

  • Kafka Streams

Community Driven NiFi Tools and Services

  • Embedded ZooKeeper

  • Sensitive key migration toolkit

Community Driven NiFi Processors

  • AttributeRollingWindow

  • AWSCredentialsProviderControllerService

  • CompareFuzzyHash

  • ConsumeAzureEventHub

  • ConsumeEWS

  • ConsumeIMAP

  • ConsumeKafka_0_11

  • ConsumeKafkaRecord_0_11

  • ConsumePOP3

  • ConvertExcelToCSVProcessor

  • CountText

  • DebugFlow

  • DeleteDynamoDB

  • DeleteGCSObject

  • DeleteHDFS

  • DeleteMongo

  • DeleteRethinkDB

  • ExecuteFlumeSink

  • ExecuteFlumeSource

  • ExecuteSparkInteractive

  • ExtractCCDAAttributes

  • ExtractEmailAttachments

  • ExtractEmailHeaders

  • ExtractMediaMetadata

  • ExtractTNEFAttachments

  • FetchAzureBlobStorage

  • FetchGCSObject

  • FuzzyHashContent

  • GetDynamoDB

  • GetHDFSEvents

  • GetRethinkDB

  • GetSNMP

  • InvokeGRPC

  • ISPEnrichIP

  • InferAvroSchema

  • ListenBeats

  • ListenGRPC

  • ListenLumberjack

  • ListenSMTP

  • ListAzureBlobStorage

  • ListGCSBucket

  • ListS3

  • LogMessage

  • ModifyBytes

  • MoveHDFS

  • PublishKafka_0_11

  • PublishKafkaRecord_0_11

  • PutKudu

  • PutMongoRecord

  • PutRethinkDB

  • OrcFormatConversion

  • PutAzureBlobStorage

  • PutDynamoDB

  • PutGCSObject

  • PutIgniteCache

  • PutKinesisFirehose

  • PutKinesisStream

  • PutLambda

  • PutSlack

  • PutTCP

  • PutUDP

  • QueryDNS

  • SetSNMP

  • SpringContextProcessor

  • StoreInKiteDataset

[Note]Note

HDF 3.1.x does not support Hive 2. As a result, NiFi Processors working with Hive (PutHiveQL, PutHiveStreaming, SelectHiveQL) and the NiFi Controller Service HiveConnectionPool may not support Hive 2.

Community Driven NiFi Controller Services

  • AWSCredentialsProviderControllerService

  • ConfluentSchemaRegistry

  • GCPCredentialsControllerService

  • GraphiteMetricReporterService

  • IPLookupService

  • JettyWebSocketClient

  • JettyWebSocketServer

  • LivySessionController

  • MongoDBControllerService

  • MongoDBLookupService

  • PropertiesFileLookupService

  • RedisConnectionPoolService

  • RedisDistributedMapCacheClientService

  • SimpleCsvFileLookupService

  • SimpleKeyValueLookupService

  • XMLFileLookupService

Community Driven NiFi Reporting Tasks

  • DataDogReportingTask

  • MetricsReportingTask

  • SiteToSiteBulletinReportingTask

  • SiteToSiteStatusReportingTask

  • StandardGangliaReporter

​Unsupported Customizations

Hortonworks cannot guarantee that default NiFi processors are compatible with proprietary protocol implementations or proprietary interface extensions. For example, we support interfaces like JMS and JDBC that are built around standards, specifications, or open protocols. But we do not support customizations of those interfaces, or proprietary extensions built on top of those interfaces.

​Deprecated Technologies

This section points out any technology from previous releases that has been deprecated or removed from this release (operating systems, Java versions, databases, product features). Use this section as a guide for your implementation plans.

Deprecated

Technology that Hortonworks is removing in a future release. Deprecated items are supported until they are removed; deprecation gives you time to plan for removal.

Removed

Technology that Hortonworks has removed from production and is no longer supported.

​

Table 1.2. Deprecated Operating Systems

Operating System

Release DeprecatedRelease Removed
Ubuntu 12HDF 3.0.0HDF 3.0.0
Debian 6HDF 2.1.2HDF 3.0.0

​

Table 1.3. Deprecated NiFi Processors

Processor

Release Deprecated
ConvertCSVToAvroHDF 3.0.0
ConvertJSONToAvroHDF 3.0.0
GetKafka HDF 2.0.0
PutKafkaHDF 2.0.0
EvaluateRegularExpressionHDF 1.0.0

​

Table 1.4. Deprecated Kafka APIs

APIRelease DeprecatedUse Instead
kafka.producer.ProducerHDF 3.1.0org.apache.kafka.clients.producer.KafkaProducer
kafka.consumer.SimpleConsumerHDF 3.1.0org.apache.kafka.clients.consumer.KafkaConsumer
SecurityProtocol.PLAINTEXTSASLHDF 3.1.0SASL_PLAINTEXT

​HDF Repository Locations

Use the following table to identify the HDF 3.1.2 repository location for your operating system and operational objectives. HDF 3.1.2 supports the following operating systems:

​

Table 1.5. RHEL/Oracle Linux/CentOS 6 HDF repository & additional download locations

OS FormatDownload location
RHEL/Oracle Linux/CentOS 6 (64-bit):HDF Build number3.1.2.0-7
HDF Base URL http://public-repo-1.hortonworks.com/HDF/centos6/3.x/updates/3.1.2.0
HDF Repohttp://public-repo-1.hortonworks.com/HDF/centos6/3.x/updates/3.1.2.0/hdf.repo
RPM tarballhttp://public-repo-1.hortonworks.com/HDF/centos6/3.x/updates/3.1.2.0/HDF-3.1.2.0-centos6-rpm.tar.gz
Tars tarballhttp://public-repo-1.hortonworks.com/HDF/centos6/3.x/updates/3.1.2.0/HDF-3.1.2.0-centos6-tars-tarball.tar.gz
HDF Management Packhttp://public-repo-1.hortonworks.com/HDF/centos6/3.x/updates/3.1.2.0/tars/hdf_ambari_mp/hdf-ambari-mpack-3.1.2.0-7.tar.gz
MiNiFi C++http://public-repo-1.hortonworks.com/HDF/centos6/3.x/updates/3.1.2.0/tars/nifi-minifi-cpp/nifi-minifi-cpp-0.4.0-bin.tar.gz
HDP and Ambari Repositories
Ambarihttp://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.6.2.0/ambari.repo
HDP http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.6.5.0/hdp.repo
HDP-UTILS Base URLhttp://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos6
OS Agnostic Downloads
NiFi only
NiFi Toolkit
Docker Hub

https://hub.docker.com/r/hortonworks/nifi/

Tags are latest and 3.1.2.0

NiFi Registry
MiNiFi Java Agent
MiNiFi Toolkit
iOS/Android Libraries

​

Table 1.6. RHEL/Oracle Linux/CentOS 7 HDF repository & additional download locations

OS FormatDownload location
RHEL/Oracle Linux/CentOS 7 (64-bit):HDF Build Number3.1.2.0-7
HDF Base URLhttp://public-repo-1.hortonworks.com/HDF/centos7/3.x/updates/3.1.2.0
HDF Repohttp://public-repo-1.hortonworks.com/HDF/centos7/3.x/updates/3.1.2.0/hdf.repo
RPM tarballhttp://public-repo-1.hortonworks.com/HDF/centos7/3.x/updates/3.1.2.0/HDF-3.1.2.0-centos7-rpm.tar.gz
Tars tarballhttp://public-repo-1.hortonworks.com/HDF/centos7/3.x/updates/3.1.2.0/HDF-3.1.2.0-centos7-tars-tarball.tar.gz
HDF Management Packhttp://public-repo-1.hortonworks.com/HDF/centos7/3.x/updates/3.1.2.0/tars/hdf_ambari_mp/hdf-ambari-mpack-3.1.2.0-7.tar.gz
MiNiFi C++ Agenthttp://public-repo-1.hortonworks.com/HDF/centos7/3.x/updates/3.1.2.0/tars/nifi-minifi-cpp/nifi-minifi-cpp-0.4.0-bin.tar.gz
HDP and Ambari Repositories
Ambari http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.2.0/ambari.repo
HDP http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/hdp.repo
HDP-UTILS Base URLhttp://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7
OS Agnostic Downloads
NiFi only
NiFi Toolkit
Docker Hub

https://hub.docker.com/r/hortonworks/nifi/

Tags are latest and 3.1.2.0

NiFi Registry
MiNiFi Java Agent
MiNiFi Toolkit
iOS/Android Libraries

​

Table 1.7. SLES 11 SP3/SP4 HDF repository & additional download locations

OS FormatDownload location
SUSE Enterprise Linux 11 SP3, SP4 HDF Build Number3.1.2.0-7
HDF Base URLhttp://public-repo-1.hortonworks.com/HDF/suse11sp3/3.x/updates/3.1.2.0
Repohttp://public-repo-1.hortonworks.com/HDF/suse11sp3/3.x/updates/3.1.2.0/hdf.repo
RPM tarball http://public-repo-1.hortonworks.com/HDF/suse11sp3/3.x/updates/3.1.2.0/HDF-3.1.2.0-suse11sp3-rpm.tar.gz
Tars tarballhttp://public-repo-1.hortonworks.com/HDF/suse11sp3/3.x/updates/3.1.2.0/HDF-3.1.2.0-suse11sp3-tars-tarball.tar.gz
HDF Management Packhttp://public-repo-1.hortonworks.com/HDF/suse11sp3/3.x/updates/3.1.2.0/tars/hdf_ambari_mp/hdf-ambari-mpack-3.1.2.0-7.tar.gz
HDP and Ambari Repositories
Ambari http://public-repo-1.hortonworks.com/ambari/suse11/2.x/updates/2.6.2.0/ambari.repo
HDP http://public-repo-1.hortonworks.com/HDP/suse11sp3/2.x/updates/2.6.5.0/hdp.repo
HDP-UTILS Base URLhttp://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/suse11sp3
OS Agnostic Downloads
NiFi only
NiFi Toolkit
Docker Hub

https://hub.docker.com/r/hortonworks/nifi/

Tags are latest and 3.1.2.0

NiFi Registry
MiNiFi Java Agent
MiNiFi Toolkit
iOS/Android Libraries

​

Table 1.8. SLES 12 HDF repository & additional download locations

OS FormatDownload location
SUSE Linux Enterprise Server (SLES) v12 SP1HDF Build Number3.1.2.0-7
HDF Base URLhttp://public-repo-1.hortonworks.com/HDF/sles12/3.x/updates/3.1.2.0
Repohttp://public-repo-1.hortonworks.com/HDF/sles12/3.x/updates/3.1.2.0/hdf.repo
RPM tarballhttp://public-repo-1.hortonworks.com/HDF/sles12/3.x/updates/3.1.2.0/HDF-3.1.2.0-sles12-rpm.tar.gz
Tars tarballhttp://public-repo-1.hortonworks.com/HDF/sles12/3.x/updates/3.1.2.0/HDF-3.1.2.0-sles12-tars-tarball.tar.gz
HDF Management Packhttp://public-repo-1.hortonworks.com/HDF/sles12/3.x/updates/3.1.2.0/tars/hdf_ambari_mp/hdf-ambari-mpack-3.1.2.0-7.tar.gz
HDP and Ambari Repositories
Ambari http://public-repo-1.hortonworks.com/ambari/sles12/2.x/updates/2.6.2.0/ambari.repo
HDP http://public-repo-1.hortonworks.com/HDP/sles12/2.x/updates/2.6.5.0/hdp.repo
HDP-UTILS Base URLhttp://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/sles12
OS Agnostic Downloads
NiFi only
NiFi Toolkit
Docker Hub

https://hub.docker.com/r/hortonworks/nifi/

Tags are latest and 3.1.2.0

NiFi Registry
MiNiFi Java Agent
MiNiFi Toolkit
iOS/Android Libraries

​

Table 1.9. Ubuntu 14 HDF repository & additional download locations

OS FormatDownload location
Ubuntu Trusty (14.04) (64-bit)HDF Build Number3.1.2.0-7
HDF Base URLhttp://public-repo-1.hortonworks.com/HDF/ubuntu14/3.x/updates/3.1.2.0
Repohttp://public-repo-1.hortonworks.com/HDF/ubuntu14/3.x/updates/3.1.2.0/hdf.list
Deb tarballhttp://public-repo-1.hortonworks.com/HDF/ubuntu14/3.x/updates/3.1.2.0/HDF-3.1.2.0-ubuntu14-deb.tar.gz
Tars tarballhttp://public-repo-1.hortonworks.com/HDF/ubuntu14/3.x/updates/3.1.2.0/HDF-3.1.2.0-ubuntu14-tars-tarball.tar.gz
HDF Management Packhttp://public-repo-1.hortonworks.com/HDF/ubuntu14/3.x/updates/3.1.2.0/tars/hdf_ambari_mp/hdf-ambari-mpack-3.1.2.0-7.tar.gz
HDP and Ambari Repositories
Ambarihttp://public-repo-1.hortonworks.com/ambari/ubuntu14/2.x/updates/2.6.2.0/ambari.list
HDP http://public-repo-1.hortonworks.com/HDP/ubuntu14/2.x/updates/2.6.5.0/hdp.list
HDP-UTILS Base URLhttp://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu14
OS Agnostic Downloads
NiFi only
NiFi Toolkit
Docker Hub

https://hub.docker.com/r/hortonworks/nifi/

Tags are latest and 3.1.2.0

NiFi Registry
MiNiFi Java Agent
MiNiFi Toolkit
iOS/Android Libraries

​

Table 1.10. Ubuntu 16 HDF repository & additional download locations

OS FormatDownload location
Ubuntu Trusty 16 (64-bit)HDF Build Number3.1.2.0-7
HDF Base URLhttp://public-repo-1.hortonworks.com/HDF/ubuntu16/3.x/updates/3.1.2.0
Repohttp://public-repo-1.hortonworks.com/HDF/ubuntu16/3.x/updates/3.1.2.0/hdf.list
Deb tarballhttp://public-repo-1.hortonworks.com/HDF/ubuntu16/3.x/updates/3.1.2.0/HDF-3.1.2.0-ubuntu16-deb.tar.gz
Tars tarballhttp://public-repo-1.hortonworks.com/HDF/ubuntu16/3.x/updates/3.1.2.0/HDF-3.1.2.0-ubuntu16-tars-tarball.tar.gz
HDF Management Packhttp://public-repo-1.hortonworks.com/HDF/ubuntu16/3.x/updates/3.1.2.0/tars/hdf_ambari_mp/hdf-ambari-mpack-3.1.2.0-7.tar.gz
MiNiFi C++ Agenthttp://public-repo-1.hortonworks.com/HDF/ubuntu16/3.x/updates/3.1.2.0/tars/nifi-minifi-cpp/nifi-minifi-cpp-0.4.0-bin.tar.gz
HDP and Ambari Repositories
Ambarihttp://public-repo-1.hortonworks.com/ambari/ubuntu16/2.x/updates/2.6.2.0/ambari.list
HDP http://public-repo-1.hortonworks.com/HDP/ubuntu16/2.x/updates/2.6.5.0/hdp.list
HDP-UTILS Base URLhttp://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu16
OS Agnostic Downloads
NiFi only
NiFi Toolkit
Docker Hub

https://hub.docker.com/r/hortonworks/nifi/

Tags are latest and 3.1.2.0

NiFi Registry
MiNiFi Java Agent
MiNiFi Toolkit
iOS/Android Libraries

​

Table 1.11. Debian 7 HDF repository & additional download locations

OS FormatDownload location
Debian 7HDF Build Number3.1.2.0-7
HDF Base URLhttp://public-repo-1.hortonworks.com/HDF/debian7/3.x/updates/3.1.2.0
Repohttp://public-repo-1.hortonworks.com/HDF/debian7/3.x/updates/3.1.2.0/hdf.list
Deb tarballhttp://public-repo-1.hortonworks.com/HDF/debian7/3.x/updates/3.1.2.0/HDF-3.1.2.0-debian7-deb.tar.gz
Tars tarballhttp://public-repo-1.hortonworks.com/HDF/debian7/3.x/updates/3.1.2.0/HDF-3.1.2.0-debian7-tars-tarball.tar.gz
HDF Management Packhttp://public-repo-1.hortonworks.com/HDF/debian7/3.x/updates/3.1.2.0/tars/hdf_ambari_mp/hdf-ambari-mpack-3.1.2.0-7.tar.gz
HDP and Ambari Repositories
Ambari http://public-repo-1.hortonworks.com/ambari/debian7/2.x/updates/2.6.2.0/ambari.list
HDPhttp://public-repo-1.hortonworks.com/HDP/debian7/2.x/updates/2.6.5.0/hdp.list
HDP-UTILS Base URLhttp://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/debian7
OS Agnostic Downloads
NiFi only
NiFi Toolkit
Docker Hub

https://hub.docker.com/r/hortonworks/nifi/

Tags are latest and 3.1.2.0

NiFi Registry
MiNiFi Java Agent
MiNiFi Toolkit
iOS/Android Libraries

​

Table 1.12. NiFi and MiNiFi MSI files

File Location
NiFi MSI http://public-repo-1.hortonworks.com/HDF/windows/3.x/updates/3.1.2.0/nifi-3.1.2.0-7.msi
MiNiFi MSIhttp://public-repo-1.hortonworks.com/HDF/windows/3.x/updates/3.1.2.0/minifi-3.1.2.0-7.msi

​HDF Repository Location for IBM Power Systems

​

Table 1.13. RHEL 7 HDF repository & additional download locations

OS FormatDownload location
RHEL 7 (64-bit):HDF Build Number3.1.2.0-7
HDF Base URLhttp://public-repo-1.hortonworks.com/HDF/centos7-ppc/3.x/updates/3.1.2.0
HDF Repohttp://public-repo-1.hortonworks.com/HDF/centos7-ppc/3.x/updates/3.1.2.0/hdf.repo
RPM tarballhttp://public-repo-1.hortonworks.com/HDF/centos7-ppc/3.x/updates/3.1.2.0/HDF-3.1.2.0-centos7-ppc-rpm.tar.gz
Tars tarballhttp://public-repo-1.hortonworks.com/HDF/centos7-ppc/3.x/updates/3.1.2.0/HDF-3.1.2.0-centos7-ppc-tars-tarball.tar.gz
HDF Management Packhttp://public-repo-1.hortonworks.com/HDF/centos7-ppc/3.x/updates/3.1.2.0/tars/hdf_ambari_mp/hdf-ambari-mpack-3.1.2.0-7.tar.gz
HDP and Ambari Repositories
Ambari https://public-repo-1.hortonworks.com/ambari/centos7-ppc/2.x/updates/2.6.2.0/ambari.repo
HDP http://public-repo-1.hortonworks.com/HDP/centos7-ppc/2.x/updates/2.6.5.0/hdp.repo
HDP-UTILShttp://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7-ppc/hdp-utils.repo

​Common Vulnerabilities and Exposures

The following CVEs have been fixed in HDF 3.1.2.

​CVE-2018-1310

Summary: Apache NiFi JMS Deserialization issue because of ActiveMQ client vulnerability
Severity: Moderate
Versions Affected: Apache NiFi 0.1.0 - 1.5.0
Description: Malicious JMS content could cause denial of service. See ActiveMQ CVE-2015-5254 announcement for more information.
Mitigation:The fix to upgrade the activemq-client library to 5.15.3 was applied to the HDF 3.1.2 release. HDF users should upgrade to HDF 3.1.2.

​CVE-2017-8028

Summary: Apache NiFi LDAP TLS issue because of Spring Security LDAP vulnerability
Severity: Severe
Versions Affected: Apache NiFi 0.1.0 - 1.5.0
Description: Spring Security LDAP library was not enforcing credential authentication after TLS handshake negotiation. See NVD CVE-2017-8028 disclosure for more information.
Mitigation: The fix to upgrade the spring-ldap library to 2.3.2.RELEASE+ was applied to the HDF 3.1.2 release. HDF users should upgrade to HDF 3.1.2.

​CVE-2018-1324

Summary: Apache NiFi Denial of service issue because of commons-compress vulnerability
Severity: Low
Versions Affected: Apache NiFi 0.1.0 - 1.5.0
Description: A vulnerability in the commons-compress library could cause denial of service. See commons-compress CVE-2018-1324 announcement for more information.
Mitigation: The fix to upgrade the commons-compress library to 1.16.1 was applied to the HDF 3.1.2 release. HDF users should upgrade to HDF 3.1.2.

The following CVEs have been fixed in HDF 3.1.0.

​CVE-2017-12632

Summary: Apache NiFi host header poisoning issue
Severity: Medium
Versions Affected: Apache NiFi 0.1.0 - 1.4.0, HDF 1.x, 2.x, 3.0.x
Description: A malicious host header in an incoming HTTP request could cause NiFi to load resources from an external server.
Mitigation: The fix to sanitize host headers and compare to a controlled whitelist was applied on the Apache NiFi 1.5.0 release. HDF users should upgrade to HDF 3.1.0.

​CVE-2017-15697

Summary: Apache NiFi XSS issue in context path handling
Severity: Moderate
Versions Affected: Apache NiFi 1.0.0 - 1.4.0, HDF 2.0.0 - 3.0.x
Description: A malicious X-ProxyContextPath or X-Forwarded-Context header containing external resources or embedded code could cause remote code execution.
Mitigation: The fix to properly handle these headers was applied on the Apache NiFi 1.5.0 release. HDF users should upgrade to HDF 3.1.0.

​CVE-2017-12623

Summary: Apache NiFi XXE issue in template XML upload
Severity: Important
Versions Affected: Apache NiFi 1.0.0 - 1.3.0; HDF 2.x, 3.0.0 - 3.0.1.1
Impact: Any authenticated user could upload a template which contained malicious code and accessed sensitive files via an XML External Entity (XXE) attack.
Mitigation: The fix to properly handle XML External Entities was applied on the Apache NiFi 1.4.0 release. Users running a prior 1.x release should upgrade to the appropriate release. HDF users should upgrade to HDF 3.1.0.

​CVE-2017-15703

Summary: Apache NiFi Java deserialization issue in template XML upload
Severity: Moderate
Versions Affected: Apache NiFi 1.0.0 - 1.3.0; HDF 2.x, 3.0.0 - 3.0.1.1
Description: Any authenticated user (valid client certificate but without ACL permissions) could upload a template which contained malicious code and caused a denial of service via Java deserialization attack.
Mitigation: The fix to properly handle Java deserialization was applied on the Apache NiFi 1.4.0 release. HDF users should upgrade to HDF 3.1.0.

​Known Issues

Hortonworks Bug ID

Apache JIRA

Component

Summary

    
BUG-97319 Ambari/NiFi

Issue: After you have upgraded HDF services on an HDP cluster, you may experience issues starting NiFi.

Problem: After upgrade, NiFi starts but Ambari reports that it is stopped. The command line NiFi service is running on the host.

Workaround: To work around this issue, click the Ambari Restart All option for NiFi. NiFi starts successfully.

BUG-94090 SAM

Issue: You may be unable to import some topologies from HDF 3.0.x

Problem: When you have a new HDF 3.1.0 installation, you cannot import topologies created in HDF 3.0.x.

Workaround: To work around this issue, you can upgrade your existing version of HDF to HDF 3.1.0.

  NiFi

Issue: When you have upgraded HDF services on your HDP cluster, the NiFi version number displayed in Ambari does not change from NiFi 1.2.0 to NiFi 1.5.0.

Workaround: After performing the upgrade scenario for HDF services on an HDP cluster as documented in the Ambari Managed HDF Upgrade documentation, disregard the NiFi version displayed in Ambari. NiFi has been successfully upgraded.

  NiFi

Issue: Unable to load native library.

Associated error message: You may encounter an error message similar to one of the following:

  • Failed to write to HDFS of libhadoop was built without snappy support... Native snappy library not available: this version of libhadoop was built without snappy support.

  • Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: Native Library /path/to/hadoop/lib/native/libhadoop.so.1.0.0 already loaded in another classloader

Problem: Native libraries can only be loaded into the Java runtime by a single ClassLoader. Once a ClassLoader loads a native library, subsequent attempts to load that same native library by a different ClassLoader will fail. If multiple components in NiFi attempt to load a native library, it will fail to do so and the components may fail to execute if it attempts to use those native libraries. Whether those components fail ultimately depend on the order they are loaded and executed which is not guaranteed by NiFi. This will occur most frequently with components that leverage the native Hadoop library. These components include but are not limited to:

  • NiFi Ranger Authorizer

  • Any HDFS Processors

  • Any HBase Processors

  • Any Hive Processors

Workaround: This issue is most frequently encountered using an HDFS Processor with a Compression Codec that Hadoop leverages native libraries to implement. In this scenario, the dataflow can be updated to utilize a CompressContent Processor before/after the HDFS Processor to apply the desired compression. The CompressContent does not leverage native libraries and will not be susceptible to this known issue.

BUG-94989 Storm/Ambari

Issue: When performing an Ambari managed rolling upgrade, you may encounter an inaccurate warning message indicating that Storm topologies need to be stopped.

Associated error message: Storm topologies need to stop.

Workaround: You may safely ignore this error message. Storm topologies do not need to be stopped before performing a rolling upgrade.

 KNOX-1108NiFi/Knox

Issue: NiFi/Knox integration does not support HA.

In NiFiHaDispatch, executeRequest is overridden and does not have the try/catch block in DefaultHaDispatch's executeRequest method which is used to catch exceptions and begin the fail over process.

Workaround: There is no workaround for this issue.

BUG-90903N/ANiFi/Knox

Issue: Knox HA failover for NiFi is not supported.

Workaround: There is no workaround for this issue.

BUG-63132N/AStorm

Summary: Solr bolt does not run in a Kerberos environment.

Associated error message: The following is an example:

[ERROR] Request to collection hadoop_logs failed due to (401) org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http:[...] Error 401 Authentication required

Workaround: None at this time.

​Third-Party Licenses

HDF 3.1.2 deploys numerous third-party licenses and dependencies, all of which are compatible with the Apache software license. For complete third-party license information, see the licenses and notice files contained within the distribution.

© 2012–2020, Cloudera, Inc.
Document licensed under the Creative Commons Attribution ShareAlike 4.0 License.
Cloudera.com | Documentation | Support | Community