# Copyright 2012, Hortonworks Inc.  All rights reserved.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# http://www.apache.org/licenses/LICENSE-2.0
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

RELEASE NOTES: Hortonworks Data Platform with gsInstaller powered by Apache Hadoop

Product Version: HDP-1.1.0.15
============================================
This release of Hortonworks Data Platform (HDP) deploys the following Hadoop-related components:

* Apache Hadoop 1.0.3
* Apache HBase 0.92.1
* Apache Pig 0.9.2
* Apache ZooKeeper 3.3.4
* Apache HCatalog 0.4.0
* Apache Hive 0.9.0
* Templeton 0.1.4
* Apache Oozie 3.1.3
* Apache Sqoop 1.4.2
* Hortonworks Management Center (HMC) 1.0.2
* Apache Flume 1.2.0
* HA-Monitor 0.1.0 

Third party components:
* Talend Open Studio for Big Data 5.1.1
* Ganglia 3.2.0
* Nagios 3.2.3


Patch Information
----------------------------------------   
* Hadoop is patched to include the following:
  - High Availability (HA) fixes: HDFS-3522, HDFS-3521, HDFS-1108, HDFS-3551, HDFS-528, HDFS-3667, HDFS-3516, HDFS-3696, HDFS-3658, MAPREDUCE-4328, MAPREDUCE-3837, MAPREDUCE-4328, MAPREDUCE-4603, and HADOOP-8656.
  - Performance improvements: HDFS-2465, HDFS-2751, HDFS-496, MAPREDUCE-782, MAPREDUCE-1906, MAPREDUCE-4399, MAPREDUCE-4400, MAPREDUCE-3289, MAPREDUCE-3278, HADOOP-7753, and HADOOP-8617, 
  - Bug Fixes: HDFS-3846 and MAPREDUCE-4558.

* HBase is patched to include HBASE-6447, HBASE-6450, HBASE-6334, HBASE-4470, HBASE-6460, HBASE-6552, HBASE-6512, HBASE-6308, HBASE-6576, HBASE-6565, HBASE-6538, HBASE-6608, HBASE-6503, HBASE-5714, HBASE-6631, and HBASE-6632.

* Hive is patched to include HIVE-3008, HIVE-3063, HIVE-3076, HIVE-3168, HIVE-3246, HIVE-3153, HIVE-3291, and HIVE-3098.

* Oozie is patched to include OOZIE-698, OOZIE-697, OOZIE-810, and OOZIE-863.

* Sqoop is patched to include SQOOP-578, SQOOP-579, SQOOP-580, SQOOP-582, and SQOOP-462.

* HCatalog is patched to include HCATALOG-448, HCATALOG-350, HCATALOG-436, HCATALOG-471, and HCATALOG-464.

* Pig is patched to include PIG-2766.

* Ambari is patched to include AMBARI-664, AMBARI-641, AMBARI-628, AMBARI-633, and AMBARI-597.

Minimum system requirements
==============================================
Operating Systems Requirements
------------------------------------
The following operating systems are supported:
* Red Hat compatible systems
  * 64-bit Red Hat Enterprise Linux (RHEL) v5.*, v6.*
  * 64-bit CentOS v5.*, v6.*
* SUSE systems:
  * 64-bit SUSE Linux Enterprise Server 11 (SLES 11) Service pack 1
  
IMPORTANT: All hosts in the cluster must run the same OS, version and patch sets.

Hardware requirements
----------------------
Although there is no single hardware requirement for installing HDP, there are some basic guide­lines. 
You can see sample setups here: http://docs.hortonworks.com/CURRENT/About_Hortonworks_Data_Platform/Hardware_Recommendations_For_Apache_Hadoop.htm

Ensure that you choose the appropriate number of host machines for your cluster:
* For evaluation purpose, you can use a single machine (see: http://docs.hortonworks.com/CURRENT/Deploying_Hortonworks_Data_Platform/Using_gsInstaller/Deploying_Single_Node_Cluster/Deploying_Single_Node_Cluster.htm).

Software requirements
----------------------
On each of your hosts:
* yum (RHEL) or zypper (SLES)
* rpm
* scp
* curl
* unzip
* tar
* wget
* pdsh

* Ensure that all the nodes have JDK v 1.6 update 31 installed. 
  Also ensure that for all the nodes, the JAVA_HOME variable points to a common location (for example: /usr/java/default). 
  To install Java Development Kit (JDK), see: http://docs.hortonworks.com/CURRENT/Deploying_Hortonworks_Data_Platform/Using_gsInstaller/System_Requirements_For_Test_And_Production_Clusters.htm.

Database requirements
----------------------
* To be able to use Hive Metastore, you must deploy MySQL instance on your Hive Metastore host machine. 
  To install MySQL instance, see http://docs.hortonworks.com/CURRENT/Deploying_Hortonworks_Data_Platform/Using_gsInstaller/System_Requirements_For_Test_And_Production_Clusters.htm.

Pre-deployment planning
----------------------
* Ensure you use the fully qualified domain name (FQDN) for all the host machines. 
  Note that only alphanumeric, hyphen (“-”), and period (“.”) characters are allowed in a valid FQDN.

* All the host machines in your cluster must be configured for DNS and Reverse DNS.

* Ensure that the Network Time Protocol (NTP) is enabled for your cluster.

* In environments with no access to the Internet, ensure that you make one of your master nodes as NTP server.
   For more details, see: http://docs.hortonworks.com/CURRENT/Deploying_Hortonworks_Data_Platform/Using_gsInstaller/System_Requirements_For_Test_And_Production_Clusters.htm.

Improvements
=============================================
* Introduced storage-based authorization for Hive with HCatalog. 
  For details, see: http://docs.hortonworks.com/HCatalog/CURRENT/authorization.html

* Introduced high availability feature using VMware and Red Hat Enterprise Linux. 
  For details, see: http://docs.hortonworks.com/CURRENT/HDP_HA/HDP_High_Availability/High_Availability_For_Hadoop/High_Availability_For_Hadoop.htm

* Added support for Apache Flume NG. 
  For details, see: http://docs.hortonworks.com/CURRENT/Data_Integration_Services_With_HDP/Using_Apache_FlumeNG/install_configure_flume.htm

* Added support to install HDP manually using RPMs. 
  For details, see: http://docs.hortonworks.com/CURRENT/Deploying_Hortonworks_Data_Platform/Installing_Manually/rpminstall/rpm_getting_ready/rpm_getting_ready.htm
  
Known issues
=============================================
* The ALTER INDEX command in an automated script that also contains CREATE INDEX command will fail for Hive. 
  The workaround is to either use the ALTER INDEX command in an interactive shell or add this command in a separate script file.

* Hive and HCatalog authorizations are based on permissions in the underlying storage system and so are not affected by account-management DDL statements such as GRANT and REVOKE. 
  For details, see:  http://docs.hortonworks.com/HCatalog/CURRENT/authorization.html

* Templeton's job submission APIs (beta) do not work in secure mode.

* Using init.d scripts for starting and stopping services is not recommended.     

* Pig or MapReduce jobs get incorrect data when reading binary data type from the HCatalog table. For more details, see: https://issues.apache.org/jira/browse/HCATALOG-430.

Hortonworks offers technical support subscriptions for Hortonworks Data Platform.
For more information, please visit http://hortonworks.com/support/ or contact us directly at info@hortonworks.com.