5. Known Issues

HDP smoke tests fail when running as a non-hadoop user.

Problem: MSI installer does not set permissions to allow users other than the hadoop user to run jobs on the HDP 2.0 cluster for Windows.

Workaround: As the hadoop user, set permissions for the /mapred directory in Hadoop:

C:\hdp\hadoop-2.2.0.2.0.6.0-0009>hadoop fs -chown 757 /mapred

You should now be able to run smoke tests as Administrator or another non-hadoop user that has been added to the Hadoop group.

Windows Firewall Stealth Mode interferes with an RPC interaction between the MapReduce client, the MapReduce YARN Application Master, and the MapReduce History Server. The symptom of this problem is that the submitter of a MapReduce job appears to stall right after job completion. During the stall, the console logs show repeated re-attempts of an RPC connection. Under default configuration, the period of this stall can last up to 15 minutes.

Problem: Windows Firewall Stealth Mode intentionally drops outbound TCP RST packets if a client initiates a connection to a destination port that has no server listening. You can read more from the Micosoft Technet site here. This behavior interferes with the design of the MapReduce client, which initiates a failover to the History Server after detecting that the connection to the Application Master is no longer available. By dropping outbound TCP RST packets under Stealth Mode, the client does not observe an immediate connection refused. Instead, it must wait for a longer connection timeout period before initiating failover to the History Server.

Workaround: Disable Stealth Mode by setting registry key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\SharedAccess\Parameters\FirewallPolicy\PublicProfile with the DisableStealthMode value. For details, see Disable Stealth Mode. Restart the Windows Firewall service to make this change take effect.

Running MapReduce jobs using pipes is currently not supported on Windows

Non-default compression codecs that are based on zlib or snappy are currently not supported on Windows

It is possible to encounter the following exception while starting the Hive command line interface (CLI)

FAILED: Error in metadata: javax.jdo.JDOFatalDataStoreException: DERBY SQL error: SQLCODE: -1, SQLSTATE: XJ041, SQLERRMC: 
    Failed to create database 'metastore_db', see the next exception for details.
    ::SQLSTATE: XBM0JDirectory C:\Hadoop\hive-0.9.0\bin\metastore_db already exists.

This typically happens when the user attempts to install and uninstall HDP repeatedly. In such cases, the directories for Hive might not get deleted properly on uninstall. You can use any of the following workarounds:

HDP creates some files in the HDFS directory (C:\hdfs) that are not deleted on uninstall. This issue is observed when hadoop.tmp.dir is not defined to point to the C:\hadoop location. There is no impact on the deployment of your cluster. However, it is recommended that you manually delete the files in the HDFS directory (C:\hdfs) after you uninstall HDP.

Migrating data and tables from SqlLite to Oracle must be performed manually to prevent data loss.

HADOOP-717: NameNode Upgrade Error Messges convert to Warnings

HADOOP-722: YARN client returns with application not found error if RM restarted wihin 5s of job submission

HADOOP-682: A lot of retries seen while running Pig tests causing tests to be slow

ONPREM-397: Cancelling installation midway hangs

ONPREM-400: Installer sometimes hangs during unzip phase

ONPREM-402: Services being configured to restart immediately might cause issues with HA