Hortonworks Data Platform for HDInsight
Also available as:
PDF

HDInsight Fixed Issues

The fixed issues listed here are issues that were specifically addressed to support HDInsight. For fixed issues in Ambari, see the Ambari HDInsight Release Notes.

HDP 3.0.2 (Atlantic)

Incorrect Results

Bug ID Apache JIRA Component Summary
HIVE-1112 N/A Hive Incorrect results with query result cache
HIVE-1118 N/A Hive Show tables not listing the table when queries on the table work
HIVE-1119 N/A Hive TPCDS - query91 returns 0 rows (incorrect results)
SPARK-300 N/A Spark Spark history server UI doesnt show all applications in secure spark 2.2

Other

Bug ID Apache JIRA Component Summary
BUG-114180 HBASE-21466 HBase WALProcedureStore uses wrong FileSystem if wal.dir is not under rootdir
BUG-115734 HBASE-21523, HBASE-21524 HBase Chatty DEBUG logging in Master log
HBASE-536 N/A HBase System table hbase:namepace left in transition after upgrade from HBase 1.1 to HBase 2.0
HBASE-540 N/A HBase HMaster fails in HDP 3.0 with HBase 2.0 when using Azure managed disk for WAL logs
HBASE-544 N/A HBase Regression on split_2 on WASB
HDFS-197 N/A Hadoop Common Bug in hadoop-httpfs systemd service scripts
HDFS-199 N/A Hadoop Common WASB2 java.lang.NoSuchMethodError: org.apache.commons.codec.binary.Base64.isBase64
HDFS-200 N/A Hadoop Common HDP 3.0.2.0-13 cluster fails with AzureException: No credentials found for account mycluster in the configuration
HDFS-202 N/A Hadoop Common hadoop-httpsfs.service file doesn't exist on 3.0 cluster
HIVE-1017 N/A Hive The property StatusFolder that unicode content will be changed when we sumit job with powershell CLI (status folder isn't generated if the statusfolder only unicode content)
HIVE-1110 N/A Hive Bucket file name issues on HDI atlantic
HIVE-1114 N/A Hive Hive CLI does not work on the LLAP cluster for 3.0
HIVE-1120 N/A Hive HiveServer2 Interactive stops during deployment phase
HIVE-1136 N/A Hive Hive throws OutOfMemoryError on ABFS during QueryResultsCache operation
HIVE-1154 N/A Hive Permission Denied for Create External Table with fallback authorization
HIVE-1165 HIVE-21214 Hive HiveServer2 split 2 empty bucket tests fail due to missing data from output
OOZIE-371 N/A Oozie Oozie fails to get job information from Hadoop on ABFS
PHOENIX-134 N/A Phoenix Phoenix IOException while using ABFS
RANGER-174 N/A Ranger Ranger start may be prevented by leftover DB entry
SPARK-307 N/A Spark Error in custom script action
SPARK-310 N/A Spark STS shows ambari alerts
SPARK-316 N/A Spark Backport BUG-110970 into HDI
SPARK-319 N/A Spark Secure Spark cluster creation failure with WASB
WASB-136 N/A Spark enable use of HDP nameservice filesystem paths in WASB2 (abfs and abfss protocols)
YARN-320 N/A Spark METRICS_COLLECTOR is not starting on HDInsight Atlantic ABFS unsecure runs
YARN-322 N/A Spark Spark UI failed due to certificate verification error
ZOOK-67 N/A Spark zoo.cfg does not exist in /usr/hdp/current/zookeeper-client/conf dir

Performance

Bug ID Apache JIRA Component Summary
WASB-132 N/A N/A WASB: improve listStatus performance (Port HADOOP-15547)

Query Failure

Bug ID Apache JIRA Component Summary
HDFS-201 N/A Hadoop Common Hive jobs on ABFS 3.0 clusters fail with NoSuchMethodError from commons.codec
BUG-96841 HIVE-20510, HIVE-20540 Hive Bucketing: _bucket_number is marked as a constant, not a virtual column

Security

Bug ID Apache JIRA Component Summary
HIVE-1113 N/A Hive Incorrect property value for hive.security.metastore.authorization.manager - Atlantic HDI
HIVE-1140 N/A Hive HiveServer2 cannot start in WASB Secure for Atlantic clusters
RANGER-118 N/A Hive Maintain parity with Ranger-HDFS policy-spec wrt recursive and wildcard
SPARK-317 N/A Spark Livy broken in secure spark clusters
QE-21187 N/A HDFS, MapReduce, YARN YARN and HDFS test failures due to OpenSSL logging
BUG-118138 N/A Ranger Incorrect foreign key in MSSQL

Stability

Bug ID Apache JIRA Component Summary
BUG-113464 HIVE-20834 Hive Hive QueryResultCache entries keeping reference to SemanticAnalyzer from cached query
BUG-114044 RANGER-2286 Ranger Ranger start may be prevented by leftover database entry
ZEP-65 N/A Zeppelin Zeppelin modifies the authentication cache on logout, which breaks behavior for HDInsight

Supportability

Bug ID Apache JIRA Component Summary
HIVE-1116 N/A Hive Kill API support on WebHCat for Hive
HIVE-1129 N/A Hive Hive JDBC jars for Beeline not available for download in public URL
HIVE-1149 N/A Hive AzureDB schema no longer packaged with Hive
ZEP-80 N/A Zeppelin Kerberos (SPNEGO) Support
ZEP-81 N/A Zeppelin Zeppelin UI doesn't load fully on Chrome (Works on Edge)

Usability

Bug ID Apache JIRA Component Summary
SPARK-309 N/A Spark beeline on port 10002 doesnt work

HDP 3.1.2 (Celtic)

Other

Bug ID Apache JIRA Component Summary
BUG-118672 HADOOP-16182 Hadoop Common Update abfs storage back-end with "close" flag when application is done writing to a file
BUG-118762 HADOOP-16157 Hadoop Common [Clean-up] Remove NULL check before instanceof in AzureNativeFileSystemStore
BUG-118763 HADOOP-15813 Hadoop Common Enable more reliable SSL connection reuse
BUG-118764 HADOOP-16163 Hadoop Common NPE in setup/teardown of ITestAbfsDelegationTokens
BUG-118770 HADOOP-16068 Hadoop Common ABFS Authentication and Delegation Token plugins to optionally be bound to specific URI of the store
BUG-118772 HADOOP-16169 Hadoop Common ABFS: Bug fix for getPathProperties
BUG-118774 HADOOP-16105 Hadoop Common WASB in secure mode does not set connectingUsingSAS
BUG-118779 HADOOP-16104 Hadoop Common WASB tests to downgrade to skip when test a/c is namespace enabled
BUG-118780 HADOOP-15582 Hadoop Common Document ABFS
BUG-118846 N/A Hadoop Common WebHcat Hive calls fail due to missing HS2 URL
BUG-118965 HADOOP-15994 Hadoop Common Upgrade Jackson2 to 2.9.8
WASB-140 N/A Hadoop Common wildfly logs to the console
QE-1097 N/A Hadoop Common Enable Hive Warehouse Connector (Spark2LLAP) test suite
HBASE-548 N/A HBase test_runIntegrationTestZKAndFSPermissions fails on secure environments
BUG-118417 N/A Hive Hive Metastore can't convert nvarchar to CLOB on SQL Server in Celtic HDInsight
HIVE-1167 N/A Hive WebHCat doesn't autostart
HIVE-1169 N/A Hive Turn-on point lookup optimization
HIVE-1173 N/A Hive TPCDS tests fails in setup phase on ABFS
HIVE-1168 N/A Hive MSSQL upgrade scripts have some issue (2.1.2000 to 3.1.0)
HIVE-1176 N/A Hive Wrong file owner when Hive table is created with CTAS.
HIVE-1180 N/A Hive Hive Warehouse Connector select() issue after count()
BUG-119402 N/A Hive Disable HIVE_SERVER 1+ cardinality requirement for HDP 3.1.2 for HDInsight
QE-22489 N/A Hive Hive Tez queue tests fail on secure configurations
BUG-119332 HIVE-21507 Hive Hive swallows NPE if no delegation token is found
BUG-118846 N/A Hive WebHCat Hive calls fail due to missing HiveServer2 URL
BUG-113948 N/A Hive, Spark Spark2LLAP test failures (Spark 2.4): JVM related errors, test_tpcds[query*.sql] and test_etl_scala[*]
BUG-117414 AMBARI-25168 Kafka Leader for a topic is assigned 'None'
DEPLOY-337 N/A N/A Deploy failure on both U10 and Celtic
DEPLOY-342 N/A N/A New deploy tool for Celtic (fix for deploy-337)
DEPLOY-345 N/A N/A Deployment failure for hadoop_spark clustertype for Celtic with hwxtools-201902240021 because of invalid version
DEPLOY-346 N/A N/A Invalid TaskLaunchCmdOpts defined for Vertex Map : Invalid/conflicting GC options found
QE-21402 N/A N/A Admin password changed on HDI for Ranger
QE-22409 N/A N/A Certification tests need some amendments due to 2.4 Spark UI change
QE-22459 N/A N/A Speed up Spark2Hive TPCDS tests on HDI
QE-22470 N/A N/A XaAgents on HDInsight: tests are aborted
QE-22472 N/A N/A Sqoop tests fail due to malformed SQL query
QE-22552 N/A N/A System test test_complex_intra_queue_preemption_userlimit[USERLIMIT_FIRST] fails
QE-22558 N/A N/A Kyro serialization issue in spark-shell with -i option
QE-22504 N/A Oozie, Sqoop Implement Oozie - Sqoop system test which covers Sqoop Hive import using delegation token
DEPLOY-349 N/A Spark Fix for SPARK-331
RMP-12966 N/A Spark Upgrade Spark to V2.4 on HDP-3.1.1 for MSFT
SPARK-331 N/A Spark Spark catalog in Hive metastore is not available at first attempt
SPARK-332 N/A Spark Spark Thrift Server goes down when Yarn is restarted
QE-1103 N/A Spark Spark2LLAP TPC-DS failures due to empty values saved as "" in CSV
BUG-113248 N/A Spark Spark2 test failures (Spark 2.4): HBase test failures, test_spark_hbase and test_hbasesource
QE-22558 N/A Spark Kyro serialization issue in spark-shell with -i option
SQOOP-118 N/A Sqoop Sqoop: test_copyParquetTableToHive[mysql] fails
QE-22122 N/A Storm Fix WASB Storm HDFS tests
QE-22561 N/A Storm WasbAzureIaasSink is missing from the classpath

Security

Bug ID Apache JIRA Component Summary
OOZIE-379 N/A Oozie Oozie Hive jobs are failing in secure environment with mapreduce.job.map.class is incompatible with map compatibility mode
BUG-118138 N/A Ranger Incorrect foreign key in MSSQL
DEPLOY-344 N/A Ranger Ranger is unable to start due to incorrect password
BUG-118666 RANGER-2356 Ranger External user's email address can be edited
QE-1102 N/A YARN ACL issues in Yarn
QE-21187 N/A YARN, Hadoop Common YARN and HDFS test failures due to OpenSSL logging

Usability

Bug ID Apache JIRA Component Summary
RANGER-179 N/A Ranger Admin: External user's email can be edited