7. Known Issues for HDP 2.2.6

Hortonworks Bug ID

Apache JIRA

Component/s

Summary

BUG-18450HIVE-7160HiveVectorization Udf: GenericUDFConcat, is not supported for String+Long
BUG-19255 ZookeeperUninstalling the ZooKeeper RPM does not remove all ZooKeeper directories
BUG-19911 FalconEnsure falcon windows installer takes secure cluster installation into account
BUG-19912 FalconDocument how to configure Falcon on Kerberized HDP for Windows
BUG-19930 FalconDocument entity management dashboard
BUG-20856 HiveColumn names are missing from join expression in Map join with CBO enabled
BUG-20866 MahoutMahout trainnb command fails with -c option
BUG-21166 PhoenixtestGetCaseInsensitiveDynCol fails intermittently throwing AssertionError
BUG-21254 HBasehbck tool fixMeta flag not able to recreate deleted entries from hbase:meta table when just one column is deleted
BUG-21468 HiveWith CBO enabled queries fail with Failed to breakup Windowing invocations into Groups
BUG-22522 YARNMR Job failed when run via oozie and NN and RM were being killed
BUG-22807 YARNNon-secure slider app processes run with yarn user.
BUG-22841 HBaseHBase test failed with Insufficient permissions (user=hrt_qa@HWQE.HORTONWORKS.COM error
BUG-23260HIVE-11421HiveBUG-22878 insert after alter table issue
BUG-23527 OozieOozie config referred to versioned hadoop config
BUG-23619 HBaseOn Windows, tests fail complaining about Cannot run program /usr/bin/env
BUG-23655 OozieOozie stuck on mapreduce job because workflow state isn't changing from RUNNING to KILLED
BUG-23889 KnoxWindows security scripting should setup Knox to use ActiveDirectory for LDAP
BUG-23897 HBaseHBase REST server continuously restarting in secure windows env
BUG-23981HADOOP-7844=HBaseIOException: Invalid HFile block magic in Windows unit tests
BUG-24136 FalconSpecial/utf8 chars should not be allowed in pipeline name
BUG-24518 TezTez needs to support Application Tag
BUG-24589 FalconDR - set up replication for HDFS
BUG-24590 FalconDR - set up replication for Table
BUG-24591 FalconBUG-24589 UI Mock for setting up HDFS Replication
BUG-24592 FalconBUG-24589 Rest API design for HDFS replication
BUG-24594 FalconBUG-24590 UI Mock for setup table replication
BUG-24595 FalconBUG-24590 Rest API design for set up table replication
BUG-24597 FalconDR - delete replication for HDFS
BUG-24598 FalconDR - delete replication for Table
BUG-24831 TezTEZ windows test framework needs to respect ${fs.defaultFS} in tez configs.
BUG-25049 KnoxTwo WebHdfsHaFuncTest are unstable and should be fixed and enabled
BUG-25064 HiveHive : Q51 fails in CBO with "Failed to breakup Windowing invocations into Groups. At least 1 group must only depend on input columns"
BUG-25194 TezAM running out of memory for partitioned bucketed tables
BUG-25380 TezFor hive to work with Virtual columns and parittion names, we need information available within a record-reader
BUG-25451 Accumulo[Moved to Dal] Integrate Accumulo in RU Automation Frame work
BUG-25961 HBaseHBase tests failing with org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out; 10000ms
BUG-26243 HiveCBO : Q51 fails in explain with Failed to breakup Windowing invocations into Groups.
BUG-26307 FalconAs a user, I need to be able to replicate the action of creating new database at the target Hive DB
BUG-26309 FalconAs a user, I need to be able to replicate the action of dropping a database at the target Hive DB
BUG-26310 FalconAs a user, I need to be able to replicate the action of altering database at the target Hive DB
BUG-26311 FalconAs a user, I need to be able to replicate the action of creating new table at the target Hive DB
BUG-26312 FalconAs a user, I need to be able to replicate the action of dropping a table at the target Hive DB
BUG-26313 FalconAs a user, I need to be able to replicate the action of altering a table at the target Hive DB
BUG-26314 FalconAs a user, I need to be able to replicate the action of inserting rows into unpartitioned table at the target Hive DB
BUG-26315 FalconAs a user, I need to be able to replicate the action of inserting and overwriting rows into unpartitioned table at the target Hive DB
BUG-26318 FalconAs a user, I need to be able to replicate the action of altering a partition in a table at the target Hive DB
BUG-26319 FalconAs a user, I need to be able to replicate the action of inserting rows into a partition of a table at the target Hive DB
BUG-26320 FalconAs a user, I need to be able to replicate the action of inserting and overwriting rows into a partition of a table at the target Hive DB
BUG-26322 FalconAs a user, I need to be able to replicate Date partitions at the target Hive DB
BUG-26323 FalconAs a user, I need to be able to replicate Site partitions at the target Hive DB
BUG-26324 FalconAs a user, I need to be able to replicate Regex partitions at the target Hive DB
BUG-26325 FalconAs a user, I need to be able to replicate custom non standard partitions at the target Hive DB
BUG-26326 FalconAs a user, I need to be able to replicate multiple tables at once in a database at the target Hive DB
BUG-26327 FalconAs a user, I need to be able to replicate multiple table names (defined by regular expressions) at the target Hive DB
BUG-26328 FalconAs a user, I need to be able to replicate Avro partitions at the target Hive DB
BUG-26329 FalconAs a user, I need to be able to replicate dynamic partitions at the target Hive DB
BUG-26330 FalconAs a user, I need to be able to replicate hive roles at the target Hive DB
BUG-26419 TezTez application fails with diff client and AM version
BUG-26428 RangerXaAdmin and XaAgent jobs for Debian failing on Unix clusters.
BUG-26661 PhoenixWindows runs failed with "region isn't co-located with its data" exception
BUG-26944 Falcon, OozieE0803: IO error, null during scheduling feed on a cluster with wire enc on
BUG-27089 KnoxKnox YarnSysTest skipped in secure mode for gsInstaller test config
BUG-27091 KnoxKnox YarnSysTest skipped in secure mode for ambari test config
BUG-27094 KnoxKnox HBase tests skipped for ambari test config
BUG-27194 YARN[YARN-2821] Failures encountered on Distributed Shell on windows
BUG-27256 HBaseCreate mr based Hbase Long running application for Rollingupgrade
BUG-27479 Falconfalcon idempotency is being violated
BUG-27507 HiveHive CLI returns 0 even in case of failures
BUG-27509HBASE-12464HBasetest_ReplicaCopyTable_verifyCopyTable[tableone] fails after trying several times for 'Failed to find location, tableName=hbase:meta, row='
BUG-27513 StormStorm-slider default config has ganglia error in supervisor.log
BUG-27562 TezWordCount on Tez with non-nightly failed with NPE
BUG-27582 HiveBeeline Client Connections Errors - Obscure errors and exception swallow - 1
BUG-27590 YARNDistributed shell job does not execute python script in Windows
BUG-27632 ZookeeperAdd feature of retry in zookeeper client
BUG-27636HIVE-10500HiveOracle: intemittent acid_concurrency test failures due to NoSuchLockException
BUG-27638 FalconDEV - As a user i need a login page when i run Falcon as a standalone app
BUG-27639 FalconDEV - As a user, i need a way to logout when logged in the standalone version of Falcon
BUG-27656HBASE-12472, HBASE-13192HBaseIntegrationTestBulkLoad fails on mapreduce
BUG-27708 FalconDEV - When uploading an XML of a cluster that already exists (same name) the system allows the operation
BUG-27734 FalconDEV - Column titles should have 14px size
BUG-27737 FalconDEV - When loading the dashboard, the systen should show a spinning loader for tables that are updating, and just a plain message for columns without entities
BUG-27756 HBasetest_IntegrationTestMultiGetRegionReplicas slowness
BUG-27769 Sqoop[Rolling upgrade] [Test issue] Sqoop Long running job finishes before upgrade of all component finishes
BUG-27858 HBaseHBase HA tests fail with Table Namespace Manager not ready yet
BUG-27893 PhoenixtestCurrentDate and testCurrentTime are failing due to NTP on windows server
BUG-27927 FalconDEV - Correct the steps information
BUG-27928 FalconDEV - The form needs a "Target Clusters" title and "add cluster" button in step 4 of feed setup
BUG-27929 FalconDEV - Put a label before the input and add archive option in step 4 of feed setup
BUG-27933 FalconDEV - Each cluster or input container must have it's title
BUG-27936 FalconDEV - Correct vertical spacings between labels and inputs in all setups
BUG-28011 PhoenixtestInnerJoinWithPreFilters and testJoinOverSaltedTables are failing with SequenceNotFoundException exception
BUG-28031 KnoxKnox rolling upgrade system test support for wire encryption
BUG-28039KNOX-534KnoxKnox Failed Authentication Auditing
BUG-28049 Falcon, RelEnglineage is not enabled for Falcon in MSI
BUG-28080FALCON-885FalconMerge falcon-885 to internal repo for Champlain-M10
BUG-28218 HiveHive CLI hangs
BUG-28276 FalconFalcon actions intermittently fail with java.net.SocketTimeoutException on windows
BUG-28344 FalconDEV - XML editor should only be able to change it's vertical size, not horizontal
BUG-28348 FalconDEV - Increase vertical spacing in the space for the action buttons
BUG-28394 FalconDEV - Correct font sizes in forms
BUG-28614 OozieOozie property oozie.authentication.kerberos.principal is being used for authentication of webhdfs endpoint
BUG-28664 FalconDEV - Fix layout for user generated inputs
BUG-28977FALCON-959FalconUnable to update feed's acl to a different user of different group
BUG-29261 PhoenixTests are failing with ERROR 1200 and then raising an error due to jobManager rejection
BUG-29287 FalconBackport https://issues.apache.org/jira/browse/FALCON-829
BUG-29427HIVE-9235HiveTurn off Parquet Vectorization until all data types work: DECIMAL, DATE, TIMESTAMP, CHAR, and VARCHAR
BUG-29730 StormStorm Hive,Hbase, Hadoop connector should support user specified UGI in non-secure env
BUG-29864FALCON-1101FalconCluster submission in falcon does not create an owned-by edge
BUG-29903 HBaseIntegrationTestMTTR slowness
BUG-30055 FalconFalcon recipe should have rest api
BUG-30784ZOOKEEPER-1952Zookeeperzookeeper.log.file property is not respected; log output goes only to the zookeeper.out
BUG-30984HIVE-10521HiveHive compaction tests failing
BUG-31349 Oozieoozie job in running state is causing intermittent timeout failures
BUG-31353 HBaseIntegrationTestRegionReplicaPerf region not serving
BUG-31426 Oozie, RelEngOozie failed to start due to "SA" schema not created
BUG-31497 Phoenix[RollingUpgrade] Phoenix intermittently fails to return every column on sql queries
BUG-31522 TezYou cannot search by the name of 'Queue'
BUG-31565 TezWhen you kill a job in the middle of running it, it Diagnostics shows 'killed/failed due to:null".
BUG-31624 HiveDropping a 2000+ partition table takes over 10 minutes
BUG-31671 HiveHS2 concurr tests intermittently fail with http mode on secure Openstack throwing org.apache.hive.service.auth.HttpAuthenticationException: Kerberos authentication failed
BUG-31676 FalconFalcon jar uploading occasionally fails
BUG-31936 HBasetest_hbaseBigLinkedList[Loop] TimeoutException
BUG-31943 HBasetest_IntegrationTestRegionReplicas region not serving
BUG-32026 Oozietest_mapred_via_oozie_ha[oozie-nn-rm-5-jobs] FAILED
BUG-32195 HBaseNeed to add hbase-slider RU downgrade
BUG-32509 HBasetest_runIntegrationTestMTTRwithDistributedLogs due to failure in testKillRsHoldingMeta
BUG-32688 Stormkafka/api/OffsetRequest is not packaged in storm-kafka-example jar
BUG-32763 Oozieyum cannot find Xvfb for centos06
BUG-32768FALCON-954FalconCLONE - Secure Kerberos setup : Falcon should periodically revalidate security credentials.
BUG-32902 FalconFalcon pom.xml should not use exact HDP versions for Hive,Oozie and Hadoop
BUG-33054 RelEng, SparkSpark examples are packaged differently on Windows
BUG-33056 Spark{start,stop}-thriftserver equivalent script on Windows is missing
BUG-33113 HBaseHBase-HA master fail over immediately
BUG-33168 Hivehive authorization checks on configs should not be done if authorization is diabled
BUG-33180 Oozieoozie logging tests fail on MR framework with postgres 9
BUG-33278HBASE-13239HBaseHBASE grants at specific column level does not work for Groups
BUG-33335 Sparkspark make-distribution.sh doesn't evaluate SPARK_HIVE on OSX
BUG-33338HIVE-10528HiveHiveserver2 in HTTP mode is not applying auth_to_local rules
BUG-33764 HDP stackHDP-Select needs to handle user-created directories in /usr/hdp
BUG-35528 Ranger

Users must remove empty user entries from their MYSQL database

Error: Internal Exception: java.sql.SQLException: Connections could not be acquired from the underlying database!

BUG-35942 YARNUsers must manually configure ZooKeeper security with ResourceManager High Availability.
BUG-38088HDFS-7553HDFS

Issue: DataNode restart fails during Rolling Upgrade from 2.2.x-2.2.6 to 2.3.x.

Details: Not all writers have responder running, but sendOOB() tries anyway. This causes NPE and the shutdown thread dies, halting the shutdown after only shutting down DataXceiverServer.

Workaround: Two possible workarounds:

  • Skip the failed datanodes, downgrade them and try to upgrade them again.

  • Manually restart the failed datanodes.

Related bugs: BUG-38088, BUG-36247, BUG-38009

BUG-40259 Ranger

Issue: Upgrade to HDP 2.2.6 breaks usersync.

Details: Ranger in HDP 2.2.6 introduced authentication between usersync and the admin REST API. If you upgrade to HDP 2.2.6, there will be a previous version of /etc/ranger/usersync/conf/unixauthservice.properties and {{ /etc/ranger/admin/conf/security-applicationContext.xml}} in place. Because setup.sh treats these files as templates, it expects certain base content to be there in order to generate correct configurations. It looks like there is new configuration in the default templates (the ones in /usr/hdp/2.2.6.0-2800/etc/ranger/...) that will not get pulled into the live configs. This results in usersync pulling 401 errors every time it tries to add/modify a user.

Workaround:

  1. Apply to the copy files:

    /usr/hdp/2.2.6.0-2800/etc/ranger/usersync/conf/unixauthservice.properties -> /etc/ranger/usersync/conf/unixauthservice.properties
    /usr/hdp/2.2.6.0-2800/etc/ragner/admin/conf/security-applicationContext.xml -> /etc/ranger/admin/conf/security-applicationContext.xml
  2. Rerun setup.sh.

  3. Restart admin.

  4. Restart usersync.

BUG-50531 Kafka

Kafka file system support

Issue: Encrypted file systems such as SafenetFS are not supported for Kafka. Index file corruption can occur.

For more information, see: Install Kafka.