Migrating SAM and Schema Registry data to HDF 3.1.0
About This Task
HDF 3.1 does not support installing SAM or Schema Registry services on an HDP cluster. As a result, you cannot upgrade SAM and Schema Registry from HDF 3.0.x, once they are installed on an HDP cluster. If you want to upgrade existing SAM and Schema Registry services installed on an HDP cluster, you must:
Backup necessary SAM and Schema Registry data
Create an HDF 3.0.2 cluster
Migrate SAM and Schema Registry to your new HDF 3.0.2 cluster
Upgrade the HDF 3.0.2 cluster to HDF 3.1
Backing up SAM and Schema Registry data
Backup up your SAM and Schema Registry datastores. See the following database provider documentation for backup instructions:
MySQL 5.5 or higher – https://dev.mysql.com/doc/refman/5.5/en/backup-and-recovery.html
Postgres 9.5 or higher – https://www.postgresql.org/docs/9.5/static/backup.html
Oracle 12c and 11g Release 2
If SAM or Schema Registry reference any jars using local storage, copy and move these jars to your new server location.
Shut down SAM and Schema Registry on the HDF 3.0.2 cluster if it is still running. Ensure that you have also stopped any SAM applications.
Creating a new HDF 3.0.x cluster
Create a 3.0.x HDF cluster, restore the database, validate SAM and Schema Registry functionality, and then upgrade the new cluster to HDF 3.1. Installing the a cluster at the same version from which you are migrating allows you to validate that application metadata was migrated successfully before upgrading.
Install a new HDF 3.0.x cluster. For instructions, see the installation documentation available for the appropriate version of HDF 3.0.x.
Before you install SAM and Schema Registry on your new HDF cluster, set up the new destination databases and restore from the backups you made. See the database provider documentation for restore instructions.
Install SAM and Schema Registry using an Ambari instance configured with the HDF version consistent with your source cluster. Ensure that SAM and Schema Registry are configured to use the databases you just restored.
For SAM, complete the following post-migration configuration updates:
Confirm that the environment, service pools, and applications were successfully imported.
To use services (Storm, Kafka, etc.) on your HDF cluster, create a new Service pool that references the new cluster
As needed, edit the imported applications to select the new cluster and update any configurations.
For Schema Registry, ensure that it is running with all schemas visible and accessible.
Upgrade to HDF 3.1
Using Ambari, upgrade this new HDF 3.0.x cluster to HDF 3.1.0. Use the instructions contained in Upgrading an HDF Cluster.
After you have completed the upgrade to an HDF 3.1.0 cluster, use SAM to create a new Service Pool and Environment that uses the services from HDF 3.1.0.
Edit any SAM applications to use this environment to ensure that you are using the versions of Storm and Kafka that you have deployed with HDF 3.1.0.