Hortonworks Data Platform

Installing Hadoop Using Apache Ambari

2014-04-25


Contents

I. Setting Up Ambari
I.1. Getting Ready
1. Understand the Basics
2. Meet Minimum System Requirements
2.1. Hardware Recommendations
2.2. Operating Systems Requirements
2.3. Browser Requirements
2.4. Software Requirements
2.5. JDK Requirements
2.6. Database Requirements
3. Collect Information
4. Prepare the Environment
4.1. Check Existing Installs
4.2. Set Up Password-less SSH
4.3. Enable NTP on the Cluster and on the Browser Host
4.4. Check DNS
4.5. Configuring iptables
4.6. Disable SELinux and PackageKit and check the umask Value
5. Optional: Configure Ambari for Local Repositories
I.2. Running the Ambari Installer
1. Set Up the Bits
1.1. RHEL/CentOS/Oracle Linux 5.x
1.2. RHEL/CentOS/Oracle Linux 6.x
1.3. SLES 11
2. Set Up the Server
2.1. Setup Options
3. Optional: Set Up LDAP or Active Directory Authentication
4. Optional: Set Up Security for Ambari
4.1. Set Up HTTPS for Ambari Server
4.2. Set Up HTTPS for Ganglia
4.3. Set Up HTTPS for Nagios
4.4. Optional: Encrypt Database and LDAP Passwords
5. Optional: Set Up Two-Way SSL Between Ambari Server and Ambari Agents
6. Optional: Change the Ambari Server Port
7. Start the Ambari Server
II. Deploying, Configuring, and Upgrading Ambari for Hadoop 2.x
II.3. Installing, Configuring, and Deploying the Cluster for Hadoop 2.x
1. Log into Apache Ambari
2. Welcome
3. Select Stack
4. Install Options
5. Confirm Hosts
6. Choose Services
7. Assign Masters
8. Assign Slaves and Clients
9. Customize Services
9.1. Service Users and Groups
9.2. Properties That Depend on Service Usernames/Groups
10. Review
11. Install, Start and Test
12. Summary
II.4. Troubleshooting Ambari Deployments for Hadoop 2.x
1. Getting the Logs
2. Quick Checks
3. Specific Issues
3.1. Problem: Browser crashed before Install Wizard completed
3.2. Problem: Install Wizard reports that the cluster install has failed
3.3. Problem: “Unable to create new native thread” exceptions in HDFS DataNode logs or those of any system daemon
3.4. Problem: The “yum install ambari-server” Command Fails
3.5. Problem: HDFS Smoke Test Fails
3.6. Problem: The HCatalog Daemon Metastore Smoke Test Fails
3.7. Problem: MySQL and Nagios fail to install on RightScale CentOS 5 images on EC2
3.8. Problem: Trouble starting Ambari on system reboot
3.9. Problem: Metrics and Host information display incorrectly in Ambari Web
3.10. Problem: On SUSE 11 Ambari Agent crashes within the first 24 hours
3.11. Problem: Attempting to Start HBase REST server causes either REST server or Ambari Web to fail
3.12. Problem: Multiple Ambari Agent processes are running, causing re-register
3.13. Problem: Some graphs do not show a complete hour of data until the cluster has been running for an hour
3.14. Problem: After performing a cluster install the Nagios server is not started
3.15. Problem: A service with a customized service user is not appearing properly in Ambari Web
3.16. Problem: Updated configuration changes are not pushed to client/gateway nodes
II.5. Appendix: Upgrading Ambari Server to 1.4.1
II.6. Appendix: Upgrading the HDP Stack from 1.3.2 or later to 2.0.6
1. Preparing for the Upgrade
2. Setting Up the Ambari Repository
3. Upgrading to Ambari 1.4.1
4. Upgrading the Stack
4.1. Prepare for the Stack Upgrade
4.2. Upgrade the Stack
4.3. Add YARN/MR2 and Update Configurations
4.4. Complete the Stack Upgrade
II.7. Appendix: Configuring Ports for Hadoop 2.x
1. HDFS Ports
2. MapReduce Ports
3. YARN Ports
4. Hive Ports
5. HBase Ports
6. ZooKeeper Ports
7. WebHCat Port
8. Ganglia Ports
9. MySQL Port
10. Ambari Ports
11. Nagios Ports
II.8. Appendix: NameNode High Availabilty
1. Setting Up NameNode High Availabilty
1.1. Rolling Back NameNode HA
III. Deploying, Configuring, and Upgrading Ambari for Hadoop 1.x
III.9. Installing, Configuring, and Deploying the Cluster for Hadoop 1.x
1. Log into Apache Ambari
2. Welcome
3. Select Stack
4. Install Options
5. Confirm Hosts
6. Choose Services
7. Assign Masters
8. Assign Slaves and Clients
9. Customize Services
9.1. Service Users and Groups
9.2. Properties That Depend on Service Usernames/Groups
9.3. Recommended Memory Configurations for the MapReduce Service
10. Review
11. Install, Start and Test
12. Summary
III.10. Troubleshooting Ambari Deployments
1. Getting the Logs
2. Quick Checks
3. Specific Issues
3.1. Problem: Browser crashed before Install Wizard completed
3.2. Problem: Install Wizard reports that the cluster install has failed
3.3. Problem: “Unable to create new native thread” exceptions in HDFS DataNode logs or those of any system daemon
3.4. Problem: The “yum install ambari-server” Command Fails
3.5. Problem: HDFS Smoke Test Fails
3.6. Problem: The HCatalog Daemon Metastore Smoke Test Fails
3.7. Problem: MySQL and Nagios fail to install on RightScale CentOS 5 images on EC2
3.8. Problem: Trouble starting Ambari on system reboot
3.9. Problem: Metrics and Host information display incorrectly in Ambari Web
3.10. Problem: On SUSE 11 Ambari Agent crashes within the first 24 hours
3.11. Problem: Attempting to Start HBase REST server causes either REST server or Ambari Web to fail
3.12. Problem: Multiple Ambari Agent processes are running, causing re-register
3.13. Problem: Some graphs do not show a complete hour of data until the cluster has been running for an hour
3.14. Problem: After performing a cluster install the Nagios server is not started
3.15. Problem: A service with a customized service user is not appearing properly in Ambari Web
3.16. Problem: Updated configuration changes are not pushed to client/gateway nodes
III.11. Appendix: Upgrading Ambari Server to 1.2.5
III.12. Appendix: Upgrading the HDP Stack to 1.3.3
1. Preparing for the Upgrade
2. Setting Up the Ambari Repository
3. Upgrading Ambari
4. Upgrading the Stack (from 1.2.* to 1.3.3)
5. Upgrading the Stack (from 1.3.0 to 1.3.3)
III.13. Appendix: Configuring Ports
1. HDFS Ports
2. MapReduce Ports
3. Hive Ports
4. HBase Ports
5. ZooKeeper Ports
6. WebHCat Port
7. Ganglia Ports
8. MySQL Port
9. Ambari Ports
10. Nagios Ports
IV. Additional Tasks with Ambari
IV.14. Appendix: Installing Ambari Agents Manually
1. RHEL/CentOS/Oracle Linux 5.x and 6.x
2. SLES
IV.15. Appendix: Using Custom Hostnames
IV.16. Appendix: Upgrading Operating Systems on an Ambari-based Hadoop Installation
IV.17. Appendix: Moving the Ambari Server
1. Back up Current Data
2. Update Agents
3. Install the New Server and Populate the Databases
IV.18. Appendix: Using Non-Default Databases
1. Hive/HCatalog
1.1. Troubleshooting Hive/HCatalog
2. Oozie
2.1. Troubleshooting Oozie
3. Ambari
3.1. Troubleshooting Ambari
IV.19. Setting Up Kerberos for Use with Ambari
1. Setting Up Kerberos for Hadoop 2.x
1.1. Preparing Kerberos
1.2. Setting Up Hadoop Users
1.3. Enabling Kerberos Security
2. Setting Up Kerberos for Hadoop 1.x
2.1. Preparing Kerberos
2.2. Setting Up Hadoop Users
2.3. Enabling Kerberos Security

loading table of contents...