Workflow Management
Also available as:
PDF
loading table of contents...

Contents

1. Workflow Manager Basics
Workflow Manager Design Component
Workflow Manager Dashboard Component
Workflow Action Types
2. Content Roadmap for Workflow Manager
3. Quick Start
Configuring a Workflow Manager Ambari View
UI Elements of Workflow Manager
Understanding the Design Component
Using the Design Component
Using the Dashboard Component
Dashboard Job Details Tabs
Details Available for Workflow, Coordinator, and Bundle Jobs
Details Available Only for Workflow Jobs
Details Available Only for Coordinator Jobs
Details Available Only for Bundle Jobs
4. Designing Workflows Using the Design Component
Create a Workflow
Add Nodes to a Workflow
Add Action Nodes
Add Fork Control Nodes
Add Decision Control Nodes
Save a Workflow Draft
Validate and Save a Workflow
Import an Existing Workflow
Export a Workflow
Submitting and Executing a Workflow
Submit and Execute an Existing Workflow
Submit and Execute a New Workflow
Execute a Submitted Workflow
Modifying Workflows
Copy or Move Nodes on a Workflow Graph
Remove Nodes from a Workflow Graph
Resize and Move Graph Images
Reusing Node Configurations as Assets
Save an Ambari Database Asset to Use Within the WFM Instance
Save an Asset to Shared Storage to Use Across WFM Instances
Import an Asset from the Ambari Database Linked to a Single WFM Instance
Import a Shared Asset File from Shared Storage
Managing Assets
Creating Coordinators and Bundles
Create a Coordinator
Create a Bundle
View the XML Code
5. Monitoring Jobs Using the Dashboard
Verify the Status of a Job
View Job Details and Logs
Identify the Location of a Job XML File
Troubleshooting Job Errors
Basic Job Troubleshooting
Search the Log Output
Open an Existing Workflow to Edit
6. Sample ETL Use Case
Configure the Cluster
Create an HDFS Directory for Each New User
Create a Proxy User
Copy Files
Create and Submit the Workflow
Access the Workflow Manager View
Create the Sqoop Action to Extract Data
Create the Hive Action to Transform the Data
Create the Sqoop Action to Load the Data
Submit and Execute the Workflow Job
Monitor the Workflow Job
7. Workflow Parameters
Hive Action Parameters
Hive2 Action Parameters
Sqoop Action Parameters
Pig Action Parameters
Sub-Workflow Action Parameters
Java Action Parameters
Shell Action Parameters
DistCp Action Parameters
Map-Reduce (MR) Action Parameters
SSH Action Parameters
Spark Action Parameters
File System (FS) Action Parameters
Submit Dialog Parameters
8. Settings Menu Parameters
Global Configuration Parameters
Workflow Credentials Parameters
SLA for Workflow Parameters
Workflow Parameters
Workflow and Action Versions
How SLA Works
9. Job States
10. Workflow Manager Files

List of Tables

2.1. Workflow Manager Content roadmap
7.1. Hive Action, General Parameters
7.2. Hive Action, Transition Parameters
7.3. Hive Action, Advanced Properties Parameters
7.4. Hive Action, Configuration Parameters
7.5. Hive2 Action, General Parameters
7.6. Hive2 Action, Transition Parameters
7.7. Hive2 Action, Advanced Properties Parameters
7.8. Hive2 Action, Configuration Parameters
7.9. Sqoop Action, General Parameters
7.10. Sqoop Action, Transition Parameters
7.11. Sqoop Action, Advanced Properties Parameters
7.12. Sqoop Action, Configuration Parameters
7.13. Pig Action, General Parameters
7.14. Pig Action, Transition Parameters
7.15. Pig Action, Advanced Properties Parameters
7.16. Pig Action, Configuration Parameters
7.17. Sub-Workflow Action, General Parameters
7.18. Sub-Workflow Action, Configuration Parameters
7.19. Sub-Workflow Action, Transition Parameters
7.20. Java Action, General Parameters
7.21. Java Action, Transition Parameters
7.22. Java Action, Advanced Properties Parameters
7.23. Java Action, Configuration Parameters
7.24. Shell Action, General Parameters
7.25. Shell Action, Transition Parameters
7.26. Shell Action, Advanced Properties Parameters
7.27. Shell Action, Configuration Parameters
7.28. DistCp Action, General Parameters
7.29. DistCp Action, Transition Parameters
7.30. DistCp Action, Advanced Properties Parameters
7.31. DistCp Action, Configuration Parameters
7.32. MR Action, General Parameters
7.33. MR Action, Transition Parameters
7.34. MR Action, Advanced Properties Parameters
7.35. MR Action, Configuration Parameters
7.36. SSH Action, General Parameters
7.37. SSH Action, Transition Parameters
7.38. Spark Action, General Parameters
7.39. Spark Action, Transition Parameters
7.40. Spark Action, Advanced Properties Parameters
7.41. Spark Action, Configuration Parameters
7.42. FS Action, General Parameters
7.43. FS Action, Transition Parameters
7.44. FS Action, Advanced Properties Parameters
7.45. FS Action, Configuration Parameters
7.46. Submit Dialog, Configuration Parameters
8.1. Global Configuration Parameters
8.2. HCat Workflow Credentials Parameters
8.3. Hive2 Workflow Credentials Parameters
8.4. HBase Workflow Credentials Parameters
8.5. SLA for Workflows Parameters
8.6. Workflow Configuration Parameters