Scaling Namespaces and Optimizing Data Storage
Also available as:
PDF
loading table of contents...

Namespace view in a federation

Each of the multiple NameNodes in a federation has its own namespace. The NameNodes share the DataNodes of the cluster. Using ViewFs, you can create a cluster namespace view that is similar to a namespace in a cluster without support for federation.

Consider a federation with namespace volumes where each volume contains data distinct from that of others. For example, a federation containing namespace volumes such as /data, /project, /user, and /tmp. Using ViewFs, you can create a cluster namespace view that defines mount points for the namespace volumes at the root level or below. The following figure shows a mount table for the namespace volumes /data, /project, /user, and /tmp:

For the cluster, you can set the default file system to the ViewFs mount table in the core-site.xml file, as shown in the following example:
<property>
  <name>fs.default.name</name>
  <value>viewfs://clusterX</value>
</property>
In this example, the authority following the viewfs:// scheme in the URI is the mount table name. For easier management of the federation, you should consider specifying the cluster name as the mount table name. In your Hadoop environment, ensure that the configuration files for all the gateway nodes contain the mount tables for all the clusters such that, for each cluster, the default file system is set to the ViewFs mount table for that cluster.
Note
Note
After setting the default file system to the ViewFs mount table for a given cluster, you must remap any directory paths mounted earlier to point to the corresponding namespaces that contain those directories.