Managing Data Operating System
Also available as:
PDF
loading table of contents...

Intra-Queue Preemption Based on Application Priorities

Enabling preemption on a queue depending on application priorities ensures that higher-priority applications can preempt resources from lower-priority applications when required.

Example of resource consumption on a queue without preemption

Consider a queue qA with root.qA.capacity configured at 70%. Consider applications submitted in the following order:
  1. A user submits an application app1 with priority p1. Because no other application is running on the queue, app1 uses a majority of the resources available on the queue.
  2. Shortly after app1 starts running, the user submits another application app2 with the same priority as app1. In this situation, app2 uses the resources that remain on the queue.
  3. The user submits a third application app3 with a higher priority p3.

    If preemption is not enabled on the queue, the lower priority applications app1 and app2 consume all of the queue's available capacity leaving the higher priority application app3 starved of resources.

The following table explains the resource distribution between the three applications:
Application Priority Consumed Resources Pending Resources
app1 p1 50 20
app2 p1 20 20
app3 p3 0 80

Example of resource consumption on a queue with preemption

Consider the same queue and applications with priorities as the previous example.

If preemption based on application priority is enabled on the queue, resources are preempted from app1 and app2 for the higher-priority app3 to run.

The following table explains the resource distribution between the three applications when preemption is enabled:
Application Priority Preempted Resources Consumed Resources Pending Resources
app1 p1 16 34 36
app2 p1 19 1 39
app3 p3 0 35 45