Manager-Worker Pattern in Distributed & Parallel Computing

When confronted with a daunting problem beyond individual capacity, the Manager-Worker pattern offers a solution by dividing the workload for a collaborative effort.  Key components of this pattern:

• A problem divisible to independent tasks.

• A manager acting as the central coordinator, dividing the problem, distributing tasks, collating results, and overseeing the overall workflow.

• Multiple workers executing the assigned tasks.

Additional characteristics often include ordered-data processing and identical workers.

Benefits:

• Scalability:  Easily add workers to handle larger or more complex problems.

• Speed:  Parallel processing significantly reduces overall execution time.

• Efficiency:  Maximize processing power using multiple resources.

Considerations:

• Overhead:  The worker computation must outweigh the overhead of communication between manager and workers.

• Communication:

  • For large assignment data, storing and sending workers its location is advisable.

  • Assignments may be pushed by the manager to the workers or pulled by the workers from the manager, depending on which side knows the other.  The choice could affect the load—the trigger side works at their own pace, whereas the other side may be over- or under-whelmed.

• State Management:  Keeping the problem state in memory, or a durable storage depends on how big it is and on the fault tolerance solution.

• Load Balancing:  The manager or workers idle while the other side is overwhelmed means inefficiencies.  When the workers idle, let the manager distribute larger or more tasks to the workers, and vice versa.

• Failure Management:  Machines can fail and will fail over the long run.  Usually workers are stateless, and their failures are simpler to overcome.  The manager is stateful, and a fault tolerance solution needs to ensure the integrity of the state.

Applications include problems divisible to independent tasks, such as matrix multiplication, batch processing of images or video frames, and log processing, etc.

See also:

A demonstration in Python.

Jorge L. Ortega-Arjona (2004). The Manager Workers Pattern. In Proceedings of the 9th European Conference on Pattern Languages of Programms (EuroPLoP '2004), Irsee, Germany, July 7-11, 2004 (pp. 53–64). UVK - Universitaetsverlag Konstanz.

Weber, R. (2021, January 13).  Manager-Worker Communication Patterns.  Level Up Coding.

 https://levelup.gitconnected.com/manager-worker-communication-patterns-c3580b9db5db