4.3 Parallel and Distributed Computing

Enduring Understanding

Parallel and distributed computing leverage multiple computers to more quickly solve complex problems or process large data sets.

Learning Objective

For sequential, parallel, and distributed computing:

a. Compare problem solutions.

b. Determine the efficiency of solutions.

Essential Knowledge

Sequential computing is a computational model in which operations are performed in order one at a time.

Parallel computing is a computational model where the program is broken into multiple smaller sequential computing operations, some of which are performed simultaneously

Distributed computing is a computational model in which multiple devices are used to run a program.

Comparing efficiency of solutions can be done by comparing the time it takes them to perform the same task.

A sequential solution takes as long as the sum of all of its steps.

A parallel computing solution takes as long as its sequential tasks plus the longest of its parallel tasks.

The “speedup” of a parallel solution is measured in the time it took to complete the task sequentially divided by the time it took to complete the task when done in parallel.

Learning Objective

Describe benefits and challenges of parallel and distributed computing. 

Essential Knowledge

Parallel computing consists of a parallel portion and a sequential portion.

Solutions that use parallel computing can scale more effectively than solutions that use sequential computing.

Distributed computing allows problems to be solved that could not be solved on a single computer because of either the processing time or storage needs involved.

Distributed computing allows much larger problems to be solved quicker than they could be solved using a single computer.

When increasing the use of parallel computing in a solution, the efficiency of the solution is still limited by the sequential portion. This means that at some point, adding parallel portions will no longer meaningfully increase efficiency