Processor Computational Models
- Speedup and Efficiency: Speedup measures performance gains by dividing the time taken on one CPU by the time taken on multiple processors. Efficiency evaluates the speedup achieved per individual processor.
$$
S = \frac{T(1)}{T(n)}
$$
$$
Efficiency = \frac{S(n)}{n}
$$
- Equal Duration Model: This model assumes a task is split into perfectly equal subtasks executed by individual processors. The base time taken per processor is calculated without overhead:
$$
t_m = \frac{t_s}{n}
$$
When factoring in communication overhead time, the perfect speedup drops. This model is considered unrealistic because algorithms typically contain non-dividable serial components.
$$
S(n) = \frac{n}{1 + \left(n \times \frac{t_c}{t_s}\right)}
$$
- Parallel Computation With Serial Sections (Amdahl's Model): This accounts for a specific fraction of a task that must remain strictly sequential, while the rest can be processed concurrently. When accounting for communication overhead, the speedup is intrinsically limited:
$$
S(n) = \frac{n}{f(n-1) + 1 + n\left(\frac{t_c}{t_s}\right)}
$$
As the number of processors approaches infinity, the maximum possible speedup hits a strict ceiling:
$$
S_{max} = \frac{1}{f + \left(\frac{t_c}{t_s}\right)}
$$
Arguments For Parallel Architectures
- Grosch's Law: An older principle stating that computing power scales with the square of the system's cost, implying a computer must be four times faster to justify costing twice as much. Parallel architectures effectively repeal this law by clustering cheaper processors together.
$$
P = K \times C^2
$$
- Amdahl's Law: Argues that overall performance improvement is bottlenecked by the fraction of the program that is naturally serial. The speedup is strictly limited regardless of how many processors are added to the system.
$$
S = \frac{N}{\beta N + (1-\beta)}
$$
- Gustafson-Barsis's Law: Counters Amdahl by arguing that the problem size naturally scales up alongside the number of processors. By keeping parallel time fixed and scaling serial time backward, significantly higher performance gains are possible.
$$
S = N - (N-1)\alpha
$$