Cluster Computing: A collection of independent, stand-alone computers that are connected together over a local network to function as a single, integrated computing resource. They serve as a cost-effective alternative to expensive, specialized parallel machines.
Grid Computing: A geographically distributed computing platform made up of interconnected resources across multiple locations, leveraging network infrastructure and the Internet to handle large-scale computations.

Flynn's taxonomy

a classification system for computer architectures based on the number of concurrent instruction streams and data streams

SISD (Single Instruction, Single Data)

Overview: A traditional sequential uniprocessor machine where one instruction processes one data stream at a time.
Examples: Classic von Neumann architecture, early single-core personal computers.

SIMD (Single Instruction, Multiple Data)

Overview: A parallel architecture where a single instruction is applied to multiple data items simultaneously, commonly used in vector and array processing.
Examples: Modern GPUs, CPU vector extensions like AVX — Matrix mul — ILLIAC IV.

consists of 2 parts:

The Front-End Computer (also called the Host Computer or Scalar Unit/Control Unit)
The Processor Array (a grid or set of Processing Elements / PEs)

MISD (Multiple Instructions, Single Data) — NOT COVERED

Overview: A model where different instructions operate on the same single data stream. Largely theoretical or used for specialized fault-tolerant systems.
Examples: Redundant fault-tolerant systems — modern AI systems.

MIMD (Multiple Instructions, Multiple Data)

Overview: A system where multiple independent processors execute different instructions on different data sets simultaneously.