MapReduce

MapReduce is a programming model and processing framework, popularized by Google’s 2004 paper and later implemented in open source by Apache Hadoop, for computing over massive datasets in parallel across a cluster of commodity machines. It breaks a job into two phases: a map phase that transforms input records into intermediate key/value pairs, and a reduce phase that aggregates those pairs into a final result. The framework itself handles the hard parts — distribution, scheduling, and fault tolerance — so developers can focus on the map and reduce functions.

• Map phase - Applies a function to each input record, emitting intermediate key/value pairs.
• Reduce phase - Groups by key and aggregates the intermediate values into final output.
• Automatic parallelism and fault tolerance - The framework partitions work, reschedules failed tasks, and moves computation to the data.
• Commodity-cluster scale - Designed to run across many inexpensive machines rather than one large server.

MapReduce shaped a generation of big-data infrastructure and the batch analytics that sit behind many data APIs, even as newer engines like Spark have largely superseded it for interactive workloads. In the API world it usually lives one layer down — the batch pipeline that computes the aggregates, indexes, and datasets an API then serves — which is exactly how it appeared in early cloud reference architectures like AWS Elastic MapReduce. Understanding the model still helps when reasoning about how large, derived datasets behind an API are produced and refreshed.