What a Processor Cache Can and Cannot Hide
What a Processor Cache Can and Cannot Hide
A cache shortens trips to memory when a workload revisits data in predictable ways.
Processors keep recently used instructions and data close to their cores. A cache hit avoids a longer memory lookup, while a miss sends the request to another cache level or main memory.
A workload that walks a large dataset in an irregular pattern can quickly exhaust this advantage. Two threads sharing a cache can also evict each other's data.
Compare latency and bandwidth as the working set grows. Changes at cache boundaries reveal more about a workload than a single score taken at one data size.
