Open Source Summit + Embedded Linux Conference North America 2026

Caching has traditionally relied on exact matches: the same input produces the same cached output. AI systems challenge this assumption by introducing semantic similarity — requests that are different on the surface but equivalent in meaning. This talk explores how caching is evolving to support AI workloads, from classical key-value strategies to semantic caching using vector search. We'll walk through a practical architecture that layers exact and semantic caches in front of an expensive model and demonstrate how hybrid caching can reduce cost and latency. This talk will explore multiple open-source systems, such as OpenSearch and Valkey, and discuss the tradeoffs that they provide and when they matter.