Caching the uncacheable in Varnish - Thijs Feryn - NDC Oslo 2024

Web performance and caching are essential for user experience and business success - slow websites can be as damaging as downtime

Varnish Cache is a reverse caching proxy that can handle 1.3 Tb/s on a single machine, used by 22% of top 10,000 websites

Default Varnish behavior is not to cache when cookies or authorization headers are present, but this can be customized using VCL (Varnish Configuration Language)

Key strategies for caching “uncacheable” content:

• Strip unnecessary cookies while keeping essential ones
• Use ESI (Edge Side Includes) for dynamic page fragments
• Leverage synthetic responses for personalized content
• Create cache variations based on languages or other parameters
• Handle authentication at the edge using JWT validation

WebAssembly support is being added to Varnish for extended edge computing capabilities

Cache variations should be carefully managed to avoid too many unique objects that could reduce hit rates

Smart header handling is crucial:

• Vary header for creating appropriate cache variations
• Cache-Control directives for controlling caching behavior
• Surrogate-Control for CDN-specific instructions

Redis can be integrated with Varnish for session handling and dynamic data

Modern solutions like Edge Stash provide alternatives to traditional ESI for template processing at the edge

Performance optimization should focus on caching before adding servers, as horizontal scaling alone is not a sustainable solution