Varnish is a web application accelerator also known as a caching HTTP reverse proxy. You install it in front of any server that speaks HTTP and configure it to cache the contents.

It is fast with a speed factor of 300 - 1000x.

It offers flexibility due to its configuration language, VCL.

In contrast to other web accelerators, such as Squid, which began life as a client-side cache, or Apache and nginx, which are primarily origin servers, Varnish was designed as an HTTP accelerator. Varnish is focused exclusively on HTTP, unlike other proxy servers that often support FTP, SMTP and other network protocols.

Varnish stores data in virtual memory and leaves the task of deciding what is stored in memory and what gets paged out to disk to the operating system. This helps avoid the situation where the operating system starts caching data while it is moved to disk by the application.

Varnish is heavily threaded, with each client connection being handled by a separate worker thread. When the configured limit on the number of active worker threads is reached, incoming connections are placed in an overflow queue; when this queue reaches its configured limit incoming connections will be rejected.

Varnish supports load balancing using both a round robin and a random director. Load balancing [1] improves the distribution of workloads across multiple computing resources, such as computers, a computer cluster, network links, central processing units, or disk drives.[1] Load balancing aims to optimize resource use, maximize throughput, minimize response time, and avoid overload of any single resource.

Check a Page is Served from Varnish Cache

Check the X-FPFIS header: if one number: not cached by varnish / if two numbers: cached by varnish). On a fresh request (an uploaded file) try to view the headers: Network tab then click on the request and read:

Fresh request not served by Varnish.

age:0
X-FPFIS:1803895439
Cache-Control:max-age=1209600, s-maxage=0

Later, the same request is found as stored. Now, file is served by Varnish cache and is 41 seconds old.

age:41 
X-FPFIS:1803897159 1803895439
max-age=1209600, s-maxage=0

After hiting the button ‘Purge all caches’ the initial state is restored: age: 0 X-FPFIS:1803895439

Age is the amount of time since the response (or its revalidation) was generated at the origin server. Using this, intermediate proxies can estimate how old a stored response is: age: 0 it means 0 seconds old. max-age is used to define the maximum age limit: e.g.max-age: 604800 means 1 week: 604800 / 60" / 60' / 24h = 7 days.

When to Purge Files

Sites needs an occasional purge of files via provided button but also a regular purge of files: users often update files with the same name. If the users change the file names the purge is not necessary.

A custom rule can be created to purge files: do not purge all files from cache, purge files of a specific node if possible. Limit as much as possible the case of purging all files caches — it could be dozens GB. When creating the purge rules look first at the link of a uploaded files which could be like: sites/site_name/files/ and add the necessary folder name.

It is recommended an organized filesystem: /files/<content-type name>/<filename> so cache can be purged based on this pattern /files/<content-type name>/*

Rules to be tested in playground first.

Another option: to be investigated using same rules in the code to avoid human errors.

CDN Caching

CDN (Content Delivery Network) adds X-cache header to HTTP Response. X-cache:HIT means that your request was served by CDN.

References

[1] https://en.wikipedia.org/wiki/Load_balancing_(computing)