A deep dive into Shopware 6 Caching

In this blog post, I want to dive into the Shopware Caching to clarify how it works and what you have to care about while developing features. We will go from the inner to the outer caching layer in multiple steps and look into them independently.

Stage one - the object cache

First, we will start with the object cache implemented in serval “route” classes. The “route” classes/controllers are always used in the Shopware context as an entry point for the store-API. These are controllers and services simultaneously, so the default storefront and the store-API can share the same code.

Many of the “route” have an object cache. It makes it possible to cache the store-API and storefront calls in the same behavior. Also, we cache a minor part of the entire storefront application, so cache invalidation can work better and will not affect anything on the page.

Example of this behaviour:

We have here an request for a product detail page. The pager loader fetches the information from three sources. If we update the product, we have to invalidate only the cache of the product route. We can still use the cached navigation or other basic page information like currencies/languages.

The implementation

Let’s look more at the actual cache layer implementation. All route classes for the store-API have an abstract class for decoration. So Shopware has some default decorators for caching. For simplicity, we will look into the CachedCurrencyRoute.

public function load(Request $request, SalesChannelContext $context, Criteria $criteria): CurrencyRouteResponse
{
    if ($context->hasState(...$this->states)) {
        return $this->getDecorated()->load($request, $context, $criteria);
    }

    $key = $this->generateKey($request, $context, $criteria);

    $value = $this->cache->get($key, function (ItemInterface $item) use ($request, $context, $criteria) {
        $name = self::buildName($context->getSalesChannelId());
        $response = $this->tracer->trace($name, function () use ($request, $context, $criteria) {
            return $this->getDecorated()->load($request, $context, $criteria);
        });

        $item->tag($this->generateTags($request, $response, $context, $criteria));

        return CacheValueCompressor::compress($response);
    });

    return CacheValueCompressor::uncompress($value);
}

First, the decoration looks if the given context has a configured state; the entire caching layer will be skipped when states match. This check allows easily disabling caching of certain routes with configuration. The configuration can be done in a config/packages/shopware.yaml. See default config for the default configuration.

By default only two states are implemented cart-filled and logged-in see here.

As next, a cache key will be generated by the method’s input. The generateKey also fires an event to manipulate the build key. If the key hits, we return the cache; if not, we cache the result and tag it. The fired event can also extend the tags with the tags of the cache tracer, which contains all the tags of the used services.

The cache tracer is used to determine the cache tags when routes are called in routes, as the outer route needs to have the whole tags to invalidate the cache correctly. While tracing, all the usages of system config, translations, or theme variables will also be considered in the cache tags.

Stage two - the http cache

Shopware is already pretty fast with the object cache implemented in the routes. But we still need to bootstrap the application and render the twig template. To get rid of that, we can use the built-in HTTP cache, which will be in front of our controller. So in our diagramm, it will look like so:

How does Shopware decide that a page will be entirely cached?

That a page that will be cached will be decided by the controller’s response cache-control header, this header usually is not set. Shopware uses @HttpCache annotation like here in the NavigationController. That annotation is considered in the CacheResponseSubscriber, and this marks the response as cacheable.

The cache-control header

The cache-control header we set in the controller is not the same as we see in the browser. The active HTTP cache respects that header and replaces it with a private cache so the actual browser will not cache the Website and ask the server always for the request. When an external reverse proxy is enabled, this header will be passed without modification, and the external system should consider it and must set the cache-control to private again.

Cache states

Like the object cache, the HTTP cache also uses the states. The cache will be ignored when a client and the response have the same state. In default is the cache configured that when a cart is filled or the user is logged in, the http cache will be ignored. The states in the response will be set by the CacheResponseSubscriber, for the client the states are saved as a cookie and they are set in the same class in the getSystemStates method.

Cache key

The HTTP cache key is generated inside the HttpCacheKeyGenerator and uses as base the request URI and appends the current context hash to it. The context hash is taken from the sw-context-hash cookie first when it exists and fallbacks to sw-currency cookie. The sw-context-hash is only set when the customer is logged in; this order is set so to maximize the cache hit rate as all logged-out customers with the same currency have the same cache hash. When logged in, the active rules of the customer matter, but in default, the cache is skipped for logged in customers with the state configuration

Cache tags

The cache tags of the HTTP cache are built from all the caches in the entire stack. If a store-API route has been called and it has cached something, this route’s tags will also be considered for the HTTP cache. Also, there are tracers for system config and Symfony translator to trace all used config keys/translations in the template to invalidate this page when it changes. This is built by an custom CacheDecorator which writes all used tages into a CacheTagCollection where at the end of the request the http cache gets the tags from.

Final setup - reverse http cache

We can now serve requests fast, but we still need to bootstrap the Symfony container and do a lookup in a Redis/Filesystem, serve the request with PHP and maybe replace some CSRF placeholders with a session. To serve the entire page without loading on our actual server, we can use a reverse HTTP cache like Varnish or external services like Fastly.

With Varnish, our Infrastructure would look like this:

In the Varnish case, our Browser talks to a Webserver like Nginx to do SSL termination (so our Website has https). The Nginx forwards the request to the Varnish, and it forwards it conditionally to the actual Shopware server.

In the best case, the Varnish has the Website cached and delivers the cached Website without talking with our Shopware server. The cache-control header is used to tell Varnish which pages should be cached. As we learned in the previous section that the cache-control is always private to the end-user, we need to configure Shopware properly to forward the cache-control without modifying it.

This is explicitly done with the storefront.reverse_proxy.enabled config.

storefront:
      csrf:
        mode: ajax
    reverse_proxy:
        enabled: true
        hosts: [ "http://varnish" ]
        redis_url: "redis://redis"

To invalidate, later, the pages on the reverse proxy cache Shopware needs to know all cache server hosts and a Redis server to hold the mapping of cache-key to URL. A normal site cache can have up to 100 tags in Shopware, which doesn’t work well with the Varnish provided configuration so that we can hold this better in Redis. An external service like Fastly is not required as they support larger tag amounts.

CSRF and external Caching

As the last step, we need to configure that the CSRF handling happens with ajax. By default, Shopware generates CSRF placeholders in the HTML response, caches the response, and replaces it afterward before serving it to the actual client with the real CSRF tokens. This is very smart and makes the Caching a lot easier; if you are interested in how it is working, you can find it in the CsrfPlaceholderHandler. As we serve the cached page directly in Varnish, we cannot generate CSRF tokens. To fix this behavior, we can switch the CSRF implementation to ajax. So instead of generating CSRF tokens into the cached HTML page, we do an ajax to get the CSRF token for the form we are submitting.

The cache key and the states

The Shopware stack is not called when the cache key is generated, so we must implement it in the reverse cache itself. In Varnish, we would do this in the vcl_hash subroutine.

sub vcl_hash {
    if (req.http.cookie ~ "sw-cache-hash=") {
        hash_data("+context=" + regsub(req.http.cookie, "^.*?sw-cache-hash=([^;]*);*.*$", "\1"));
    } elseif (req.http.cookie ~ "sw-currency=") {
        hash_data("+currency=" + regsub(req.http.cookie, "^.*?sw-currency=([^;]*);*.*$", "\1"));
    }
}

As for the standard built-in HTTP cache, we first consider the sw-cache-hash and the sw-currency to maximize the cache hit rate.

The same process must be done for the states to skip the cache for specific situations with a vcl_hit subroutine.

sub vcl_hit {
# When the request has an sw-states cookie
  if (req.http.cookie ~ "sw-states=") {
     set req.http.states = regsub(req.http.cookie, "^.*?sw-states=([^;]*);*.*$", "\1");

# When the client and the response has the same state skip it
     if (req.http.states ~ "logged-in" && obj.http.sw-invalidation-states ~ "logged-in" ) {
        return (pass);
     }

# When the client and the response has the same state skip it
     if (req.http.states ~ "cart-filled" && obj.http.sw-invalidation-states ~ "cart-filled" ) {
        return (pass);
     }
  }
}

This configuration can be done more dynamically with Varnish Plus, as you can split their strings.