java-topology/docs/tickets/varnish-0001-ban-check-object-linear-ban-list-walk.md

varnish-0001: BAN_CheckObject linear ban list walk per cache hit

Target: varnish-cache File: bin/varnishd/cache/cache_ban.c Function: BAN_CheckObject Lines: 688–693 Severity: HIGH CWE: CWE-407 (Inefficient Algorithmic Complexity)

Description

On every cache hit, Varnish calls BAN_CheckObject to determine whether the cached object is covered by a newer ban. The function walks the global ban list from ban_start to the object's own ban pointer:

for (b = b0; b != bn; b = VTAILQ_NEXT(b, list)) {
    CHECK_OBJ_NOTNULL(b, BAN_MAGIC);
    if (b->flags & BANS_FLAG_COMPLETED)
        continue;
    if (ban_evaluate(wrk, b->spec, oc, req->http, &tests))
        break;
}

ban_evaluate itself evaluates the ban expression (URL/header regex or field comparison) against the object. The list is kept in insertion order (newest first). An object created at ban-list position K must check K pending bans on every hit until the lurker catches up.

With B pending bans and H cache hits/second, total ban_evaluate calls = O(B × H). A sustained ban workload (e.g., frequent content invalidation) where the lurker falls behind creates a compounding penalty.

Impact

• Workload: 1 000 pending bans × 10 000 cache hits/second = 10 000 000 ban_evaluate calls/second before lurker catches up.
• ban_evaluate walks the binary ban-spec and may execute regex per call.
• The per-request path holds oc->objhead->mtx for the duration of the loop.

Fix

Two-stage approach:

1. The lurker already attempts to pre-test objects and move their oc->ban pointer forward. Tuning ban_lurker_batch and ban_lurker_sleep downward reduces the backlog.
2. Structural fix: index bans by the object fields they test (URL prefix, specific header name). A trie keyed on URL or a per-field hash map lets BAN_CheckObject skip non-matching bans in O(log B) or O(1) instead of O(B).

Patch

See defects/varnish/patch/varnish-0001.patch

Unit Test

See defects/varnish/unit/VarnishBanCheckAlgorithmTest.java