UNDF: UNDF-2026-000000557

UNDF: (pending)

traefik-MOADX-0001: CNAMEFlatten — cache stampede + data race O(N) concurrent DNS lookups

MOAD-0006 Candidate — The Thundering Herd

Field	Value
ID	traefik-MOADX-0001
Severity	HIGH
Ecosystem	Traefik
Language	Go
File	`pkg/middlewares/requestdecorator/hostresolver.go`
Lines	37–75
Pattern	get → miss → expensive DNS lookup → set (no singleflight, no mutex)
Trigger	N concurrent goroutines all hit same hostname with cold/expired cache
Compute cost	Full CNAME-chain DNS resolution via `cnameResolve()` with 30-second timeout per DNS server

Defect

CNAMEFlatten is called by RequestDecorator.ServeHTTP on every inbound HTTP request in the entrypoint's handler chain. The Resolver struct is a single instance shared across all goroutines for that entrypoint (created once in NewTCPEntryPoint, passed to requestdecorator.New).

Race 1 — lazy cache init data race:

// pkg/middlewares/requestdecorator/hostresolver.go:38-39
func (hr *Resolver) CNAMEFlatten(ctx context.Context, host string) string {
    if hr.cache == nil {
        hr.cache = cache.New(30*time.Minute, 5*time.Minute)  // DATA RACE
    }
    // ...
}

Multiple goroutines read hr.cache == nil simultaneously, then all write hr.cache = .... Go's race detector flags this immediately. The *cache.Cache pointer is written without any synchronization.

Race 2 — cache stampede on miss:

// pkg/middlewares/requestdecorator/hostresolver.go:50-68
value, found := hr.cache.Get(host)   // thread-safe individually
if found {
    return value.(string)
}

// NO LOCK OR SINGLEFLIGHT AROUND THIS BLOCK:
for depth := range hr.ResolvDepth {
    resolv, err := cnameResolve(ctx, request, hr.ResolvConfig) // expensive DNS call
    // ...
}

hr.cache.Set(host, result, cacheDuration)  // thread-safe individually

go-cache's Get and Set are individually mutex-protected, but the check-then-compute-then-set sequence is not atomic. When N goroutines all request the same hostname simultaneously on a cold or expired cache entry, all N pass the found == false check and all N execute cnameResolve() in parallel — each doing up to ResolvDepth DNS round-trips (each with 30-second timeouts).

Grower signal: Works fine at low traffic. Under a traffic spike to a new backend hostname, or when the 30-minute TTL expires on a popular hostname, all concurrent requests trigger DNS lookups simultaneously. With ResolvDepth=5 (default) and N=1000 concurrent goroutines, this is 5000 concurrent DNS UDP/TCP connections.

Fix

Option A — singleflight.Group (preferred, matches Traefik's existing pattern):

import "golang.org/x/sync/singleflight"

type Resolver struct {
    CnameFlattening bool
    ResolvConfig    string
    ResolvDepth     int
    cache           *cache.Cache
    cacheOnce       sync.Once
    group           singleflight.Group
}

func (hr *Resolver) CNAMEFlatten(ctx context.Context, host string) string {
    hr.cacheOnce.Do(func() {
        hr.cache = cache.New(30*time.Minute, 5*time.Minute)
    })

    if value, found := hr.cache.Get(host); found {
        return value.(string)
    }

    result, _, _ := hr.group.Do(host, func() (any, error) {
        // Only one goroutine resolves; others wait and share the result.
        res := host
        req := host
        cacheDuration := 0 * time.Second
        for depth := range hr.ResolvDepth {
            resolv, err := cnameResolve(ctx, req, hr.ResolvConfig)
            if err != nil || resolv == nil {
                break
            }
            res = resolv.Record
            if depth == 0 {
                cacheDuration = resolv.TTL
            }
            req = resolv.Record
        }
        hr.cache.Set(host, res, cacheDuration)
        return res, nil
    })
    return result.(string)
}

Option B — sync.Once for init + sync.Map as cache:

Replace *cache.Cache with sync.Map and use a per-key singleflight.Group. Traefik already uses singleflight in basic_auth.go (line 118) and healthcheck.go — the pattern is established in the codebase.

Impact

Data race: Go runtime will panic under -race flag; undefined behavior in production (cache pointer torn write).
Stampede: N×ResolvDepth DNS connections on cache miss. At N=1000 goroutines with ResolvDepth=5, a single cache expiry produces 5000 DNS queries from one Traefik instance. DNS resolver exhaustion, connection table blowup, upstream SERVFAIL cascade.
Fix speedup: After fix, cache miss costs 1 DNS lookup regardless of concurrent request count.

4.6 KiB Raw Blame History Unescape Escape