java-topology/whitepaper/outreach/opensmtpd.md

2.3 KiB
Raw Blame History

OpenSMTPD — CWE-407 Disclosure Brief

2026-03-27 · Patch available — awaiting upstream merge

Finding

One O(n²) defect in OpenSMTPD's ruleset matching. ruleset_match() uses TAILQ_FOREACH to linearly walk all R rules for every envelope processed, causing O(R) per message and O(R×M) under load. Patch ready for upstream review.

The Defects

opensmtpd-0001 (PATCHED — HIGH): ruleset.c:234

/* Inside ruleset_match() — per envelope: */
TAILQ_FOREACH(rule, &env->sc_rules, r_entry) {
    /* O(R) full rule scan per envelope */
    if (rule_matches(rule, envelope)) {
        return rule;
    }
}

TAILQ_FOREACH walks all R rules linearly on every envelope match. For M messages and R rules: O(R × M) total. Measured ratio: 146×.

Complexity Proof

For R=146 rules:

  • Per envelope: O(R) full scan
  • Fixed: domain dispatch dict → O(1) for domain-matched rules
  • 146× measured ratio.

Impact

All OpenSMTPD deployments with complex ruleset configurations. OpenSMTPD is the default MTA on OpenBSD and is used on many security-focused servers. Rule evaluation runs on every message processed — incoming, outgoing, and forwarded. Servers with large rulesets (many virtual domains, complex filtering rules) hit worst case on every message.

The Fix

Pre-build a domain-indexed dispatch dictionary from the rules at startup:

/* Before */
TAILQ_FOREACH(rule, &env->sc_rules, r_entry) {
    if (rule_matches(rule, envelope)) return rule;
}

/* After */
/* CWE-407 fix: domain dispatch dict for O(1) initial lookup, then filtered scan. */
struct rule_list *domain_rules = dict_get(&env->sc_rules_by_domain, domain);
if (domain_rules) {
    TAILQ_FOREACH(rule, domain_rules, r_entry) {
        if (rule_matches(rule, envelope)) return rule;
    }
}

Patch

defects/opensmtpd/patch/opensmtpd-0001-ruleset-dispatch-dict.patch

What We Ask

  1. Confirm receipt and assign a GitHub Security Advisory or issue reference.
  2. Validate the patch against your ruleset matching test suite.
  3. Assess CVE eligibility — fires on every message processed with a large ruleset.
  4. Coordinate a disclosure date — we are targeting 90 days from first contact.

Contact: see cover email. This brief is confidential until coordinated disclosure.