# OpenSSL — CWE-407 Disclosure Brief

**Project:** OpenSSL
**Disclosure date:** 2026-03-27
**Severity:** HIGH
**Speedup:** varies with n×m, n², C×S
**Status:** PATCHED

---

## Finding

OpenSSL contains three independent quadratic-complexity defects in cipher negotiation and SRTP profile matching within `ssl/ssl_ciph.c` and `ssl/statem/extensions_srvr.c`. Two are in cipher suite deduplication during TLS connection setup (one TLS 1.2, one TLS 1.3), and one is in SRTP profile matching for DTLS/WebRTC connections. All three execute on the hot path of TLS handshake processing.

## The Defect(s)

| ID | Location | Pattern | Complexity |
|----|----------|---------|------------|
| openssl-0001 | `ssl/ssl_ciph.c` | `SSL_get_shared_ciphers()` O(n×m) scan per connection when server stack unsorted | O(n×m) |
| openssl-0002 | `ssl/ssl_ciph.c` | `ciphersuite_cb` TLS 1.3 dedup O(n²) during config parsing | O(n²) |
| openssl-0003 | `ssl/statem/extensions_srvr.c` | `tls_parse_ctos_use_srtp()` outer while over C client IDs × inner for over S server profiles | O(C×S) |

## Complexity Proof

**openssl-0001** — Let n = size of client cipher list, m = size of server cipher list.

`SSL_get_shared_ciphers()` finds the intersection by iterating all n client ciphers and for each scanning all m server ciphers linearly:

```
For client_cipher_1: scan m server ciphers
For client_cipher_2: scan m server ciphers
...
For client_cipher_n: scan m server ciphers
Total: n × m comparisons per connection
```

With a hash set of server cipher IDs (built once at config time), each client cipher lookup is O(1), reducing total work to O(n+m).

**openssl-0002** — Let n = number of TLS 1.3 ciphersuites in the configured list.

The `ciphersuite_cb` deduplication iterates the accumulating output list for each new candidate, scanning up to n entries:

```
Total: 0 + 1 + ... + (n-1) = n(n-1)/2 = O(n²)
```

A bitmask indexed by the cipher table index (bounded by the small fixed TLS 1.3 suite count) reduces each check to O(1).

**openssl-0003** — Let C = number of client SRTP profile IDs, S = number of server SRTP profiles.

`tls_parse_ctos_use_srtp()` iterates client IDs in an outer `while` loop and for each scans the server profile list in an inner `for` loop:

```
Total: C × S comparisons per DTLS handshake
```

A 32-slot Knuth hash set (the SRTP profile space is small) reduces this to O(C+S).

## Impact

**openssl-0001** affects all TLS 1.2 connections where the server cipher stack is not pre-sorted or filtered. High-connection-rate servers (HTTPS load balancers, API gateways, CDN edge nodes) pay this on every handshake.

**openssl-0002** affects OpenSSL configuration parsing and TLS context setup when custom TLS 1.3 ciphersuite lists are configured. Applications that dynamically create SSL_CTX objects (multi-tenant servers, connection pooling) are most affected.

**openssl-0003** affects all DTLS/WebRTC connections using SRTP (RFC 5764). Media servers, SFUs, and WebRTC gateways processing many simultaneous DTLS handshakes are most affected. C = 20 client profiles × S = 10 server profiles = 200 comparisons per handshake, reduced to 30 with the hash set.

## The Fix

**openssl-0001:** Pre-build a hash set of server cipher IDs at `SSL_CTX` configuration time. Use it in `SSL_get_shared_ciphers()`.

**openssl-0002:** Use a `uint64_t` bitmask indexed by position in the fixed TLS 1.3 cipher table for O(1) dedup.

**openssl-0003:** Build a 32-slot Knuth multiplicative hash set of server profile IDs before the outer client loop.

## Patch

```diff
# openssl-0001: ssl/ssl_ciph.c
- for (i = 0; i < client_len; i++) {
-     for (j = 0; j < server_len; j++) {
-         if (client[i]->id == server[j]->id) { ... }
-     }
- }
+ /* pre-build server_id_set at ctx init */
+ for (i = 0; i < client_len; i++) {
+     if (hash_set_contains(server_id_set, client[i]->id)) { ... }
+ }

# openssl-0002: ssl/ssl_ciph.c
- /* TLS 1.3 ciphersuite dedup — O(n²) */
- for (i = 0; i < num; i++) {
-     for (j = 0; j < i; j++) {
-         if (list[j] == list[i]) { duplicate = 1; break; }
-     }
- }
+ /* bitmask dedup — O(n) */
+ uint64_t seen = 0;
+ for (i = 0; i < num; i++) {
+     int idx = cipher_table_index(list[i]);
+     if (seen & (1ULL << idx)) { duplicate = 1; continue; }
+     seen |= (1ULL << idx);
+ }

# openssl-0003: ssl/statem/extensions_srvr.c
- while (PACKET_get_net_2(&profiles_list, &id)) {
-     for (i = 0; i < server->srtp_profiles->num; i++) {
-         if (server->srtp_profiles->profiles[i].id == id) { match = 1; }
-     }
- }
+ /* build 32-slot hash set of server profile IDs first */
+ srtp_hash_set_t srv_set;
+ srtp_hash_init(&srv_set, server->srtp_profiles);
+ while (PACKET_get_net_2(&profiles_list, &id)) {
+     if (srtp_hash_contains(&srv_set, id)) { match = 1; break; }
+ }
```

## What We Ask

Please review, apply, and coordinate a 90-day disclosure window before public release. Reply to security@undefect.com.

---

*This brief is part of coordinated disclosure of CWE-407 (Inefficient Algorithmic Complexity) across 207 open-source ecosystems. Full report: https://undefect.com*
