java-topology/whitepaper/vectors/compiler/scala3.rst

Scala 3 (``scalac``) — CWE-407 Analysis
=========================================

.. contents:: :local:

Overview
--------

Scala 3 (Dotty) is the current production Scala compiler. Its type inference engine
solves type parameter ordering constraints via ``OrderingConstraint``, which tracks
``<:<`` relationships between type parameters. The constraint-solving code is invoked
on every call to ``minLower`` / ``minUpper``, which are hot paths during implicit
resolution and HKT (higher-kinded type) elaboration.

**Status: 1 confirmed defect — CRITICAL severity, PATCHED**

This is the highest-severity site in the entire CWE-407 survey: **O(n³)** complexity,
not O(n²). The defect triples the exponent relative to the canonical CWE-407 pattern.

Confirmed Defects
-----------------

.. list-table::
   :header-rows: 1
   :widths: 15 40 20 10 15

   * - ID
     - File:Line
     - Pattern
     - Complexity
     - Severity
   * - scala3-0001
     - ``core/OrderingConstraint.scala:243``
     - ``List[TypeParamRef].contains`` in ``isLess`` inside ``minLower``/``minUpper``
     - O(n³)
     - **CRITICAL**

Defect Detail — scala3-0001
-----------------------------

``OrderingConstraint`` maintains ordering relations between type parameters as the
type inference engine adds ``<:<`` facts. The central membership test is ``isLess``::

   // OrderingConstraint.scala:243 — THE DEFECT
   def isLess(param1: TypeParamRef, param2: TypeParamRef): Boolean =
     upper(param1).contains(param2)   // O(n) List scan

``upper(param)`` retrieves the stored upper-bound set for a type parameter.
The storage type is:

.. code-block:: scala

   private type ParamOrdering = ArrayValuedMap[List[TypeParamRef]]

``List.contains`` is a sequential scan — O(n) where n = number of constrained
type parameters.

``isLess`` is called from ``minLower`` and ``minUpper``:

.. code-block:: scala

   def minUpper(param: TypeParamRef): List[TypeParamRef] = {
     val all = upper(param)                         // O(n)
     all.filterNot(p => all.exists(isLess(_, p)))  // O(n) calls × O(n) each = O(n²)
   }

For n constrained type parameters, ``minUpper`` costs O(n²). Called for each of n
parameters: **O(n³) total**.

Complexity Analysis
-------------------

Let V = number of currently constrained type parameters.

.. list-table::
   :header-rows: 1
   :widths: 30 35 35

   * - Operation
     - Before (defective)
     - After (patched)
   * - ``isLess(p1, p2)``
     - O(V) — List.contains linear scan
     - O(1) — Set.contains hash lookup
   * - ``minUpper(param)`` / ``minLower(param)``
     - O(V²) — O(V) loop × O(V) isLess
     - O(V) — O(V) loop × O(1) isLess
   * - Full constraint solve (n params)
     - O(V³)
     - O(V²)

At V=50 (realistic for Cats/Shapeless HKT chains):

- Before: 125,000 operations
- After: 2,500 operations
- **Speedup: 50×**

At V=100:

- Before: 1,000,000 operations
- After: 10,000 operations
- **Speedup: 100×**

The Fix
-------

Change ``ParamOrdering`` storage from ``List[TypeParamRef]`` to ``Set[TypeParamRef]``.
Add private ``upperSet``/``lowerSet`` accessors for O(1) internal lookup.
Keep public API returning ``List[TypeParamRef]`` via ``.toList`` for backward compatibility.

.. code-block:: scala

   // Before — ParamOrdering
   private type ParamOrdering = ArrayValuedMap[List[TypeParamRef]]

   // After
   private type ParamOrdering = ArrayValuedMap[Set[TypeParamRef]]

   // New private accessors (no .toList overhead in isLess)
   private def lowerSet(param: TypeParamRef): Set[TypeParamRef] =
     lowerLens(this, param.binder, param.paramNum)
   private def upperSet(param: TypeParamRef): Set[TypeParamRef] =
     upperLens(this, param.binder, param.paramNum)

   // isLess — O(n) → O(1)
   def isLess(param1: TypeParamRef, param2: TypeParamRef): Boolean =
     upperSet(param1).contains(param2)  // O(1)

Changed sites in ``OrderingConstraint.scala``:

.. list-table::
   :header-rows: 1
   :widths: 40 30 30

   * - Site
     - Before
     - After
   * - ``ParamOrdering`` type alias
     - ``ArrayValuedMap[List[...]]``
     - ``ArrayValuedMap[Set[...]]``
   * - ``lowerLens`` / ``upperLens`` ``initial``
     - ``Nil``
     - ``Set.empty``
   * - ``isLess``
     - ``upper(p).contains(p2)`` O(n)
     - ``upperSet(p).contains(p2)`` O(1)
   * - ``order()`` concat (lines 627-628)
     - ``newUpper ::: existing``
     - ``existing ++ newUpper``
   * - ``removeParamFrom`` in ``replace()``
     - ``ps.filterConserve(param ne _)``
     - ``ps - param``
   * - ``removeFromBoundss`` in ``remove()``
     - ``Array[List[...]]`` + ``filterConserve``
     - ``Array[Set[...]]`` + ``filterNot``

Semantics are preserved: ordering sets never contain duplicate ``TypeParamRef``
entries by construction (``order()`` checks ``isLess`` before adding). Converting
from ``Set`` to ``List`` at the public API boundary is safe — callers only iterate.

Patch
-----

``defects/scala3/patch/scala3-0001-ordering-constraint-set.patch``

102-line unified diff. Apply with:

.. code-block:: bash

   cd /path/to/scala3
   git apply /path/to/java-topology/defects/scala3/patch/scala3-0001-ordering-constraint-set.patch

Why CRITICAL
------------

This is the only O(n³) site in the survey. All other CWE-407 sites are O(n²).
The cubic exponent means:

- At V=10: 10× slower than necessary
- At V=100: 100× slower than necessary
- At V=1000: 1000× slower than necessary

Scala 3 projects that use Cats, Shapeless, or ZIO — all of which rely on implicit
resolution over many constrained type parameters — are directly impacted. Users
who observe non-linear compile time scaling with added type parameter constraints
are hitting this defect.

Scan Details
------------

* Scanner: targeted ``grep`` for ``List[TypeParamRef].contains`` and ``isLess`` call chain
* File: ``compiler/src/dotty/tools/dotc/core/OrderingConstraint.scala``
* Scanned: ``compiler/src/dotty/tools/dotc/core/``, ``typer/``
* Result: 1 confirmed defect