russell@unturf.com
3e60bef516
Pure-Python (stdlib only). Uses chunk-level corpus baseline so terms concentrated in fewer chunks outrank common ones. Default top-K = 16. Output cores are comma-separated keyword lists — extreme compression toward the tweet/haiku end of the planet metaphor. Same source can now carry both a first-sentence-v1 core AND a tfidf-keywords-v1 core, each derived independently and Merkle-signed back to the same surface. The 'contributing_chunk_indices' for TF-IDF is every chunk that contains at least one of the top-K keywords — proof binding remains honest and cryptographically tight.
20 lines
632 B
Python
20 lines
632 B
Python
"""Distillation: surface docs -> core docs, Merkle-signed back."""
|
|
|
|
from aborist.distill.base import DistillationResult, Distiller
|
|
from aborist.distill.first_sentence import FirstSentenceDistiller
|
|
from aborist.distill.tfidf import TfidfKeywordDistiller
|
|
|
|
__all__ = [
|
|
"DistillationResult",
|
|
"Distiller",
|
|
"FirstSentenceDistiller",
|
|
"TfidfKeywordDistiller",
|
|
]
|
|
|
|
|
|
def get_distiller(name: str) -> Distiller:
|
|
if name == FirstSentenceDistiller.name:
|
|
return FirstSentenceDistiller()
|
|
if name == TfidfKeywordDistiller.name:
|
|
return TfidfKeywordDistiller()
|
|
raise ValueError(f"unknown distiller: {name}")
|