engineering/remarkbox
1
0
Fork 0
Code Issues 3 Pull requests 1 Projects Releases Packages Wiki Activity Actions

fix: export markdown from data_html, not data+source_format

Browse source 
Observed defect: a node's source_format column can disagree with the
actual bytes in data. A Collatz thread on foxhop.net has data filled
with RST (======, ---, :: blocks) yet source_format="markdown". The
earlier helper trusted source_format and returned the RST untouched,
so GET /api/v1/export/.../thread.md served raw RST to anyone who asked
for markdown.

data_html is the canonical rendered form — set at every write via
set_data() regardless of input syntax — so converting data_html → markdown
via pandoc sidesteps the label mismatch entirely. Falls back to raw
data if data_html is missing or pandoc fails.
This commit is contained in:
russell@unturf.com 2026-04-24 15:09:31 -04:00
parent f5e453d6ba
commit dded1622e8
1 changed files with 24 additions and 22 deletions
Download patch file Download diff file Expand all files Collapse all files

46
remarkbox/lib/pandoc.py
View file

@ -249,30 +249,32 @@ def convert(source, from_format="markdown", to_format="html5", title=None, stand
def _node_data_as_markdown(node): def _node_data_as_markdown(node):
"""Return node.data rendered as markdown. """Return node content rendered as markdown.
Nodes authored in non-markdown source_formats (rst, mediawiki, latex, html, The canonical rendered form of a node is node.data_html — produced at
...) keep their raw source in .data. For export we need real markdown, so write time regardless of what source syntax the author used (markdown,
convert through pandoc when needed. A pandoc failure falls back to the raw rst, mediawiki, latex, html...). Converting HTML → markdown always
source — better a rough dump than an empty export. yields real markdown, and sidesteps bugs where a node's source_format
label disagrees with the actual bytes stored in node.data (observed
in the wild: RST content labelled source_format="markdown").
Falls back to raw node.data if data_html is absent or the conversion
fails — better a rough dump than an empty export.
""" """
if not node.data: html = getattr(node, "data_html", None)
return "" if html:
src_fmt = (getattr(node, "source_format", None) or "markdown").lower() try:
if src_fmt in ("markdown", "gfm", "commonmark", "commonmark_x"): return convert(
return node.data html, from_format="html", to_format="markdown",
try: standalone=False,
return convert( ).rstrip()
node.data, from_format=src_fmt, to_format="markdown", except Exception:
standalone=False, log.exception(
).rstrip() "Pandoc failed converting node %s data_html to markdown; "
except Exception: "falling back to raw source",
log.exception( node.id,
"Pandoc failed converting node %s from %s to markdown; " )
"falling back to raw source", return (node.data or "")
node.id, src_fmt,
)
return node.data
def node_tree_to_markdown(root_node, nodes, include_root=True): def node_tree_to_markdown(root_node, nodes, include_root=True):