For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (
- [ ]) syntax for tracking.
Goal: Add a dependency-free Markdown-to-ContentState importer to draftjs_exporter, with a two-phase parse-then-filter architecture.
Architecture: MarkdownImporter wires together MarkdownParser (CommonMark core → valid ContentState) and ContentStateFilter (declarative rules → filtered ContentState). Entity creation for links/images goes through configurable resolver chains; a shipped scheme_resolver helper handles internal-URL syntaxes like wagtail://image?id=10. A configurable inline_html_styles whitelist maps paired HTML tags (<sup>) to inline styles. Spec: docs/superpowers/specs/2025-07-21-markdown-importer-design.md.
Tech Stack: Python 3.10+ stdlib only (re, urllib.parse), pytest + unittest-style test classes, Hypothesis for property tests, ruff + mypy + ty for lint/type-check.
- No external dependencies in production code. Stdlib only.
- Python floor: 3.10 (walrus,
X | None, andmatchare allowed). - Core classes must use
__slots__. - Docstrings required on all public modules, classes, methods, functions (Google style; types in annotations, not docstrings).
- Type annotations required on production code; must pass
ruff check,ruff format --check,mypy draftjs_exporter tests, andty check. - Tests use
unittest.TestCaseclasses namedTest*; test functions/methodstest_*. - Run tests with
just test <path>(strict:PYTHONDEVMODE=1, warnings as errors). - Coverage target: 100% on all new modules.
- Commit messages: short, capitalized, imperative summaries (e.g., "Add Markdown parser inline emphasis support").
- Sentence case everywhere, no Title Case.
- MarkdownParseError rarity: CommonMark is total — nearly every string is valid Markdown. Unclosed fenced code blocks parse to EOF (CommonMark behavior), not an error.
MarkdownParseErrorfires when an entity resolver raises or returns a malformed resolution; the block parser attaches line numbers. It is also the designated error type for custom parser engines. - Atomic block pattern: thematic breaks and standalone images produce Draft.js atomic blocks exactly like the existing fixtures:
text: " ",entityRanges: [{"offset": 0, "length": 1, "key": N}], entity withmutability: "IMMUTABLE". - Inline images (image inside a text paragraph) produce an inline IMAGE entity whose range covers the alt text in the block text.
- Entity defaults: links →
MUTABLE, images →IMMUTABLE(matches existing fixtures). - Block keys: deterministic, sequential, 5-char zero-padded (
"00000","00001", …). Snapshot tests normalize keys before comparison.
New files:
draftjs_exporter/markdown_parser/__init__.py—MarkdownParser,ParserConfigdraftjs_exporter/markdown_parser/builder.py—ContentStateBuilderdraftjs_exporter/markdown_parser/inline.py—InlineParserdraftjs_exporter/markdown_parser/blocks.py—BlockParserdraftjs_exporter/markdown_parser/resolvers.py—EntityResolution,EntityResolver,resolve, defaults,scheme_resolverdraftjs_exporter/contentstate_filter/__init__.py—ContentStateFilter,FilterRule,FilterAction,FilterCallbackdraftjs_exporter/markdown_importer/__init__.py—MarkdownImporter,ImporterConfigtests/markdown_parser/__init__.py(empty),test_builder.py,test_resolvers.py,test_scheme_resolver.py,test_inline.py,test_inline_html.py,test_blocks.py,test_parser.py,test_parser_config.py,test_parser_errors.pytests/contentstate_filter/__init__.py(empty),test_filter.py,test_filter_rules.pytests/markdown_importer/__init__.py(empty),test_importer.pytests/test_imports.py,tests/test_imports.jsondocs/markdown-importer.md
Modified files:
draftjs_exporter/error.py— addMarkdownParseErrordraftjs_exporter/__init__.py— new exportstests/test_exports.json— add"import"overrides on 4 fixturestests/test_properties.py— add importer/filter propertiesexample.py— add import demomkdocs.yml— nav entry.agents/skills/draftjs_exporter/SKILL.md— document new APICHANGELOG.md— feature entry
Files:
- Modify:
draftjs_exporter/error.py - Test:
tests/test_error.py(new)
Interfaces:
-
Produces:
MarkdownParseError(message: str, line: int | None = None)with.messageand.lineattributes; subclassesExporterException. -
Step 1: Write the failing test
"""Tests for exporter exception types."""
import unittest
from draftjs_exporter.error import ExporterException, MarkdownParseError
class TestMarkdownParseError(unittest.TestCase):
def test_message_only(self):
err = MarkdownParseError("bad input")
self.assertEqual(str(err), "bad input")
self.assertEqual(err.message, "bad input")
self.assertIsNone(err.line)
def test_with_line(self):
err = MarkdownParseError("bad input", line=7)
self.assertEqual(str(err), "line 7: bad input")
self.assertEqual(err.line, 7)
def test_is_exporter_exception(self):
self.assertIsInstance(MarkdownParseError("x"), ExporterException)- Step 2: Run test to verify it fails
Run: just test tests/test_error.py
Expected: FAIL — ImportError: cannot import name 'MarkdownParseError'
- Step 3: Implement
Replace the contents of draftjs_exporter/error.py with:
"""Custom exceptions raised by the exporter."""
class ExporterException(Exception):
"""Base exception for all exporter errors."""
class ConfigException(ExporterException):
"""Raised when the exporter configuration is invalid or unsupported."""
class MarkdownParseError(ExporterException):
"""Raised when Markdown input cannot be parsed.
Carries an optional 1-based line number pointing at the source of
the failure.
"""
__slots__ = ("line", "message")
def __init__(self, message: str, line: int | None = None) -> None:
"""Initialize the error with a message and optional line number.
Parameters:
message: Human-readable description of the failure.
line: 1-based source line number, if known.
"""
self.message = message
self.line = line
super().__init__(f"line {line}: {message}" if line is not None else message)- Step 4: Run test to verify it passes
Run: just test tests/test_error.py
Expected: PASS (3 tests)
- Step 5: Commit
git add draftjs_exporter/error.py tests/test_error.py
git commit -m "Add MarkdownParseError exception type"Files:
- Create:
draftjs_exporter/markdown_parser/__init__.py(docstring-only placeholder for now) - Create:
draftjs_exporter/markdown_parser/resolvers.py - Test:
tests/markdown_parser/__init__.py(empty),tests/markdown_parser/test_resolvers.py,tests/markdown_parser/test_scheme_resolver.py
Interfaces:
-
Produces:
EntityResolution(TypedDict, total=False): keystype: str,data: dict[str, Any],mutability: MutabilityEntityResolver: TypeAlias = Callable[[str, str], EntityResolution | None]resolve(chain: list[EntityResolver], url: str, label: str, default: EntityResolver) -> EntityResolutiondefault_link_resolver(url: str, label: str) -> EntityResolution— LINK,{"url": url}, MUTABLEdefault_image_resolver(url: str, alt: str) -> EntityResolution— IMAGE,{"src": url, "alt": alt}, IMMUTABLEscheme_resolver(scheme: str, type_map: dict[str, str], coerce: dict[str, Callable[[str], Any]] | None = None, label_key: str | None = None, mutability: Mutability = "MUTABLE") -> EntityResolver
-
Step 1: Write the failing tests
tests/markdown_parser/__init__.py — empty file.
tests/markdown_parser/test_resolvers.py:
"""Tests for entity resolver chains and default resolvers."""
import unittest
from draftjs_exporter.markdown_parser.resolvers import (
default_image_resolver,
default_link_resolver,
resolve,
)
class TestDefaultLinkResolver(unittest.TestCase):
def test_returns_link_entity(self):
self.assertEqual(
default_link_resolver("https://example.com", "example"),
{
"type": "LINK",
"data": {"url": "https://example.com"},
"mutability": "MUTABLE",
},
)
class TestDefaultImageResolver(unittest.TestCase):
def test_returns_image_entity(self):
self.assertEqual(
default_image_resolver("/media/a.jpg", "an alt"),
{
"type": "IMAGE",
"data": {"src": "/media/a.jpg", "alt": "an alt"},
"mutability": "IMMUTABLE",
},
)
class TestResolve(unittest.TestCase):
def test_empty_chain_uses_default(self):
result = resolve([], "/x", "lbl", default_link_resolver)
self.assertEqual(result["data"], {"url": "/x"})
def test_first_match_wins(self):
calls = []
def first(url, label):
calls.append("first")
return {"type": "DOCUMENT", "data": {"id": 1}}
def second(url, label):
calls.append("second")
return {"type": "LINK", "data": {}}
result = resolve([first, second], "/x", "lbl", default_link_resolver)
self.assertEqual(result["type"], "DOCUMENT")
self.assertEqual(calls, ["first"])
def test_none_defers_to_next(self):
def defer(url, label):
return None
result = resolve([defer], "/x", "lbl", default_link_resolver)
self.assertEqual(result["type"], "LINK")tests/markdown_parser/test_scheme_resolver.py:
"""Tests for the scheme_resolver helper."""
import unittest
from draftjs_exporter.markdown_parser.resolvers import scheme_resolver
class TestSchemeResolver(unittest.TestCase):
def setUp(self):
self.resolve = scheme_resolver(
"wagtail",
{"page": "LINK", "document": "DOCUMENT", "image": "IMAGE"},
coerce={"id": int},
label_key="alt",
)
def test_scheme_mismatch_returns_none(self):
self.assertIsNone(self.resolve("https://example.com", "x"))
def test_unmapped_host_returns_none(self):
self.assertIsNone(self.resolve("wagtail://unknown?id=1", "x"))
def test_host_maps_to_entity_type(self):
result = self.resolve("wagtail://page?id=3", "label")
self.assertEqual(result["type"], "LINK")
self.assertEqual(result["data"], {"id": 3})
def test_query_params_become_data(self):
result = self.resolve("wagtail://image?id=10&alt=alt&format=left", "alt")
self.assertEqual(
result["data"], {"id": 10, "alt": "alt", "format": "left"}
)
def test_label_fills_label_key_when_absent(self):
result = self.resolve("wagtail://image?id=10", "my alt")
self.assertEqual(result["data"], {"id": 10, "alt": "my alt"})
def test_label_does_not_override_query_param(self):
result = self.resolve("wagtail://image?id=10&alt=fromquery", "frommd")
self.assertEqual(result["data"]["alt"], "fromquery")
def test_empty_label_not_injected(self):
result = self.resolve("wagtail://image?id=10", "")
self.assertNotIn("alt", result["data"])
def test_percent_decoding(self):
result = self.resolve("wagtail://page?id=1&url=https%3A%2F%2Fa.b%2F", "x")
self.assertEqual(result["data"]["url"], "https://a.b/")
def test_coercion_error_raises_value_error(self):
with self.assertRaises(ValueError):
self.resolve("wagtail://page?id=abc", "x")
def test_default_mutability(self):
result = self.resolve("wagtail://page?id=3", "x")
self.assertEqual(result["mutability"], "MUTABLE")
def test_custom_mutability(self):
resolve = scheme_resolver("wagtail", {"image": "IMAGE"}, mutability="IMMUTABLE")
result = resolve("wagtail://image?id=1", "x")
self.assertEqual(result["mutability"], "IMMUTABLE")- Step 2: Run tests to verify they fail
Run: just test tests/markdown_parser/
Expected: FAIL — ModuleNotFoundError: No module named 'draftjs_exporter.markdown_parser'
- Step 3: Implement
draftjs_exporter/markdown_parser/__init__.py:
"""Markdown parsing engine: converts CommonMark core to Draft.js ContentState."""draftjs_exporter/markdown_parser/resolvers.py:
"""Entity resolvers: map Markdown link and image URLs to Draft.js entities."""
from collections.abc import Callable
from typing import Any, TypeAlias, TypedDict
from urllib.parse import parse_qsl, urlparse
from draftjs_exporter.constants import ENTITY_TYPES
from draftjs_exporter.types import Mutability
class EntityResolution(TypedDict, total=False):
"""How a link or image URL should be converted into a Draft.js entity."""
type: str
"""The entity type, e.g. ``LINK``, ``IMAGE``, ``DOCUMENT``, ``EMBED``."""
data: dict[str, Any]
"""The entity data payload."""
mutability: Mutability
"""The entity mutability. Defaults to ``MUTABLE`` when omitted."""
EntityResolver: TypeAlias = Callable[[str, str], "EntityResolution | None"]
"""Resolve a URL and its label into an entity, or return None to defer."""
def default_link_resolver(url: str, label: str) -> EntityResolution:
"""Resolve any URL into a standard ``LINK`` entity.
Parameters:
url: The link URL from the Markdown source.
label: The link text.
Returns:
A ``LINK`` resolution with the URL in its data.
"""
return {
"type": ENTITY_TYPES.LINK,
"data": {"url": url},
"mutability": "MUTABLE",
}
def default_image_resolver(url: str, alt: str) -> EntityResolution:
"""Resolve any URL into a standard ``IMAGE`` entity.
Parameters:
url: The image URL from the Markdown source.
alt: The image alt text.
Returns:
An ``IMAGE`` resolution with ``src`` and ``alt`` in its data.
"""
return {
"type": ENTITY_TYPES.IMAGE,
"data": {"src": url, "alt": alt},
"mutability": "IMMUTABLE",
}
def resolve(
chain: list[EntityResolver],
url: str,
label: str,
default: EntityResolver,
) -> EntityResolution:
"""Run a resolver chain, falling back to the default resolver.
Parameters:
chain: Resolvers tried in order; the first non-None result wins.
url: The URL to resolve.
label: The link text or image alt text.
default: Resolver used when every chain entry defers.
Returns:
The winning resolution, or the default resolution.
"""
for resolver in chain:
resolution = resolver(url, label)
if resolution is not None:
return resolution
return default(url, label)
def scheme_resolver(
scheme: str,
type_map: dict[str, str],
coerce: dict[str, Callable[[str], Any]] | None = None,
label_key: str | None = None,
mutability: Mutability = "MUTABLE",
) -> EntityResolver:
"""Build a resolver for internal URLs like ``scheme://kind?key=value``.
The URL host selects the entity type via ``type_map``. Query string
parameters become entity data, optionally converted per key via
``coerce``. When ``label_key`` is set, a non-empty Markdown label
fills that data key if the query string did not provide it.
Parameters:
scheme: The URL scheme to match, e.g. ``"wagtail"``.
type_map: Mapping of URL host to entity type.
coerce: Optional per-key converters for query string values.
label_key: Optional data key filled from the Markdown label.
mutability: Mutability for produced resolutions.
Returns:
A resolver that defers (returns None) for non-matching URLs.
"""
converters = coerce if coerce is not None else {}
def resolver(url: str, label: str) -> EntityResolution | None:
parsed = urlparse(url)
if parsed.scheme != scheme:
return None
entity_type = type_map.get(parsed.netloc)
if entity_type is None:
return None
data: dict[str, Any] = {}
for key, value in parse_qsl(parsed.query, keep_blank_values=True):
converter = converters.get(key)
data[key] = converter(value) if converter is not None else value
if label_key is not None and label and label_key not in data:
data[label_key] = label
return {"type": entity_type, "data": data, "mutability": mutability}
return resolver- Step 4: Run tests to verify they pass
Run: just test tests/markdown_parser/
Expected: PASS (16 tests)
- Step 5: Lint and commit
just lint
git add draftjs_exporter/markdown_parser/ tests/markdown_parser/
git commit -m "Add entity resolvers for Markdown import"Files:
- Create:
draftjs_exporter/markdown_parser/builder.py - Test:
tests/markdown_parser/test_builder.py
Interfaces:
-
Consumes: nothing from earlier tasks.
-
Produces:
ContentStateBuilderwith:add_entity(type_: str, data: dict[str, Any], mutability: Mutability = "MUTABLE") -> int— returns integer entity keyadd_block(type_: str, text: str = "", depth: int = 0, inline_style_ranges: list[InlineStyleRange] | None = None, entity_ranges: list[EntityRange] | None = None) -> Nonebuild() -> ContentState
-
Step 1: Write the failing test
"""Tests for the ContentState builder."""
import unittest
from draftjs_exporter.markdown_parser.builder import ContentStateBuilder
class TestContentStateBuilder(unittest.TestCase):
def test_empty_build(self):
builder = ContentStateBuilder()
self.assertEqual(builder.build(), {"blocks": [], "entityMap": {}})
def test_block_keys_are_sequential(self):
builder = ContentStateBuilder()
builder.add_block("unstyled", "a")
builder.add_block("unstyled", "b")
keys = [b["key"] for b in builder.build()["blocks"]]
self.assertEqual(keys, ["00000", "00001"])
self.assertEqual(len(set(keys)), 2)
def test_block_defaults(self):
builder = ContentStateBuilder()
builder.add_block("unstyled", "a")
block = builder.build()["blocks"][0]
self.assertEqual(block["depth"], 0)
self.assertEqual(block["inlineStyleRanges"], [])
self.assertEqual(block["entityRanges"], [])
def test_add_entity_returns_int_keys(self):
builder = ContentStateBuilder()
first = builder.add_entity("LINK", {"url": "/a"})
second = builder.add_entity("IMAGE", {"src": "/b"}, "IMMUTABLE")
self.assertEqual((first, second), (0, 1))
entity_map = builder.build()["entityMap"]
self.assertEqual(
entity_map["0"],
{"type": "LINK", "mutability": "MUTABLE", "data": {"url": "/a"}},
)
self.assertEqual(entity_map["1"]["mutability"], "IMMUTABLE")
def test_full_block(self):
builder = ContentStateBuilder()
key = builder.add_entity("LINK", {"url": "/a"})
builder.add_block(
"unstyled",
"text",
depth=2,
inline_style_ranges=[{"offset": 0, "length": 2, "style": "BOLD"}],
entity_ranges=[{"offset": 0, "length": 4, "key": key}],
)
block = builder.build()["blocks"][0]
self.assertEqual(block["depth"], 2)
self.assertEqual(block["entityRanges"][0]["key"], 0)- Step 2: Run test to verify it fails
Run: just test tests/markdown_parser/test_builder.py
Expected: FAIL — ModuleNotFoundError: No module named 'draftjs_exporter.markdown_parser.builder'
- Step 3: Implement
draftjs_exporter/markdown_parser/builder.py:
"""Accumulates Draft.js blocks and entities into a ContentState."""
from typing import Any
from draftjs_exporter.types import (
Block,
ContentState,
Entity,
EntityRange,
InlineStyleRange,
Mutability,
)
class ContentStateBuilder:
"""Build a ContentState block by block with consistent keys.
Entity keys are assigned as monotonically increasing integers in
order of first use. Block keys are deterministic, sequential, and
unique within the built ContentState.
"""
__slots__ = ("blocks", "entity_map", "_block_counter")
def __init__(self) -> None:
"""Initialize an empty builder."""
self.blocks: list[Block] = []
self.entity_map: dict[str, Entity] = {}
self._block_counter = 0
def add_entity(
self,
type_: str,
data: dict[str, Any],
mutability: Mutability = "MUTABLE",
) -> int:
"""Register an entity and return its integer key.
Parameters:
type_: The entity type, e.g. ``LINK``.
data: The entity data payload.
mutability: The entity mutability.
Returns:
The integer key used in entity ranges.
"""
key = len(self.entity_map)
self.entity_map[str(key)] = {
"type": type_,
"mutability": mutability,
"data": data,
}
return key
def add_block(
self,
type_: str,
text: str = "",
depth: int = 0,
inline_style_ranges: list[InlineStyleRange] | None = None,
entity_ranges: list[EntityRange] | None = None,
) -> None:
"""Append a block to the ContentState.
Parameters:
type_: The Draft.js block type.
text: The block's plain text.
depth: Nesting depth for list items.
inline_style_ranges: Style ranges over the text.
entity_ranges: Entity ranges over the text.
"""
self.blocks.append(
{
"key": f"{self._block_counter:05d}",
"text": text,
"type": type_,
"depth": depth,
"inlineStyleRanges": (
inline_style_ranges if inline_style_ranges is not None else []
),
"entityRanges": entity_ranges if entity_ranges is not None else [],
}
)
self._block_counter += 1
def build(self) -> ContentState:
"""Return the accumulated ContentState."""
return {"blocks": self.blocks, "entityMap": self.entity_map}- Step 4: Run test to verify it passes
Run: just test tests/markdown_parser/test_builder.py
Expected: PASS (5 tests)
- Step 5: Commit
git add draftjs_exporter/markdown_parser/builder.py tests/markdown_parser/test_builder.py
git commit -m "Add ContentState builder for Markdown parser"Files:
- Create:
draftjs_exporter/markdown_parser/inline.py - Test:
tests/markdown_parser/test_inline.py
Interfaces:
- Consumes:
ContentStateBuilder(Task 3); resolvers (Task 2). - Produces:
InlineParser, constructed with keyword-only args (all required):emphasis: bool, code_inline: bool, links: bool, images: bool, line_breaks: bool, inline_html_styles: dict[str, str], link_resolvers: list[EntityResolver], image_resolvers: list[EntityResolver], builder: ContentStateBuilder.parse(text: str) -> tuple[str, list[InlineStyleRange], list[EntityRange]]resolve_image_entity(url: str, alt: str) -> int— used by BlockParser (Task 11) for atomic images.
This task implements the skeleton plus escapes, code spans, and hard breaks. Emphasis (Task 5), links/images (Task 6), and inline HTML (Task 7) are added to the same file; their dispatch hooks are present but return None (literal passthrough) until implemented. Test helper builds a parser with everything enabled:
- Step 1: Write the failing tests
"""Tests for inline Markdown parsing."""
import unittest
from draftjs_exporter.markdown_parser.builder import ContentStateBuilder
from draftjs_exporter.markdown_parser.inline import InlineParser
def make_parser(**overrides):
"""Build an InlineParser with all constructs enabled."""
config = {
"emphasis": True,
"code_inline": True,
"links": True,
"images": True,
"line_breaks": True,
"inline_html_styles": {},
"link_resolvers": [],
"image_resolvers": [],
"builder": ContentStateBuilder(),
}
config.update(overrides)
return InlineParser(**config)
class TestPlainText(unittest.TestCase):
def test_plain_text_passes_through(self):
text, styles, entities = make_parser().parse("hello world")
self.assertEqual(text, "hello world")
self.assertEqual(styles, [])
self.assertEqual(entities, [])
class TestEscapes(unittest.TestCase):
def test_escaped_char_is_literal(self):
text, styles, _ = make_parser().parse(r"\*not italic\*")
self.assertEqual(text, "*not italic*")
self.assertEqual(styles, [])
def test_backslash_before_non_escapable_is_literal(self):
text, _, _ = make_parser().parse(r"\a")
self.assertEqual(text, r"\a")
class TestCodeSpans(unittest.TestCase):
def test_code_span(self):
text, styles, _ = make_parser().parse("a `bc` d")
self.assertEqual(text, "a bc d")
self.assertEqual(styles, [{"offset": 2, "length": 2, "style": "CODE"}])
def test_code_span_contents_are_literal(self):
text, styles, _ = make_parser().parse("`**not bold**`")
self.assertEqual(text, "**not bold**")
self.assertEqual(styles, [{"offset": 0, "length": 12, "style": "CODE"}])
def test_unmatched_backtick_is_literal(self):
text, styles, _ = make_parser().parse("a `b")
self.assertEqual(text, "a `b")
self.assertEqual(styles, [])
def test_code_disabled(self):
text, styles, _ = make_parser(code_inline=False).parse("`x`")
self.assertEqual(text, "`x`")
self.assertEqual(styles, [])
class TestHardBreaks(unittest.TestCase):
def test_two_spaces_before_newline_are_stripped(self):
text, _, _ = make_parser().parse("a \nb")
self.assertEqual(text, "a\nb")
def test_soft_break_kept(self):
text, _, _ = make_parser().parse("a\nb")
self.assertEqual(text, "a\nb")
def test_single_trailing_space_kept(self):
text, _, _ = make_parser().parse("a \nb")
self.assertEqual(text, "a \nb")- Step 2: Run tests to verify they fail
Run: just test tests/markdown_parser/test_inline.py
Expected: FAIL — ModuleNotFoundError: No module named 'draftjs_exporter.markdown_parser.inline'
- Step 3: Implement
draftjs_exporter/markdown_parser/inline.py:
"""Inline Markdown parsing: emphasis, code spans, links, images, inline HTML."""
import re
from typing import TypeAlias
from draftjs_exporter.constants import INLINE_STYLES
from draftjs_exporter.error import MarkdownParseError
from draftjs_exporter.markdown_parser.builder import ContentStateBuilder
from draftjs_exporter.markdown_parser.resolvers import (
EntityResolution,
EntityResolver,
default_image_resolver,
default_link_resolver,
resolve,
)
from draftjs_exporter.types import EntityRange, InlineStyleRange
Span: TypeAlias = tuple[int, int, str, "str | int"]
"""An inline annotation: ``(offset, length, kind, payload)``.
``kind`` is ``"style"`` (payload: style name) or ``"entity"`` (payload:
integer entity key).
"""
ESCAPABLE = frozenset("\\`*{}_[]<>()#+-.!|\"")
"""Punctuation characters that can be backslash-escaped per CommonMark."""
TAG_RE = re.compile(r"<([a-zA-Z][a-zA-Z0-9]*)>")
"""Matches an HTML opening tag without attributes."""
class InlineParser:
"""Parse inline Markdown constructs into text with style/entity ranges.
The parser is a character-by-character recursive descent scanner.
Delimiter runs (``*``, ``_``) match by exact length: ``**`` only
closes ``**``, not two adjacent ``*`` runs. Intraword emphasis and
other flanking-rule subtleties are intentionally not supported.
"""
__slots__ = (
"emphasis",
"code_inline",
"links",
"images",
"line_breaks",
"inline_html_styles",
"link_resolvers",
"image_resolvers",
"builder",
)
def __init__(
self,
*,
emphasis: bool,
code_inline: bool,
links: bool,
images: bool,
line_breaks: bool,
inline_html_styles: dict[str, str],
link_resolvers: list[EntityResolver],
image_resolvers: list[EntityResolver],
builder: ContentStateBuilder,
) -> None:
"""Initialize the parser with feature toggles and resolvers.
Parameters:
emphasis: Parse ``*italic*`` / ``**bold**`` constructs.
code_inline: Parse backtick code spans.
links: Parse ``[label](url)`` links.
images: Parse ```` images.
line_breaks: Strip two-space hard break markers.
inline_html_styles: Whitelist of HTML tags to inline styles.
link_resolvers: Resolver chain for link URLs.
image_resolvers: Resolver chain for image URLs.
builder: The builder entities are registered on.
"""
self.emphasis = emphasis
self.code_inline = code_inline
self.links = links
self.images = images
self.line_breaks = line_breaks
self.inline_html_styles = inline_html_styles
self.link_resolvers = link_resolvers
self.image_resolvers = image_resolvers
self.builder = builder
def parse(
self, text: str
) -> tuple[str, list[InlineStyleRange], list[EntityRange]]:
"""Convert Markdown inline syntax to plain text plus ranges.
Parameters:
text: The Markdown source of a single block.
Returns:
The plain text, its inline style ranges, and its entity ranges.
"""
plain, spans = self._parse(text)
styles: list[InlineStyleRange] = []
entities: list[EntityRange] = []
for offset, length, kind, payload in spans:
if kind == "style":
styles.append(
{"offset": offset, "length": length, "style": str(payload)}
)
else:
entities.append(
{"offset": offset, "length": length, "key": int(payload)}
)
styles.sort(key=lambda r: r["offset"])
entities.sort(key=lambda r: r["offset"])
return plain, styles, entities
def resolve_image_entity(self, url: str, alt: str) -> int:
"""Register an image entity through the resolver chain.
Parameters:
url: The image URL.
alt: The image alt text.
Returns:
The integer key of the newly registered entity.
"""
return self._resolve_entity(url, alt, self.image_resolvers, default_image_resolver)
def _resolve_entity(
self,
url: str,
label: str,
resolvers: list[EntityResolver],
default: EntityResolver,
) -> int:
"""Resolve a URL into an entity and register it on the builder."""
try:
resolution: EntityResolution = resolve(resolvers, url, label, default)
except MarkdownParseError:
raise
except Exception as err:
raise MarkdownParseError(
f"Entity resolver failed for URL {url!r}: {err}"
) from err
entity_type = resolution.get("type")
if not isinstance(entity_type, str) or not entity_type:
raise MarkdownParseError(
f"Entity resolver for URL {url!r} must return a 'type'"
)
data = resolution.get("data", {})
if not isinstance(data, dict):
raise MarkdownParseError(
f"Entity resolver for URL {url!r} must return dict 'data'"
)
return self.builder.add_entity(
entity_type, data, resolution.get("mutability", "MUTABLE")
)
def _parse(self, text: str) -> tuple[str, list[Span]]:
"""Scan text, returning output characters and annotation spans."""
out: list[str] = []
spans: list[Span] = []
i = 0
n = len(text)
while i < n:
ch = text[i]
# Backslash escapes.
if ch == "\\" and i + 1 < n and text[i + 1] in ESCAPABLE:
out.append(text[i + 1])
i += 2
continue
# Code spans.
if ch == "`" and self.code_inline:
end = text.find("`", i + 1)
if end != -1:
content = text[i + 1 : end]
start = len(out)
out.extend(content)
spans.append((start, len(content), "style", INLINE_STYLES.CODE))
i = end + 1
continue
# Images: 
if self.images and ch == "!" and i + 1 < n and text[i + 1] == "[":
result = self._link_target(text, i + 1)
if result is not None:
alt, url, end = result
start = len(out)
out.extend(alt)
key = self._resolve_entity(
url, alt, self.image_resolvers, default_image_resolver
)
spans.append((start, len(alt), "entity", key))
i = end
continue
# Links: [label](url)
if self.links and ch == "[":
result = self._link_target(text, i)
if result is not None:
label_src, url, end = result
label_plain, label_spans = self._parse(label_src)
start = len(out)
out.extend(label_plain)
spans.extend(
(s + start, length, kind, payload)
for s, length, kind, payload in label_spans
)
key = self._resolve_entity(
url, label_plain, self.link_resolvers, default_link_resolver
)
spans.append((start, len(label_plain), "entity", key))
i = end
continue
# Emphasis: * _ ** __ *** ___
if self.emphasis and ch in "*_":
consumed = self._parse_emphasis(text, i, out, spans)
if consumed is not None:
i = consumed
continue
# Whitelisted inline HTML tags.
if ch == "<" and self.inline_html_styles:
consumed = self._parse_inline_html(text, i, out, spans)
if consumed is not None:
i = consumed
continue
# Hard line breaks: strip 2+ trailing spaces before newline.
if ch == "\n" and self.line_breaks:
spaces = 0
j = len(out) - 1
while j >= 0 and out[j] == " ":
spaces += 1
j -= 1
if spaces >= 2:
del out[j + 1 :]
out.append("\n")
i += 1
continue
out.append(ch)
i += 1
return "".join(out), spans
@staticmethod
def _link_target(text: str, i: int) -> tuple[str, str, int] | None:
"""Parse ``[label](url)`` starting at the opening bracket.
Parameters:
text: The full source text.
i: Index of the ``[`` character.
Returns:
``(label, url, end_index)`` or None when the construct does
not parse. Labels containing ``](`` and URLs containing
``)`` are not supported.
"""
close = text.find("](", i)
if close == -1:
return None
paren = text.find(")", close + 2)
if paren == -1:
return None
return text[i + 1 : close], text[close + 2 : paren], paren + 1
def _parse_emphasis(
self, text: str, i: int, out: list[str], spans: list[Span]
) -> int | None:
"""Parse an emphasis delimiter run. Implemented in Task 5."""
return None
def _parse_inline_html(
self, text: str, i: int, out: list[str], spans: list[Span]
) -> int | None:
"""Parse a whitelisted inline HTML tag. Implemented in Task 7."""
return None- Step 4: Run tests to verify they pass
Run: just test tests/markdown_parser/test_inline.py
Expected: PASS (10 tests)
- Step 5: Commit
git add draftjs_exporter/markdown_parser/inline.py tests/markdown_parser/test_inline.py
git commit -m "Add inline parser: escapes, code spans, hard breaks"Files:
- Modify:
draftjs_exporter/markdown_parser/inline.py(replace_parse_emphasisstub, add_find_closing) - Test:
tests/markdown_parser/test_inline.py(append test classes)
Interfaces:
-
Consumes/Produces: unchanged. Delimiter runs match by exact length: 1 → ITALIC, 2 → BOLD, 3 → BOLD + ITALIC. Runs longer than 3 are literal.
-
Step 1: Write the failing tests
Append to tests/markdown_parser/test_inline.py:
class TestEmphasis(unittest.TestCase):
def test_italic_star(self):
text, styles, _ = make_parser().parse("a *b* c")
self.assertEqual(text, "a b c")
self.assertEqual(styles, [{"offset": 2, "length": 1, "style": "ITALIC"}])
def test_italic_underscore(self):
text, styles, _ = make_parser().parse("_b_")
self.assertEqual(styles, [{"offset": 0, "length": 1, "style": "ITALIC"}])
def test_bold_stars(self):
text, styles, _ = make_parser().parse("**bold**")
self.assertEqual(text, "bold")
self.assertEqual(styles, [{"offset": 0, "length": 4, "style": "BOLD"}])
def test_bold_underscores(self):
text, styles, _ = make_parser().parse("__bold__")
self.assertEqual(styles, [{"offset": 0, "length": 4, "style": "BOLD"}])
def test_bold_italic_triple(self):
text, styles, _ = make_parser().parse("***both***")
self.assertEqual(text, "both")
self.assertEqual(
styles,
[
{"offset": 0, "length": 4, "style": "BOLD"},
{"offset": 0, "length": 4, "style": "ITALIC"},
],
)
def test_nested_italic_in_bold(self):
text, styles, _ = make_parser().parse("**a *b* c**")
self.assertEqual(text, "a b c")
self.assertEqual(
styles,
[
{"offset": 0, "length": 5, "style": "BOLD"},
{"offset": 2, "length": 1, "style": "ITALIC"},
],
)
def test_bold_inside_italic(self):
text, styles, _ = make_parser().parse("*a **b** c*")
self.assertEqual(text, "a b c")
self.assertIn({"offset": 0, "length": 5, "style": "ITALIC"}, styles)
self.assertIn({"offset": 2, "length": 1, "style": "BOLD"}, styles)
def test_unmatched_delimiter_is_literal(self):
text, styles, _ = make_parser().parse("a *b")
self.assertEqual(text, "a *b")
self.assertEqual(styles, [])
def test_mismatched_run_length_is_literal(self):
text, styles, _ = make_parser().parse("**a*")
self.assertEqual(text, "**a*")
self.assertEqual(styles, [])
def test_run_longer_than_three_is_literal(self):
text, styles, _ = make_parser().parse("****a****")
self.assertEqual(text, "****a****")
self.assertEqual(styles, [])
def test_offsets_after_emphasis(self):
text, styles, _ = make_parser().parse("**b** x *i*")
self.assertEqual(text, "b x i")
self.assertEqual(
styles,
[
{"offset": 0, "length": 1, "style": "BOLD"},
{"offset": 4, "length": 1, "style": "ITALIC"},
],
)
def test_emphasis_disabled(self):
text, styles, _ = make_parser(emphasis=False).parse("**a**")
self.assertEqual(text, "**a**")
self.assertEqual(styles, [])- Step 2: Run tests to verify they fail
Run: just test tests/markdown_parser/test_inline.py -k Emphasis
Expected: FAIL — most emphasis tests fail (delimiters remain literal)
- Step 3: Implement
In draftjs_exporter/markdown_parser/inline.py, replace the _parse_emphasis stub and add _find_closing:
def _parse_emphasis(
self, text: str, i: int, out: list[str], spans: list[Span]
) -> int | None:
"""Parse an emphasis delimiter run at index i.
Delimiter runs match by exact length: a run of 2 only closes a
run of 2. Runs longer than 3 are treated as literal text.
Parameters:
text: The full source text.
i: Index of the first delimiter character.
out: Output characters accumulated so far.
spans: Spans accumulated so far.
Returns:
The index after the closing delimiter, or None when the run
does not form emphasis (it is then emitted literally).
"""
ch = text[i]
n = len(text)
run = 1
while i + run < n and text[i + run] == ch:
run += 1
if run > 3:
return None
marker = ch * run
end = self._find_closing(text, i + run, marker)
if end == -1:
return None
inner_plain, inner_spans = self._parse(text[i + run : end])
start = len(out)
out.extend(inner_plain)
spans.extend(
(s + start, length, kind, payload)
for s, length, kind, payload in inner_spans
)
styles_by_run = {
1: [INLINE_STYLES.ITALIC],
2: [INLINE_STYLES.BOLD],
3: [INLINE_STYLES.BOLD, INLINE_STYLES.ITALIC],
}
for style in styles_by_run[run]:
spans.append((start, len(inner_plain), "style", style))
return end + run
@staticmethod
def _find_closing(text: str, start: int, marker: str) -> int:
"""Find the closing delimiter, skipping longer runs for singles.
Parameters:
text: The full source text.
start: Index to start searching from.
marker: The exact delimiter run to find.
Returns:
The index of the closing delimiter, or -1 when absent.
"""
i = start
while True:
end = text.find(marker, i)
if end == -1:
return -1
if len(marker) == 1:
ch = marker
part_of_longer_run = (end > 0 and text[end - 1] == ch) or (
end + 1 < len(text) and text[end + 1] == ch
)
if part_of_longer_run:
i = end + 1
continue
return end- Step 4: Run tests to verify they pass
Run: just test tests/markdown_parser/test_inline.py
Expected: PASS (22 tests)
- Step 5: Commit
git add draftjs_exporter/markdown_parser/inline.py tests/markdown_parser/test_inline.py
git commit -m "Add inline parser emphasis support"Files:
- Modify:
draftjs_exporter/markdown_parser/inline.py(no code change needed — dispatch already wired in Task 4; this task only adds tests, plus fixes if tests reveal issues) - Test:
tests/markdown_parser/test_inline.py(append),tests/markdown_parser/test_parser_errors.py(new — resolver failure wrapping)
Interfaces:
-
Consumes: resolvers (Task 2). Produces: unchanged.
-
Step 1: Write the failing tests
Append to tests/markdown_parser/test_inline.py:
def parse_with_builder(**overrides):
"""Parse with a fresh builder, returning parse results and builder."""
builder = ContentStateBuilder()
parser = make_parser(builder=builder, **overrides)
return parser, builder
class TestLinks(unittest.TestCase):
def test_simple_link(self):
parser, builder = parse_with_builder()
text, styles, entities = parser.parse("[example](https://example.com)")
self.assertEqual(text, "example")
self.assertEqual(styles, [])
self.assertEqual(entities, [{"offset": 0, "length": 7, "key": 0}])
self.assertEqual(
builder.entity_map["0"],
{
"type": "LINK",
"mutability": "MUTABLE",
"data": {"url": "https://example.com"},
},
)
def test_link_with_styled_label(self):
parser, builder = parse_with_builder()
text, styles, entities = parser.parse("[**bold**](/url)")
self.assertEqual(text, "bold")
self.assertEqual(styles, [{"offset": 0, "length": 4, "style": "BOLD"}])
self.assertEqual(entities, [{"offset": 0, "length": 4, "key": 0}])
def test_link_offsets_in_text(self):
parser, builder = parse_with_builder()
text, _, entities = parser.parse("see [docs](/d) now")
self.assertEqual(text, "see docs now")
self.assertEqual(entities, [{"offset": 4, "length": 4, "key": 0}])
def test_links_disabled(self):
parser, builder = parse_with_builder(links=False)
text, _, entities = parser.parse("[a](/b)")
self.assertEqual(text, "[a](/b)")
self.assertEqual(entities, [])
def test_custom_link_resolver(self):
def wagtail(url, label):
if url.startswith("wagtail://"):
return {"type": "DOCUMENT", "data": {"id": 1}}
return None
parser, builder = parse_with_builder(link_resolvers=[wagtail])
text, _, entities = parser.parse("[file](wagtail://document?id=1)")
self.assertEqual(builder.entity_map["0"]["type"], "DOCUMENT")
def test_resolver_deferring_falls_back_to_default(self):
parser, builder = parse_with_builder(
link_resolvers=[lambda url, label: None]
)
parser.parse("[a](/b)")
self.assertEqual(builder.entity_map["0"]["type"], "LINK")
class TestImages(unittest.TestCase):
def test_inline_image(self):
parser, builder = parse_with_builder()
text, _, entities = parser.parse("a  b")
self.assertEqual(text, "a alt b")
self.assertEqual(entities, [{"offset": 2, "length": 3, "key": 0}])
self.assertEqual(
builder.entity_map["0"],
{
"type": "IMAGE",
"mutability": "IMMUTABLE",
"data": {"src": "/img.jpg", "alt": "alt"},
},
)
def test_images_disabled(self):
parser, _ = parse_with_builder(images=False)
text, _, entities = parser.parse("")
self.assertEqual(text, "")
self.assertEqual(entities, [])
def test_resolve_image_entity(self):
parser, builder = parse_with_builder()
key = parser.resolve_image_entity("/x.jpg", "alt text")
self.assertEqual(key, 0)
self.assertEqual(builder.entity_map["0"]["type"], "IMAGE")Create tests/markdown_parser/test_parser_errors.py:
"""Tests for MarkdownParseError surfaces in the inline parser."""
import unittest
from draftjs_exporter.error import MarkdownParseError
from draftjs_exporter.markdown_parser.builder import ContentStateBuilder
from tests.markdown_parser.test_inline import make_parser
class TestResolverErrors(unittest.TestCase):
def test_raising_resolver_wrapped(self):
def bad(url, label):
raise RuntimeError("boom")
parser = make_parser(
builder=ContentStateBuilder(), link_resolvers=[bad]
)
with self.assertRaises(MarkdownParseError) as ctx:
parser.parse("[a](/b)")
self.assertIn("boom", str(ctx.exception))
def test_resolution_without_type_rejected(self):
parser = make_parser(
builder=ContentStateBuilder(),
link_resolvers=[lambda url, label: {"data": {}}],
)
with self.assertRaises(MarkdownParseError):
parser.parse("[a](/b)")
def test_resolution_with_non_dict_data_rejected(self):
parser = make_parser(
builder=ContentStateBuilder(),
link_resolvers=[lambda url, label: {"type": "LINK", "data": "nope"}],
)
with self.assertRaises(MarkdownParseError):
parser.parse("[a](/b)")- Step 2: Run tests to verify they pass (already-implemented dispatch)
Run: just test tests/markdown_parser/test_inline.py tests/markdown_parser/test_parser_errors.py
Expected: PASS. If any fail, fix inline.py — the dispatch for links/images was written in Task 4, so these tests validate it.
- Step 3: Commit
git add tests/markdown_parser/test_inline.py tests/markdown_parser/test_parser_errors.py draftjs_exporter/markdown_parser/inline.py
git commit -m "Add inline parser link and image tests"Files:
- Modify:
draftjs_exporter/markdown_parser/inline.py(replace_parse_inline_htmlstub) - Test:
tests/markdown_parser/test_inline_html.py(new)
Interfaces:
-
Produces: unchanged. Tags match exactly
<tag>(no attributes); content is parsed recursively; unmatched or non-whitelisted tags are literal. -
Step 1: Write the failing tests
tests/markdown_parser/test_inline_html.py:
"""Tests for the inline HTML style whitelist."""
import unittest
from tests.markdown_parser.test_inline import make_parser
SUP_SUB = {"sup": "SUPERSCRIPT", "sub": "SUBSCRIPT"}
class TestInlineHtml(unittest.TestCase):
def test_whitelisted_tag_produces_style(self):
text, styles, _ = make_parser(inline_html_styles=SUP_SUB).parse(
"a <sup>2</sup> b"
)
self.assertEqual(text, "a 2 b")
self.assertEqual(
styles, [{"offset": 2, "length": 1, "style": "SUPERSCRIPT"}]
)
def test_recursive_content(self):
text, styles, _ = make_parser(inline_html_styles=SUP_SUB).parse(
"<sup>**bold**</sup>"
)
self.assertEqual(text, "bold")
self.assertIn({"offset": 0, "length": 4, "style": "SUPERSCRIPT"}, styles)
self.assertIn({"offset": 0, "length": 4, "style": "BOLD"}, styles)
def test_tag_with_attributes_is_literal(self):
text, styles, _ = make_parser(inline_html_styles=SUP_SUB).parse(
'<sup class="x">2</sup>'
)
self.assertEqual(text, '<sup class="x">2</sup>')
self.assertEqual(styles, [])
def test_non_whitelisted_tag_is_literal(self):
text, styles, _ = make_parser(inline_html_styles=SUP_SUB).parse(
"<b>bold</b>"
)
self.assertEqual(text, "<b>bold</b>")
self.assertEqual(styles, [])
def test_unclosed_tag_is_literal(self):
text, styles, _ = make_parser(inline_html_styles=SUP_SUB).parse("<sup>2")
self.assertEqual(text, "<sup>2")
self.assertEqual(styles, [])
def test_empty_whitelist_means_literal(self):
text, styles, _ = make_parser().parse("<sup>2</sup>")
self.assertEqual(text, "<sup>2</sup>")
self.assertEqual(styles, [])- Step 2: Run tests to verify they fail
Run: just test tests/markdown_parser/test_inline_html.py
Expected: FAIL — tags pass through literally
- Step 3: Implement
Replace the _parse_inline_html stub in draftjs_exporter/markdown_parser/inline.py:
def _parse_inline_html(
self, text: str, i: int, out: list[str], spans: list[Span]
) -> int | None:
"""Parse a whitelisted inline HTML tag at index i.
Only exact ``<tag>`` openers (no attributes) with a matching
``</tag>`` closer are recognized. Anything else is literal text,
so unwhitelisted or malformed HTML carries no markup semantics.
Parameters:
text: The full source text.
i: Index of the ``<`` character.
out: Output characters accumulated so far.
spans: Spans accumulated so far.
Returns:
The index after the closing tag, or None when the tag does
not parse as a whitelisted construct.
"""
match = TAG_RE.match(text, i)
if match is None:
return None
tag = match.group(1)
style = self.inline_html_styles.get(tag)
if style is None:
return None
closing = f"</{tag}>"
end = text.find(closing, match.end())
if end == -1:
return None
inner_plain, inner_spans = self._parse(text[match.end() : end])
start = len(out)
out.extend(inner_plain)
spans.extend(
(s + start, length, kind, payload)
for s, length, kind, payload in inner_spans
)
spans.append((start, len(inner_plain), "style", style))
return end + len(closing)- Step 4: Run tests to verify they pass
Run: just test tests/markdown_parser/
Expected: PASS (all inline + error tests)
- Step 5: Commit
git add draftjs_exporter/markdown_parser/inline.py tests/markdown_parser/test_inline_html.py
git commit -m "Add inline HTML style whitelist to inline parser"Files:
- Create:
draftjs_exporter/markdown_parser/blocks.py - Test:
tests/markdown_parser/test_blocks.py
Interfaces:
- Consumes:
InlineParser,ContentStateBuilder. - Produces:
BlockParser, constructed with keyword-only args (all required):headings: bool, blockquote: bool, code_fenced: bool, thematic_break: bool, unordered_list: bool, ordered_list: bool, images: bool, inline: InlineParser, builder: ContentStateBuilder.parse(text: str) -> None— appends blocks to the builder.
This task implements the skeleton with paragraphs, ATX headings, and thematic breaks. Blockquotes/fences (Task 9), lists (Task 10), and standalone images (Task 11) follow. Test helper:
- Step 1: Write the failing tests
tests/markdown_parser/test_blocks.py:
"""Tests for block-level Markdown parsing."""
import unittest
from draftjs_exporter.markdown_parser.blocks import BlockParser
from draftjs_exporter.markdown_parser.builder import ContentStateBuilder
from draftjs_exporter.markdown_parser.inline import InlineParser
def parse(markdown, **overrides):
"""Parse Markdown into a ContentState with all constructs enabled."""
builder = ContentStateBuilder()
inline = InlineParser(
emphasis=True,
code_inline=True,
links=True,
images=overrides.get("images", True),
line_breaks=True,
inline_html_styles={},
link_resolvers=[],
image_resolvers=[],
builder=builder,
)
config = {
"headings": True,
"blockquote": True,
"code_fenced": True,
"thematic_break": True,
"unordered_list": True,
"ordered_list": True,
"images": True,
"inline": inline,
"builder": builder,
}
config.update(overrides)
BlockParser(**config).parse(markdown)
return builder.build()
def block_types(cs):
"""Extract the block types of a ContentState."""
return [b["type"] for b in cs["blocks"]]
class TestParagraphs(unittest.TestCase):
def test_single_paragraph(self):
cs = parse("hello")
self.assertEqual(block_types(cs), ["unstyled"])
self.assertEqual(cs["blocks"][0]["text"], "hello")
def test_blank_lines_split_paragraphs(self):
cs = parse("a\n\nb")
self.assertEqual(len(cs["blocks"]), 2)
def test_soft_wrapped_lines_join_with_newline(self):
cs = parse("a\nb")
self.assertEqual(len(cs["blocks"]), 1)
self.assertEqual(cs["blocks"][0]["text"], "a\nb")
def test_empty_input_produces_no_blocks(self):
self.assertEqual(parse("")["blocks"], [])
self.assertEqual(parse("\n\n\n")["blocks"], [])
class TestHeadings(unittest.TestCase):
def test_all_levels(self):
names = ["one", "two", "three", "four", "five", "six"]
for level in range(1, 7):
cs = parse(f"{'#' * level} Title")
self.assertEqual(block_types(cs), [f"header-{names[level - 1]}"])
def test_heading_content_is_inline_parsed(self):
cs = parse("## **Bold** title")
block = cs["blocks"][0]
self.assertEqual(block["text"], "Bold title")
self.assertEqual(
block["inlineStyleRanges"],
[{"offset": 0, "length": 4, "style": "BOLD"}],
)
def test_closing_hashes_stripped(self):
cs = parse("# Title #")
self.assertEqual(cs["blocks"][0]["text"], "Title")
def test_no_space_after_hash_is_paragraph(self):
cs = parse("#notaheading")
self.assertEqual(block_types(cs), ["unstyled"])
def test_heading_then_paragraph(self):
cs = parse("# T\n\ntext")
self.assertEqual(block_types(cs), ["header-one", "unstyled"])
class TestThematicBreaks(unittest.TestCase):
def test_dashes(self):
cs = parse("a\n\n---\n\nb")
self.assertEqual(
block_types(cs), ["unstyled", "atomic", "unstyled"]
)
def test_atomic_block_shape(self):
cs = parse("---")
block = cs["blocks"][0]
self.assertEqual(block["text"], " ")
self.assertEqual(
block["entityRanges"], [{"offset": 0, "length": 1, "key": 0}]
)
self.assertEqual(
cs["entityMap"]["0"],
{"type": "HORIZONTAL_RULE", "mutability": "IMMUTABLE", "data": {}},
)
def test_stars_and_underscores(self):
self.assertEqual(block_types(parse("***")), ["atomic"])
self.assertEqual(block_types(parse("___")), ["atomic"])
def test_spaced_markers(self):
self.assertEqual(block_types(parse("- - -")), ["atomic"])- Step 2: Run tests to verify they fail
Run: just test tests/markdown_parser/test_blocks.py
Expected: FAIL — ModuleNotFoundError
- Step 3: Implement
draftjs_exporter/markdown_parser/blocks.py:
"""Block-level Markdown parsing: lines to Draft.js blocks."""
import re
from typing import Any
from draftjs_exporter.constants import BLOCK_TYPES, ENTITY_TYPES
from draftjs_exporter.error import MarkdownParseError
from draftjs_exporter.markdown_parser.builder import ContentStateBuilder
from draftjs_exporter.markdown_parser.inline import InlineParser
from draftjs_exporter.types import Mutability
ATX_RE = re.compile(r"^(#{1,6})[ \t]+(.*?)[ \t]*#*[ \t]*$")
"""ATX heading with optional closing hash sequence."""
HR_RE = re.compile(r"^[ \t]*((\*[ \t]*){3,}|(-[ \t]*){3,}|(_[ \t]*){3,})$")
"""Thematic break: 3+ of the same ``*``, ``-``, or ``_`` marker."""
FENCE_RE = re.compile(r"^[ \t]*(```+|~~~+)(.*)$")
"""Fenced code block opener or closer."""
QUOTE_RE = re.compile(r"^[ \t]*>[ \t]?(.*)$")
"""Blockquote line, capturing the content after the marker."""
ULIST_RE = re.compile(r"^([ \t]*)[-*+][ \t]+(.*)$")
"""Unordered list item, capturing indent and content."""
OLIST_RE = re.compile(r"^([ \t]*)\d{1,9}[.)][ \t]+(.*)$")
"""Ordered list item, capturing indent and content."""
STANDALONE_IMAGE_RE = re.compile(r"^!\[(.*)\]\((.*)\)$")
"""A paragraph consisting of exactly one image."""
HEADING_TYPES = [
BLOCK_TYPES.HEADER_ONE,
BLOCK_TYPES.HEADER_TWO,
BLOCK_TYPES.HEADER_THREE,
BLOCK_TYPES.HEADER_FOUR,
BLOCK_TYPES.HEADER_FIVE,
BLOCK_TYPES.HEADER_SIX,
]
"""Heading block types indexed by ATX level minus one."""
class BlockParser:
"""Parse Markdown line by line into Draft.js blocks.
The parser tracks block constructs that span lines (lists,
blockquotes, fenced code) and delegates inline content to the
inline parser. It emits blocks directly onto the builder — there
is no intermediate AST.
"""
__slots__ = (
"headings",
"blockquote",
"code_fenced",
"thematic_break",
"unordered_list",
"ordered_list",
"images",
"inline",
"builder",
)
def __init__(
self,
*,
headings: bool,
blockquote: bool,
code_fenced: bool,
thematic_break: bool,
unordered_list: bool,
ordered_list: bool,
images: bool,
inline: InlineParser,
builder: ContentStateBuilder,
) -> None:
"""Initialize the parser with feature toggles and helpers.
Parameters:
headings: Parse ATX headings.
blockquote: Parse ``>`` blockquotes.
code_fenced: Parse fenced code blocks.
thematic_break: Parse thematic breaks.
unordered_list: Parse unordered lists.
ordered_list: Parse ordered lists.
images: Convert standalone images to atomic blocks.
inline: The inline parser for block content.
builder: The builder blocks are appended to.
"""
self.headings = headings
self.blockquote = blockquote
self.code_fenced = code_fenced
self.thematic_break = thematic_break
self.unordered_list = unordered_list
self.ordered_list = ordered_list
self.images = images
self.inline = inline
self.builder = builder
def parse(self, text: str) -> None:
"""Parse Markdown source, appending blocks to the builder.
Parameters:
text: Markdown with line endings normalized to ``\\n``.
"""
lines = text.split("\n")
i = 0
n = len(lines)
while i < n:
line = lines[i]
if not line.strip():
i += 1
continue
if self.code_fenced and (match := FENCE_RE.match(line)):
i = self._parse_fence(lines, i, match)
continue
if self.headings and (match := ATX_RE.match(line)):
self._add_text_block(
HEADING_TYPES[len(match.group(1)) - 1], match.group(2), i
)
i += 1
continue
if self.thematic_break and HR_RE.match(line):
self._add_atomic(ENTITY_TYPES.HORIZONTAL_RULE, {}, "IMMUTABLE")
i += 1
continue
if self.blockquote and QUOTE_RE.match(line):
i = self._parse_quote(lines, i)
continue
if self._is_list_item(line):
i = self._parse_list(lines, i)
continue
i = self._parse_paragraph(lines, i)
def _add_text_block(
self, type_: str, source: str, line_index: int, depth: int = 0
) -> None:
"""Inline-parse source text and append a block.
Parameters:
type_: The Draft.js block type.
source: The Markdown source of the block's content.
line_index: 0-based source line, for error reporting.
depth: Nesting depth for list items.
"""
try:
text, styles, entities = self.inline.parse(source)
except MarkdownParseError as err:
if err.line is None:
raise MarkdownParseError(err.message, line=line_index + 1) from err
raise
self.builder.add_block(
type_,
text,
depth=depth,
inline_style_ranges=styles,
entity_ranges=entities,
)
def _add_atomic(
self, entity_type: str, data: dict[str, Any], mutability: Mutability
) -> None:
"""Append an atomic block carrying a single entity.
Parameters:
entity_type: The entity type.
data: The entity data.
mutability: The entity mutability.
"""
key = self.builder.add_entity(entity_type, data, mutability)
self.builder.add_block(
BLOCK_TYPES.ATOMIC,
" ",
entity_ranges=[{"offset": 0, "length": 1, "key": key}],
)
def _is_list_item(self, line: str) -> bool:
"""Return whether the line starts an enabled list item."""
return bool(
(self.unordered_list and ULIST_RE.match(line))
or (self.ordered_list and OLIST_RE.match(line))
)
def _starts_block(self, line: str) -> bool:
"""Return whether the line starts a block construct."""
return bool(
(self.code_fenced and FENCE_RE.match(line))
or (self.headings and ATX_RE.match(line))
or (self.thematic_break and HR_RE.match(line))
or (self.blockquote and QUOTE_RE.match(line))
or self._is_list_item(line)
)
def _parse_paragraph(self, lines: list[str], i: int) -> int:
"""Parse consecutive plain lines into one paragraph block."""
start = i
collected = [lines[i]]
i += 1
while i < len(lines) and lines[i].strip() and not self._starts_block(lines[i]):
collected.append(lines[i])
i += 1
source = "\n".join(collected)
image = (
STANDALONE_IMAGE_RE.match(source.strip()) if self.images else None
)
if image is not None:
key = self.inline.resolve_image_entity(image.group(2), image.group(1))
self.builder.add_block(
BLOCK_TYPES.ATOMIC,
" ",
entity_ranges=[{"offset": 0, "length": 1, "key": key}],
)
else:
self._add_text_block(BLOCK_TYPES.UNSTYLED, source, start)
return i
def _parse_fence(self, lines: list[str], i: int, match: re.Match[str]) -> int:
"""Parse a fenced code block. Implemented in Task 9."""
raise NotImplementedError
def _parse_quote(self, lines: list[str], i: int) -> int:
"""Parse a blockquote. Implemented in Task 9."""
raise NotImplementedError
def _parse_list(self, lines: list[str], i: int) -> int:
"""Parse a list. Implemented in Task 10."""
raise NotImplementedError- Step 4: Run tests to verify they pass
Run: just test tests/markdown_parser/test_blocks.py
Expected: PASS
- Step 5: Commit
git add draftjs_exporter/markdown_parser/blocks.py tests/markdown_parser/test_blocks.py
git commit -m "Add block parser: paragraphs, headings, thematic breaks"Files:
-
Modify:
draftjs_exporter/markdown_parser/blocks.py(implement_parse_fence,_parse_quote) -
Test:
tests/markdown_parser/test_blocks.py(append) -
Step 1: Write the failing tests
class TestBlockquotes(unittest.TestCase):
def test_single_line_quote(self):
cs = parse("> quoted")
self.assertEqual(block_types(cs), ["blockquote"])
self.assertEqual(cs["blocks"][0]["text"], "quoted")
def test_multiline_quote_joins(self):
cs = parse("> a\n> b")
self.assertEqual(len(cs["blocks"]), 1)
self.assertEqual(cs["blocks"][0]["text"], "a\nb")
def test_empty_quote_line_splits_blocks(self):
cs = parse("> a\n>\n> b")
self.assertEqual(block_types(cs), ["blockquote", "blockquote"])
def test_quote_content_is_inline_parsed(self):
cs = parse("> **bold**")
self.assertEqual(cs["blocks"][0]["text"], "bold")
self.assertEqual(
cs["blocks"][0]["inlineStyleRanges"],
[{"offset": 0, "length": 4, "style": "BOLD"}],
)
def test_quote_without_space(self):
cs = parse(">quoted")
self.assertEqual(cs["blocks"][0]["text"], "quoted")
class TestFencedCode(unittest.TestCase):
def test_backtick_fence(self):
cs = parse("```\ncode line\n```")
self.assertEqual(block_types(cs), ["code-block"])
self.assertEqual(cs["blocks"][0]["text"], "code line")
def test_tilde_fence(self):
cs = parse("~~~\ncode\n~~~")
self.assertEqual(block_types(cs), ["code-block"])
def test_info_string_ignored(self):
cs = parse("```python\nx = 1\n```")
self.assertEqual(cs["blocks"][0]["text"], "x = 1")
def test_multiline_code(self):
cs = parse("```\na\nb\n```")
self.assertEqual(cs["blocks"][0]["text"], "a\nb")
def test_unclosed_fence_parses_to_eof(self):
cs = parse("```\ncode")
self.assertEqual(block_types(cs), ["code-block"])
self.assertEqual(cs["blocks"][0]["text"], "code")
def test_code_content_not_inline_parsed(self):
cs = parse("```\n**not bold**\n```")
self.assertEqual(cs["blocks"][0]["text"], "**not bold**")
self.assertEqual(cs["blocks"][0]["inlineStyleRanges"], [])
def test_closing_fence_must_match_marker(self):
cs = parse("```\na\n~~~\nb\n```")
self.assertEqual(cs["blocks"][0]["text"], "a\n~~~\nb")- Step 2: Run tests to verify they fail
Run: just test tests/markdown_parser/test_blocks.py -k "Blockquote or Fenced"
Expected: FAIL — NotImplementedError
- Step 3: Implement
Replace the two stubs in draftjs_exporter/markdown_parser/blocks.py:
def _parse_fence(self, lines: list[str], i: int, match: re.Match[str]) -> int:
"""Parse a fenced code block starting at line i.
Unclosed fences parse to end of input, per CommonMark.
Parameters:
lines: All source lines.
i: Index of the opening fence line.
match: The fence opener match.
Returns:
The index of the first line after the block.
"""
fence = match.group(1)
marker = fence[0]
size = len(fence)
body: list[str] = []
i += 1
while i < len(lines):
close = FENCE_RE.match(lines[i])
if (
close is not None
and close.group(1)[0] == marker
and len(close.group(1)) >= size
):
i += 1
break
body.append(lines[i])
i += 1
self.builder.add_block(BLOCK_TYPES.CODE, "\n".join(body))
return i
def _parse_quote(self, lines: list[str], i: int) -> int:
"""Parse consecutive blockquote lines into blockquote blocks.
Quoted lines join with newlines. A quoted line with no content
(``>`` alone) splits the quote into separate blocks.
Parameters:
lines: All source lines.
i: Index of the first quoted line.
Returns:
The index of the first line after the quote.
"""
start = i
quote_lines: list[str] = []
while i < len(lines):
match = QUOTE_RE.match(lines[i])
if match is None:
break
quote_lines.append(match.group(1))
i += 1
paragraph: list[str] = []
for offset, content in enumerate(quote_lines):
if not content.strip():
if paragraph:
self._add_text_block(
BLOCK_TYPES.BLOCKQUOTE, "\n".join(paragraph), start + offset
)
paragraph = []
else:
paragraph.append(content)
if paragraph:
self._add_text_block(
BLOCK_TYPES.BLOCKQUOTE, "\n".join(paragraph), start + len(quote_lines) - 1
)
return i- Step 4: Run tests to verify they pass
Run: just test tests/markdown_parser/test_blocks.py
Expected: PASS
- Step 5: Commit
git add draftjs_exporter/markdown_parser/blocks.py tests/markdown_parser/test_blocks.py
git commit -m "Add block parser blockquotes and fenced code"Files:
-
Modify:
draftjs_exporter/markdown_parser/blocks.py(implement_parse_list) -
Test:
tests/markdown_parser/test_blocks.py(append) -
Step 1: Write the failing tests
class TestLists(unittest.TestCase):
def test_unordered_flat(self):
cs = parse("- a\n- b")
self.assertEqual(
block_types(cs), ["unordered-list-item", "unordered-list-item"]
)
self.assertEqual([b["depth"] for b in cs["blocks"]], [0, 0])
def test_ordered_flat(self):
cs = parse("1. a\n2. b")
self.assertEqual(block_types(cs), ["ordered-list-item"] * 2)
def test_ordered_with_paren_delimiter(self):
cs = parse("1) a")
self.assertEqual(block_types(cs), ["ordered-list-item"])
def test_all_bullet_markers(self):
for marker in "*+-":
cs = parse(f"{marker} item")
self.assertEqual(block_types(cs), ["unordered-list-item"])
def test_nested_unordered(self):
cs = parse("- a\n - b\n - c")
self.assertEqual([b["depth"] for b in cs["blocks"]], [0, 1, 2])
def test_nested_then_back_to_top(self):
cs = parse("- a\n - b\n- c")
self.assertEqual([b["depth"] for b in cs["blocks"]], [0, 1, 0])
def test_mixed_kinds_by_indent(self):
cs = parse("- a\n 1. b")
self.assertEqual(
block_types(cs), ["unordered-list-item", "ordered-list-item"]
)
self.assertEqual(cs["blocks"][1]["depth"], 1)
def test_list_content_is_inline_parsed(self):
cs = parse("- **bold**")
self.assertEqual(cs["blocks"][0]["text"], "bold")
def test_blank_line_ends_list(self):
cs = parse("- a\n\nparagraph")
self.assertEqual(block_types(cs), ["unordered-list-item", "unstyled"])
def test_paragraph_after_list_without_blank_line(self):
cs = parse("- a\nparagraph")
self.assertEqual(block_types(cs), ["unordered-list-item", "unstyled"])
def test_list_between_paragraphs(self):
cs = parse("intro\n\n- item\n\noutro")
self.assertEqual(
block_types(cs), ["unstyled", "unordered-list-item", "unstyled"]
)- Step 2: Run tests to verify they fail
Run: just test tests/markdown_parser/test_blocks.py -k Lists
Expected: FAIL — NotImplementedError
- Step 3: Implement
Replace the _parse_list stub in draftjs_exporter/markdown_parser/blocks.py:
def _parse_list(self, lines: list[str], i: int) -> int:
"""Parse consecutive list items with indent-based depth tracking.
Depth derives from a stack of indent widths: deeper indents
push, shallower indents pop. Continuation lines (indented
content without a marker) are not supported — they end the
list and become paragraphs.
Parameters:
lines: All source lines.
i: Index of the first list item line.
Returns:
The index of the first line after the list.
"""
stack: list[int] = []
while i < len(lines):
line = lines[i]
if not line.strip():
break
unordered = ULIST_RE.match(line) if self.unordered_list else None
ordered = OLIST_RE.match(line) if self.ordered_list else None
match = unordered or ordered
if match is None:
break
indent = len(match.group(1).replace("\t", " "))
while stack and indent < stack[-1]:
stack.pop()
if not stack or indent > stack[-1]:
stack.append(indent)
type_ = (
BLOCK_TYPES.UNORDERED_LIST_ITEM
if unordered
else BLOCK_TYPES.ORDERED_LIST_ITEM
)
self._add_text_block(type_, match.group(2), i, depth=len(stack) - 1)
i += 1
return i- Step 4: Run tests to verify they pass
Run: just test tests/markdown_parser/test_blocks.py
Expected: PASS
- Step 5: Commit
git add draftjs_exporter/markdown_parser/blocks.py tests/markdown_parser/test_blocks.py
git commit -m "Add block parser lists with depth tracking"Files:
-
Modify:
draftjs_exporter/markdown_parser/blocks.py(no change expected — standalone image logic shipped in Task 8; this task validates it and covers config toggles) -
Test:
tests/markdown_parser/test_blocks.py(append),tests/markdown_parser/test_parser_config.py(new) -
Step 1: Write the failing tests
Append to tests/markdown_parser/test_blocks.py:
class TestStandaloneImages(unittest.TestCase):
def test_standalone_image_is_atomic(self):
cs = parse("")
block = cs["blocks"][0]
self.assertEqual(block["type"], "atomic")
self.assertEqual(block["text"], " ")
self.assertEqual(
block["entityRanges"], [{"offset": 0, "length": 1, "key": 0}]
)
self.assertEqual(
cs["entityMap"]["0"],
{
"type": "IMAGE",
"mutability": "IMMUTABLE",
"data": {"src": "/img.jpg", "alt": "alt text"},
},
)
def test_image_with_text_stays_inline(self):
cs = parse("look  here")
self.assertEqual(block_types(cs), ["unstyled"])
self.assertEqual(cs["blocks"][0]["text"], "look alt here")
def test_images_disabled_stays_text(self):
cs = parse("", images=False)
self.assertEqual(block_types(cs), ["unstyled"])
self.assertEqual(cs["blocks"][0]["text"], "")Create tests/markdown_parser/test_parser_config.py:
"""Tests for parser feature toggles at the block level."""
import unittest
from tests.markdown_parser.test_blocks import block_types, parse
class TestBlockToggles(unittest.TestCase):
def test_headings_disabled(self):
cs = parse("# Title", headings=False)
self.assertEqual(block_types(cs), ["unstyled"])
self.assertEqual(cs["blocks"][0]["text"], "# Title")
def test_blockquote_disabled(self):
cs = parse("> quote", blockquote=False)
self.assertEqual(block_types(cs), ["unstyled"])
def test_code_fenced_disabled(self):
cs = parse("```\ncode\n```", code_fenced=False)
self.assertEqual(block_types(cs), ["unstyled"])
def test_thematic_break_disabled(self):
cs = parse("a\n\n---", thematic_break=False)
self.assertEqual(block_types(cs), ["unstyled"])
self.assertEqual(cs["blocks"][0]["text"], "a\n---")
def test_unordered_list_disabled(self):
cs = parse("- item", unordered_list=False)
self.assertEqual(block_types(cs), ["unstyled"])
self.assertEqual(cs["blocks"][0]["text"], "- item")
def test_ordered_list_disabled(self):
cs = parse("1. item", ordered_list=False)
self.assertEqual(block_types(cs), ["unstyled"])
def test_disabled_construct_does_not_end_paragraph(self):
cs = parse("text\n# heading", headings=False)
self.assertEqual(len(cs["blocks"]), 1)
self.assertEqual(cs["blocks"][0]["text"], "text\n# heading")Note: thematic_break_disabled — with the break disabled, --- continues the paragraph. code_fenced_disabled produces ```\ncode\n``` as paragraph text.
- Step 2: Run tests
Run: just test tests/markdown_parser/
Expected: PASS (logic shipped in Task 8; fix blocks.py if any test exposes a bug)
- Step 3: Commit
git add tests/markdown_parser/
git commit -m "Add standalone image atomic blocks and parser toggle tests"Files:
- Modify:
draftjs_exporter/markdown_parser/__init__.py(full implementation) - Test:
tests/markdown_parser/test_parser.py(new), extendtests/markdown_parser/test_parser_errors.py
Interfaces:
-
Consumes: all parser modules.
-
Produces:
ParserConfig(TypedDict, total=False)— keys per spec;MarkdownParser(config: ParserConfig | None = None)withparse(markdown: str) -> ContentState. -
Step 1: Write the failing tests
tests/markdown_parser/test_parser.py:
"""Tests for the assembled MarkdownParser."""
import unittest
from draftjs_exporter.markdown_parser import MarkdownParser
class TestMarkdownParser(unittest.TestCase):
def test_empty_config_uses_defaults(self):
cs = MarkdownParser().parse("# Hi\n\nSome **bold** text.")
self.assertEqual(
[b["type"] for b in cs["blocks"]], ["header-one", "unstyled"]
)
def test_none_config_uses_defaults(self):
cs = MarkdownParser(None).parse("text")
self.assertEqual(cs["blocks"][0]["text"], "text")
def test_crlf_normalized(self):
cs = MarkdownParser().parse("a\r\n\r\nb")
self.assertEqual(len(cs["blocks"]), 2)
def test_non_string_input_raises_type_error(self):
with self.assertRaises(TypeError):
MarkdownParser().parse(None) # type: ignore[arg-type]
def test_config_toggle_passed_through(self):
cs = MarkdownParser({"headings": False}).parse("# Title")
self.assertEqual(cs["blocks"][0]["type"], "unstyled")
def test_resolvers_passed_through(self):
from draftjs_exporter.markdown_parser.resolvers import scheme_resolver
parser = MarkdownParser(
{
"link_resolvers": [
scheme_resolver("wagtail", {"page": "LINK"}, coerce={"id": int})
]
}
)
cs = parser.parse("[label](wagtail://page?id=3)")
self.assertEqual(cs["entityMap"]["0"]["data"], {"id": 3})
def test_structural_invariants(self):
cs = MarkdownParser().parse(
"# T\n\n- a\n - b\n\n[link](/x) and \n\n---"
)
keys = [b["key"] for b in cs["blocks"]]
self.assertEqual(len(keys), len(set(keys)))
referenced = {
str(r["key"]) for b in cs["blocks"] for r in b["entityRanges"]
}
self.assertEqual(set(cs["entityMap"].keys()), referenced)
for block in cs["blocks"]:
text_length = len(block["text"])
for r in block["inlineStyleRanges"] + block["entityRanges"]:
self.assertGreaterEqual(r["offset"], 0)
self.assertLessEqual(r["offset"] + r["length"], text_length)Append to tests/markdown_parser/test_parser_errors.py:
class TestLineNumbers(unittest.TestCase):
def test_resolver_error_gets_line_number(self):
from draftjs_exporter.markdown_parser import MarkdownParser
def bad(url, label):
raise RuntimeError("boom")
parser = MarkdownParser({"link_resolvers": [bad]})
with self.assertRaises(MarkdownParseError) as ctx:
parser.parse("first\n\nsecond [a](/b)")
self.assertEqual(ctx.exception.line, 3)- Step 2: Run tests to verify they fail
Run: just test tests/markdown_parser/test_parser.py tests/markdown_parser/test_parser_errors.py
Expected: FAIL — ImportError: cannot import name 'MarkdownParser'
- Step 3: Implement
Replace draftjs_exporter/markdown_parser/__init__.py:
"""Markdown parsing engine: converts CommonMark core to Draft.js ContentState."""
from typing import TypedDict
from draftjs_exporter.markdown_parser.blocks import BlockParser
from draftjs_exporter.markdown_parser.builder import ContentStateBuilder
from draftjs_exporter.markdown_parser.inline import InlineParser
from draftjs_exporter.markdown_parser.resolvers import (
EntityResolution as EntityResolution,
)
from draftjs_exporter.markdown_parser.resolvers import (
EntityResolver as EntityResolver,
)
from draftjs_exporter.markdown_parser.resolvers import (
scheme_resolver as scheme_resolver,
)
from draftjs_exporter.types import ContentState
class ParserConfig(TypedDict, total=False):
"""Options controlling which Markdown constructs are recognized."""
headings: bool
blockquote: bool
code_fenced: bool
thematic_break: bool
unordered_list: bool
ordered_list: bool
emphasis: bool
code_inline: bool
links: bool
images: bool
line_breaks: bool
link_resolvers: list[EntityResolver]
image_resolvers: list[EntityResolver]
inline_html_styles: dict[str, str]
class MarkdownParser:
"""Parse Markdown text into a Draft.js ContentState.
Supports the CommonMark core: paragraphs, ATX headings, blockquotes,
fenced code, thematic breaks, lists, emphasis, code spans, links,
images, and hard line breaks. Every input produces either a
structurally valid ContentState or a ``MarkdownParseError``.
"""
__slots__ = ("config",)
def __init__(self, config: ParserConfig | None = None) -> None:
"""Initialize the parser with the given configuration.
Parameters:
config: Feature toggles and entity resolvers. Missing keys
use defaults that enable all constructs.
"""
self.config = config if config is not None else ParserConfig()
def parse(self, markdown: str) -> ContentState:
"""Parse Markdown source into a ContentState.
Parameters:
markdown: The Markdown text to parse.
Returns:
A structurally valid Draft.js ContentState.
Raises:
TypeError: If ``markdown`` is not a string.
MarkdownParseError: If an entity resolver fails.
"""
if not isinstance(markdown, str):
raise TypeError(
f"Expected str, got {type(markdown).__name__}"
)
text = markdown.replace("\r\n", "\n").replace("\r", "\n")
builder = ContentStateBuilder()
images = self.config.get("images", True)
inline = InlineParser(
emphasis=self.config.get("emphasis", True),
code_inline=self.config.get("code_inline", True),
links=self.config.get("links", True),
images=images,
line_breaks=self.config.get("line_breaks", True),
inline_html_styles=self.config.get("inline_html_styles", {}),
link_resolvers=self.config.get("link_resolvers", []),
image_resolvers=self.config.get("image_resolvers", []),
builder=builder,
)
blocks = BlockParser(
headings=self.config.get("headings", True),
blockquote=self.config.get("blockquote", True),
code_fenced=self.config.get("code_fenced", True),
thematic_break=self.config.get("thematic_break", True),
unordered_list=self.config.get("unordered_list", True),
ordered_list=self.config.get("ordered_list", True),
images=images,
inline=inline,
builder=builder,
)
blocks.parse(text)
return builder.build()- Step 4: Run tests to verify they pass
Run: just test tests/markdown_parser/
Expected: PASS (whole suite)
- Step 5: Lint and commit
just lint
git add draftjs_exporter/markdown_parser/ tests/markdown_parser/
git commit -m "Add MarkdownParser with parser config"Files:
- Create:
draftjs_exporter/contentstate_filter/__init__.py - Test:
tests/contentstate_filter/__init__.py(empty),tests/contentstate_filter/test_filter.py,tests/contentstate_filter/test_filter_rules.py
Interfaces:
-
Produces:
FilterCallback: TypeAlias = Callable[[Any], Any]FilterAction: TypeAlias = Literal["remove", "keep", "demote"] | FilterCallbackFilterRule(TypedDict):type: Literal["block", "inline_style", "entity"],match: str,action: FilterActionContentStateFilter(rules: list[FilterRule] | None = None)withapply(content_state: ContentState) -> ContentState— returns a new ContentState; input is not mutated.
-
Step 1: Write the failing tests
tests/contentstate_filter/test_filter.py:
"""Tests for ContentState filtering."""
import unittest
from draftjs_exporter.contentstate_filter import ContentStateFilter
def cs_with_blocks(*blocks):
"""Build a ContentState from blocks with an empty entity map."""
return {"blocks": list(blocks), "entityMap": {}}
def make_block(type_, text="x", depth=0, styles=None, entities=None):
"""Build a single Draft.js block."""
return {
"key": "aaaaa",
"text": text,
"type": type_,
"depth": depth,
"inlineStyleRanges": styles or [],
"entityRanges": entities or [],
}
class TestBlockRules(unittest.TestCase):
def test_remove_block(self):
cs = cs_with_blocks(make_block("header-one"), make_block("unstyled"))
result = ContentStateFilter(
[{"type": "block", "match": "header-one", "action": "remove"}]
).apply(cs)
self.assertEqual([b["type"] for b in result["blocks"]], ["unstyled"])
def test_keep_is_noop(self):
cs = cs_with_blocks(make_block("header-one"))
result = ContentStateFilter(
[{"type": "block", "match": "header-one", "action": "keep"}]
).apply(cs)
self.assertEqual(len(result["blocks"]), 1)
def test_demote_headings(self):
cs = cs_with_blocks(
make_block("header-one"),
make_block("header-three"),
)
result = ContentStateFilter(
[{"type": "block", "match": "header-one", "action": "demote"}]
).apply(cs)
self.assertEqual(
[b["type"] for b in result["blocks"]], ["header-two", "header-three"]
)
def test_unmatched_blocks_kept(self):
cs = cs_with_blocks(make_block("header-two"))
result = ContentStateFilter(
[{"type": "block", "match": "header-one", "action": "remove"}]
).apply(cs)
self.assertEqual(len(result["blocks"]), 1)
def test_callable_replaces_block(self):
def replace(block):
return {**block, "type": "unstyled"}
cs = cs_with_blocks(make_block("header-one"))
result = ContentStateFilter(
[{"type": "block", "match": "header-one", "action": replace}]
).apply(cs)
self.assertEqual(result["blocks"][0]["type"], "unstyled")
def test_callable_none_removes(self):
cs = cs_with_blocks(make_block("unstyled"))
result = ContentStateFilter(
[{"type": "block", "match": "unstyled", "action": lambda b: None}]
).apply(cs)
self.assertEqual(result["blocks"], [])
def test_input_not_mutated(self):
block = make_block("header-one")
cs = cs_with_blocks(block)
ContentStateFilter(
[{"type": "block", "match": "header-one", "action": "demote"}]
).apply(cs)
self.assertEqual(block["type"], "header-one")
class TestInlineStyleRules(unittest.TestCase):
def test_remove_style(self):
block = make_block(
"unstyled",
text="abc",
styles=[
{"offset": 0, "length": 1, "style": "BOLD"},
{"offset": 1, "length": 1, "style": "ITALIC"},
],
)
result = ContentStateFilter(
[{"type": "inline_style", "match": "BOLD", "action": "remove"}]
).apply(cs_with_blocks(block))
self.assertEqual(
result["blocks"][0]["inlineStyleRanges"],
[{"offset": 1, "length": 1, "style": "ITALIC"}],
)
def test_callable_renames_style(self):
block = make_block(
"unstyled", text="a", styles=[{"offset": 0, "length": 1, "style": "X"}]
)
result = ContentStateFilter(
[
{
"type": "inline_style",
"match": "X",
"action": lambda style: "BOLD",
}
]
).apply(cs_with_blocks(block))
self.assertEqual(
result["blocks"][0]["inlineStyleRanges"][0]["style"], "BOLD"
)
class TestEntityRules(unittest.TestCase):
def setUp(self):
self.cs = {
"blocks": [
make_block(
"unstyled",
text="ab",
entities=[{"offset": 0, "length": 1, "key": 0}],
)
],
"entityMap": {
"0": {"type": "LINK", "mutability": "MUTABLE", "data": {"url": "/x"}}
},
}
def test_remove_entity(self):
result = ContentStateFilter(
[{"type": "entity", "match": "LINK", "action": "remove"}]
).apply(self.cs)
self.assertEqual(result["blocks"][0]["entityRanges"], [])
self.assertEqual(result["entityMap"], {})
def test_unmatched_entity_kept(self):
result = ContentStateFilter(
[{"type": "entity", "match": "IMAGE", "action": "remove"}]
).apply(self.cs)
self.assertEqual(result["entityMap"], self.cs["entityMap"])
def test_callable_replaces_entity(self):
result = ContentStateFilter(
[
{
"type": "entity",
"match": "LINK",
"action": lambda e: {**e, "data": {"url": "/y"}},
}
]
).apply(self.cs)
self.assertEqual(result["entityMap"]["0"]["data"], {"url": "/y"})
def test_orphaned_range_dropped(self):
self.cs["blocks"][0]["entityRanges"] = [{"offset": 0, "length": 1, "key": 9}]
result = ContentStateFilter([]).apply(self.cs)
self.assertEqual(result["blocks"][0]["entityRanges"], [])
class TestDepthNormalization(unittest.TestCase):
def test_removed_parent_clamps_depth(self):
cs = cs_with_blocks(
make_block("unordered-list-item", depth=0),
make_block("unordered-list-item", depth=1),
make_block("unordered-list-item", depth=2),
)
result = ContentStateFilter(
[
{
"type": "block",
"match": "unordered-list-item",
"action": lambda b: None
if b["depth"] == 0
else b,
}
]
).apply(cs)
self.assertEqual([b["depth"] for b in result["blocks"]], [0, 1])
def test_depth_reset_after_non_list_block(self):
cs = cs_with_blocks(
make_block("unstyled"),
make_block("unordered-list-item", depth=2),
)
result = ContentStateFilter([]).apply(cs)
self.assertEqual(result["blocks"][1]["depth"], 0)tests/contentstate_filter/test_filter_rules.py:
"""Tests for filter rule validation."""
import unittest
from draftjs_exporter.contentstate_filter import ContentStateFilter
from draftjs_exporter.error import ConfigException
class TestRuleValidation(unittest.TestCase):
def test_invalid_rule_type(self):
with self.assertRaises(ConfigException):
ContentStateFilter([{"type": "nope", "match": "x", "action": "keep"}]) # type: ignore[typeddict-item]
def test_invalid_action(self):
with self.assertRaises(ConfigException):
ContentStateFilter([{"type": "block", "match": "x", "action": "nope"}]) # type: ignore[typeddict-item]
def test_demote_requires_header_block(self):
with self.assertRaises(ConfigException):
ContentStateFilter(
[{"type": "block", "match": "unstyled", "action": "demote"}]
)
def test_demote_header_six_rejected(self):
with self.assertRaises(ConfigException):
ContentStateFilter(
[{"type": "block", "match": "header-six", "action": "demote"}]
)
def test_demote_on_non_block_rejected(self):
with self.assertRaises(ConfigException):
ContentStateFilter(
[{"type": "entity", "match": "header-one", "action": "demote"}]
)
def test_no_rules_is_identity(self):
cs = {"blocks": [], "entityMap": {}}
self.assertEqual(ContentStateFilter().apply(cs), cs)
self.assertEqual(ContentStateFilter(None).apply(cs), cs)
def test_callable_returning_garbage_rejected(self):
cs = {
"blocks": [
{
"key": "a",
"text": "x",
"type": "unstyled",
"depth": 0,
"inlineStyleRanges": [],
"entityRanges": [],
}
],
"entityMap": {},
}
with self.assertRaises(ConfigException):
ContentStateFilter(
[{"type": "block", "match": "unstyled", "action": lambda b: 42}]
).apply(cs)- Step 2: Run tests to verify they fail
Run: just test tests/contentstate_filter/
Expected: FAIL — ModuleNotFoundError
- Step 3: Implement
draftjs_exporter/contentstate_filter/__init__.py:
"""Filter Draft.js ContentState with declarative rules."""
import copy
from collections.abc import Callable
from typing import Any, Literal, TypeAlias, TypedDict
from draftjs_exporter.constants import BLOCK_TYPES
from draftjs_exporter.error import ConfigException
from draftjs_exporter.types import Block, ContentState, Entity
FilterCallback: TypeAlias = Callable[[Any], Any]
"""Custom rule action: receives the matched object, returns a replacement or None."""
FilterAction: TypeAlias = Literal["remove", "keep", "demote"] | FilterCallback
"""Predefined action name or custom callback."""
class FilterRule(TypedDict):
"""A single filtering rule."""
type: Literal["block", "inline_style", "entity"]
"""Which kind of object the rule matches."""
match: str
"""The block type, style name, or entity type to match."""
action: FilterAction
"""What to do with matching objects."""
HEADER_DEMOTION = {
BLOCK_TYPES.HEADER_ONE: BLOCK_TYPES.HEADER_TWO,
BLOCK_TYPES.HEADER_TWO: BLOCK_TYPES.HEADER_THREE,
BLOCK_TYPES.HEADER_THREE: BLOCK_TYPES.HEADER_FOUR,
BLOCK_TYPES.HEADER_FOUR: BLOCK_TYPES.HEADER_FIVE,
BLOCK_TYPES.HEADER_FIVE: BLOCK_TYPES.HEADER_SIX,
}
"""Heading demotion targets. ``header-six`` cannot be demoted."""
LIST_BLOCKS = frozenset(
{BLOCK_TYPES.UNORDERED_LIST_ITEM, BLOCK_TYPES.ORDERED_LIST_ITEM}
)
"""Block types whose depth participates in list nesting."""
VALID_ACTIONS = frozenset({"remove", "keep", "demote"})
class ContentStateFilter:
"""Apply declarative rules to a ContentState.
Rules run in definition order per object. Objects without a
matching rule are kept. The filter always produces a structurally
valid ContentState: entity ranges and the entity map stay in sync,
and list depths are re-normalized after removals.
"""
__slots__ = ("rules",)
def __init__(self, rules: list[FilterRule] | None = None) -> None:
"""Initialize the filter, validating rules eagerly.
Parameters:
rules: The rules to apply, in order.
Raises:
ConfigException: If a rule is malformed.
"""
self.rules = rules if rules is not None else []
for rule in self.rules:
self._validate(rule)
@staticmethod
def _validate(rule: FilterRule) -> None:
"""Check a rule for structural validity.
Parameters:
rule: The rule to validate.
Raises:
ConfigException: If the rule type or action is invalid.
"""
if rule["type"] not in ("block", "inline_style", "entity"):
raise ConfigException(f"Invalid filter rule type: {rule['type']!r}")
action = rule["action"]
if not callable(action) and action not in VALID_ACTIONS:
raise ConfigException(f"Invalid filter rule action: {action!r}")
if action == "demote" and (
rule["type"] != "block" or rule["match"] not in HEADER_DEMOTION
):
raise ConfigException(
'"demote" only applies to header-one through header-five blocks'
)
def apply(self, content_state: ContentState) -> ContentState:
"""Apply all rules, returning a new ContentState.
Parameters:
content_state: The ContentState to filter. Not mutated.
Returns:
The filtered ContentState.
"""
entity_map_in = content_state.get("entityMap", {})
block_rules = self._rules_by_match("block")
style_rules = self._rules_by_match("inline_style")
entity_rules = self._rules_by_match("entity")
out_blocks: list[Block] = []
replacements: dict[str, Entity] = {}
used_keys: set[str] = set()
for block in content_state.get("blocks", []):
kept = self._apply_block_rule(copy.deepcopy(block), block_rules)
if kept is None:
continue
self._apply_style_rules(kept, style_rules)
self._apply_entity_rules(
kept, entity_map_in, entity_rules, replacements, used_keys
)
out_blocks.append(kept)
self._normalize_depths(out_blocks)
entity_map_out = {}
for key in used_keys:
entity = replacements.get(key, entity_map_in[key])
entity_map_out[key] = entity
return {"blocks": out_blocks, "entityMap": entity_map_out}
def _rules_by_match(
self, kind: str
) -> dict[str, list[FilterAction]]:
"""Group actions for a rule kind by match value."""
grouped: dict[str, list[FilterAction]] = {}
for rule in self.rules:
if rule["type"] == kind:
grouped.setdefault(rule["match"], []).append(rule["action"])
return grouped
@staticmethod
def _run_actions(
value: Any, actions: list[FilterAction], kind: str
) -> Any:
"""Run a chain of actions over a matched object.
Parameters:
value: The matched object.
actions: The actions to run in order.
kind: Rule kind, for error messages.
Returns:
The transformed object, or None when removed.
Raises:
ConfigException: If a callback returns an invalid value.
"""
current = value
for action in actions:
if current is None:
break
if action == "keep":
continue
if action == "remove":
current = None
continue
if action == "demote":
current = {**current, "type": HEADER_DEMOTION[current["type"]]}
continue
current = action(current)
valid = (
current is None
or (kind == "inline_style" and isinstance(current, str))
or (kind != "inline_style" and isinstance(current, dict))
)
if not valid:
raise ConfigException(
f"Filter callback for {kind} rule returned invalid value"
)
return current
def _apply_block_rule(
self, block: Block, rules: dict[str, list[FilterAction]]
) -> Block | None:
"""Apply block rules to a single block."""
actions = rules.get(block.get("type", ""), [])
if not actions:
return block
result = self._run_actions(block, actions, "block")
if result is not None and "type" not in result:
raise ConfigException("Filter block callback must return a block")
return result
def _apply_style_rules(
self, block: Block, rules: dict[str, list[FilterAction]]
) -> None:
"""Apply inline style rules to a block's style ranges."""
ranges = block.get("inlineStyleRanges", [])
if not ranges or not rules:
return
kept = []
for style_range in ranges:
actions = rules.get(style_range["style"], [])
if not actions:
kept.append(style_range)
continue
result = self._run_actions(style_range["style"], actions, "inline_style")
if result is not None:
kept.append({**style_range, "style": result})
block["inlineStyleRanges"] = kept
def _apply_entity_rules(
self,
block: Block,
entity_map: dict[str, Entity],
rules: dict[str, list[FilterAction]],
replacements: dict[str, Entity],
used_keys: set[str],
) -> None:
"""Apply entity rules to a block's entity ranges."""
kept = []
for entity_range in block.get("entityRanges", []):
key = str(entity_range["key"])
entity = entity_map.get(key)
if entity is None:
# Orphaned range: drop to keep output valid.
continue
actions = rules.get(entity.get("type", ""), [])
if not actions:
kept.append(entity_range)
used_keys.add(key)
continue
result = self._run_actions(copy.deepcopy(entity), actions, "entity")
if result is not None:
if "type" not in result:
raise ConfigException(
"Filter entity callback must return an entity"
)
kept.append(entity_range)
used_keys.add(key)
replacements[key] = result
block["entityRanges"] = kept
@staticmethod
def _normalize_depths(blocks: list[Block]) -> None:
"""Clamp list depths so nesting never skips a level.
Parameters:
blocks: Blocks to normalize in place.
"""
last_list_depth = -1
for block in blocks:
if block.get("type") in LIST_BLOCKS:
depth = block.get("depth", 0)
if depth > last_list_depth + 1:
depth = last_list_depth + 1
block["depth"] = depth
last_list_depth = depth
else:
last_list_depth = -1- Step 4: Run tests to verify they pass
Run: just test tests/contentstate_filter/
Expected: PASS
- Step 5: Lint and commit
just lint
git add draftjs_exporter/contentstate_filter/ tests/contentstate_filter/
git commit -m "Add ContentStateFilter with declarative rules"Files:
- Create:
draftjs_exporter/markdown_importer/__init__.py - Modify:
draftjs_exporter/__init__.py - Test:
tests/markdown_importer/__init__.py(empty),tests/markdown_importer/test_importer.py, extendtests/test_init.py
Interfaces:
-
Consumes: everything.
-
Produces:
ImporterConfig(TypedDict, total=False)—parser: str,parser_config: ParserConfig,filter_rules: list[FilterRule];MarkdownImporter(config)withimport_markdown(markdown: str) -> ContentState. -
Step 1: Write the failing tests
tests/markdown_importer/test_importer.py:
"""Tests for the MarkdownImporter public API."""
import unittest
from draftjs_exporter.markdown_importer import MarkdownImporter
class TestMarkdownImporter(unittest.TestCase):
def test_default_import(self):
cs = MarkdownImporter().import_markdown("# Hello\n\nWorld")
self.assertEqual(
[b["type"] for b in cs["blocks"]], ["header-one", "unstyled"]
)
def test_none_config(self):
cs = MarkdownImporter(None).import_markdown("text")
self.assertEqual(cs["blocks"][0]["text"], "text")
def test_filter_rules_applied(self):
importer = MarkdownImporter(
{
"filter_rules": [
{"type": "block", "match": "header-one", "action": "demote"}
]
}
)
cs = importer.import_markdown("# Hello")
self.assertEqual(cs["blocks"][0]["type"], "header-two")
def test_parser_config_applied(self):
importer = MarkdownImporter({"parser_config": {"headings": False}})
cs = importer.import_markdown("# Hello")
self.assertEqual(cs["blocks"][0]["type"], "unstyled")
def test_custom_parser_dotted_path(self):
importer = MarkdownImporter(
{"parser": "draftjs_exporter.markdown_parser.MarkdownParser"}
)
cs = importer.import_markdown("text")
self.assertEqual(cs["blocks"][0]["text"], "text")
def test_parse_error_propagates(self):
from draftjs_exporter.error import MarkdownParseError
def bad(url, label):
raise RuntimeError("boom")
importer = MarkdownImporter({"parser_config": {"link_resolvers": [bad]}})
with self.assertRaises(MarkdownParseError):
importer.import_markdown("[a](/b)")
def test_wagtail_style_end_to_end(self):
from draftjs_exporter.markdown_parser import scheme_resolver
importer = MarkdownImporter(
{
"parser_config": {
"image_resolvers": [
scheme_resolver(
"wagtail",
{"image": "IMAGE"},
coerce={"id": int},
label_key="alt",
mutability="IMMUTABLE",
)
]
}
}
)
cs = importer.import_markdown(
""
)
self.assertEqual(
cs["entityMap"]["0"]["data"],
{"id": 10, "alt": "alt", "format": "left"},
)Check tests/test_init.py first, then extend it to assert the new names are importable from draftjs_exporter and listed in __all__.
- Step 2: Run tests to verify they fail
Run: just test tests/markdown_importer/
Expected: FAIL — ModuleNotFoundError
- Step 3: Implement
draftjs_exporter/markdown_importer/__init__.py:
"""Public Markdown importer: parses Markdown, then filters the ContentState."""
from typing import TypedDict
from draftjs_exporter.contentstate_filter import ContentStateFilter, FilterRule
from draftjs_exporter.markdown_parser import MarkdownParser, ParserConfig
from draftjs_exporter.types import ContentState
from draftjs_exporter.utils.module_loading import import_string
DEFAULT_PARSER = "draftjs_exporter.markdown_parser.MarkdownParser"
"""Dotted path of the built-in parser engine."""
class ImporterConfig(TypedDict, total=False):
"""Configuration for the Markdown importer."""
parser: str
"""Dotted path of the parser engine class. Defaults to the built-in parser."""
parser_config: ParserConfig
"""Options passed to the parser engine constructor."""
filter_rules: list[FilterRule]
"""Rules applied to the parsed ContentState."""
class MarkdownImporter:
"""Import Markdown text as a Draft.js ContentState.
Combines a parser engine (Markdown to ContentState) with a filter
(content policy on the result). The parser is referenced by dotted
path so alternative engines can be swapped in.
"""
__slots__ = ("parser", "filter")
def __init__(self, config: ImporterConfig | None = None) -> None:
"""Initialize the importer with the given configuration.
Parameters:
config: Parser engine, parser options, and filter rules.
"""
if config is None:
config = {}
parser_class = import_string(config.get("parser", DEFAULT_PARSER))
self.parser: MarkdownParser = parser_class(config.get("parser_config"))
self.filter = ContentStateFilter(config.get("filter_rules"))
def import_markdown(self, markdown: str) -> ContentState:
"""Parse Markdown and apply filter rules.
Parameters:
markdown: The Markdown text to import.
Returns:
The parsed, filtered ContentState.
Raises:
MarkdownParseError: If the input cannot be parsed.
"""
return self.filter.apply(self.parser.parse(markdown))Modify draftjs_exporter/__init__.py — add imports after the existing ones:
from draftjs_exporter.contentstate_filter import (
ContentStateFilter as ContentStateFilter,
)
from draftjs_exporter.contentstate_filter import FilterRule as FilterRule
from draftjs_exporter.error import MarkdownParseError as MarkdownParseError
from draftjs_exporter.markdown_importer import ImporterConfig as ImporterConfig
from draftjs_exporter.markdown_importer import MarkdownImporter as MarkdownImporter
from draftjs_exporter.markdown_parser import EntityResolution as EntityResolution
from draftjs_exporter.markdown_parser import EntityResolver as EntityResolver
from draftjs_exporter.markdown_parser import MarkdownParser as MarkdownParser
from draftjs_exporter.markdown_parser import ParserConfig as ParserConfig
from draftjs_exporter.markdown_parser import scheme_resolver as scheme_resolverAnd append to __all__ (before the closing bracket):
# Importer
"MarkdownImporter",
"ImporterConfig",
"MarkdownParser",
"ParserConfig",
"ContentStateFilter",
"FilterRule",
"MarkdownParseError",
"EntityResolution",
"EntityResolver",
"scheme_resolver",- Step 4: Run tests to verify they pass
Run: just test tests/markdown_importer/ tests/test_init.py
Expected: PASS
- Step 5: Lint and commit
just lint
git add draftjs_exporter/ tests/markdown_importer/ tests/test_init.py
git commit -m "Add MarkdownImporter public API"Files:
- Create:
tests/test_imports.py,tests/test_imports.json - Modify:
tests/test_exports.json(add"import"overrides)
Interfaces:
-
Consumes:
MarkdownImporter. Uses the round-trip config: default parser +inline_html_styleswhitelist for the tags the Markdown exporter emits:u, sup, sub, mark, q, small, samp, ins, del, kbd. -
Step 1: Write the snapshot test harness
tests/test_imports.py:
import json
import os
import unittest
from draftjs_exporter.markdown_importer import MarkdownImporter
from draftjs_exporter.markdown_parser import scheme_resolver
fixtures_path = os.path.join(os.path.dirname(__file__), "test_exports.json")
with open(fixtures_path) as f:
export_fixtures = json.loads(f.read())
imports_path = os.path.join(os.path.dirname(__file__), "test_imports.json")
with open(imports_path) as f:
import_fixtures = json.loads(f.read())
# Tags the Markdown exporter emits as inline HTML fallback (see the
# exporter's markdown fallbacks module). Whitelisting them lets most
# styles round-trip.
ROUNDTRIP_INLINE_HTML_STYLES = {
"u": "UNDERLINE",
"sup": "SUPERSCRIPT",
"sub": "SUBSCRIPT",
"mark": "MARK",
"q": "QUOTATION",
"small": "SMALL",
"samp": "SAMPLE",
"ins": "INSERT",
"del": "DELETE",
"kbd": "KEYBOARD",
}
def make_importer():
"""Build the importer used for round-trip snapshot tests."""
return MarkdownImporter(
{
"parser_config": {
"inline_html_styles": ROUNDTRIP_INLINE_HTML_STYLES,
}
}
)
def normalize(content_state):
"""Rewrite block and entity keys for deterministic comparison.
Block keys become sequential; entity keys are remapped in order of
first appearance in entity ranges, and the entity map is rebuilt
to match.
"""
entity_map = content_state.get("entityMap", {})
key_map: dict[str, str] = {}
blocks = []
for index, block in enumerate(content_state.get("blocks", [])):
ranges = []
for entity_range in block.get("entityRanges", []):
old_key = str(entity_range["key"])
if old_key not in key_map:
key_map[old_key] = str(len(key_map))
ranges.append({**entity_range, "key": int(key_map[old_key])})
blocks.append(
{
"key": f"{index:05d}",
"text": block.get("text", ""),
"type": block.get("type", "unstyled"),
"depth": block.get("depth", 0),
"inlineStyleRanges": block.get("inlineStyleRanges", []),
"entityRanges": ranges,
**({"data": block["data"]} if block.get("data") else {}),
}
)
new_map = {}
for old_key, new_key in key_map.items():
if old_key in entity_map:
new_map[new_key] = entity_map[old_key]
return {"blocks": blocks, "entityMap": new_map}
class TestRoundTrip(unittest.TestCase):
"""Import the recorded Markdown output of every export fixture."""
def test_round_trip(self):
importer = make_importer()
for fixture in export_fixtures:
markdown = fixture["output"]["markdown"]
expected = fixture.get("import", fixture["content_state"])
with self.subTest(fixture=fixture["label"]):
self.assertEqual(
normalize(importer.import_markdown(markdown)),
normalize(expected),
)
class TestDirectImports(unittest.TestCase):
"""Import hand-written Markdown covering importer-only behavior."""
def test_imports(self):
for fixture in import_fixtures:
config = fixture.get("config", {})
if config.get("wagtail_resolvers"):
config = {
**config,
"parser_config": {
**config.get("parser_config", {}),
"link_resolvers": [
scheme_resolver(
"wagtail",
{"page": "LINK", "document": "DOCUMENT"},
coerce={"id": int},
)
],
"image_resolvers": [
scheme_resolver(
"wagtail",
{"image": "IMAGE", "media": "EMBED"},
coerce={"id": int},
label_key="alt",
mutability="IMMUTABLE",
)
],
},
}
config.pop("wagtail_resolvers")
importer = MarkdownImporter(config)
with self.subTest(fixture=fixture["label"]):
self.assertEqual(
normalize(importer.import_markdown(fixture["markdown"])),
normalize(fixture["content_state"]),
)- Step 2: Add
"import"overrides totests/test_exports.json
Run the harness first — 14 of 18 fixtures should pass as-is. For the 4 fixtures below, add an "import" key with the expected post-import ContentState. Generate each override by printing normalize(make_importer().import_markdown(fixture["output"]["markdown"])), then verify by hand that the differences from content_state match exactly the documented information loss:
- "Style map defaults" — only difference: the STRIKETHROUGH range is gone, and the block text contains the literal
~4~instead of4(single-tilde strikethrough is GFM, out of scope). All HTML-tag styles (<u>,<sup>,<sub>,<mark>,<q>,<small>,<samp>,<ins>,<del>,<kbd>) must round-trip with ranges intact. - "Multiple decorators" — only difference: a LINK entity appears for
http://www.google.com#world(the export fixture's linkify composite decorator created the link; the original content state has plain text). - "From https://github.com/icelab/…" — differences: the IMAGE entity keeps only
src(plusaltif present in the Markdown) —caption,rightsHolder,featuredare lost; the atomic block text becomes" "(was" :)"). - "Big content export" — differences: the EMBED entity and its atomic block are gone (the export config discards EMBED); the IMAGE entity keeps only
srcandalt(width/heightlost); the HIGHLIGHT style is lost (exported as<strong style="…">, which has attributes → literal text, so the text also gains the literal tag characters); STRIKETHROUGH lost with literal~…~text; KBD round-trips via the whitelist; HORIZONTAL_RULE round-trips exactly.
Any other difference indicates a parser bug — fix the parser, not the fixture.
- Step 3: Create
tests/test_imports.json
Direct import fixtures. Cases (each with label, markdown, content_state; config where noted):
- "Italic with stars" —
*a*→ ITALIC range overa. - "Bold with underscores" —
__a__→ BOLD range. - "Plus bullet list" —
+ a→unordered-list-item. - "Paren ordered list" —
1) a→ordered-list-item. - "Tilde fence" —
~~~\nx\n~~~→code-blockwith textx. - "Star thematic break" —
***→ atomic HORIZONTAL_RULE. - "Inline HTML styles" — markdown
x <sup>2</sup>, config{"parser_config": {"inline_html_styles": {"sup": "SUPERSCRIPT"}}}→ SUPERSCRIPT range over2. - "Wagtail internal image" — markdown
, config{"wagtail_resolvers": true}→ atomic block, IMAGE entity data{"id": 10, "format": "left", "alt": "alt"}(label fillsaltsince query omits it). - "Wagtail page link" — markdown
[label](wagtail://page?id=3), config{"wagtail_resolvers": true}→ LINK entity data{"id": 3}. - "Filter demote in importer" — markdown
# T, config{"filter_rules": [{"type": "block", "match": "header-one", "action": "demote"}]}→header-twoblock.
Full example for fixture 1 (others follow the same shape):
[
{
"label": "Italic with stars",
"markdown": "*a*",
"content_state": {
"entityMap": {},
"blocks": [
{
"key": "00000",
"text": "a",
"type": "unstyled",
"depth": 0,
"inlineStyleRanges": [
{ "offset": 0, "length": 1, "style": "ITALIC" }
],
"entityRanges": []
}
]
}
}
]- Step 4: Run tests
Run: just test tests/test_imports.py
Expected: PASS (all 18 round-trip + 10 direct cases)
- Step 5: Commit
git add tests/test_imports.py tests/test_imports.json tests/test_exports.json
git commit -m "Add importer snapshot tests with fixture round-trips"Files:
- Modify:
tests/test_properties.py(append) - Modify:
tests/strategies.py(append — safe-text strategy + importer-friendly content states)
Interfaces:
-
Consumes:
MarkdownImporter,ContentStateFilter, existingcontent_statesstrategy,MARKDOWN_CONFIG. -
Step 1: Extend
tests/strategies.py
Append:
# Text that survives a Markdown export → import round-trip unchanged:
# no Markdown-special characters, no leading/trailing whitespace quirks.
safe_block_text = st.text(
alphabet=st.characters(
whitelist_categories=["Ll", "Lu", "Nd"], whitelist_characters=" "
),
min_size=1,
max_size=20,
).map(str.strip).filter(bool)
ROUNDTRIP_BLOCK_TYPES = [
BLOCK_TYPES.UNSTYLED,
BLOCK_TYPES.HEADER_ONE,
BLOCK_TYPES.BLOCKQUOTE,
BLOCK_TYPES.UNORDERED_LIST_ITEM,
BLOCK_TYPES.ORDERED_LIST_ITEM,
]
ROUNDTRIP_STYLES = [INLINE_STYLES.BOLD, INLINE_STYLES.ITALIC, INLINE_STYLES.CODE]
@st.composite
def roundtrip_blocks(draw: st.DrawFn) -> dict[str, Any]:
"""A block whose type, text, and styles survive a Markdown round-trip."""
text = draw(safe_block_text)
return {
"key": draw(
st.text(
alphabet=st.characters(min_codepoint=97, max_codepoint=122),
min_size=5,
max_size=5,
)
),
"text": text,
"type": draw(st.sampled_from(ROUNDTRIP_BLOCK_TYPES)),
"depth": 0,
"inlineStyleRanges": draw(
st.lists(
ranges(
text,
st.builds(lambda s: {"style": s}, st.sampled_from(ROUNDTRIP_STYLES)),
),
max_size=2,
)
),
"entityRanges": [],
}
@st.composite
def roundtrip_content_states(draw: st.DrawFn) -> dict[str, Any]:
"""Content states limited to constructs both engines support."""
return {
"entityMap": {},
"blocks": draw(st.lists(roundtrip_blocks(), min_size=0, max_size=4)),
}- Step 2: Add property tests
Append to tests/test_properties.py:
from draftjs_exporter.html import HTML
from draftjs_exporter.markdown import CONFIG as MARKDOWN_CONFIG
from draftjs_exporter.markdown_importer import MarkdownImporter
from draftjs_exporter.contentstate_filter import ContentStateFilter
from tests.strategies import content_states, roundtrip_content_states
class TestImporterProperties(unittest.TestCase):
@given(roundtrip_content_states())
def test_round_trip_preserves_block_types_and_text(self, content_state):
"""Export → import keeps block types and text for safe content."""
markdown = HTML(MARKDOWN_CONFIG).render(content_state)
result = MarkdownImporter().import_markdown(markdown)
self.assertEqual(
[b["type"] for b in result["blocks"]],
[b["type"] for b in content_state["blocks"]],
)
self.assertEqual(
[b["text"] for b in result["blocks"]],
[b["text"] for b in content_state["blocks"]],
)
@given(content_states())
def test_filter_produces_valid_content_state(self, content_state):
"""Filtered output never has orphaned entity ranges."""
filter_ = ContentStateFilter(
[{"type": "block", "match": BLOCK_TYPES.HEADER_ONE, "action": "remove"}]
)
result = filter_.apply(content_state)
entity_map = result.get("entityMap", {})
for block in result["blocks"]:
text_length = len(block.get("text", ""))
for r in block.get("entityRanges", []):
self.assertIn(str(r["key"]), entity_map)
self.assertLessEqual(r["offset"] + r["length"], text_length)
for r in block.get("inlineStyleRanges", []):
self.assertLessEqual(r["offset"] + r["length"], text_length)
@given(content_states())
def test_filter_idempotent(self, content_state):
"""Applying the same filter twice yields the same result."""
filter_ = ContentStateFilter(
[{"type": "inline_style", "match": INLINE_STYLES.BOLD, "action": "remove"}]
)
once = filter_.apply(content_state)
twice = filter_.apply(once)
self.assertEqual(once, twice)Note: BLOCK_TYPES and INLINE_STYLES are already imported in test_properties.py — reuse those imports. If the round-trip property fails on blockquote/list depth edge cases, constrain depth generation rather than weakening the assertion.
- Step 3: Run
Run: just test tests/test_properties.py
Expected: PASS
- Step 4: Commit
git add tests/test_properties.py tests/strategies.py
git commit -m "Add importer and filter property-based tests"Files:
-
Modify:
example.py -
Step 1: Add the import demo
At the end of the __main__ block in example.py, after the Markdown export section, add:
# --- Markdown import ---
from draftjs_exporter import MarkdownImporter
from draftjs_exporter.markdown_parser import scheme_resolver
importer = MarkdownImporter(
{
"parser_config": {
"image_resolvers": [
scheme_resolver(
"wagtail",
{"image": "IMAGE"},
coerce={"id": int},
label_key="alt",
mutability="IMMUTABLE",
)
],
},
"filter_rules": [
{"type": "block", "match": BLOCK_TYPES.HEADER_ONE, "action": "demote"},
],
}
)
imported = importer.import_markdown(markdown_output)
print("=== Markdown import ===") # noqa: T201
print(json.dumps(imported, indent=2)) # noqa: T201Move the from draftjs_exporter import MarkdownImporter import to the top-level import block (ruff E402); keep scheme_resolver imported from draftjs_exporter.markdown_parser alongside the existing markdown_link import.
- Step 2: Run the example
Run: uv run example.py docs/example.json
Expected: prints HTML, Markdown, then the imported ContentState JSON; exits 0.
- Step 3: Commit
git add example.py
git commit -m "Add Markdown import demo to example.py"Files:
-
Create:
docs/markdown-importer.md -
Modify:
mkdocs.yml,CHANGELOG.md,.agents/skills/draftjs_exporter/SKILL.md -
Step 1: Write
docs/markdown-importer.md
Follow docs/style-guide.md (sentence case, American English). Sections:
- Intro — experimental status (mirroring the Markdown export page), one-paragraph overview of parse → filter.
- Getting started —
MarkdownImporter().import_markdown(md)minimal example with output. - Supported Markdown — the construct table from the spec; explicitly list out-of-scope items (reference links, Setext, indented code, tables, autolinks).
- Parser configuration — every
ParserConfigkey with defaults; a toggles example ("headings": False). - Entity resolution — resolver chain concept, defaults,
scheme_resolverwith thewagtail://example from the spec, writing a custom resolver (signature + contract: return resolution orNone). - Inline HTML —
inline_html_styleswhitelist, safety model (literal text otherwise), example. - Filtering —
FilterRulereference: rule kinds, actions (remove/keep/demote/callable), the heading-demote example, depth re-normalization note, standalone use ofContentStateFilter. - Errors —
MarkdownParseErrorshape and when it fires. - Custom parser engines — the
parserdotted-path config and the engine contract (__init__(config),parse(str) -> ContentState).
- Step 2: Register in nav and changelog
mkdocs.yml — after the Markdown support line:
- Markdown importer: markdown-importer.mdCHANGELOG.md — under the unreleased section, add a feature entry: "Add experimental Markdown importer: MarkdownImporter converts Markdown to Draft.js ContentState, with configurable entity resolvers, inline HTML style whitelist, and ContentStateFilter for content policy."
- Step 3: Update the skill
.agents/skills/draftjs_exporter/SKILL.md:
-
Add a quick-reference row: "Import Markdown as ContentState |
MarkdownImporter({}).import_markdown(md)| Markdown importer" -
Add rows for
scheme_resolverusage andContentStateFilterdemote example. -
Extend the Public API list with the new exports.
-
Description frontmatter: add
markdown_to_content_statetrigger — update thedescriptionfield to mention importing Markdown. -
Step 4: Build docs
Run: just docs-build
Expected: builds with --strict, no warnings.
- Step 5: Commit
git add docs/markdown-importer.md mkdocs.yml CHANGELOG.md .agents/skills/draftjs_exporter/SKILL.md
git commit -m "Document the Markdown importer"- Step 1: Full test suite
Run: just test
Expected: PASS, no warnings.
- Step 2: Coverage
Run: just test-coverage
Expected: 100% on draftjs_exporter/markdown_parser/, draftjs_exporter/contentstate_filter/, draftjs_exporter/markdown_importer/. Fill gaps with targeted tests — do not lower the target.
- Step 3: Lint and types
Run: just lint
Expected: clean (ruff check, ruff format --check, mypy, ty).
- Step 4: Compatibility suite
Run: just test-compatibility
Expected: PASS on Python 3.10 with pinned old deps.
- Step 5: Commit any fixes
git add -A
git commit -m "Polish Markdown importer for release"- Spec coverage: parser (Tasks 1–12), resolvers + scheme helper (2, 6, 14), inline HTML (7), filter (13), importer API (14), all test layers (15, 16 + unit tests per task), example (17), docs (18), changelog/skill (18).
- Known deviations from spec: none beyond the two clarifications listed at the top (MarkdownParseError rarity; atomic block pattern).
- Type consistency check:
resolve_image_entity,_add_text_block,ParserConfigkeys, andFilterRulefields are used identically across tasks.