Sapling/Docs/editor-investigation.md

Editor Investigation

Date: 2026-05

Status: Milestone 2 validation

Related decisions:

• D-008: Hybrid Markdown Editing Is a Core Feature
• D-009: The Editor Is the Highest-Priority System
• D-010: Git Access Must Be Abstracted
• D-012: Platform Focus
• D-013: Editor Technology Selection

Question

Can Sapling support a high-quality hybrid Markdown editing experience where the active line shows Markdown source and inactive lines show rendered Markdown?

Prototype Summary

The Milestone 1 prototype uses a SwiftUI application shell with a native text view bridge:

• macOS: NSTextView through NSViewRepresentable
• iOS path: UITextView through UIViewRepresentable
• shared editor state in SaplingEditor
• document loading and saving through HybridMarkdownEditorViewModel
• line tracking through EditorState, EditorLine, and EditorSelection
• in-place line styling through NSTextStorage

The prototype validates that native text views can support the first version of Sapling's editor without coupling the app directly to AppKit.

Current Architecture

flowchart TD
    App["SwiftUI app shell"] --> Editor["HybridMarkdownEditor"]
    Editor --> Bridge["Native text view bridge"]
    Bridge --> Mac["NSTextView on macOS"]
    Bridge --> IOS["UITextView on iOS"]
    Editor --> Coordinator["EditorCoordinator"]
    Coordinator --> State["EditorState"]
    State --> Document["EditorDocument"]
    State --> Lines["EditorLine array"]
    State --> Selection["EditorSelection"]
    Bridge --> TextKit["Text storage and selection APIs"]

The important boundary is the Sapling editor abstraction. Application code talks to HybridMarkdownEditorViewModel and editor state, not to NSTextView.

Architectures Explored

SwiftUI TextEditor

SwiftUI TextEditor was already present in the early prototype and worked for simple active-line editing.

Benefits:

• Minimal SwiftUI integration.
• Fast to prototype.
• Good enough for plain text entry.

Limitations:

• Does not expose reliable low-level selection and cursor APIs.
• Does not provide direct TextKit access.
• Makes line-level source/render switching difficult.
• Offers limited control over attributed rendering and layout.

Conclusion: TextEditor is not suitable as Sapling's primary editor implementation.

NSTextView With Attributed Styling

The implemented spike keeps Markdown source as the actual text buffer and applies visual styling to inactive lines with NSTextStorage.

Benefits:

• Native cursor movement remains intact.
• Native selection ranges remain tied to the real source buffer.
• Undo, find, spell checking, and keyboard navigation come from AppKit.
• Active line detection can be derived from selection location.
• Inactive lines can be styled differently from the active source line.

Limitations:

• Styling alone cannot remove Markdown delimiters without affecting layout.
• It cannot render block elements such as images, tables, or Mermaid diagrams as real embedded views.
• Changing font sizes per line affects line height and may create visual jumps.
• Reapplying attributes on every selection change must be optimized for large documents.

Conclusion: This is sufficient for Milestone 1 and likely sufficient for an early Milestone 2 implementation of headings, emphasis, links, lists, and task lists.

Line Replacement

Line replacement would swap inactive Markdown source with rendered text while preserving the source elsewhere.

Benefits:

• More closely matches the desired hybrid model.
• Can hide Markdown delimiters from inactive lines.

Risks:

• Cursor offsets no longer map directly to source offsets.
• Selection crossing active and inactive lines becomes complex.
• Undo behavior may become fragile.
• TextKit may fight source/render buffer divergence.

Conclusion: Worth exploring after the attributed-string path, but risky as the primary approach.

Overlay Rendering

Overlay rendering would keep source text in the editor but draw rendered content above inactive lines.

Benefits:

• Source buffer remains canonical.
• Rendered views could support richer blocks.
• Cursor math can still use the real text buffer.

Risks:

• Requires precise layout synchronization with TextKit.
• Hit testing and selection feedback are difficult.
• Accessibility must be designed carefully.

Conclusion: Promising for richer Milestone 3 rendering, but heavier than needed for Milestone 2.

Mixed Text and Render Layers

A custom layout could combine text editing lines with rendered SwiftUI/AppKit views.

Benefits:

• Maximum control over rendering.
• Can support images, diagrams, tables, and custom blocks.

Risks:

• Requires building editor infrastructure Sapling does not yet have.
• Cursor movement, selection, IME, undo, accessibility, and text input become product-critical systems.

Conclusion: This may be necessary in the long term, but it should not be the first Milestone 2 implementation.

TextKit Findings

NSTextView gives Sapling the hooks needed for the prototype:

• selectedRange() gives the cursor and selection range.
• NSTextStorage allows line-level attributes without replacing source text.
• NSTextViewDelegate reports text and selection changes.
• Native editing behavior remains available while the app tracks editor state.

The key finding is that Sapling can preserve a single source buffer and still present different active and inactive line states. This reduces risk compared with maintaining separate source and render buffers.

Finding #1 — Selection Feedback Loop

Root cause:

The first NSTextView bridge treated every selection notification as user intent, even when the selection change was caused by Sapling itself while restoring selection after an attribute pass.

The recursive path was:

textViewDidChangeSelection()
-> selection binding update
-> EditorState update
-> @Published state refresh
-> SwiftUI updateNSView()
-> applyHybridAttributes()
-> setSelectedRange()
-> textViewDidChangeSelection()

This created an unbounded feedback loop and could crash immediately after launch while the hybrid styling pass and SwiftUI refresh cycle bounced selection updates back and forth.

Architecture impact:

• Selection is not just editor state; it is also a native text system side effect.
• Attribute rendering can produce selection notifications even when the user's logical selection did not change.
• SwiftUI refreshes must not be allowed to feed native programmatic changes back into EditorState as if they were user edits.
• Hybrid editing remains viable, but the platform adapter must own reentrancy control.

Chosen solution:

• Programmatic text, selection, and attribute changes are wrapped in a coordinator transaction.
• Selection delegate callbacks are ignored while that transaction is active.
• EditorCoordinator.updateSelection and updateSource ignore no-op updates before publishing.
• Attribute passes restore selection only if the text view actually changed it.
• The bridge skips redundant restyling when both source text and active line are unchanged.

This keeps the canonical source and selection model in EditorState, while making the native adapter responsible for distinguishing user-originated changes from TextKit side effects.

Alternative solutions considered:

• Temporary delegate removal: workable, but broader than needed and easy to forget when adding new native callbacks.
• Debounced rendering: useful for future performance work, but it would only delay the recursion instead of fixing the feedback path.
• Never restoring selection after attributes: avoids this crash, but risks cursor drift if TextKit changes selection during editing.
• Moving selection outside published editor state: reduces SwiftUI refresh pressure, but weakens the architecture because active-line rendering depends on selection.

Result:

The app now survives launch, typing, arrow-key movement, and select-all smoke testing without re-entering the selection loop.

Finding #2 — Writing Comfort Matters More Than Features

Root cause:

After the selection recursion fix, the editor was technically functional but still uncomfortable. The three-column layout left the editor fighting for space, the inspector panel kept Git-oriented information visible during writing, and the text column was too close to the window edges.

Milestone 1 needs to validate whether a person wants to keep writing in Sapling. That requires layout and typography work even before advanced Markdown behavior exists.

Layout experiments:

• The three-column split view made the editor feel secondary.
• Removing the inspector from the default writing surface immediately improved focus.
• A two-column sidebar/detail split gives the editor roughly 70-80% of the default window.
• The sidebar remains collapsible for focused writing.
• The editor should be the detail column, not the middle column in a sidebar/content/inspector arrangement.

Sidebar impact:

The workspace sidebar is useful for opening documents, but it should not dominate Milestone 1. Git status and project inspection are distractions during editor validation and should stay hidden until later milestones.

Cursor behavior:

Native NSTextView keyboard behavior remains the right default. Arrow keys, Shift-selection, Option-arrow word navigation, Home, End, Page Up, and Page Down should be allowed to flow through AppKit instead of being reimplemented in SwiftUI.

Smoke testing after the layout and typography changes covered:

• Basic typing.
• Left and right arrow movement.
• Home and End.
• Page Up and Page Down.
• Option + Arrow movement.
• Shift + Arrow selection.

No selection recursion or immediate cursor instability appeared during the smoke pass.

Typography observations:

• A 14-point editor made the prototype feel cramped.
• A 16-point base font with more line and paragraph spacing is more comfortable for writing.
• A readable-width text column is more important than using all available horizontal space.
• Generous document padding makes the editor feel like a writing surface instead of a debug widget.
• The active-line highlight should stay subtle; it is there to orient the writer, not decorate the page.

Status bar observations:

The status bar should remain quiet. Line number, column number, total line count, and saved/modified state are enough for Milestone 1. Showing rendered previews in the status bar added visual noise without improving writing flow.

Architecture impact:

• Editor usability depends on app layout, not only text view internals.
• The native text view bridge can support comfortable writing if it controls text insets, readable width, and cursor visibility.
• Keyboard navigation should be validated through native behavior first.
• More features should wait until the writing surface itself feels calm.

Result:

The prototype now opens into a focused writing layout with a larger text area, collapsible sidebar, readable line width, generous padding, a subtle current-line treatment, and a realistic sample document for testing.

Finding #3 — Hybrid Editing Architecture

Milestone 2 keeps one canonical Markdown source buffer inside the native text view. The editor does not maintain a second rendered document, and it does not replace inactive lines with different strings. Active-line source mode and inactive-line rendered mode are presentation states derived from the same source.

flowchart TD
    TextView["NSTextView / UITextView source buffer"] --> Delegate["Native delegate callbacks"]
    Delegate --> Coordinator["Bridge coordinator"]
    Coordinator --> ViewModel["HybridMarkdownEditorViewModel"]
    ViewModel --> State["EditorState"]
    State --> Tracker["EditorActiveLineTracker"]
    Tracker --> Lines["EditorLine ranges and modes"]
    Lines --> Styler["MarkdownTextStyler"]
    Styler --> TextStorage["NSTextStorage attributes"]
    TextStorage --> TextView
    Coordinator --> Metrics["EditorInstrumentationSnapshot"]

The active-line model now has a single source of truth:

• EditorState.selection stores the current source-buffer selection.
• EditorActiveLineTracker maps the selection location to one line index.
• EditorState.activeLineIndex is derived from that mapping after every source or selection update.
• EditorLine.mode is rebuilt from the active line and source ranges.
• The native bridge only applies attributes from that state; it does not decide which line is active.

This avoids recursive update loops because the native adapter still owns programmatic-update suppression. Text, selection, and attribute changes caused by Sapling are wrapped in coordinator transactions and ignored by delegate callbacks. User-originated changes flow back to the view model only when the source or selection actually changed.

sequenceDiagram
    participant User
    participant Native as Native text view
    participant Bridge as Coordinator
    participant State as EditorState
    participant Render as Attribute styler

    User->>Native: Move cursor or edit text
    Native->>Bridge: Delegate notification
    Bridge->>State: Update source or selection
    State->>State: Derive activeLineIndex
    State->>Render: Apply attributes for active/inactive lines
    Render->>Native: Preserve selection and visible origin

Implementation scope is intentionally minimal:

• Headings are visually emphasized on inactive lines.
• Bold, italic, and inline code receive inline attributes.
• Markdown delimiters are deemphasized, not hidden.
• Links, tasks, lists, code blocks, tables, images, Mermaid, LaTeX, and attachments are not promoted in the hybrid renderer for this phase.

Cursor and selection behavior remain stable because the visible text length is still the source text length. Delimiters are styled but not removed, so source offsets, native selection ranges, undo grouping, and TextKit cursor movement continue to share the same coordinate system.

Finding #4 — Rendering Strategy Evaluation

Milestone 2 compared four rendering strategies against the current product risk: editing correctness is more important than rendering quality.

Approach	Complexity	Performance	Cursor stability	Maintainability	Finding
A. Attributed-string rendering	Low to medium	Dirty native attribute passes after initial render; state line tracking remains O(n)	Strong because source offsets are unchanged	Strong while rendering is inline and line-level	Chosen for Milestone 2
B. Overlay rendering	High	Potentially good with cached layout, but expensive to synchronize	Medium risk from hit testing, selection painting, and accessibility	Medium; requires layout synchronization code	Defer until rich block rendering
C. Line replacement	High	Could be efficient per line, but requires source/render mapping	High risk because cursor offsets diverge from visible text	Fragile around undo, IME, and cross-line selection	Not appropriate for current milestone
D. Mixed editor/render layers	Very high	Potentially best long term with a custom layout engine	Unknown until Sapling owns text input, IME, selection, undo, accessibility	Expensive; effectively a custom editor engine	Long-term fallback only

Approach A was selected because it validates Sapling's core hybrid interaction while preserving native text editing. It does not prove that full rendered blocks can live inside NSTextView, but it does prove that active-line source editing and inactive-line presentation can coexist without destabilizing cursor and selection behavior.

quadrantChart
    title Rendering Strategy Risk
    x-axis Low cursor stability --> High cursor stability
    y-axis Low implementation complexity --> High implementation complexity
    quadrant-1 "Good validation path"
    quadrant-2 "Powerful but expensive"
    quadrant-3 "Avoid"
    quadrant-4 "Defer"
    "Attributed attributes": [0.82, 0.28]
    "Overlay rendering": [0.55, 0.72]
    "Line replacement": [0.28, 0.68]
    "Mixed custom layers": [0.45, 0.92]

The important tradeoff is honest: attributed rendering cannot hide Markdown syntax without leaving visible gaps or breaking cursor math. For Milestone 2, that is acceptable. It is better to keep the editor believable and predictable than to make inactive lines look fully rendered at the cost of offset instability.

Finding #5 — Performance Characteristics

Instrumentation was added at the editor boundary:

• Source changes are counted in EditorInstrumentationSnapshot.sourceChangeCount.
• Selection changes are counted in selectionChangeCount.
• Active-line transitions are counted in activeLineChangeCount.
• Native attribute passes are counted in renderPassCount.
• Full native attribute passes are counted in fullRenderCount.
• Dirty lines touched by native rendering are counted in totalDirtyLineCount and lastDirtyLineCount.
• Scroll-origin restoration is counted in scrollRestorationCount.
• Each render pass records reason, duration, source character count, line count, dirty line count, active line index, whether it was a full render, and whether scroll position was restored.

The counters are intentionally not @Published. They can be inspected by tests, logs, or future debug UI without causing every render pass to trigger another SwiftUI update.

Automated validation added in SaplingEditorTests covers:

• Cursor-derived active-line transitions.
• Selection clamping after deletion.
• Trailing blank-line range tracking.
• Heading, bold, italic, and inline-code render plans.
• Links and tasks remaining unsupported in the Milestone 2 renderer.
• A 2,100-line prototype document shape.

Current measured automated results on May 29, 2026:

• swift test: 13 tests passed.
• Large render-plan test: 2,100 lines processed in roughly 0.04 to 0.06 seconds during the XCTest run.
• Full suite runtime: roughly 0.05 to 0.07 seconds after build.

Prototype documents were added for repeatable manual validation:

• Docs/EditorPrototypes/hybrid-small-50.md
• Docs/EditorPrototypes/hybrid-medium-500.md
• Docs/EditorPrototypes/hybrid-large-2100.md

Scroll stability work:

• Attribute passes restore the pre-render visible origin.
• Selection restoration after styling does not call scroll-to-visible.
• Explicit programmatic selection changes still scroll the selected range into view.

This means rendering updates should not yank the viewport while the user scrolls or while inactive-line attributes refresh. Native cursor movement still gets to scroll naturally when the insertion point moves outside the visible area.

Resolved performance limitation in Milestone 2.5:

The native styler no longer applies a full-buffer attribute pass for every source edit or active-line change. Initial renders and programmatic full text replacements still rebuild the full attributed string. User edits and active-line navigation now produce a dirty-line invalidation plan, and the text storage resets and restyles only those line ranges.

Remaining performance limitation:

EditorState.updateSource and dirty-plan construction still derive the full line model from the source string. The expensive native attributed-string rebuild is avoided, but state rebuilding remains O(n). This is visible in the 10,000-line typing proxy measurement below and should be the next measured optimization before richer block rendering.

Finding #6 — Dirty-Line Rendering

Milestone 2 rendered every line on source changes and active-line changes. The coordinator avoided redundant SwiftUI restyles with a lastStyledText and lastStyledActiveLineIndex cache, but any actual edit still called MarkdownTextStyler.apply(...) over the full NSTextStorage.

Milestone 2.5 adds EditorDirtyLineInvalidator, which compares the previously styled source and active line against the current source and active line. It emits an EditorDirtyLineInvalidationPlan containing:

• render reason
• full-render flag
• dirty line indexes
• changed UTF-16 range for source edits

The native adapter now follows this path:

flowchart TD
    Native["NSTextView text storage"] --> Coordinator["Native coordinator"]
    Coordinator --> Cache["Last styled text + active line"]
    Cache --> Invalidator["EditorDirtyLineInvalidator"]
    Invalidator --> Plan["Dirty-line invalidation plan"]
    Plan --> Styler["MarkdownTextStyler"]
    Styler --> DirtyRanges["Reset base attributes only on dirty line ranges"]
    DirtyRanges --> RenderLine["Apply source/rendered styling to dirty lines"]
    RenderLine --> Metrics["EditorRenderPassMetric"]

Observed behavior from automated tests:

User action	Render reason	Lines touched
Initial editor load	initial	all lines
Move active line from A to B	activeLineChange	A and B
Edit one line	sourceChange	edited line plus immediate neighbors
Insert a line break	sourceChange	split line, new active line, and affected neighbor
Redundant SwiftUI view update	viewUpdate	zero; no styling pass

The neighboring-line policy is intentionally conservative. Current Milestone 2 Markdown spans do not cross line boundaries, but line breaks and future block parsing can make nearby lines visually affected. The policy avoids full-document rendering while keeping enough context for line joins and splits.

Finding #7 — Large Document Performance

Stress documents were added for repeatable validation:

• Docs/EditorPrototypes/hybrid-stress-1000.md
• Docs/EditorPrototypes/hybrid-stress-5000.md
• Docs/EditorPrototypes/hybrid-stress-10000.md

Automated measurement command:

SAPLING_EDITOR_PRINT_METRICS=1 swift test --filter EditorLargeDocumentValidationTests/testLargeDocumentOpenAndDirtyRenderPlanning

Measured on May 29, 2026 in a debug SwiftPM test run:

Lines	Open proxy	Typing proxy	Dirty render-plan proxy	Dirty lines	Full render
1,000	1.769 ms	11.770 ms	2.804 ms	3	no
5,000	9.016 ms	59.068 ms	9.348 ms	3	no
10,000	17.642 ms	119.364 ms	19.115 ms	3	no

What these numbers measure:

• Open proxy: creating EditorState and its line model.
• Typing proxy: updating source, rebuilding editor state, and creating a dirty-line plan.
• Dirty render-plan proxy: line tracking plus render-plan creation for dirty lines.
• Render frequency: one source edit schedules one render metric with three dirty lines and isFullRender == false.

What these numbers do not measure:

• Real AppKit layout time in a visible window.
• GPU compositing or actual scroll wheel latency.
• IME composition behavior.

Evidence:

• 10,000-line documents remain tractable for state creation in debug tests.
• Native attributed-string work is now bounded to three dirty lines for a single-line edit.
• The remaining typing proxy cost scales linearly because the state model still rebuilds all lines.
• Scroll stability is covered by repeatable instrumentation tests: dirty render passes restore the captured scroll origin/content offset and increment scrollRestorationCount.

Finding #8 — Overlay Architecture Exploration

No overlay renderer was implemented. The spike evaluated how it would fit beside the current attributed-string path.

Approach A: attributed-string rendering keeps all presentation inside TextKit.

flowchart LR
    Source["Markdown source string"] --> TextStorage["NSTextStorage"]
    TextStorage --> Attributes["Line + inline attributes"]
    Attributes --> TextView["NSTextView display, cursor, selection"]

Approach B: overlay rendering keeps the source editor native and draws rendered inactive content in a synchronized layer.

flowchart TD
    Source["Markdown source string"] --> TextView["NSTextView source buffer"]
    TextView --> Layout["TextKit layout manager"]
    Layout --> LineRects["Line fragment rects"]
    Source --> Parser["Markdown line/block parser"]
    Parser --> RenderCache["Rendered overlay cache"]
    LineRects --> Overlay["Overlay view layer"]
    RenderCache --> Overlay
    Overlay --> HitTesting["Hit testing + accessibility mapping"]
    HitTesting --> TextView

Evidence-based comparison:

Question	Attributed-string rendering	Overlay rendering
Would it improve performance?	Dirty-line invalidation already removes full attributed-string rebuilds for current inline styling.	Could help rich blocks by rendering only visible overlays, but requires layout cache invalidation and visible-range tracking.
Would it improve cursor stability?	Strong today because text, cursor, and selection share one source coordinate system.	Source buffer remains canonical, but hit testing, selection painting, and accessibility must map overlay geometry back to source ranges.
What complexity appears?	Attribute scopes, dirty ranges, and TextKit line metrics.	Overlay lifecycle, z-order, scroll synchronization, invalidation, accessibility, hit testing, and mismatch between rendered block heights and source line heights.

Overlay rendering is still promising for future images, diagrams, and rich blocks. It should not replace the current renderer before those features exist because the current bottleneck is state line-model rebuilding, not attributed styling for dirty lines.

Finding #9 — Editor Scalability Assessment

Current assessment:

• NSTextView remains a viable foundation for the next milestone.
• The editor now avoids full attributed-string rebuilds for user edits and active-line moves.
• Cursor and selection state remain stable because source offsets are unchanged.
• Scroll restoration remains explicit and counted.
• Large documents open in the automated proxy tests, including 10,000 generated lines.

Scalability limits:

• Source updates still rebuild EditorState.lines for the whole document.
• Dirty rendering still computes line positions from the full source before touching only dirty ranges.
• Real scroll responsiveness still needs visible-window AppKit measurement, not only model-level tests.
• Rich block rendering may require overlay or mixed-layer work once block heights diverge from source line heights.

Recommended next measured work before Milestone 3 rendering complexity:

1. Add native edited-range capture from TextKit callbacks so state updates can avoid full line-model reconstruction.
2. Add an AppKit UI benchmark harness that opens the 1,000/5,000/10,000-line files in a visible editor and records scroll and typing latency.
3. Keep overlay rendering as a technical option for rich blocks, but do not introduce it for current heading/emphasis/code styling.

Finding #10 — Newline Semantics Matter

Milestone 2.6 found that the 5 MB benchmark document appeared to render entirely in source mode. Profiling showed that the editor line model contained one source line and zero rendered lines, even though the file had 51,482 physical lines.

Root cause:

EditorActiveLineTracker searched Swift String values for Character("\n") and used String.split(separator: "\n") for active-line lookup. This is incorrect for editor line segmentation because Swift String is a collection of extended grapheme clusters, not raw bytes or UTF-16 code units. A CRLF sequence ("\r\n") is treated as a single newline grapheme cluster according to Unicode text segmentation behavior. Searching that String for Character("\n") does not find the LF code unit inside the CRLF grapheme.

Evidence from Milestone 2.6:

CR=51481 LF=51481
"a\r\nb".firstIndex(of: "\n") == nil
"a\r\nb".split(separator: "\n").count == 1
(NSString("a\r\nb")).components(separatedBy: "\n").count == 2

Architecture impact:

• The benchmark file collapsed to one EditorLine.
• The whole document became the active source line.
• Rendered mode was absent because there were no inactive lines to render.
• Dirty-line invalidation technically returned one dirty line, but that dirty line was the whole document.
• TextKit layout and attributed styling measured the cost of one giant wrapped line, not the intended hybrid editor model.

Milestone 2.7 corrected this by introducing DocumentLineIndex and LineEndingStrategy.

flowchart TD
    Source["Markdown source"] --> Index["DocumentLineIndex"]
    Index --> UTF16["UTF-16 line-ending scan"]
    UTF16 --> LF["LF"]
    UTF16 --> CRLF["CRLF"]
    UTF16 --> CR["CR"]
    LF --> Boundaries["DocumentLineBoundary ranges"]
    CRLF --> Boundaries
    CR --> Boundaries
    Boundaries --> Active["Active-line lookup"]
    Boundaries --> Lines["EditorLine array"]
    Lines --> Hybrid["1 source line, inactive rendered lines"]

Corrected validation:

• LF-only, CRLF-only, CR-only, mixed newline, and trailing blank line tests pass.
• Docs/Benchmarks/5mb.md now segments into 51,482 editor lines.
• Corrected rendered-mode trace: 51,481 rendered lines and 1 source line.
• Active-line lookup maps positions inside CRLF line-ending ranges to the preceding line and the position after CRLF to the next line.

Corrected benchmark impact:

Measurement	Before CRLF fix	After CRLF fix
Editor lines	1	51,482
Rendered/source trace	0 rendered, 1 source	51,481 rendered, 1 source
Dirty typing render	887.213 ms	55.805 ms
Layout after typing	490.160 ms	0.903 ms
Dirty invalidation typing	566.852 ms	980.651 ms
Full TextKit layout	766.331 ms	1,320.499 ms
Measured total	4,376.271 ms	5,674.222 ms

The corrected benchmark is slower overall because it now measures real per-line work across 51,482 editor lines instead of one accidental giant line. That is the correct baseline for future optimization work.

Lesson:

Future editor systems must not infer logical lines by searching Swift String values for newline characters. Editor line segmentation must go through DocumentLineIndex, which uses explicit UTF-16 line-ending handling compatible with TextKit selection ranges and NSRange.

Finding #11 — Incremental Line Index

Milestone 2.9 implemented the highest-impact optimization identified in Milestone 2.8: a persistent line index used by active-line lookup, selection mapping, dirty invalidation, and dirty styling.

Architecture:

flowchart TD
    TextKit["NSTextView / UITextView edited range"] --> Edit["DocumentLineIndexEdit"]
    Edit --> Index["Persistent DocumentLineIndex"]
    Index --> Active["Active-line lookup"]
    Index --> Dirty["Dirty-line invalidation"]
    Index --> Styling["Dirty-line styling"]
    Index --> State["EditorState line count and active column"]

DocumentLineIndex now provides:

• line count
• line boundary lookup
• offset-to-line mapping
• line-to-offset mapping
• affected-line mapping with neighboring-line expansion
• incremental local replacement using DocumentLineIndexEdit

Update strategy:

1. Capture the native edited range and replacement string from the platform text view delegate.
2. Expand the scan window to the edited line, adjacent lines, and line-ending boundary neighbors.
3. Rescan only that local window using UTF-16 line-ending rules.
4. Replace the affected boundary slice in the persistent index.
5. Use the updated index for active-line lookup, dirty-line invalidation, and dirty styling.

The editor still stores the full source string because TextKit and persistence require it. The optimized path now accepts TextKit's already-edited source instead of reconstructing it from the old source inside the index.

Complexity impact:

Operation	Before	After
Active-line lookup	O(document) scan	O(log line count) binary lookup
Selection update	O(document) line rebuild	O(log line count) active-line lookup
Dirty invalidation for typing	O(document) prefix/suffix diff + line rebuild	O(log line count + affected lines)
Dirty styling setup	O(document) line-list rebuild	O(dirty lines)
Source edit index update	O(document) full line rebuild	local rescan plus boundary offset maintenance

5 MB benchmark impact:

Measurement	Before Milestone 2.9	After Milestone 2.9	Improvement
active_line_lookup	191.751 ms	0.001 ms	99.999%
selection_update	398.262 ms	0.002 ms	99.999%
dirty_line_invalidation_click	224.418 ms	0.001 ms	99.999%
typing_state_update	416.540 ms	0.102 ms	99.976%
dirty_line_invalidation_typing	1,019.380 ms	0.003 ms	99.999%
render_update_typing_dirty	53.512 ms	0.796 ms	98.512%

Tradeoffs:

• The line index is now stateful and must remain synchronized with native text edits.
• Programmatic full-source replacement still falls back to a full index rebuild.
• Correctness depends on preserving the Milestone 2.7 UTF-16 newline semantics for LF, CRLF, CR, mixed endings, and trailing blank lines.

Validation:

• Incremental line-index tests compare local edits against full rebuilds.
• Tests cover insertion, deletion, replacement, CRLF boundary edits, mixed line endings, and a 10,000-line edit.
• Cursor, dirty invalidation, scroll stability, and large-document tests continue to pass.

Conclusion:

The main Milestone 2.8 Sapling-side interaction bottlenecks have been removed from the measured 5 MB typing and selection paths. After this change, the dominant measured costs are TextKit full/cold layout and full-document initial rendering/planning work, not active-line tracking or dirty invalidation.

Finding #12 — Markdown Rendering Expansion

Milestone 3 expands the attributed hybrid renderer from the Milestone 2 proof of concept into a more complete Markdown writing surface. The implementation still keeps one canonical source buffer inside the native text view. It does not replace Markdown source with rendered strings, and it does not introduce overlay views or a custom editor engine.

Supported rendering plans now cover:

• ATX headings
• bold and italic emphasis using asterisks or underscores
• inline code
• blockquotes
• horizontal rules
• unordered lists
• ordered lists
• nested list indentation
• task lists
• Markdown links
• automatic links
• fenced code blocks with language labels
• Markdown table rows and divider rows

Architecture:

flowchart TD
    TextKit["NSTextView / UITextView source buffer"] --> Index["DocumentLineIndex"]
    Index --> Lines["EditorLine values"]
    Lines --> Parser["HybridMarkdownLineRenderer"]
    Parser --> Plans["HybridMarkdownLineRenderPlan"]
    Plans --> Styler["MarkdownTextStyler"]
    Styler --> Attributes["NSTextStorage attributes"]
    Attributes --> TextKit

HybridMarkdownLineRenderer remains a line-oriented parser. It emits line kinds and inline spans with NSRange values in source-buffer coordinates. MarkdownTextStyler applies fonts, colors, paragraph styles, background treatments, underlines, and strikethroughs to those source ranges.

The active line continues to receive source styling. Inactive lines receive rendered-style attributes. Because the actual text is not replaced, cursor locations, selections, undo, TextKit layout ranges, dirty-line invalidation, and persistence all remain tied to the same source string.

Tradeoffs:

• Markdown delimiters are visually de-emphasized rather than removed. This is less visually complete than Typora-style replacement, but it avoids source/render offset mapping.
• Task lists show checkbox state with styling on [ ] and [x]; interactive toggling is deferred.
• Horizontal rules are represented with attributed separator styling over the source marker instead of a true drawn rule.
• Tables use monospaced attributed rendering for readability. They do not yet become independent grid views.
• Fenced code block contents are styled when render plans are produced with contiguous line context. Dirty-line rendering avoids document-wide context scans, so long code-block context remains conservative around local edits.
• Markdown links and automatic links are visually distinct. Full click handling is deferred unless it can be added without destabilizing editing.
• Reference-style link definitions remain unsupported in the hybrid renderer. The 5 MB benchmark is dominated by reference-style definitions, so automatic-link parsing intentionally skips those lines for now.

Scalability impact:

The expanded renderer was validated with:

• sample document benchmark
• Docs/EditorPrototypes/hybrid-large-2100.md
• Docs/Benchmarks/5mb.md

Release benchmark run on May 31, 2026 after parser guards:

Scenario	Lines	Measured total	Render planning	Initial styling	Dirty typing render
sample document	54	10.151 ms	1.005 ms	3.941 ms	0.265 ms
2,100-line prototype	2,101	413.814 ms	88.227 ms	96.668 ms	0.369 ms
5 MB benchmark	51,482	3,157.751 ms	244.277 ms	371.099 ms	0.985 ms

Tracked 5 MB interaction metrics remained within the Milestone 2.9 responsiveness envelope:

Operation	Time
active-line lookup	0.000 ms
selection update	0.002 ms
dirty click invalidation	0.001 ms
typing state update	0.189 ms
dirty typing invalidation	0.003 ms
dirty typing render	0.985 ms

The main remaining costs are still full/cold TextKit layout and initial full-document work, not active-line switching or dirty edit rendering. Parser guards were necessary because the first expanded renderer performed too much per-line Markdown detection on the 5 MB reference-definition benchmark. After guarding by first token and skipping reference-style definitions for automatic-link parsing, the measured 5 MB total remained below the Milestone 2.9 reported baseline.

Validation fixtures:

• Docs/EditorPrototypes/markdown-technical-documentation.md
• Docs/EditorPrototypes/markdown-meeting-notes.md
• Docs/EditorPrototypes/markdown-research-notes.md
• Docs/EditorPrototypes/markdown-project-plan.md
• Docs/EditorPrototypes/markdown-long-form-writing.md

Unsupported elements explicitly deferred:

• images
• wikilinks
• Mermaid
• LaTeX
• callouts
• attachments
• audio
• video

Conclusion:

Sapling now renders enough standard Markdown structure for documentation, technical notes, project notes, meeting notes, and long-form writing to feel substantially closer to a real Markdown editor while preserving the validated hybrid editing model. The renderer remains deliberately attributed and source-preserving until richer block layout can be proven without harming cursor stability, selection stability, typing latency, or scrolling behavior.

Finding #13 — Rendering vs Syntax Styling

Milestone 3.1 corrects the first Markdown expansion from syntax-colored Markdown into meaning-oriented rendered lines. The editor still keeps one canonical Markdown source buffer in the native text view, but inactive lines now hide more structural syntax instead of only dimming it.

Previous behavior:

• Headings were styled with larger fonts while the leading # marker remained visible.
• Bold, italic, and inline-code delimiters remained visible in inactive rendered lines.
• Fenced code blocks colored the fence syntax, but the opening and closing fences still read as source text.
• Task lists were detected by the renderer, but the inactive line did not show a distinct task state treatment.
• Dirty rendering of lines inside fenced code blocks could lose block context when the dirty slice did not include the opening fence.

Corrected behavior:

• Inactive headings hide the heading marker and following separator while preserving heading level, spacing, and hierarchy.
• Inline emphasis and code delimiters are hidden on inactive lines; the content keeps the bold, italic, or monospace treatment.
• Fenced code blocks receive a dedicated monospace block treatment. Fence markers are hidden in rendered mode, and an opening-fence language label remains visible when present.
• Task-list bullets are hidden. The [x] or [ ] token receives a visible state treatment, and completed task content is struck through.
• Active lines always remain source text, so the same heading, task, code fence, or inline markup becomes editable Markdown when the cursor enters that line.

Render invalidation audit:

Initial rendering was not being skipped. Full renders still style every indexed line, and EditorDirtyLineInvalidationTests.testInitialRenderTouchesEveryLine covers that contract. The most visible heading case is the expected hybrid behavior when the heading is the active line: active lines intentionally show source. Moving focus away causes the line to become inactive and therefore rendered.

The real invalidation gap was block context for dirty slices. The renderer already understood fenced code blocks during full contiguous rendering, but a dirty update for only a code-content line could be planned without its surrounding fences. The fix adds contextual dirty rendering only when a dirty line is near a fence marker. That path asks DocumentLineIndex for enough source context to decide whether the dirty line is inside a fenced block, avoiding a document-wide refresh or unconditional full-source scan.

Rendering architecture:

flowchart TD
    Source["Native text storage source"] --> Index["DocumentLineIndex"]
    Index --> Dirty["Dirty line invalidation"]
    Dirty --> Context{"Fence nearby?"}
    Context -->|No| LinePlans["Line-only render plans"]
    Context -->|Yes| ContextPlans["Contextual code-block plans"]
    LinePlans --> Styler["MarkdownTextStyler"]
    ContextPlans --> Styler
    Styler --> Hidden["Hidden syntax attributes"]
    Styler --> Meaning["Rendered meaning attributes"]
    Hidden --> Source
    Meaning --> Source

The correction is still source-preserving. Hidden syntax uses temporary text attributes, not string replacement. This keeps TextKit source offsets, cursor movement, selections, undo, and persistence aligned with the Markdown file.

Tradeoffs:

• Hidden delimiters still occupy a tiny layout footprint because the backing text remains present. This is intentional until replacement or overlay rendering is proven stable.
• Task lists do not yet replace [x] and [ ] with true ☑ and ☐ glyphs. AppKit text attachments require replacement characters, which would break the source-offset model. The current rendered mode hides the list marker and styles the task-state token instead.
• Code blocks are rendered as attributed text blocks, not independent block widgets. Padding is represented through paragraph and background styling.
• Long-range dirty code-block context is resolved only when the dirty line is near fence syntax. This keeps normal typing and active-line switching cheap; broader block invalidation can be added later if editing fences inside very long code blocks exposes stale styling.
• Blockquotes, lists, horizontal rules, tables, and links remain attributed renderings rather than custom layout objects.

Validation:

• Focused renderer tests cover heading marker hiding, active-line source preservation, inline delimiter hiding, task-state styling, code-fence hiding, and dirty code-block context recovery.
• Full test suite on May 31, 2026: 58 XCTest cases passed.
• Release benchmark on May 31, 2026:

Scenario	Lines	Measured total	Render planning	Initial styling	Dirty typing render
sample document	54	13.506 ms	1.264 ms	4.733 ms	0.330 ms
2,100-line prototype	2,101	423.835 ms	88.610 ms	98.276 ms	0.368 ms
5 MB benchmark	51,482	3,275.857 ms	248.582 ms	361.581 ms	1.002 ms

Tracked 5 MB interaction metrics:

Operation	Time
active-line lookup	0.001 ms
selection update	0.003 ms
dirty click invalidation	0.001 ms
typing state update	0.116 ms
dirty typing invalidation	0.004 ms
dirty typing render	1.002 ms

Conclusion:

Milestone 3.1 moves Sapling's inactive-line display closer to rendered Markdown while preserving the validated hybrid editing contract. The renderer now hides the most distracting source syntax for headings, inline markup, and fenced code blocks, and it gives task lists visible state. The remaining limitations are deliberate source-preserving tradeoffs rather than missed parser features.

Finding #14 — Rendered Document Lifecycle

Milestone 3.2 makes rendered state explicit instead of leaving it implicit inside styling code. Before this change, the native adapter decided whether a line was source or rendered inside MarkdownTextStyler.apply(...) by comparing each line index to the active line. That worked, but it left presentation state spread across delegate timing, invalidation caches, and styler branches.

The new lifecycle model is:

• RenderedLineState.source: exactly one active line derived from the current native selection and DocumentLineIndex.
• RenderedLineState.rendered: every other visible line.
• DocumentPresentationState: deterministic projection from (DocumentLineIndex, activeLineIndex, optional dirty line indexes) to presentation lines and semantic rendered elements.
• DocumentPresentationLine: one source line plus its line state, render plan, and semantic rendered elements.

Every line in a presentation pass is now in exactly one state. The state is data-driven by current document text and current selection, not by previous focus history or click history.

stateDiagram-v2
    [*] --> Rendered: line.index != activeLineIndex
    [*] --> Source: line.index == activeLineIndex
    Rendered --> Source: selection enters line
    Source --> Rendered: selection leaves line
    Rendered --> Rendered: source edit elsewhere
    Source --> Source: source edit on active line

Render pipeline after Milestone 3.2:

flowchart TD
    NativeSelection["Native text selection"] --> Active["DocumentLineIndex.lineIndex(containing:)"]
    Source["Markdown source"] --> LineIndex["DocumentLineIndex"]
    LineIndex --> Presentation["DocumentPresentationState"]
    Active --> Presentation
    Dirty["Dirty line indexes"] --> Presentation
    Presentation --> Lines["DocumentPresentationLine"]
    Lines --> Styler["MarkdownTextStyler"]
    Lines --> Elements["RenderedDocumentElement"]
    Elements --> Widgets["Rendered widgets / overlays"]
    Styler --> TextStorage["NSTextStorage attributes"]

Invalidation audit:

• Initial rendering is still a full presentation pass. EditorDirtyLineInvalidator emits .initial with every line index, and the presentation model assigns one source line and all remaining lines as rendered.
• Source edits still use dirty-line invalidation and preserve neighbor-line coverage for constructs affected by adjacent text.
• Active-line transitions still dirty only the previous and current active line.
• View updates with unchanged source and unchanged active line still skip styling.
• The important timing fix is that native adapters now derive the active line from the current native text selection immediately before building the invalidation plan. Selection-change styling no longer depends on waiting for SwiftUI binding propagation to update the activeLineIndex value.

Rendered element architecture:

DocumentPresentationState now emits semantic RenderedDocumentElement values for supported constructs:

• heading
• task
• codeBlock
• link
• inlineCode
• blockquote
• horizontalRule
• listItem
• tableRow

Plain paragraphs are intentionally not materialized as semantic elements in normal passes. They are represented by their DocumentPresentationLine and render plan, which avoids allocating tens of thousands of no-op paragraph elements in large documents.

Code block lifecycle:

Fenced code blocks are represented as semantic multi-line RenderedCodeBlockElement values with:

• all participating line indexes
• the source range spanning the block
• the optional language range

The attributed renderer still styles code blocks line-by-line inside TextKit, but the document presentation model now has a block-level representation that future widgets can consume without reparsing source text.

Tradeoffs:

• Text remains source-preserving. Hidden syntax is still done with attributes, not source replacement.
• The native text view remains the scroll, selection, undo, and IME owner.
• Semantic elements are only emitted for constructs that need rendered behavior or future widget hooks.
• Dirty presentation only resolves code-block context when nearby fence syntax makes block context relevant; this preserves the Milestone 2.9 dirty-render scalability model.

Validation:

• DocumentPresentationStateTests verify one state per line, deterministic projection for identical input, semantic heading/task/link elements, semantic code-block state, and dirty code-block context recovery.
• Existing cursor, dirty invalidation, scroll stability, rendering, large-document, and performance tests continue to pass.

Before/after release benchmark comparison:

Scenario	Milestone 3.1 total	Milestone 3.2 total	Milestone 3.1 dirty typing render	Milestone 3.2 dirty typing render
sample document	13.506 ms	10.013 ms	0.330 ms	0.220 ms
2,100-line prototype	423.835 ms	477.423 ms	0.368 ms	0.383 ms
5 MB benchmark	3,275.857 ms	3,614.818 ms	1.002 ms	1.049 ms

Tracked 5 MB interaction metrics after Milestone 3.2:

Operation	Time
active-line lookup	0.000 ms
selection update	0.005 ms
dirty click invalidation	0.001 ms
typing state update	0.077 ms
dirty typing invalidation	0.003 ms
dirty typing render	1.049 ms

Conclusion:

The presentation lifecycle is now explicit and deterministic. Opening a document, moving the cursor, and editing source all project through the same state model: source for the active line, rendered for inactive lines. The measured cost is a modest increase in initial large-document work, while dirty interaction remains within the Milestone 2.9 responsiveness envelope.

Finding #15 — Interactive Rendered Elements

Milestone 3.2 introduces task lists as the first interactive rendered element. The Markdown source remains authoritative; the checkbox is only a rendered control backed by a known source range.

Checklist architecture on macOS:

sequenceDiagram
    participant User
    participant Checkbox as Native NSButton checkbox
    participant Coordinator as NSTextView Coordinator
    participant Source as Markdown source
    participant Index as DocumentLineIndex
    participant Presentation as DocumentPresentationState
    participant Styler as MarkdownTextStyler

    User->>Checkbox: click
    Checkbox->>Coordinator: toggle RenderedTaskElement
    Coordinator->>Source: replace checkbox range with [ ] or [x]
    Coordinator->>Index: apply DocumentLineIndexEdit
    Coordinator->>Presentation: rebuild affected presentation
    Presentation->>Styler: rendered/source line states
    Styler->>Checkbox: sync overlay state and position

Implementation details:

• Rendered task lines produce RenderedTaskElement with marker, checkbox, content, checked state, and nesting level.
• The macOS NSTextView adapter creates native NSButton checkbox overlays for inactive rendered task lines.
• Clicking the checkbox replaces the Markdown checkbox source range with [x] or [ ].
• The edit flows through the same source update, line-index update, dirty invalidation, and restyling path as typing.
• When the line becomes active, it returns to source mode and the overlay is removed for that line.

This is deliberately an overlay, not a text replacement. TextKit remains responsible for the source buffer and selection. The overlay consumes semantic rendered element data and writes back to source through DocumentLineIndexEdit.

Current platform scope:

• macOS has the native interactive checklist overlay.
• iOS keeps the attributed task-list fallback for now; it needs a UITextView overlay equivalent before task checkboxes are interactive there.

Consistency:

• Headings, inline Markdown, links, task lists, and code blocks now all pass through DocumentPresentationState.
• Active lines show source.
• Inactive lines show rendered presentation.
• Widgets are attached to semantic rendered elements, not to ad hoc regex matches in the native view.

Future extensibility:

The rendered element architecture can support images, wiki links, Mermaid, LaTeX, and embeds without fundamental redesign. The evidence is structural:

• The presentation layer already separates semantic element detection from platform rendering.
• Elements carry source ranges, so widgets can update or inspect Markdown while preserving source as truth.
• Code blocks already prove multi-line semantic elements can coexist with line-level dirty styling.
• Checklist overlays prove rendered controls can live above the source-preserving text view and write back through normal edit/invalidation paths.

Future rich elements will still need feature-specific parsers, layout, accessibility, hit testing, and caching. Those are additions to the rendered element set and platform widget layer, not a replacement of the hybrid editor architecture.

AttributedString and NSAttributedString

Swift AttributedString is useful for renderer-facing APIs and SwiftUI previews.

NSAttributedString and NSTextStorage are still required for the actual native editor bridge because the text view operates on TextKit types.

Recommended split:

• Use AttributedString in cross-platform renderer interfaces.
• Use NSAttributedString and NSTextStorage inside platform adapters.
• Keep conversion and styling code contained in SaplingEditor.

Cursor Challenges

The prototype validates basic active-line detection:

• The selected range location maps to a line index.
• The active line switches as the cursor moves.
• The active line can be styled as source while other lines are styled as rendered.

Open challenges:

• Multi-line selection across source and rendered lines.
• Selection after visually hidden or replaced Markdown delimiters.
• Cursor placement if future phases hide links, list markers, task markers, or other delimiters.
• IME composition and marked text handling.
• Very large documents where full state line-model rebuilding on each source change is too expensive.

Cursor correctness is the main reason to keep the canonical source text inside NSTextView for now.

Rendering Challenges

The current spike uses line-level styling, not full Markdown rendering.

Validated:

• Headings can be styled by line.
• Bold, italic, and inline code can be styled inline.
• Unsupported constructs can remain plain source text without breaking the editor model.
• Active source line can remain monospaced and visibly distinct.

Not validated:

• True delimiter removal without cursor side effects.
• Embedded rendered blocks inside the text layout.
• Images, tables, code block widgets, Mermaid, LaTeX, and attachments.
• Precise layout matching between rendered inactive lines and raw active lines.

Hybrid Editing Feasibility

Hybrid editing is achievable for Milestone 2 when the first implementation is scoped to line-level attributed styling.

The safest path is:

1. Keep the Markdown source as the only editable text buffer.
2. Detect the active line from the native selection range.
3. Apply source styling to the active line.
4. Apply rendered-style attributes to inactive lines.
5. Delay true line replacement and overlay rendering until selection behavior is better understood.

This does not produce perfect rendered Markdown, but it validates the writing feel and cursor behavior before Sapling invests in a custom editor engine.

Recommended Path Forward

Continue with NSTextView for the next editor milestone on macOS and keep the iOS UITextView adapter available behind the same editor abstraction.

Immediate next steps:

• Capture native edited ranges so source updates can avoid rebuilding the full line model.
• Manually test undo/redo, IME input, and keyboard navigation with the stress prototype files.
• Add visible AppKit scroll and typing benchmarks for the 1,000/5,000/10,000-line documents.
• Explore delimiter hiding with temporary attributes only after selection mapping is robust.

Future investigation:

• Use TextKit layout fragments or overlay views for rich block rendering.
• Prototype image and code block rendering outside the text buffer.
• Reevaluate whether a custom editor engine is required after Milestone 2.

Decision

NSTextView can continue to support Sapling's roadmap through the next phase.

Evidence:

• The active-line source and inactive-line presentation model works while preserving a single source buffer.
• Cursor and selection ranges remain native source ranges because rendering does not replace text.
• Scroll position can be preserved around rendering updates.
• Redundant render passes are avoided when source text and active line are unchanged.
• Dirty-line invalidation avoids full attributed-string rebuilds for user edits and active-line transitions.
• The 1,000/5,000/10,000-line stress documents are tractable in automated model-level validation, with 10,000-line open proxy time measured at 17.642 ms.

Sapling should not begin a custom editor engine now. The evidence supports continuing with native text systems while measuring state-line rebuilding and richer overlay experiments.

A future custom editor engine may still be required for high-fidelity block rendering, especially images, tables, Mermaid, LaTeX, attachments, and fully hidden delimiters. That decision should be made only after overlay rendering and native edited-range state updates have been measured against real documents.

The architecture should continue with a SwiftUI shell, a Sapling editor abstraction, and native platform text views hidden behind replaceable adapters.