# Editor Investigation Date: 2026-05 Status: Milestone 2 validation Related decisions: - D-008: Hybrid Markdown Editing Is a Core Feature - D-009: The Editor Is the Highest-Priority System - D-010: Git Access Must Be Abstracted - D-012: Platform Focus - D-013: Editor Technology Selection ## Question Can Sapling support a high-quality hybrid Markdown editing experience where the active line shows Markdown source and inactive lines show rendered Markdown? ## Prototype Summary The Milestone 1 prototype uses a SwiftUI application shell with a native text view bridge: - macOS: `NSTextView` through `NSViewRepresentable` - iOS path: `UITextView` through `UIViewRepresentable` - shared editor state in `SaplingEditor` - document loading and saving through `HybridMarkdownEditorViewModel` - line tracking through `EditorState`, `EditorLine`, and `EditorSelection` - in-place line styling through `NSTextStorage` The prototype validates that native text views can support the first version of Sapling's editor without coupling the app directly to AppKit. ## Current Architecture ```mermaid flowchart TD App["SwiftUI app shell"] --> Editor["HybridMarkdownEditor"] Editor --> Bridge["Native text view bridge"] Bridge --> Mac["NSTextView on macOS"] Bridge --> IOS["UITextView on iOS"] Editor --> Coordinator["EditorCoordinator"] Coordinator --> State["EditorState"] State --> Document["EditorDocument"] State --> Lines["EditorLine array"] State --> Selection["EditorSelection"] Bridge --> TextKit["Text storage and selection APIs"] ``` The important boundary is the Sapling editor abstraction. Application code talks to `HybridMarkdownEditorViewModel` and editor state, not to `NSTextView`. ## Architectures Explored ### SwiftUI TextEditor SwiftUI `TextEditor` was already present in the early prototype and worked for simple active-line editing. Benefits: - Minimal SwiftUI integration. - Fast to prototype. - Good enough for plain text entry. Limitations: - Does not expose reliable low-level selection and cursor APIs. - Does not provide direct TextKit access. - Makes line-level source/render switching difficult. - Offers limited control over attributed rendering and layout. Conclusion: `TextEditor` is not suitable as Sapling's primary editor implementation. ### NSTextView With Attributed Styling The implemented spike keeps Markdown source as the actual text buffer and applies visual styling to inactive lines with `NSTextStorage`. Benefits: - Native cursor movement remains intact. - Native selection ranges remain tied to the real source buffer. - Undo, find, spell checking, and keyboard navigation come from AppKit. - Active line detection can be derived from selection location. - Inactive lines can be styled differently from the active source line. Limitations: - Styling alone cannot remove Markdown delimiters without affecting layout. - It cannot render block elements such as images, tables, or Mermaid diagrams as real embedded views. - Changing font sizes per line affects line height and may create visual jumps. - Reapplying attributes on every selection change must be optimized for large documents. Conclusion: This is sufficient for Milestone 1 and likely sufficient for an early Milestone 2 implementation of headings, emphasis, links, lists, and task lists. ### Line Replacement Line replacement would swap inactive Markdown source with rendered text while preserving the source elsewhere. Benefits: - More closely matches the desired hybrid model. - Can hide Markdown delimiters from inactive lines. Risks: - Cursor offsets no longer map directly to source offsets. - Selection crossing active and inactive lines becomes complex. - Undo behavior may become fragile. - TextKit may fight source/render buffer divergence. Conclusion: Worth exploring after the attributed-string path, but risky as the primary approach. ### Overlay Rendering Overlay rendering would keep source text in the editor but draw rendered content above inactive lines. Benefits: - Source buffer remains canonical. - Rendered views could support richer blocks. - Cursor math can still use the real text buffer. Risks: - Requires precise layout synchronization with TextKit. - Hit testing and selection feedback are difficult. - Accessibility must be designed carefully. Conclusion: Promising for richer Milestone 3 rendering, but heavier than needed for Milestone 2. ### Mixed Text and Render Layers A custom layout could combine text editing lines with rendered SwiftUI/AppKit views. Benefits: - Maximum control over rendering. - Can support images, diagrams, tables, and custom blocks. Risks: - Requires building editor infrastructure Sapling does not yet have. - Cursor movement, selection, IME, undo, accessibility, and text input become product-critical systems. Conclusion: This may be necessary in the long term, but it should not be the first Milestone 2 implementation. ## TextKit Findings `NSTextView` gives Sapling the hooks needed for the prototype: - `selectedRange()` gives the cursor and selection range. - `NSTextStorage` allows line-level attributes without replacing source text. - `NSTextViewDelegate` reports text and selection changes. - Native editing behavior remains available while the app tracks editor state. The key finding is that Sapling can preserve a single source buffer and still present different active and inactive line states. This reduces risk compared with maintaining separate source and render buffers. ## Finding #1 — Selection Feedback Loop Root cause: The first `NSTextView` bridge treated every selection notification as user intent, even when the selection change was caused by Sapling itself while restoring selection after an attribute pass. The recursive path was: ```text textViewDidChangeSelection() -> selection binding update -> EditorState update -> @Published state refresh -> SwiftUI updateNSView() -> applyHybridAttributes() -> setSelectedRange() -> textViewDidChangeSelection() ``` This created an unbounded feedback loop and could crash immediately after launch while the hybrid styling pass and SwiftUI refresh cycle bounced selection updates back and forth. Architecture impact: - Selection is not just editor state; it is also a native text system side effect. - Attribute rendering can produce selection notifications even when the user's logical selection did not change. - SwiftUI refreshes must not be allowed to feed native programmatic changes back into `EditorState` as if they were user edits. - Hybrid editing remains viable, but the platform adapter must own reentrancy control. Chosen solution: - Programmatic text, selection, and attribute changes are wrapped in a coordinator transaction. - Selection delegate callbacks are ignored while that transaction is active. - `EditorCoordinator.updateSelection` and `updateSource` ignore no-op updates before publishing. - Attribute passes restore selection only if the text view actually changed it. - The bridge skips redundant restyling when both source text and active line are unchanged. This keeps the canonical source and selection model in `EditorState`, while making the native adapter responsible for distinguishing user-originated changes from TextKit side effects. Alternative solutions considered: - Temporary delegate removal: workable, but broader than needed and easy to forget when adding new native callbacks. - Debounced rendering: useful for future performance work, but it would only delay the recursion instead of fixing the feedback path. - Never restoring selection after attributes: avoids this crash, but risks cursor drift if TextKit changes selection during editing. - Moving selection outside published editor state: reduces SwiftUI refresh pressure, but weakens the architecture because active-line rendering depends on selection. Result: The app now survives launch, typing, arrow-key movement, and select-all smoke testing without re-entering the selection loop. ## Finding #2 — Writing Comfort Matters More Than Features Root cause: After the selection recursion fix, the editor was technically functional but still uncomfortable. The three-column layout left the editor fighting for space, the inspector panel kept Git-oriented information visible during writing, and the text column was too close to the window edges. Milestone 1 needs to validate whether a person wants to keep writing in Sapling. That requires layout and typography work even before advanced Markdown behavior exists. Layout experiments: - The three-column split view made the editor feel secondary. - Removing the inspector from the default writing surface immediately improved focus. - A two-column sidebar/detail split gives the editor roughly 70-80% of the default window. - The sidebar remains collapsible for focused writing. - The editor should be the detail column, not the middle column in a sidebar/content/inspector arrangement. Sidebar impact: The workspace sidebar is useful for opening documents, but it should not dominate Milestone 1. Git status and project inspection are distractions during editor validation and should stay hidden until later milestones. Cursor behavior: Native `NSTextView` keyboard behavior remains the right default. Arrow keys, Shift-selection, Option-arrow word navigation, Home, End, Page Up, and Page Down should be allowed to flow through AppKit instead of being reimplemented in SwiftUI. Smoke testing after the layout and typography changes covered: - Basic typing. - Left and right arrow movement. - Home and End. - Page Up and Page Down. - Option + Arrow movement. - Shift + Arrow selection. No selection recursion or immediate cursor instability appeared during the smoke pass. Typography observations: - A 14-point editor made the prototype feel cramped. - A 16-point base font with more line and paragraph spacing is more comfortable for writing. - A readable-width text column is more important than using all available horizontal space. - Generous document padding makes the editor feel like a writing surface instead of a debug widget. - The active-line highlight should stay subtle; it is there to orient the writer, not decorate the page. Status bar observations: The status bar should remain quiet. Line number, column number, total line count, and saved/modified state are enough for Milestone 1. Showing rendered previews in the status bar added visual noise without improving writing flow. Architecture impact: - Editor usability depends on app layout, not only text view internals. - The native text view bridge can support comfortable writing if it controls text insets, readable width, and cursor visibility. - Keyboard navigation should be validated through native behavior first. - More features should wait until the writing surface itself feels calm. Result: The prototype now opens into a focused writing layout with a larger text area, collapsible sidebar, readable line width, generous padding, a subtle current-line treatment, and a realistic sample document for testing. ## Finding #3 — Hybrid Editing Architecture Milestone 2 keeps one canonical Markdown source buffer inside the native text view. The editor does not maintain a second rendered document, and it does not replace inactive lines with different strings. Active-line source mode and inactive-line rendered mode are presentation states derived from the same source. ```mermaid flowchart TD TextView["NSTextView / UITextView source buffer"] --> Delegate["Native delegate callbacks"] Delegate --> Coordinator["Bridge coordinator"] Coordinator --> ViewModel["HybridMarkdownEditorViewModel"] ViewModel --> State["EditorState"] State --> Tracker["EditorActiveLineTracker"] Tracker --> Lines["EditorLine ranges and modes"] Lines --> Styler["MarkdownTextStyler"] Styler --> TextStorage["NSTextStorage attributes"] TextStorage --> TextView Coordinator --> Metrics["EditorInstrumentationSnapshot"] ``` The active-line model now has a single source of truth: - `EditorState.selection` stores the current source-buffer selection. - `EditorActiveLineTracker` maps the selection location to one line index. - `EditorState.activeLineIndex` is derived from that mapping after every source or selection update. - `EditorLine.mode` is rebuilt from the active line and source ranges. - The native bridge only applies attributes from that state; it does not decide which line is active. This avoids recursive update loops because the native adapter still owns programmatic-update suppression. Text, selection, and attribute changes caused by Sapling are wrapped in coordinator transactions and ignored by delegate callbacks. User-originated changes flow back to the view model only when the source or selection actually changed. ```mermaid sequenceDiagram participant User participant Native as Native text view participant Bridge as Coordinator participant State as EditorState participant Render as Attribute styler User->>Native: Move cursor or edit text Native->>Bridge: Delegate notification Bridge->>State: Update source or selection State->>State: Derive activeLineIndex State->>Render: Apply attributes for active/inactive lines Render->>Native: Preserve selection and visible origin ``` Implementation scope is intentionally minimal: - Headings are visually emphasized on inactive lines. - Bold, italic, and inline code receive inline attributes. - Markdown delimiters are deemphasized, not hidden. - Links, tasks, lists, code blocks, tables, images, Mermaid, LaTeX, and attachments are not promoted in the hybrid renderer for this phase. Cursor and selection behavior remain stable because the visible text length is still the source text length. Delimiters are styled but not removed, so source offsets, native selection ranges, undo grouping, and TextKit cursor movement continue to share the same coordinate system. ## Finding #4 — Rendering Strategy Evaluation Milestone 2 compared four rendering strategies against the current product risk: editing correctness is more important than rendering quality. | Approach | Complexity | Performance | Cursor stability | Maintainability | Finding | | --- | --- | --- | --- | --- | --- | | A. Attributed-string rendering | Low to medium | Dirty native attribute passes after initial render; state line tracking remains O(n) | Strong because source offsets are unchanged | Strong while rendering is inline and line-level | Chosen for Milestone 2 | | B. Overlay rendering | High | Potentially good with cached layout, but expensive to synchronize | Medium risk from hit testing, selection painting, and accessibility | Medium; requires layout synchronization code | Defer until rich block rendering | | C. Line replacement | High | Could be efficient per line, but requires source/render mapping | High risk because cursor offsets diverge from visible text | Fragile around undo, IME, and cross-line selection | Not appropriate for current milestone | | D. Mixed editor/render layers | Very high | Potentially best long term with a custom layout engine | Unknown until Sapling owns text input, IME, selection, undo, accessibility | Expensive; effectively a custom editor engine | Long-term fallback only | Approach A was selected because it validates Sapling's core hybrid interaction while preserving native text editing. It does not prove that full rendered blocks can live inside `NSTextView`, but it does prove that active-line source editing and inactive-line presentation can coexist without destabilizing cursor and selection behavior. ```mermaid quadrantChart title Rendering Strategy Risk x-axis Low cursor stability --> High cursor stability y-axis Low implementation complexity --> High implementation complexity quadrant-1 "Good validation path" quadrant-2 "Powerful but expensive" quadrant-3 "Avoid" quadrant-4 "Defer" "Attributed attributes": [0.82, 0.28] "Overlay rendering": [0.55, 0.72] "Line replacement": [0.28, 0.68] "Mixed custom layers": [0.45, 0.92] ``` The important tradeoff is honest: attributed rendering cannot hide Markdown syntax without leaving visible gaps or breaking cursor math. For Milestone 2, that is acceptable. It is better to keep the editor believable and predictable than to make inactive lines look fully rendered at the cost of offset instability. ## Finding #5 — Performance Characteristics Instrumentation was added at the editor boundary: - Source changes are counted in `EditorInstrumentationSnapshot.sourceChangeCount`. - Selection changes are counted in `selectionChangeCount`. - Active-line transitions are counted in `activeLineChangeCount`. - Native attribute passes are counted in `renderPassCount`. - Full native attribute passes are counted in `fullRenderCount`. - Dirty lines touched by native rendering are counted in `totalDirtyLineCount` and `lastDirtyLineCount`. - Scroll-origin restoration is counted in `scrollRestorationCount`. - Each render pass records reason, duration, source character count, line count, dirty line count, active line index, whether it was a full render, and whether scroll position was restored. The counters are intentionally not `@Published`. They can be inspected by tests, logs, or future debug UI without causing every render pass to trigger another SwiftUI update. Automated validation added in `SaplingEditorTests` covers: - Cursor-derived active-line transitions. - Selection clamping after deletion. - Trailing blank-line range tracking. - Heading, bold, italic, and inline-code render plans. - Links and tasks remaining unsupported in the Milestone 2 renderer. - A 2,100-line prototype document shape. Current measured automated results on May 29, 2026: - `swift test`: 13 tests passed. - Large render-plan test: 2,100 lines processed in roughly 0.04 to 0.06 seconds during the XCTest run. - Full suite runtime: roughly 0.05 to 0.07 seconds after build. Prototype documents were added for repeatable manual validation: - `Docs/EditorPrototypes/hybrid-small-50.md` - `Docs/EditorPrototypes/hybrid-medium-500.md` - `Docs/EditorPrototypes/hybrid-large-2100.md` Scroll stability work: - Attribute passes restore the pre-render visible origin. - Selection restoration after styling does not call scroll-to-visible. - Explicit programmatic selection changes still scroll the selected range into view. This means rendering updates should not yank the viewport while the user scrolls or while inactive-line attributes refresh. Native cursor movement still gets to scroll naturally when the insertion point moves outside the visible area. Resolved performance limitation in Milestone 2.5: The native styler no longer applies a full-buffer attribute pass for every source edit or active-line change. Initial renders and programmatic full text replacements still rebuild the full attributed string. User edits and active-line navigation now produce a dirty-line invalidation plan, and the text storage resets and restyles only those line ranges. Remaining performance limitation: `EditorState.updateSource` and dirty-plan construction still derive the full line model from the source string. The expensive native attributed-string rebuild is avoided, but state rebuilding remains O(n). This is visible in the 10,000-line typing proxy measurement below and should be the next measured optimization before richer block rendering. ## Finding #6 — Dirty-Line Rendering Milestone 2 rendered every line on source changes and active-line changes. The coordinator avoided redundant SwiftUI restyles with a `lastStyledText` and `lastStyledActiveLineIndex` cache, but any actual edit still called `MarkdownTextStyler.apply(...)` over the full `NSTextStorage`. Milestone 2.5 adds `EditorDirtyLineInvalidator`, which compares the previously styled source and active line against the current source and active line. It emits an `EditorDirtyLineInvalidationPlan` containing: - render reason - full-render flag - dirty line indexes - changed UTF-16 range for source edits The native adapter now follows this path: ```mermaid flowchart TD Native["NSTextView text storage"] --> Coordinator["Native coordinator"] Coordinator --> Cache["Last styled text + active line"] Cache --> Invalidator["EditorDirtyLineInvalidator"] Invalidator --> Plan["Dirty-line invalidation plan"] Plan --> Styler["MarkdownTextStyler"] Styler --> DirtyRanges["Reset base attributes only on dirty line ranges"] DirtyRanges --> RenderLine["Apply source/rendered styling to dirty lines"] RenderLine --> Metrics["EditorRenderPassMetric"] ``` Observed behavior from automated tests: | User action | Render reason | Lines touched | | --- | --- | --- | | Initial editor load | `initial` | all lines | | Move active line from A to B | `activeLineChange` | A and B | | Edit one line | `sourceChange` | edited line plus immediate neighbors | | Insert a line break | `sourceChange` | split line, new active line, and affected neighbor | | Redundant SwiftUI view update | `viewUpdate` | zero; no styling pass | The neighboring-line policy is intentionally conservative. Current Milestone 2 Markdown spans do not cross line boundaries, but line breaks and future block parsing can make nearby lines visually affected. The policy avoids full-document rendering while keeping enough context for line joins and splits. ## Finding #7 — Large Document Performance Stress documents were added for repeatable validation: - `Docs/EditorPrototypes/hybrid-stress-1000.md` - `Docs/EditorPrototypes/hybrid-stress-5000.md` - `Docs/EditorPrototypes/hybrid-stress-10000.md` Automated measurement command: ```sh SAPLING_EDITOR_PRINT_METRICS=1 swift test --filter EditorLargeDocumentValidationTests/testLargeDocumentOpenAndDirtyRenderPlanning ``` Measured on May 29, 2026 in a debug SwiftPM test run: | Lines | Open proxy | Typing proxy | Dirty render-plan proxy | Dirty lines | Full render | | ---: | ---: | ---: | ---: | ---: | --- | | 1,000 | 1.769 ms | 11.770 ms | 2.804 ms | 3 | no | | 5,000 | 9.016 ms | 59.068 ms | 9.348 ms | 3 | no | | 10,000 | 17.642 ms | 119.364 ms | 19.115 ms | 3 | no | What these numbers measure: - Open proxy: creating `EditorState` and its line model. - Typing proxy: updating source, rebuilding editor state, and creating a dirty-line plan. - Dirty render-plan proxy: line tracking plus render-plan creation for dirty lines. - Render frequency: one source edit schedules one render metric with three dirty lines and `isFullRender == false`. What these numbers do not measure: - Real AppKit layout time in a visible window. - GPU compositing or actual scroll wheel latency. - IME composition behavior. Evidence: - 10,000-line documents remain tractable for state creation in debug tests. - Native attributed-string work is now bounded to three dirty lines for a single-line edit. - The remaining typing proxy cost scales linearly because the state model still rebuilds all lines. - Scroll stability is covered by repeatable instrumentation tests: dirty render passes restore the captured scroll origin/content offset and increment `scrollRestorationCount`. ## Finding #8 — Overlay Architecture Exploration No overlay renderer was implemented. The spike evaluated how it would fit beside the current attributed-string path. Approach A: attributed-string rendering keeps all presentation inside TextKit. ```mermaid flowchart LR Source["Markdown source string"] --> TextStorage["NSTextStorage"] TextStorage --> Attributes["Line + inline attributes"] Attributes --> TextView["NSTextView display, cursor, selection"] ``` Approach B: overlay rendering keeps the source editor native and draws rendered inactive content in a synchronized layer. ```mermaid flowchart TD Source["Markdown source string"] --> TextView["NSTextView source buffer"] TextView --> Layout["TextKit layout manager"] Layout --> LineRects["Line fragment rects"] Source --> Parser["Markdown line/block parser"] Parser --> RenderCache["Rendered overlay cache"] LineRects --> Overlay["Overlay view layer"] RenderCache --> Overlay Overlay --> HitTesting["Hit testing + accessibility mapping"] HitTesting --> TextView ``` Evidence-based comparison: | Question | Attributed-string rendering | Overlay rendering | | --- | --- | --- | | Would it improve performance? | Dirty-line invalidation already removes full attributed-string rebuilds for current inline styling. | Could help rich blocks by rendering only visible overlays, but requires layout cache invalidation and visible-range tracking. | | Would it improve cursor stability? | Strong today because text, cursor, and selection share one source coordinate system. | Source buffer remains canonical, but hit testing, selection painting, and accessibility must map overlay geometry back to source ranges. | | What complexity appears? | Attribute scopes, dirty ranges, and TextKit line metrics. | Overlay lifecycle, z-order, scroll synchronization, invalidation, accessibility, hit testing, and mismatch between rendered block heights and source line heights. | Overlay rendering is still promising for future images, diagrams, and rich blocks. It should not replace the current renderer before those features exist because the current bottleneck is state line-model rebuilding, not attributed styling for dirty lines. ## Finding #9 — Editor Scalability Assessment Current assessment: - `NSTextView` remains a viable foundation for the next milestone. - The editor now avoids full attributed-string rebuilds for user edits and active-line moves. - Cursor and selection state remain stable because source offsets are unchanged. - Scroll restoration remains explicit and counted. - Large documents open in the automated proxy tests, including 10,000 generated lines. Scalability limits: - Source updates still rebuild `EditorState.lines` for the whole document. - Dirty rendering still computes line positions from the full source before touching only dirty ranges. - Real scroll responsiveness still needs visible-window AppKit measurement, not only model-level tests. - Rich block rendering may require overlay or mixed-layer work once block heights diverge from source line heights. Recommended next measured work before Milestone 3 rendering complexity: 1. Add native edited-range capture from TextKit callbacks so state updates can avoid full line-model reconstruction. 2. Add an AppKit UI benchmark harness that opens the 1,000/5,000/10,000-line files in a visible editor and records scroll and typing latency. 3. Keep overlay rendering as a technical option for rich blocks, but do not introduce it for current heading/emphasis/code styling. ## Finding #10 — Newline Semantics Matter Milestone 2.6 found that the 5 MB benchmark document appeared to render entirely in source mode. Profiling showed that the editor line model contained one source line and zero rendered lines, even though the file had 51,482 physical lines. Root cause: `EditorActiveLineTracker` searched Swift `String` values for `Character("\n")` and used `String.split(separator: "\n")` for active-line lookup. This is incorrect for editor line segmentation because Swift `String` is a collection of extended grapheme clusters, not raw bytes or UTF-16 code units. A CRLF sequence (`"\r\n"`) is treated as a single newline grapheme cluster according to Unicode text segmentation behavior. Searching that `String` for `Character("\n")` does not find the LF code unit inside the CRLF grapheme. Evidence from Milestone 2.6: ```text CR=51481 LF=51481 "a\r\nb".firstIndex(of: "\n") == nil "a\r\nb".split(separator: "\n").count == 1 (NSString("a\r\nb")).components(separatedBy: "\n").count == 2 ``` Architecture impact: - The benchmark file collapsed to one `EditorLine`. - The whole document became the active source line. - Rendered mode was absent because there were no inactive lines to render. - Dirty-line invalidation technically returned one dirty line, but that dirty line was the whole document. - TextKit layout and attributed styling measured the cost of one giant wrapped line, not the intended hybrid editor model. Milestone 2.7 corrected this by introducing `DocumentLineIndex` and `LineEndingStrategy`. ```mermaid flowchart TD Source["Markdown source"] --> Index["DocumentLineIndex"] Index --> UTF16["UTF-16 line-ending scan"] UTF16 --> LF["LF"] UTF16 --> CRLF["CRLF"] UTF16 --> CR["CR"] LF --> Boundaries["DocumentLineBoundary ranges"] CRLF --> Boundaries CR --> Boundaries Boundaries --> Active["Active-line lookup"] Boundaries --> Lines["EditorLine array"] Lines --> Hybrid["1 source line, inactive rendered lines"] ``` Corrected validation: - LF-only, CRLF-only, CR-only, mixed newline, and trailing blank line tests pass. - `Docs/Benchmarks/5mb.md` now segments into 51,482 editor lines. - Corrected rendered-mode trace: 51,481 rendered lines and 1 source line. - Active-line lookup maps positions inside CRLF line-ending ranges to the preceding line and the position after CRLF to the next line. Corrected benchmark impact: | Measurement | Before CRLF fix | After CRLF fix | | --- | ---: | ---: | | Editor lines | 1 | 51,482 | | Rendered/source trace | 0 rendered, 1 source | 51,481 rendered, 1 source | | Dirty typing render | 887.213 ms | 55.805 ms | | Layout after typing | 490.160 ms | 0.903 ms | | Dirty invalidation typing | 566.852 ms | 980.651 ms | | Full TextKit layout | 766.331 ms | 1,320.499 ms | | Measured total | 4,376.271 ms | 5,674.222 ms | The corrected benchmark is slower overall because it now measures real per-line work across 51,482 editor lines instead of one accidental giant line. That is the correct baseline for future optimization work. Lesson: Future editor systems must not infer logical lines by searching Swift `String` values for newline characters. Editor line segmentation must go through `DocumentLineIndex`, which uses explicit UTF-16 line-ending handling compatible with TextKit selection ranges and `NSRange`. ## Finding #11 — Incremental Line Index Milestone 2.9 implemented the highest-impact optimization identified in Milestone 2.8: a persistent line index used by active-line lookup, selection mapping, dirty invalidation, and dirty styling. Architecture: ```mermaid flowchart TD TextKit["NSTextView / UITextView edited range"] --> Edit["DocumentLineIndexEdit"] Edit --> Index["Persistent DocumentLineIndex"] Index --> Active["Active-line lookup"] Index --> Dirty["Dirty-line invalidation"] Index --> Styling["Dirty-line styling"] Index --> State["EditorState line count and active column"] ``` `DocumentLineIndex` now provides: - line count - line boundary lookup - offset-to-line mapping - line-to-offset mapping - affected-line mapping with neighboring-line expansion - incremental local replacement using `DocumentLineIndexEdit` Update strategy: 1. Capture the native edited range and replacement string from the platform text view delegate. 2. Expand the scan window to the edited line, adjacent lines, and line-ending boundary neighbors. 3. Rescan only that local window using UTF-16 line-ending rules. 4. Replace the affected boundary slice in the persistent index. 5. Use the updated index for active-line lookup, dirty-line invalidation, and dirty styling. The editor still stores the full source string because TextKit and persistence require it. The optimized path now accepts TextKit's already-edited source instead of reconstructing it from the old source inside the index. Complexity impact: | Operation | Before | After | | --- | --- | --- | | Active-line lookup | O(document) scan | O(log line count) binary lookup | | Selection update | O(document) line rebuild | O(log line count) active-line lookup | | Dirty invalidation for typing | O(document) prefix/suffix diff + line rebuild | O(log line count + affected lines) | | Dirty styling setup | O(document) line-list rebuild | O(dirty lines) | | Source edit index update | O(document) full line rebuild | local rescan plus boundary offset maintenance | 5 MB benchmark impact: | Measurement | Before Milestone 2.9 | After Milestone 2.9 | Improvement | | --- | ---: | ---: | ---: | | `active_line_lookup` | 191.751 ms | 0.001 ms | 99.999% | | `selection_update` | 398.262 ms | 0.002 ms | 99.999% | | `dirty_line_invalidation_click` | 224.418 ms | 0.001 ms | 99.999% | | `typing_state_update` | 416.540 ms | 0.102 ms | 99.976% | | `dirty_line_invalidation_typing` | 1,019.380 ms | 0.003 ms | 99.999% | | `render_update_typing_dirty` | 53.512 ms | 0.796 ms | 98.512% | Tradeoffs: - The line index is now stateful and must remain synchronized with native text edits. - Programmatic full-source replacement still falls back to a full index rebuild. - Correctness depends on preserving the Milestone 2.7 UTF-16 newline semantics for LF, CRLF, CR, mixed endings, and trailing blank lines. Validation: - Incremental line-index tests compare local edits against full rebuilds. - Tests cover insertion, deletion, replacement, CRLF boundary edits, mixed line endings, and a 10,000-line edit. - Cursor, dirty invalidation, scroll stability, and large-document tests continue to pass. Conclusion: The main Milestone 2.8 Sapling-side interaction bottlenecks have been removed from the measured 5 MB typing and selection paths. After this change, the dominant measured costs are TextKit full/cold layout and full-document initial rendering/planning work, not active-line tracking or dirty invalidation. ## Finding #12 — Markdown Rendering Expansion Milestone 3 expands the attributed hybrid renderer from the Milestone 2 proof of concept into a more complete Markdown writing surface. The implementation still keeps one canonical source buffer inside the native text view. It does not replace Markdown source with rendered strings, and it does not introduce overlay views or a custom editor engine. Supported rendering plans now cover: - ATX headings - bold and italic emphasis using asterisks or underscores - inline code - blockquotes - horizontal rules - unordered lists - ordered lists - nested list indentation - task lists - Markdown links - automatic links - fenced code blocks with language labels - Markdown table rows and divider rows Architecture: ```mermaid flowchart TD TextKit["NSTextView / UITextView source buffer"] --> Index["DocumentLineIndex"] Index --> Lines["EditorLine values"] Lines --> Parser["HybridMarkdownLineRenderer"] Parser --> Plans["HybridMarkdownLineRenderPlan"] Plans --> Styler["MarkdownTextStyler"] Styler --> Attributes["NSTextStorage attributes"] Attributes --> TextKit ``` `HybridMarkdownLineRenderer` remains a line-oriented parser. It emits line kinds and inline spans with `NSRange` values in source-buffer coordinates. `MarkdownTextStyler` applies fonts, colors, paragraph styles, background treatments, underlines, and strikethroughs to those source ranges. The active line continues to receive source styling. Inactive lines receive rendered-style attributes. Because the actual text is not replaced, cursor locations, selections, undo, TextKit layout ranges, dirty-line invalidation, and persistence all remain tied to the same source string. Tradeoffs: - Markdown delimiters are visually de-emphasized rather than removed. This is less visually complete than Typora-style replacement, but it avoids source/render offset mapping. - Task lists show checkbox state with styling on `[ ]` and `[x]`; interactive toggling is deferred. - Horizontal rules are represented with attributed separator styling over the source marker instead of a true drawn rule. - Tables use monospaced attributed rendering for readability. They do not yet become independent grid views. - Fenced code block contents are styled when render plans are produced with contiguous line context. Dirty-line rendering avoids document-wide context scans, so long code-block context remains conservative around local edits. - Markdown links and automatic links are visually distinct. Full click handling is deferred unless it can be added without destabilizing editing. - Reference-style link definitions remain unsupported in the hybrid renderer. The 5 MB benchmark is dominated by reference-style definitions, so automatic-link parsing intentionally skips those lines for now. Scalability impact: The expanded renderer was validated with: - sample document benchmark - `Docs/EditorPrototypes/hybrid-large-2100.md` - `Docs/Benchmarks/5mb.md` Release benchmark run on May 31, 2026 after parser guards: | Scenario | Lines | Measured total | Render planning | Initial styling | Dirty typing render | | --- | ---: | ---: | ---: | ---: | ---: | | sample document | 54 | 10.151 ms | 1.005 ms | 3.941 ms | 0.265 ms | | 2,100-line prototype | 2,101 | 413.814 ms | 88.227 ms | 96.668 ms | 0.369 ms | | 5 MB benchmark | 51,482 | 3,157.751 ms | 244.277 ms | 371.099 ms | 0.985 ms | Tracked 5 MB interaction metrics remained within the Milestone 2.9 responsiveness envelope: | Operation | Time | | --- | ---: | | active-line lookup | 0.000 ms | | selection update | 0.002 ms | | dirty click invalidation | 0.001 ms | | typing state update | 0.189 ms | | dirty typing invalidation | 0.003 ms | | dirty typing render | 0.985 ms | The main remaining costs are still full/cold TextKit layout and initial full-document work, not active-line switching or dirty edit rendering. Parser guards were necessary because the first expanded renderer performed too much per-line Markdown detection on the 5 MB reference-definition benchmark. After guarding by first token and skipping reference-style definitions for automatic-link parsing, the measured 5 MB total remained below the Milestone 2.9 reported baseline. Validation fixtures: - `Docs/EditorPrototypes/markdown-technical-documentation.md` - `Docs/EditorPrototypes/markdown-meeting-notes.md` - `Docs/EditorPrototypes/markdown-research-notes.md` - `Docs/EditorPrototypes/markdown-project-plan.md` - `Docs/EditorPrototypes/markdown-long-form-writing.md` Unsupported elements explicitly deferred: - images - wikilinks - Mermaid - LaTeX - callouts - attachments - audio - video Conclusion: Sapling now renders enough standard Markdown structure for documentation, technical notes, project notes, meeting notes, and long-form writing to feel substantially closer to a real Markdown editor while preserving the validated hybrid editing model. The renderer remains deliberately attributed and source-preserving until richer block layout can be proven without harming cursor stability, selection stability, typing latency, or scrolling behavior. ## AttributedString and NSAttributedString Swift `AttributedString` is useful for renderer-facing APIs and SwiftUI previews. `NSAttributedString` and `NSTextStorage` are still required for the actual native editor bridge because the text view operates on TextKit types. Recommended split: - Use `AttributedString` in cross-platform renderer interfaces. - Use `NSAttributedString` and `NSTextStorage` inside platform adapters. - Keep conversion and styling code contained in `SaplingEditor`. ## Cursor Challenges The prototype validates basic active-line detection: - The selected range location maps to a line index. - The active line switches as the cursor moves. - The active line can be styled as source while other lines are styled as rendered. Open challenges: - Multi-line selection across source and rendered lines. - Selection after visually hidden or replaced Markdown delimiters. - Cursor placement if future phases hide links, list markers, task markers, or other delimiters. - IME composition and marked text handling. - Very large documents where full state line-model rebuilding on each source change is too expensive. Cursor correctness is the main reason to keep the canonical source text inside `NSTextView` for now. ## Rendering Challenges The current spike uses line-level styling, not full Markdown rendering. Validated: - Headings can be styled by line. - Bold, italic, and inline code can be styled inline. - Unsupported constructs can remain plain source text without breaking the editor model. - Active source line can remain monospaced and visibly distinct. Not validated: - True delimiter removal without cursor side effects. - Embedded rendered blocks inside the text layout. - Images, tables, code block widgets, Mermaid, LaTeX, and attachments. - Precise layout matching between rendered inactive lines and raw active lines. ## Hybrid Editing Feasibility Hybrid editing is achievable for Milestone 2 when the first implementation is scoped to line-level attributed styling. The safest path is: 1. Keep the Markdown source as the only editable text buffer. 2. Detect the active line from the native selection range. 3. Apply source styling to the active line. 4. Apply rendered-style attributes to inactive lines. 5. Delay true line replacement and overlay rendering until selection behavior is better understood. This does not produce perfect rendered Markdown, but it validates the writing feel and cursor behavior before Sapling invests in a custom editor engine. ## Recommended Path Forward Continue with `NSTextView` for the next editor milestone on macOS and keep the iOS `UITextView` adapter available behind the same editor abstraction. Immediate next steps: - Capture native edited ranges so source updates can avoid rebuilding the full line model. - Manually test undo/redo, IME input, and keyboard navigation with the stress prototype files. - Add visible AppKit scroll and typing benchmarks for the 1,000/5,000/10,000-line documents. - Explore delimiter hiding with temporary attributes only after selection mapping is robust. Future investigation: - Use TextKit layout fragments or overlay views for rich block rendering. - Prototype image and code block rendering outside the text buffer. - Reevaluate whether a custom editor engine is required after Milestone 2. ## Decision `NSTextView` can continue to support Sapling's roadmap through the next phase. Evidence: - The active-line source and inactive-line presentation model works while preserving a single source buffer. - Cursor and selection ranges remain native source ranges because rendering does not replace text. - Scroll position can be preserved around rendering updates. - Redundant render passes are avoided when source text and active line are unchanged. - Dirty-line invalidation avoids full attributed-string rebuilds for user edits and active-line transitions. - The 1,000/5,000/10,000-line stress documents are tractable in automated model-level validation, with 10,000-line open proxy time measured at 17.642 ms. Sapling should not begin a custom editor engine now. The evidence supports continuing with native text systems while measuring state-line rebuilding and richer overlay experiments. A future custom editor engine may still be required for high-fidelity block rendering, especially images, tables, Mermaid, LaTeX, attachments, and fully hidden delimiters. That decision should be made only after overlay rendering and native edited-range state updates have been measured against real documents. The architecture should continue with a SwiftUI shell, a Sapling editor abstraction, and native platform text views hidden behind replaceable adapters.