feror/Sapling
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
Sapling/Docs/editor-investigation.md

1471 lines
75 KiB
Markdown
Raw Blame History

# Editor Investigation
Date: 2026-05
Status: Milestone 2 validation
Related decisions:
- D-008: Hybrid Markdown Editing Is a Core Feature
- D-009: The Editor Is the Highest-Priority System
- D-010: Git Access Must Be Abstracted
- D-012: Platform Focus
- D-013: Editor Technology Selection
## Question
Can Sapling support a high-quality hybrid Markdown editing experience where the active line shows Markdown source and inactive lines show rendered Markdown?
## Prototype Summary
The Milestone 1 prototype uses a SwiftUI application shell with a native text view bridge:
- macOS: `NSTextView` through `NSViewRepresentable`
- iOS path: `UITextView` through `UIViewRepresentable`
- shared editor state in `SaplingEditor`
- document loading and saving through `HybridMarkdownEditorViewModel`
- line tracking through `EditorState`, `EditorLine`, and `EditorSelection`
- in-place line styling through `NSTextStorage`
The prototype validates that native text views can support the first version of Sapling's editor without coupling the app directly to AppKit.
## Current Architecture
```mermaid
flowchart TD
App["SwiftUI app shell"] --> Editor["HybridMarkdownEditor"]
Editor --> Bridge["Native text view bridge"]
Bridge --> Mac["NSTextView on macOS"]
Bridge --> IOS["UITextView on iOS"]
Editor --> Coordinator["EditorCoordinator"]
Coordinator --> State["EditorState"]
State --> Document["EditorDocument"]
State --> Lines["EditorLine array"]
State --> Selection["EditorSelection"]
Bridge --> TextKit["Text storage and selection APIs"]
```
The important boundary is the Sapling editor abstraction. Application code talks to `HybridMarkdownEditorViewModel` and editor state, not to `NSTextView`.
## Architectures Explored
### SwiftUI TextEditor
SwiftUI `TextEditor` was already present in the early prototype and worked for simple active-line editing.
Benefits:
- Minimal SwiftUI integration.
- Fast to prototype.
- Good enough for plain text entry.
Limitations:
- Does not expose reliable low-level selection and cursor APIs.
- Does not provide direct TextKit access.
- Makes line-level source/render switching difficult.
- Offers limited control over attributed rendering and layout.
Conclusion: `TextEditor` is not suitable as Sapling's primary editor implementation.
### NSTextView With Attributed Styling
The implemented spike keeps Markdown source as the actual text buffer and applies visual styling to inactive lines with `NSTextStorage`.
Benefits:
- Native cursor movement remains intact.
- Native selection ranges remain tied to the real source buffer.
- Undo, find, spell checking, and keyboard navigation come from AppKit.
- Active line detection can be derived from selection location.
- Inactive lines can be styled differently from the active source line.
Limitations:
- Styling alone cannot remove Markdown delimiters without affecting layout.
- It cannot render block elements such as images, tables, or Mermaid diagrams as real embedded views.
- Changing font sizes per line affects line height and may create visual jumps.
- Reapplying attributes on every selection change must be optimized for large documents.
Conclusion: This is sufficient for Milestone 1 and likely sufficient for an early Milestone 2 implementation of headings, emphasis, links, lists, and task lists.
### Line Replacement
Line replacement would swap inactive Markdown source with rendered text while preserving the source elsewhere.
Benefits:
- More closely matches the desired hybrid model.
- Can hide Markdown delimiters from inactive lines.
Risks:
- Cursor offsets no longer map directly to source offsets.
- Selection crossing active and inactive lines becomes complex.
- Undo behavior may become fragile.
- TextKit may fight source/render buffer divergence.
Conclusion: Worth exploring after the attributed-string path, but risky as the primary approach.
### Overlay Rendering
Overlay rendering would keep source text in the editor but draw rendered content above inactive lines.
Benefits:
- Source buffer remains canonical.
- Rendered views could support richer blocks.
- Cursor math can still use the real text buffer.
Risks:
- Requires precise layout synchronization with TextKit.
- Hit testing and selection feedback are difficult.
- Accessibility must be designed carefully.
Conclusion: Promising for richer Milestone 3 rendering, but heavier than needed for Milestone 2.
### Mixed Text and Render Layers
A custom layout could combine text editing lines with rendered SwiftUI/AppKit views.
Benefits:
- Maximum control over rendering.
- Can support images, diagrams, tables, and custom blocks.
Risks:
- Requires building editor infrastructure Sapling does not yet have.
- Cursor movement, selection, IME, undo, accessibility, and text input become product-critical systems.
Conclusion: This may be necessary in the long term, but it should not be the first Milestone 2 implementation.
## TextKit Findings
`NSTextView` gives Sapling the hooks needed for the prototype:
- `selectedRange()` gives the cursor and selection range.
- `NSTextStorage` allows line-level attributes without replacing source text.
- `NSTextViewDelegate` reports text and selection changes.
- Native editing behavior remains available while the app tracks editor state.
The key finding is that Sapling can preserve a single source buffer and still present different active and inactive line states. This reduces risk compared with maintaining separate source and render buffers.
## Finding #1 — Selection Feedback Loop
Root cause:
The first `NSTextView` bridge treated every selection notification as user intent, even when the selection change was caused by Sapling itself while restoring selection after an attribute pass.
The recursive path was:
```text
textViewDidChangeSelection()
-> selection binding update
-> EditorState update
-> @Published state refresh
-> SwiftUI updateNSView()
-> applyHybridAttributes()
-> setSelectedRange()
-> textViewDidChangeSelection()
```
This created an unbounded feedback loop and could crash immediately after launch while the hybrid styling pass and SwiftUI refresh cycle bounced selection updates back and forth.
Architecture impact:
- Selection is not just editor state; it is also a native text system side effect.
- Attribute rendering can produce selection notifications even when the user's logical selection did not change.
- SwiftUI refreshes must not be allowed to feed native programmatic changes back into `EditorState` as if they were user edits.
- Hybrid editing remains viable, but the platform adapter must own reentrancy control.
Chosen solution:
- Programmatic text, selection, and attribute changes are wrapped in a coordinator transaction.
- Selection delegate callbacks are ignored while that transaction is active.
- `EditorCoordinator.updateSelection` and `updateSource` ignore no-op updates before publishing.
- Attribute passes restore selection only if the text view actually changed it.
- The bridge skips redundant restyling when both source text and active line are unchanged.
This keeps the canonical source and selection model in `EditorState`, while making the native adapter responsible for distinguishing user-originated changes from TextKit side effects.
Alternative solutions considered:
- Temporary delegate removal: workable, but broader than needed and easy to forget when adding new native callbacks.
- Debounced rendering: useful for future performance work, but it would only delay the recursion instead of fixing the feedback path.
- Never restoring selection after attributes: avoids this crash, but risks cursor drift if TextKit changes selection during editing.
- Moving selection outside published editor state: reduces SwiftUI refresh pressure, but weakens the architecture because active-line rendering depends on selection.
Result:
The app now survives launch, typing, arrow-key movement, and select-all smoke testing without re-entering the selection loop.
## Finding #2 — Writing Comfort Matters More Than Features
Root cause:
After the selection recursion fix, the editor was technically functional but still uncomfortable. The three-column layout left the editor fighting for space, the inspector panel kept Git-oriented information visible during writing, and the text column was too close to the window edges.
Milestone 1 needs to validate whether a person wants to keep writing in Sapling. That requires layout and typography work even before advanced Markdown behavior exists.
Layout experiments:
- The three-column split view made the editor feel secondary.
- Removing the inspector from the default writing surface immediately improved focus.
- A two-column sidebar/detail split gives the editor roughly 70-80% of the default window.
- The sidebar remains collapsible for focused writing.
- The editor should be the detail column, not the middle column in a sidebar/content/inspector arrangement.
Sidebar impact:
The workspace sidebar is useful for opening documents, but it should not dominate Milestone 1. Git status and project inspection are distractions during editor validation and should stay hidden until later milestones.
Cursor behavior:
Native `NSTextView` keyboard behavior remains the right default. Arrow keys, Shift-selection, Option-arrow word navigation, Home, End, Page Up, and Page Down should be allowed to flow through AppKit instead of being reimplemented in SwiftUI.
Smoke testing after the layout and typography changes covered:
- Basic typing.
- Left and right arrow movement.
- Home and End.
- Page Up and Page Down.
- Option + Arrow movement.
- Shift + Arrow selection.
No selection recursion or immediate cursor instability appeared during the smoke pass.
Typography observations:
- A 14-point editor made the prototype feel cramped.
- A 16-point base font with more line and paragraph spacing is more comfortable for writing.
- A readable-width text column is more important than using all available horizontal space.
- Generous document padding makes the editor feel like a writing surface instead of a debug widget.
- The active-line highlight should stay subtle; it is there to orient the writer, not decorate the page.
Status bar observations:
The status bar should remain quiet. Line number, column number, total line count, and saved/modified state are enough for Milestone 1. Showing rendered previews in the status bar added visual noise without improving writing flow.
Architecture impact:
- Editor usability depends on app layout, not only text view internals.
- The native text view bridge can support comfortable writing if it controls text insets, readable width, and cursor visibility.
- Keyboard navigation should be validated through native behavior first.
- More features should wait until the writing surface itself feels calm.
Result:
The prototype now opens into a focused writing layout with a larger text area, collapsible sidebar, readable line width, generous padding, a subtle current-line treatment, and a realistic sample document for testing.
## Finding #3 — Hybrid Editing Architecture
Milestone 2 keeps one canonical Markdown source buffer inside the native text view. The editor does not maintain a second rendered document, and it does not replace inactive lines with different strings. Active-line source mode and inactive-line rendered mode are presentation states derived from the same source.
```mermaid
flowchart TD
TextView["NSTextView / UITextView source buffer"] --> Delegate["Native delegate callbacks"]
Delegate --> Coordinator["Bridge coordinator"]
Coordinator --> ViewModel["HybridMarkdownEditorViewModel"]
ViewModel --> State["EditorState"]
State --> Tracker["EditorActiveLineTracker"]
Tracker --> Lines["EditorLine ranges and modes"]
Lines --> Styler["MarkdownTextStyler"]
Styler --> TextStorage["NSTextStorage attributes"]
TextStorage --> TextView
Coordinator --> Metrics["EditorInstrumentationSnapshot"]
```
The active-line model now has a single source of truth:
- `EditorState.selection` stores the current source-buffer selection.
- `EditorActiveLineTracker` maps the selection location to one line index.
- `EditorState.activeLineIndex` is derived from that mapping after every source or selection update.
- `EditorLine.mode` is rebuilt from the active line and source ranges.
- The native bridge only applies attributes from that state; it does not decide which line is active.
This avoids recursive update loops because the native adapter still owns programmatic-update suppression. Text, selection, and attribute changes caused by Sapling are wrapped in coordinator transactions and ignored by delegate callbacks. User-originated changes flow back to the view model only when the source or selection actually changed.
```mermaid
sequenceDiagram
participant User
participant Native as Native text view
participant Bridge as Coordinator
participant State as EditorState
participant Render as Attribute styler
User->>Native: Move cursor or edit text
Native->>Bridge: Delegate notification
Bridge->>State: Update source or selection
State->>State: Derive activeLineIndex
State->>Render: Apply attributes for active/inactive lines
Render->>Native: Preserve selection and visible origin
```
Implementation scope is intentionally minimal:
- Headings are visually emphasized on inactive lines.
- Bold, italic, and inline code receive inline attributes.
- Markdown delimiters are deemphasized, not hidden.
- Links, tasks, lists, code blocks, tables, images, Mermaid, LaTeX, and attachments are not promoted in the hybrid renderer for this phase.
Cursor and selection behavior remain stable because the visible text length is still the source text length. Delimiters are styled but not removed, so source offsets, native selection ranges, undo grouping, and TextKit cursor movement continue to share the same coordinate system.
## Finding #4 — Rendering Strategy Evaluation
Milestone 2 compared four rendering strategies against the current product risk: editing correctness is more important than rendering quality.
| Approach | Complexity | Performance | Cursor stability | Maintainability | Finding |
| --- | --- | --- | --- | --- | --- |
| A. Attributed-string rendering | Low to medium | Dirty native attribute passes after initial render; state line tracking remains O(n) | Strong because source offsets are unchanged | Strong while rendering is inline and line-level | Chosen for Milestone 2 |
| B. Overlay rendering | High | Potentially good with cached layout, but expensive to synchronize | Medium risk from hit testing, selection painting, and accessibility | Medium; requires layout synchronization code | Defer until rich block rendering |
| C. Line replacement | High | Could be efficient per line, but requires source/render mapping | High risk because cursor offsets diverge from visible text | Fragile around undo, IME, and cross-line selection | Not appropriate for current milestone |
| D. Mixed editor/render layers | Very high | Potentially best long term with a custom layout engine | Unknown until Sapling owns text input, IME, selection, undo, accessibility | Expensive; effectively a custom editor engine | Long-term fallback only |
Approach A was selected because it validates Sapling's core hybrid interaction while preserving native text editing. It does not prove that full rendered blocks can live inside `NSTextView`, but it does prove that active-line source editing and inactive-line presentation can coexist without destabilizing cursor and selection behavior.
```mermaid
quadrantChart
title Rendering Strategy Risk
x-axis Low cursor stability --> High cursor stability
y-axis Low implementation complexity --> High implementation complexity
quadrant-1 "Good validation path"
quadrant-2 "Powerful but expensive"
quadrant-3 "Avoid"
quadrant-4 "Defer"
"Attributed attributes": [0.82, 0.28]
"Overlay rendering": [0.55, 0.72]
"Line replacement": [0.28, 0.68]
"Mixed custom layers": [0.45, 0.92]
```
The important tradeoff is honest: attributed rendering cannot hide Markdown syntax without leaving visible gaps or breaking cursor math. For Milestone 2, that is acceptable. It is better to keep the editor believable and predictable than to make inactive lines look fully rendered at the cost of offset instability.
## Finding #5 — Performance Characteristics
Instrumentation was added at the editor boundary:
- Source changes are counted in `EditorInstrumentationSnapshot.sourceChangeCount`.
- Selection changes are counted in `selectionChangeCount`.
- Active-line transitions are counted in `activeLineChangeCount`.
- Native attribute passes are counted in `renderPassCount`.
- Full native attribute passes are counted in `fullRenderCount`.
- Dirty lines touched by native rendering are counted in `totalDirtyLineCount` and `lastDirtyLineCount`.
- Scroll-origin restoration is counted in `scrollRestorationCount`.
- Each render pass records reason, duration, source character count, line count, dirty line count, active line index, whether it was a full render, and whether scroll position was restored.
The counters are intentionally not `@Published`. They can be inspected by tests, logs, or future debug UI without causing every render pass to trigger another SwiftUI update.
Automated validation added in `SaplingEditorTests` covers:
- Cursor-derived active-line transitions.
- Selection clamping after deletion.
- Trailing blank-line range tracking.
- Heading, bold, italic, and inline-code render plans.
- Links and tasks remaining unsupported in the Milestone 2 renderer.
- A 2,100-line prototype document shape.
Current measured automated results on May 29, 2026:
- `swift test`: 13 tests passed.
- Large render-plan test: 2,100 lines processed in roughly 0.04 to 0.06 seconds during the XCTest run.
- Full suite runtime: roughly 0.05 to 0.07 seconds after build.
Prototype documents were added for repeatable manual validation:
- `Docs/EditorPrototypes/hybrid-small-50.md`
- `Docs/EditorPrototypes/hybrid-medium-500.md`
- `Docs/EditorPrototypes/hybrid-large-2100.md`
Scroll stability work:
- Attribute passes restore the pre-render visible origin.
- Selection restoration after styling does not call scroll-to-visible.
- Explicit programmatic selection changes still scroll the selected range into view.
This means rendering updates should not yank the viewport while the user scrolls or while inactive-line attributes refresh. Native cursor movement still gets to scroll naturally when the insertion point moves outside the visible area.
Resolved performance limitation in Milestone 2.5:
The native styler no longer applies a full-buffer attribute pass for every source edit or active-line change. Initial renders and programmatic full text replacements still rebuild the full attributed string. User edits and active-line navigation now produce a dirty-line invalidation plan, and the text storage resets and restyles only those line ranges.
Remaining performance limitation:
`EditorState.updateSource` and dirty-plan construction still derive the full line model from the source string. The expensive native attributed-string rebuild is avoided, but state rebuilding remains O(n). This is visible in the 10,000-line typing proxy measurement below and should be the next measured optimization before richer block rendering.
## Finding #6 — Dirty-Line Rendering
Milestone 2 rendered every line on source changes and active-line changes. The coordinator avoided redundant SwiftUI restyles with a `lastStyledText` and `lastStyledActiveLineIndex` cache, but any actual edit still called `MarkdownTextStyler.apply(...)` over the full `NSTextStorage`.
Milestone 2.5 adds `EditorDirtyLineInvalidator`, which compares the previously styled source and active line against the current source and active line. It emits an `EditorDirtyLineInvalidationPlan` containing:
- render reason
- full-render flag
- dirty line indexes
- changed UTF-16 range for source edits
The native adapter now follows this path:
```mermaid
flowchart TD
Native["NSTextView text storage"] --> Coordinator["Native coordinator"]
Coordinator --> Cache["Last styled text + active line"]
Cache --> Invalidator["EditorDirtyLineInvalidator"]
Invalidator --> Plan["Dirty-line invalidation plan"]
Plan --> Styler["MarkdownTextStyler"]
Styler --> DirtyRanges["Reset base attributes only on dirty line ranges"]
DirtyRanges --> RenderLine["Apply source/rendered styling to dirty lines"]
RenderLine --> Metrics["EditorRenderPassMetric"]
```
Observed behavior from automated tests:
| User action | Render reason | Lines touched |
| --- | --- | --- |
| Initial editor load | `initial` | all lines |
| Move active line from A to B | `activeLineChange` | A and B |
| Edit one line | `sourceChange` | edited line plus immediate neighbors |
| Insert a line break | `sourceChange` | split line, new active line, and affected neighbor |
| Redundant SwiftUI view update | `viewUpdate` | zero; no styling pass |
The neighboring-line policy is intentionally conservative. Current Milestone 2 Markdown spans do not cross line boundaries, but line breaks and future block parsing can make nearby lines visually affected. The policy avoids full-document rendering while keeping enough context for line joins and splits.
## Finding #7 — Large Document Performance
Stress documents were added for repeatable validation:
- `Docs/EditorPrototypes/hybrid-stress-1000.md`
- `Docs/EditorPrototypes/hybrid-stress-5000.md`
- `Docs/EditorPrototypes/hybrid-stress-10000.md`
Automated measurement command:
```sh
SAPLING_EDITOR_PRINT_METRICS=1 swift test --filter EditorLargeDocumentValidationTests/testLargeDocumentOpenAndDirtyRenderPlanning
```
Measured on May 29, 2026 in a debug SwiftPM test run:
| Lines | Open proxy | Typing proxy | Dirty render-plan proxy | Dirty lines | Full render |
| ---: | ---: | ---: | ---: | ---: | --- |
| 1,000 | 1.769 ms | 11.770 ms | 2.804 ms | 3 | no |
| 5,000 | 9.016 ms | 59.068 ms | 9.348 ms | 3 | no |
| 10,000 | 17.642 ms | 119.364 ms | 19.115 ms | 3 | no |
What these numbers measure:
- Open proxy: creating `EditorState` and its line model.
- Typing proxy: updating source, rebuilding editor state, and creating a dirty-line plan.
- Dirty render-plan proxy: line tracking plus render-plan creation for dirty lines.
- Render frequency: one source edit schedules one render metric with three dirty lines and `isFullRender == false`.
What these numbers do not measure:
- Real AppKit layout time in a visible window.
- GPU compositing or actual scroll wheel latency.
- IME composition behavior.
Evidence:
- 10,000-line documents remain tractable for state creation in debug tests.
- Native attributed-string work is now bounded to three dirty lines for a single-line edit.
- The remaining typing proxy cost scales linearly because the state model still rebuilds all lines.
- Scroll stability is covered by repeatable instrumentation tests: dirty render passes restore the captured scroll origin/content offset and increment `scrollRestorationCount`.
## Finding #8 — Overlay Architecture Exploration
No overlay renderer was implemented. The spike evaluated how it would fit beside the current attributed-string path.
Approach A: attributed-string rendering keeps all presentation inside TextKit.
```mermaid
flowchart LR
Source["Markdown source string"] --> TextStorage["NSTextStorage"]
TextStorage --> Attributes["Line + inline attributes"]
Attributes --> TextView["NSTextView display, cursor, selection"]
```
Approach B: overlay rendering keeps the source editor native and draws rendered inactive content in a synchronized layer.
```mermaid
flowchart TD
Source["Markdown source string"] --> TextView["NSTextView source buffer"]
TextView --> Layout["TextKit layout manager"]
Layout --> LineRects["Line fragment rects"]
Source --> Parser["Markdown line/block parser"]
Parser --> RenderCache["Rendered overlay cache"]
LineRects --> Overlay["Overlay view layer"]
RenderCache --> Overlay
Overlay --> HitTesting["Hit testing + accessibility mapping"]
HitTesting --> TextView
```
Evidence-based comparison:
| Question | Attributed-string rendering | Overlay rendering |
| --- | --- | --- |
| Would it improve performance? | Dirty-line invalidation already removes full attributed-string rebuilds for current inline styling. | Could help rich blocks by rendering only visible overlays, but requires layout cache invalidation and visible-range tracking. |
| Would it improve cursor stability? | Strong today because text, cursor, and selection share one source coordinate system. | Source buffer remains canonical, but hit testing, selection painting, and accessibility must map overlay geometry back to source ranges. |
| What complexity appears? | Attribute scopes, dirty ranges, and TextKit line metrics. | Overlay lifecycle, z-order, scroll synchronization, invalidation, accessibility, hit testing, and mismatch between rendered block heights and source line heights. |
Overlay rendering is still promising for future images, diagrams, and rich blocks. It should not replace the current renderer before those features exist because the current bottleneck is state line-model rebuilding, not attributed styling for dirty lines.
## Finding #9 — Editor Scalability Assessment
Current assessment:
- `NSTextView` remains a viable foundation for the next milestone.
- The editor now avoids full attributed-string rebuilds for user edits and active-line moves.
- Cursor and selection state remain stable because source offsets are unchanged.
- Scroll restoration remains explicit and counted.
- Large documents open in the automated proxy tests, including 10,000 generated lines.
Scalability limits:
- Source updates still rebuild `EditorState.lines` for the whole document.
- Dirty rendering still computes line positions from the full source before touching only dirty ranges.
- Real scroll responsiveness still needs visible-window AppKit measurement, not only model-level tests.
- Rich block rendering may require overlay or mixed-layer work once block heights diverge from source line heights.
Recommended next measured work before Milestone 3 rendering complexity:
1. Add native edited-range capture from TextKit callbacks so state updates can avoid full line-model reconstruction.
2. Add an AppKit UI benchmark harness that opens the 1,000/5,000/10,000-line files in a visible editor and records scroll and typing latency.
3. Keep overlay rendering as a technical option for rich blocks, but do not introduce it for current heading/emphasis/code styling.
## Finding #10 — Newline Semantics Matter
Milestone 2.6 found that the 5 MB benchmark document appeared to render entirely in source mode. Profiling showed that the editor line model contained one source line and zero rendered lines, even though the file had 51,482 physical lines.
Root cause:
`EditorActiveLineTracker` searched Swift `String` values for `Character("\n")` and used `String.split(separator: "\n")` for active-line lookup. This is incorrect for editor line segmentation because Swift `String` is a collection of extended grapheme clusters, not raw bytes or UTF-16 code units. A CRLF sequence (`"\r\n"`) is treated as a single newline grapheme cluster according to Unicode text segmentation behavior. Searching that `String` for `Character("\n")` does not find the LF code unit inside the CRLF grapheme.
Evidence from Milestone 2.6:
```text
CR=51481 LF=51481
"a\r\nb".firstIndex(of: "\n") == nil
"a\r\nb".split(separator: "\n").count == 1
(NSString("a\r\nb")).components(separatedBy: "\n").count == 2
```
Architecture impact:
- The benchmark file collapsed to one `EditorLine`.
- The whole document became the active source line.
- Rendered mode was absent because there were no inactive lines to render.
- Dirty-line invalidation technically returned one dirty line, but that dirty line was the whole document.
- TextKit layout and attributed styling measured the cost of one giant wrapped line, not the intended hybrid editor model.
Milestone 2.7 corrected this by introducing `DocumentLineIndex` and `LineEndingStrategy`.
```mermaid
flowchart TD
Source["Markdown source"] --> Index["DocumentLineIndex"]
Index --> UTF16["UTF-16 line-ending scan"]
UTF16 --> LF["LF"]
UTF16 --> CRLF["CRLF"]
UTF16 --> CR["CR"]
LF --> Boundaries["DocumentLineBoundary ranges"]
CRLF --> Boundaries
CR --> Boundaries
Boundaries --> Active["Active-line lookup"]
Boundaries --> Lines["EditorLine array"]
Lines --> Hybrid["1 source line, inactive rendered lines"]
```
Corrected validation:
- LF-only, CRLF-only, CR-only, mixed newline, and trailing blank line tests pass.
- `Docs/Benchmarks/5mb.md` now segments into 51,482 editor lines.
- Corrected rendered-mode trace: 51,481 rendered lines and 1 source line.
- Active-line lookup maps positions inside CRLF line-ending ranges to the preceding line and the position after CRLF to the next line.
Corrected benchmark impact:
| Measurement | Before CRLF fix | After CRLF fix |
| --- | ---: | ---: |
| Editor lines | 1 | 51,482 |
| Rendered/source trace | 0 rendered, 1 source | 51,481 rendered, 1 source |
| Dirty typing render | 887.213 ms | 55.805 ms |
| Layout after typing | 490.160 ms | 0.903 ms |
| Dirty invalidation typing | 566.852 ms | 980.651 ms |
| Full TextKit layout | 766.331 ms | 1,320.499 ms |
| Measured total | 4,376.271 ms | 5,674.222 ms |
The corrected benchmark is slower overall because it now measures real per-line work across 51,482 editor lines instead of one accidental giant line. That is the correct baseline for future optimization work.
Lesson:
Future editor systems must not infer logical lines by searching Swift `String` values for newline characters. Editor line segmentation must go through `DocumentLineIndex`, which uses explicit UTF-16 line-ending handling compatible with TextKit selection ranges and `NSRange`.
## Finding #11 — Incremental Line Index
Milestone 2.9 implemented the highest-impact optimization identified in Milestone 2.8: a persistent line index used by active-line lookup, selection mapping, dirty invalidation, and dirty styling.
Architecture:
```mermaid
flowchart TD
TextKit["NSTextView / UITextView edited range"] --> Edit["DocumentLineIndexEdit"]
Edit --> Index["Persistent DocumentLineIndex"]
Index --> Active["Active-line lookup"]
Index --> Dirty["Dirty-line invalidation"]
Index --> Styling["Dirty-line styling"]
Index --> State["EditorState line count and active column"]
```
`DocumentLineIndex` now provides:
- line count
- line boundary lookup
- offset-to-line mapping
- line-to-offset mapping
- affected-line mapping with neighboring-line expansion
- incremental local replacement using `DocumentLineIndexEdit`
Update strategy:
1. Capture the native edited range and replacement string from the platform text view delegate.
2. Expand the scan window to the edited line, adjacent lines, and line-ending boundary neighbors.
3. Rescan only that local window using UTF-16 line-ending rules.
4. Replace the affected boundary slice in the persistent index.
5. Use the updated index for active-line lookup, dirty-line invalidation, and dirty styling.
The editor still stores the full source string because TextKit and persistence require it. The optimized path now accepts TextKit's already-edited source instead of reconstructing it from the old source inside the index.
Complexity impact:
| Operation | Before | After |
| --- | --- | --- |
| Active-line lookup | O(document) scan | O(log line count) binary lookup |
| Selection update | O(document) line rebuild | O(log line count) active-line lookup |
| Dirty invalidation for typing | O(document) prefix/suffix diff + line rebuild | O(log line count + affected lines) |
| Dirty styling setup | O(document) line-list rebuild | O(dirty lines) |
| Source edit index update | O(document) full line rebuild | local rescan plus boundary offset maintenance |
5 MB benchmark impact:
| Measurement | Before Milestone 2.9 | After Milestone 2.9 | Improvement |
| --- | ---: | ---: | ---: |
| `active_line_lookup` | 191.751 ms | 0.001 ms | 99.999% |
| `selection_update` | 398.262 ms | 0.002 ms | 99.999% |
| `dirty_line_invalidation_click` | 224.418 ms | 0.001 ms | 99.999% |
| `typing_state_update` | 416.540 ms | 0.102 ms | 99.976% |
| `dirty_line_invalidation_typing` | 1,019.380 ms | 0.003 ms | 99.999% |
| `render_update_typing_dirty` | 53.512 ms | 0.796 ms | 98.512% |
Tradeoffs:
- The line index is now stateful and must remain synchronized with native text edits.
- Programmatic full-source replacement still falls back to a full index rebuild.
- Correctness depends on preserving the Milestone 2.7 UTF-16 newline semantics for LF, CRLF, CR, mixed endings, and trailing blank lines.
Validation:
- Incremental line-index tests compare local edits against full rebuilds.
- Tests cover insertion, deletion, replacement, CRLF boundary edits, mixed line endings, and a 10,000-line edit.
- Cursor, dirty invalidation, scroll stability, and large-document tests continue to pass.
Conclusion:
The main Milestone 2.8 Sapling-side interaction bottlenecks have been removed from the measured 5 MB typing and selection paths. After this change, the dominant measured costs are TextKit full/cold layout and full-document initial rendering/planning work, not active-line tracking or dirty invalidation.
## Finding #12 — Markdown Rendering Expansion
Milestone 3 expands the attributed hybrid renderer from the Milestone 2 proof of concept into a more complete Markdown writing surface. The implementation still keeps one canonical source buffer inside the native text view. It does not replace Markdown source with rendered strings, and it does not introduce overlay views or a custom editor engine.
Supported rendering plans now cover:
- ATX headings
- bold and italic emphasis using asterisks or underscores
- inline code
- blockquotes
- horizontal rules
- unordered lists
- ordered lists
- nested list indentation
- task lists
- Markdown links
- automatic links
- fenced code blocks with language labels
- Markdown table rows and divider rows
Architecture:
```mermaid
flowchart TD
TextKit["NSTextView / UITextView source buffer"] --> Index["DocumentLineIndex"]
Index --> Lines["EditorLine values"]
Lines --> Parser["HybridMarkdownLineRenderer"]
Parser --> Plans["HybridMarkdownLineRenderPlan"]
Plans --> Styler["MarkdownTextStyler"]
Styler --> Attributes["NSTextStorage attributes"]
Attributes --> TextKit
```
`HybridMarkdownLineRenderer` remains a line-oriented parser. It emits line kinds and inline spans with `NSRange` values in source-buffer coordinates. `MarkdownTextStyler` applies fonts, colors, paragraph styles, background treatments, underlines, and strikethroughs to those source ranges.
The active line continues to receive source styling. Inactive lines receive rendered-style attributes. Because the actual text is not replaced, cursor locations, selections, undo, TextKit layout ranges, dirty-line invalidation, and persistence all remain tied to the same source string.
Tradeoffs:
- Markdown delimiters are visually de-emphasized rather than removed. This is less visually complete than Typora-style replacement, but it avoids source/render offset mapping.
- Task lists show checkbox state with styling on `[ ]` and `[x]`; interactive toggling is deferred.
- Horizontal rules are represented with attributed separator styling over the source marker instead of a true drawn rule.
- Tables use monospaced attributed rendering for readability. They do not yet become independent grid views.
- Fenced code block contents are styled when render plans are produced with contiguous line context. Dirty-line rendering avoids document-wide context scans, so long code-block context remains conservative around local edits.
- Markdown links and automatic links are visually distinct. Full click handling is deferred unless it can be added without destabilizing editing.
- Reference-style link definitions remain unsupported in the hybrid renderer. The 5 MB benchmark is dominated by reference-style definitions, so automatic-link parsing intentionally skips those lines for now.
Scalability impact:
The expanded renderer was validated with:
- sample document benchmark
- `Docs/EditorPrototypes/hybrid-large-2100.md`
- `Docs/Benchmarks/5mb.md`
Release benchmark run on May 31, 2026 after parser guards:
| Scenario | Lines | Measured total | Render planning | Initial styling | Dirty typing render |
| --- | ---: | ---: | ---: | ---: | ---: |
| sample document | 54 | 10.151 ms | 1.005 ms | 3.941 ms | 0.265 ms |
| 2,100-line prototype | 2,101 | 413.814 ms | 88.227 ms | 96.668 ms | 0.369 ms |
| 5 MB benchmark | 51,482 | 3,157.751 ms | 244.277 ms | 371.099 ms | 0.985 ms |
Tracked 5 MB interaction metrics remained within the Milestone 2.9 responsiveness envelope:
| Operation | Time |
| --- | ---: |
| active-line lookup | 0.000 ms |
| selection update | 0.002 ms |
| dirty click invalidation | 0.001 ms |
| typing state update | 0.189 ms |
| dirty typing invalidation | 0.003 ms |
| dirty typing render | 0.985 ms |
The main remaining costs are still full/cold TextKit layout and initial full-document work, not active-line switching or dirty edit rendering. Parser guards were necessary because the first expanded renderer performed too much per-line Markdown detection on the 5 MB reference-definition benchmark. After guarding by first token and skipping reference-style definitions for automatic-link parsing, the measured 5 MB total remained below the Milestone 2.9 reported baseline.
Validation fixtures:
- `Docs/EditorPrototypes/markdown-technical-documentation.md`
- `Docs/EditorPrototypes/markdown-meeting-notes.md`
- `Docs/EditorPrototypes/markdown-research-notes.md`
- `Docs/EditorPrototypes/markdown-project-plan.md`
- `Docs/EditorPrototypes/markdown-long-form-writing.md`
Unsupported elements explicitly deferred:
- images
- wikilinks
- Mermaid
- LaTeX
- callouts
- attachments
- audio
- video
Conclusion:
Sapling now renders enough standard Markdown structure for documentation, technical notes, project notes, meeting notes, and long-form writing to feel substantially closer to a real Markdown editor while preserving the validated hybrid editing model. The renderer remains deliberately attributed and source-preserving until richer block layout can be proven without harming cursor stability, selection stability, typing latency, or scrolling behavior.
## Finding #13 — Rendering vs Syntax Styling
Milestone 3.1 corrects the first Markdown expansion from syntax-colored Markdown into meaning-oriented rendered lines. The editor still keeps one canonical Markdown source buffer in the native text view, but inactive lines now hide more structural syntax instead of only dimming it.
Previous behavior:
- Headings were styled with larger fonts while the leading `#` marker remained visible.
- Bold, italic, and inline-code delimiters remained visible in inactive rendered lines.
- Fenced code blocks colored the fence syntax, but the opening and closing fences still read as source text.
- Task lists were detected by the renderer, but the inactive line did not show a distinct task state treatment.
- Dirty rendering of lines inside fenced code blocks could lose block context when the dirty slice did not include the opening fence.
Corrected behavior:
- Inactive headings hide the heading marker and following separator while preserving heading level, spacing, and hierarchy.
- Inline emphasis and code delimiters are hidden on inactive lines; the content keeps the bold, italic, or monospace treatment.
- Fenced code blocks receive a dedicated monospace block treatment. Fence markers are hidden in rendered mode, and an opening-fence language label remains visible when present.
- Task-list bullets are hidden. The `[x]` or `[ ]` token receives a visible state treatment, and completed task content is struck through.
- Active lines always remain source text, so the same heading, task, code fence, or inline markup becomes editable Markdown when the cursor enters that line.
Render invalidation audit:
Initial rendering was not being skipped. Full renders still style every indexed line, and `EditorDirtyLineInvalidationTests.testInitialRenderTouchesEveryLine` covers that contract. The most visible heading case is the expected hybrid behavior when the heading is the active line: active lines intentionally show source. Moving focus away causes the line to become inactive and therefore rendered.
The real invalidation gap was block context for dirty slices. The renderer already understood fenced code blocks during full contiguous rendering, but a dirty update for only a code-content line could be planned without its surrounding fences. The fix adds contextual dirty rendering only when a dirty line is near a fence marker. That path asks `DocumentLineIndex` for enough source context to decide whether the dirty line is inside a fenced block, avoiding a document-wide refresh or unconditional full-source scan.
Rendering architecture:
```mermaid
flowchart TD
Source["Native text storage source"] --> Index["DocumentLineIndex"]
Index --> Dirty["Dirty line invalidation"]
Dirty --> Context{"Fence nearby?"}
Context -->|No| LinePlans["Line-only render plans"]
Context -->|Yes| ContextPlans["Contextual code-block plans"]
LinePlans --> Styler["MarkdownTextStyler"]
ContextPlans --> Styler
Styler --> Hidden["Hidden syntax attributes"]
Styler --> Meaning["Rendered meaning attributes"]
Hidden --> Source
Meaning --> Source
```
The correction is still source-preserving. Hidden syntax uses temporary text attributes, not string replacement. This keeps TextKit source offsets, cursor movement, selections, undo, and persistence aligned with the Markdown file.
Tradeoffs:
- Hidden delimiters still occupy a tiny layout footprint because the backing text remains present. This is intentional until replacement or overlay rendering is proven stable.
- Task lists do not yet replace `[x]` and `[ ]` with true `☑` and `☐` glyphs. AppKit text attachments require replacement characters, which would break the source-offset model. The current rendered mode hides the list marker and styles the task-state token instead.
- Code blocks are rendered as attributed text blocks, not independent block widgets. Padding is represented through paragraph and background styling.
- Long-range dirty code-block context is resolved only when the dirty line is near fence syntax. This keeps normal typing and active-line switching cheap; broader block invalidation can be added later if editing fences inside very long code blocks exposes stale styling.
- Blockquotes, lists, horizontal rules, tables, and links remain attributed renderings rather than custom layout objects.
Validation:
- Focused renderer tests cover heading marker hiding, active-line source preservation, inline delimiter hiding, task-state styling, code-fence hiding, and dirty code-block context recovery.
- Full test suite on May 31, 2026: 58 XCTest cases passed.
- Release benchmark on May 31, 2026:
| Scenario | Lines | Measured total | Render planning | Initial styling | Dirty typing render |
| --- | ---: | ---: | ---: | ---: | ---: |
| sample document | 54 | 13.506 ms | 1.264 ms | 4.733 ms | 0.330 ms |
| 2,100-line prototype | 2,101 | 423.835 ms | 88.610 ms | 98.276 ms | 0.368 ms |
| 5 MB benchmark | 51,482 | 3,275.857 ms | 248.582 ms | 361.581 ms | 1.002 ms |
Tracked 5 MB interaction metrics:
| Operation | Time |
| --- | ---: |
| active-line lookup | 0.001 ms |
| selection update | 0.003 ms |
| dirty click invalidation | 0.001 ms |
| typing state update | 0.116 ms |
| dirty typing invalidation | 0.004 ms |
| dirty typing render | 1.002 ms |
Conclusion:
Milestone 3.1 moves Sapling's inactive-line display closer to rendered Markdown while preserving the validated hybrid editing contract. The renderer now hides the most distracting source syntax for headings, inline markup, and fenced code blocks, and it gives task lists visible state. The remaining limitations are deliberate source-preserving tradeoffs rather than missed parser features.
## Finding #14 — Rendered Document Lifecycle
Milestone 3.2 makes rendered state explicit instead of leaving it implicit inside styling code. Before this change, the native adapter decided whether a line was source or rendered inside `MarkdownTextStyler.apply(...)` by comparing each line index to the active line. That worked, but it left presentation state spread across delegate timing, invalidation caches, and styler branches.
The new lifecycle model is:
- `RenderedLineState.source`: exactly one active line derived from the current native selection and `DocumentLineIndex`.
- `RenderedLineState.rendered`: every other visible line.
- `DocumentPresentationState`: deterministic projection from `(DocumentLineIndex, activeLineIndex, optional dirty line indexes)` to presentation lines and semantic rendered elements.
- `DocumentPresentationLine`: one source line plus its line state, render plan, and semantic rendered elements.
Every line in a presentation pass is now in exactly one state. The state is data-driven by current document text and current selection, not by previous focus history or click history.
```mermaid
stateDiagram-v2
[*] --> Rendered: line.index != activeLineIndex
[*] --> Source: line.index == activeLineIndex
Rendered --> Source: selection enters line
Source --> Rendered: selection leaves line
Rendered --> Rendered: source edit elsewhere
Source --> Source: source edit on active line
```
Render pipeline after Milestone 3.2:
```mermaid
flowchart TD
NativeSelection["Native text selection"] --> Active["DocumentLineIndex.lineIndex(containing:)"]
Source["Markdown source"] --> LineIndex["DocumentLineIndex"]
LineIndex --> Presentation["DocumentPresentationState"]
Active --> Presentation
Dirty["Dirty line indexes"] --> Presentation
Presentation --> Lines["DocumentPresentationLine"]
Lines --> Styler["MarkdownTextStyler"]
Lines --> Elements["RenderedDocumentElement"]
Elements --> Widgets["Rendered widgets / overlays"]
Styler --> TextStorage["NSTextStorage attributes"]
```
Invalidation audit:
- Initial rendering is still a full presentation pass. `EditorDirtyLineInvalidator` emits `.initial` with every line index, and the presentation model assigns one source line and all remaining lines as rendered.
- Source edits still use dirty-line invalidation and preserve neighbor-line coverage for constructs affected by adjacent text.
- Active-line transitions still dirty only the previous and current active line.
- View updates with unchanged source and unchanged active line still skip styling.
- The important timing fix is that native adapters now derive the active line from the current native text selection immediately before building the invalidation plan. Selection-change styling no longer depends on waiting for SwiftUI binding propagation to update the `activeLineIndex` value.
Rendered element architecture:
`DocumentPresentationState` now emits semantic `RenderedDocumentElement` values for supported constructs:
- `heading`
- `task`
- `codeBlock`
- `link`
- `inlineCode`
- `blockquote`
- `horizontalRule`
- `listItem`
- `tableRow`
Plain paragraphs are intentionally not materialized as semantic elements in normal passes. They are represented by their `DocumentPresentationLine` and render plan, which avoids allocating tens of thousands of no-op paragraph elements in large documents.
Code block lifecycle:
Fenced code blocks are represented as semantic multi-line `RenderedCodeBlockElement` values with:
- all participating line indexes
- the source range spanning the block
- the optional language range
The attributed renderer still styles code blocks line-by-line inside TextKit, but the document presentation model now has a block-level representation that future widgets can consume without reparsing source text.
Tradeoffs:
- Text remains source-preserving. Hidden syntax is still done with attributes, not source replacement.
- The native text view remains the scroll, selection, undo, and IME owner.
- Semantic elements are only emitted for constructs that need rendered behavior or future widget hooks.
- Dirty presentation only resolves code-block context when nearby fence syntax makes block context relevant; this preserves the Milestone 2.9 dirty-render scalability model.
Validation:
- `DocumentPresentationStateTests` verify one state per line, deterministic projection for identical input, semantic heading/task/link elements, semantic code-block state, and dirty code-block context recovery.
- Existing cursor, dirty invalidation, scroll stability, rendering, large-document, and performance tests continue to pass.
Before/after release benchmark comparison:
| Scenario | Milestone 3.1 total | Milestone 3.2 total | Milestone 3.1 dirty typing render | Milestone 3.2 dirty typing render |
| --- | ---: | ---: | ---: | ---: |
| sample document | 13.506 ms | 10.013 ms | 0.330 ms | 0.220 ms |
| 2,100-line prototype | 423.835 ms | 477.423 ms | 0.368 ms | 0.383 ms |
| 5 MB benchmark | 3,275.857 ms | 3,614.818 ms | 1.002 ms | 1.049 ms |
Tracked 5 MB interaction metrics after Milestone 3.2:
| Operation | Time |
| --- | ---: |
| active-line lookup | 0.000 ms |
| selection update | 0.005 ms |
| dirty click invalidation | 0.001 ms |
| typing state update | 0.077 ms |
| dirty typing invalidation | 0.003 ms |
| dirty typing render | 1.049 ms |
Conclusion:
The presentation lifecycle is now explicit and deterministic. Opening a document, moving the cursor, and editing source all project through the same state model: source for the active line, rendered for inactive lines. The measured cost is a modest increase in initial large-document work, while dirty interaction remains within the Milestone 2.9 responsiveness envelope.
## Finding #15 — Interactive Rendered Elements
Milestone 3.2 introduces task lists as the first interactive rendered element. The Markdown source remains authoritative; the checkbox is only a rendered control backed by a known source range.
Checklist architecture on macOS:
```mermaid
sequenceDiagram
participant User
participant Checkbox as Native NSButton checkbox
participant Coordinator as NSTextView Coordinator
participant Source as Markdown source
participant Index as DocumentLineIndex
participant Presentation as DocumentPresentationState
participant Styler as MarkdownTextStyler
User->>Checkbox: click
Checkbox->>Coordinator: toggle RenderedTaskElement
Coordinator->>Source: replace checkbox range with [ ] or [x]
Coordinator->>Index: apply DocumentLineIndexEdit
Coordinator->>Presentation: rebuild affected presentation
Presentation->>Styler: rendered/source line states
Styler->>Checkbox: sync overlay state and position
```
Implementation details:
- Rendered task lines produce `RenderedTaskElement` with marker, checkbox, content, checked state, and nesting level.
- The macOS `NSTextView` adapter creates native `NSButton` checkbox overlays for inactive rendered task lines.
- Clicking the checkbox replaces the Markdown checkbox source range with `[x]` or `[ ]`.
- The edit flows through the same source update, line-index update, dirty invalidation, and restyling path as typing.
- When the line becomes active, it returns to source mode and the overlay is removed for that line.
This is deliberately an overlay, not a text replacement. TextKit remains responsible for the source buffer and selection. The overlay consumes semantic rendered element data and writes back to source through `DocumentLineIndexEdit`.
Current platform scope:
- macOS has the native interactive checklist overlay.
- iOS keeps the attributed task-list fallback for now; it needs a `UITextView` overlay equivalent before task checkboxes are interactive there.
Consistency:
- Headings, inline Markdown, links, task lists, and code blocks now all pass through `DocumentPresentationState`.
- Active lines show source.
- Inactive lines show rendered presentation.
- Widgets are attached to semantic rendered elements, not to ad hoc regex matches in the native view.
Future extensibility:
The rendered element architecture can support images, wiki links, Mermaid, LaTeX, and embeds without fundamental redesign. The evidence is structural:
- The presentation layer already separates semantic element detection from platform rendering.
- Elements carry source ranges, so widgets can update or inspect Markdown while preserving source as truth.
- Code blocks already prove multi-line semantic elements can coexist with line-level dirty styling.
- Checklist overlays prove rendered controls can live above the source-preserving text view and write back through normal edit/invalidation paths.
Future rich elements will still need feature-specific parsers, layout, accessibility, hit testing, and caching. Those are additions to the rendered element set and platform widget layer, not a replacement of the hybrid editor architecture.
## Finding #16 — Rendering Determinism
Milestone 3.3 separates the pure document render model from the editor presentation overlay. The previous lifecycle was explicit enough to decide source versus rendered lines, but it still lacked a testable model for the statement "same Markdown produces same rendered document." That made interaction-related regressions hard to isolate from real Markdown rendering changes.
Root causes found:
- Initial heading correctness depended on presentation passes being exercised through the editor path. There was no independent render model that could prove headings existed in the rendered document before focus, click, or selection changes.
- Rendered output and active-line presentation were easy to conflate. `DocumentPresentationState` intentionally depends on the active line because hybrid editing needs one source line, but the rendered document should not depend on active-line history.
- Render validation compared presentation behavior, not semantic render output. It could prove "line 2 is source" but not "this document deterministically contains heading, task, link, and code-block render nodes."
The corrected model is:
```mermaid
flowchart LR
Markdown["Markdown source"] --> Index["DocumentLineIndex"]
Index --> Model["DocumentRenderModel"]
Model --> Nodes["RenderNode values"]
Model --> Snapshot["RenderSnapshot"]
Index --> Presentation["DocumentPresentationState"]
Selection["Current editor selection"] --> Presentation
Presentation --> TextKit["Hybrid NSTextView presentation"]
```
`DocumentRenderModel` is the pure path. It receives a `DocumentLineIndex`, renders every selected line with no active line, and emits:
- `RenderedLine` values containing the source line, render plan, and semantic render nodes for that line.
- `RenderNode` values wrapping `RenderedDocumentElement` instances such as headings, tasks, links, inline code, blockquotes, list items, table rows, and code blocks.
- `RenderSnapshot`, a stable text signature of line kinds, spans, semantic nodes, source ranges, and whole-tree nodes.
`DocumentPresentationState` remains the editor path. It may depend on the current selection because active-line source mode is intentional. That interaction state is now outside the pure render model.
Determinism guarantees:
- Same Markdown source produces the same `DocumentRenderModel.snapshot`.
- Heading render nodes are produced on initial model construction without requiring focus, click, scroll, or selection changes.
- Snapshot changes are tied to Markdown changes, such as toggling `[ ]` to `[x]`.
- Dirty-line invalidation remains scoped to nearby affected lines; the pure render model does not add document-wide work to normal typing.
Validation added:
- `testDocumentRenderModelIsIndependentOfActiveLinePresentation`
- `testInitialRenderModelContainsHeadingsWithoutInteraction`
- `testRenderSnapshotChangesOnlyWhenMarkdownChanges`
- `testRenderedTaskToggleReplacementDoesNotMoveSourceRanges`
Release benchmark after Milestone 3.3:
| Scenario | Total | Dirty typing render | Dirty lines |
| --- | ---: | ---: | ---: |
| sample document | 11.207 ms | 0.268 ms | 3 |
| 2,100-line prototype | 426.829 ms | 0.357 ms | 3 |
| 5 MB benchmark | 3,154.725 ms | 0.888 ms | 3 |
The interaction path remains inside the Milestone 2.9/3.2 responsiveness envelope. The 5 MB benchmark still performs dirty typing render under 1 ms and styles only three lines.
Remaining nondeterminism boundary:
- Semantic rendering is deterministic and covered by snapshots.
- Hybrid presentation intentionally changes with the current selection because the active line must show Markdown source.
- Pixel-level TextKit layout and overlay geometry are not yet covered by automated screenshot comparison. They are constrained by source ranges and native layout APIs, but not proven by a visual snapshot test.
- iOS still uses the attributed checklist fallback; the interactive overlay is macOS-only for now.
Conclusion:
The renderer now has a deterministic semantic model. The editor presentation is a deterministic projection of document state plus the current active-line selection, which is the intended hybrid editing exception.
## Finding #17 — Interactive Widget Stability
Milestone 3.3 stabilizes checklist widgets so they behave as document interactions instead of editing interactions.
The observed checklist bug was:
```text
Before click:
☐ Move with arrow keys.
After click:
☐
Move with arrow keys.
```
Root cause:
The checkbox toggle path replaced the Markdown checkbox and then moved the text selection to the checkbox range. That made the task line become the active source line after the click. Because active lines show Markdown source and inactive lines show rendered content, the click changed line state and layout geometry even though the user only intended to toggle task state.
The corrected interaction is:
```mermaid
sequenceDiagram
participant User
participant Checkbox as Checklist overlay
participant TextView as NSTextView
participant Source as Markdown source
participant Index as DocumentLineIndex
participant Renderer as Presentation renderer
User->>Checkbox: click
Checkbox->>TextView: preserve current selection and focus state
TextView->>Source: replace [ ] with [x] or [x] with [ ]
TextView->>Index: apply same-length DocumentLineIndexEdit
TextView->>Renderer: rerender affected task line
Renderer->>Checkbox: update checkbox state at same source range
TextView->>TextView: restore previous selection and focus
```
Checklist stability guarantees:
- Clicking a rendered checkbox updates only the Markdown checkbox marker.
- The replacement is same-length (`[ ]` and `[x]`), so source ranges and layout anchors remain stable.
- The previous selection is restored after the programmatic edit.
- The text view regains first responder only when it already had focus before the click.
- The native checkbox refuses first responder, preventing the control from stealing editor focus.
- `pendingEdit` is bypassed during the programmatic toggle so the line index receives exactly one explicit `DocumentLineIndexEdit`.
This preserves the active editing line. A user can edit one paragraph, click a checklist elsewhere, and continue editing from the same selection.
Paragraph stability:
The paragraph reflow symptom was caused by widget clicks changing the active line, not by different paragraph Markdown. Once checkbox clicks preserve selection and active line, unchanged paragraphs continue through the same rendered/source state and therefore keep the same wrapping, alignment, and geometry.
Tradeoffs:
- The widget remains an overlay rather than a replacement inside the text buffer. This preserves native source selection, undo integration, IME behavior, and dirty-line invalidation.
- The checkbox toggle still modifies Markdown source immediately; undo semantics should continue through the native text system, but richer widget undo behavior needs manual testing as more widgets are added.
- Visual stability is constrained by same-length source edits and range-based overlay placement. Pixel-level regression coverage remains future work.
Applicability to future widgets:
The same interaction contract should apply to images, wiki links, embeds, Mermaid, and LaTeX:
- Render from semantic nodes.
- Keep Markdown as source of truth.
- Route widget actions through explicit source edits.
- Preserve selection, active line, and focus unless the user explicitly begins editing that element.
- Rerender only affected source ranges.
This means future rendered widgets can be added without changing the fundamental editor architecture.
## Finding #18 — Presentation Determinism
Milestone 3.4 moves determinism down from the semantic render model into the TextKit presentation layer. The render model was already stable after Milestone 3.3, but the visible editor could still diverge because focus state, stale paragraph attributes, and checklist placeholder geometry affected how the same model was materialized.
Presentation pipeline:
```mermaid
flowchart TD
Model["DocumentRenderModel / DocumentPresentationState"] --> Styler["MarkdownTextStyler"]
Styler --> Storage["NSTextStorage attributed runs"]
Storage --> Layout["NSLayoutManager + NSTextContainer"]
Layout --> Geometry["Line fragments, wrapping, baselines"]
Model --> Widgets["Rendered element overlays"]
Widgets --> Controls["Checklist NSButton controls"]
Geometry --> View["NSTextView visual output"]
Controls --> View
```
Layer audit:
| Layer | Inputs | Outputs | Cached state | Invalidation |
| --- | --- | --- | --- | --- |
| `DocumentRenderModel` | Markdown source, line index | Semantic nodes and render snapshot | None | Recreated from document state |
| `DocumentPresentationState` | Line index, effective active line | Source/rendered line states and semantic elements | None | Recreated for full or dirty line set |
| `MarkdownTextStyler` | Presentation state, dirty plan, colors | `NSTextStorage` attributes | None | Full render or dirty lines |
| TextKit layout | Attributed storage, container width | Glyphs, line fragments, wrapping | TextKit glyph/layout caches | Invalidated by attributed edits |
| Checklist overlays | Rendered task elements, source ranges | Native checkbox frames | Existing buttons keyed by line index | Full rebuild or dirty task lines |
Root causes found:
- Initial heading source display was caused by treating the text view selection as active even when the editor had not been focused by the user. Opening a document selected offset 0, so the first heading was presented as source.
- Paragraph reflow after interaction was caused by dirty styling resetting only `EditorLine.contentRange`. The newline character kept its previous paragraph style, so TextKit could lay out the same paragraph differently after active-line transitions.
- Checklist spacing could shift because the checkbox source marker was fully collapsed to a 0.1 pt hidden font while a native overlay was positioned over that range. After interaction, overlay state and collapsed placeholder geometry were not a stable visual contract.
Corrections:
- The macOS and iOS adapters no longer auto-focus the editor on document open.
- Presentation active line is now `-1` while the native text view is not first responder, so initial document load renders all lines.
- Focus begin/end events explicitly restyle the presentation, making the source/rendered transition deterministic when the user enters or leaves the editor.
- Dirty styling now resets and applies paragraph-level attributes over the full presentation paragraph range, including line endings.
- Rendered checklist controls preserve layout by hiding checkbox source text with transparent text while retaining a fixed layout placeholder instead of collapsing it completely.
Presentation snapshot validation:
`MarkdownPresentationSnapshot` captures:
- attributed runs
- font identity and point size
- hidden syntax state
- foreground/background alpha
- paragraph style values
- TextKit line fragment geometry for a fixed text container
Added tests:
- `testInitialUnfocusedPresentationRendersHeading`
- `testDirtyActiveLineRoundTripMatchesFreshPresentation`
- `testChecklistContentGeometryIsStableAcrossToggleState`
- `testParagraphGeometryIsStableAcrossRepeatedPresentation`
These tests verify that unfocused initial presentation renders headings, dirty active-line transitions can return to the exact fresh presentation signature, checklist content origin does not shift between `[ ]` and `[x]`, and repeated paragraph presentation produces identical geometry.
Release benchmark after Milestone 3.4:
| Scenario | Total | Dirty typing render | Dirty lines |
| --- | ---: | ---: | ---: |
| sample document | 11.544 ms | 0.189 ms | 3 |
| 2,100-line prototype | 399.498 ms | 0.337 ms | 3 |
| 5 MB benchmark | 3,034.613 ms | 0.863 ms | 3 |
The paragraph-range fix does not regress scalability. The 5 MB dirty typing path remains under 1 ms and still styles only three lines.
## Finding #19 — Render Model vs Presentation Layer
Milestone 3.4 clarifies the boundary between render determinism and presentation determinism.
The render model answers:
```mermaid
flowchart LR
Markdown["Markdown source"] --> RenderModel["DocumentRenderModel"]
RenderModel --> Snapshot["RenderSnapshot"]
```
The presentation layer answers:
```mermaid
flowchart LR
RenderModel["Render model + active-line state"] --> Attributes["Attributed presentation"]
Attributes --> TextKit["TextKit geometry"]
TextKit --> Overlays["Rendered widget overlays"]
Overlays --> Visual["Visual editor output"]
```
Important distinction:
- `DocumentRenderModel` is independent of interaction state.
- `DocumentPresentationState` intentionally includes the current active line when the editor is focused.
- The native presentation must be deterministic for the same Markdown, same focus state, same active line, and same container width.
The active line is now an explicit presentation input, not leaked history:
- Unfocused editor: active line `-1`, all lines rendered.
- Focused editor: selected line is source, inactive lines rendered.
- Focus lost: active line returns to `-1`, all lines rendered.
This preserves D-008 hybrid editing while avoiding the previous initial-load ambiguity.
Invalidation behavior:
- Initial document load performs a full render with no active line.
- Focus changes dirty only the previous and current active lines through the existing active-line invalidation path.
- Typing still uses `DocumentLineIndexEdit` and dirty neighbor lines.
- Checklist toggles update the Markdown checkbox range and rerender the affected task line while preserving selection and focus.
- Scrolling does not trigger rendering; it uses TextKit layout caches for visible geometry.
Remaining boundary:
Presentation determinism is now covered at the attributed-string and TextKit line-fragment level for fixed-width snapshots. It is not yet covered by end-to-end pixel screenshots of the live `NSTextView` inside a real window. The remaining possible nondeterminism is therefore platform drawing outside the attributed/layout model, such as AppKit control rasterization or OS-level antialiasing. That is a narrower visual QA problem, not a render or presentation state-model problem.
## Finding #20 — Live Editor Presentation Reproduction
Milestone 3.5 invalidated the strongest claim from Milestone 3.4: synthetic presentation snapshots were not enough. They exercised `MarkdownTextStyler` and TextKit layout, but they did not reproduce the live SwiftUI/AppKit bridge lifecycle used by `HybridMarkdownEditor`.
Live path audited:
```mermaid
flowchart TD
Document["MarkdownDocument"] --> ViewModel["HybridMarkdownEditorViewModel"]
ViewModel --> SwiftUI["HybridMarkdownEditor body"]
SwiftUI --> Representable["NativeMarkdownTextView"]
Representable --> TextView["EditorTextView / NSTextView"]
TextView --> Coordinator["NativeMarkdownTextView.Coordinator"]
Coordinator --> Styler["MarkdownTextStyler.apply"]
Styler --> Storage["NSTextStorage"]
Storage --> TextKit["NSLayoutManager / NSTextContainer"]
Coordinator --> Widgets["ChecklistOverlayButton sync"]
TextKit --> Visible["Visible editor layout"]
Widgets --> Visible
```
The divergence was:
- Milestone 3.4 tests built attributed storage directly.
- The benchmark also calls the styler directly.
- The real app installs a native text view, delegate callbacks, first-responder state, selection state, overlay controls, and SwiftUI update cycles.
Reproduction harness:
`HybridMarkdownLiveEditorHarness` now instantiates the real macOS editor pieces in debug builds:
- `EditorTextView`
- `NativeMarkdownTextView.Coordinator`
- `ComfortableEditorScrollView`
- `NSWindow`
- real `NSTextStorage`
- real checklist overlay buttons
- real coordinator calls for focus, selection, styling, and checkbox toggles
The first live regression test reproduced the heading bug before the fix:
```text
Live editor harness
-> assign document source
-> simulate app launch first responder
-> heading marker remained visible
```
Root cause:
The macOS live path installed the `NSTextView` delegate before assigning the initial document string. That allowed initial programmatic text loading to flow through live delegate behavior and mark the editor as if the user had activated editing. When the text view became first responder during launch, the default selection at offset 0 became an active source line, so the first heading appeared as raw Markdown.
Fix:
- Assign the initial string before installing the native text view delegate on macOS and iOS.
- Track user editing activation separately from first-responder state.
- First responder alone no longer creates an active source line.
- Mouse, keyboard, paste, and real text changes activate source presentation.
- Programmatic launch focus keeps all lines rendered.
Live regression tests:
- `testLiveInitialFirstResponderDoesNotShowHeadingSourceBeforeUserInteraction`
- `testLiveParagraphGeometryReturnsAfterClickAndFocusAway`
- `testLiveCheckboxClickPreservesSelectionActiveLineAndGeometry`
These tests use the live editor path, not the pure render model and not a standalone attributed-string snapshot.
Checkbox contract verified by the live harness:
- Clicking a rendered checkbox toggles Markdown source from `[ ]` to `[x]`.
- Selection remains unchanged.
- The active editing line remains unchanged.
- Checkbox label geometry remains unchanged.
- Checkbox overlay frame remains unchanged.
Manual verification checklist:
1. Launch Sapling.
2. Inspect the initially loaded Markdown document before clicking the editor.
3. Verify headings render visually and do not show leading `#` markers.
4. Click a paragraph containing bold, italic, inline code, and a link.
5. Move focus away from the paragraph.
6. Verify paragraph wrapping, spacing, and alignment return to the original rendered geometry.
7. Put the cursor on a different line from a task item.
8. Click a rendered task checkbox.
9. Verify only the checkbox state changes.
10. Verify checkbox-to-label spacing remains stable.
11. Verify the cursor remains on the original editing line.
12. Verify the task line does not become the active source line.
Current conclusion:
The live editor path now has regression coverage for the bugs that escaped Milestone 3.4. Future presentation claims should be based on live harness tests or manual app verification, not only semantic render snapshots or direct styler tests.
## AttributedString and NSAttributedString
Swift `AttributedString` is useful for renderer-facing APIs and SwiftUI previews.
`NSAttributedString` and `NSTextStorage` are still required for the actual native editor bridge because the text view operates on TextKit types.
Recommended split:
- Use `AttributedString` in cross-platform renderer interfaces.
- Use `NSAttributedString` and `NSTextStorage` inside platform adapters.
- Keep conversion and styling code contained in `SaplingEditor`.
## Cursor Challenges
The prototype validates basic active-line detection:
- The selected range location maps to a line index.
- The active line switches as the cursor moves.
- The active line can be styled as source while other lines are styled as rendered.
Open challenges:
- Multi-line selection across source and rendered lines.
- Selection after visually hidden or replaced Markdown delimiters.
- Cursor placement if future phases hide links, list markers, task markers, or other delimiters.
- IME composition and marked text handling.
- Very large documents where full state line-model rebuilding on each source change is too expensive.
Cursor correctness is the main reason to keep the canonical source text inside `NSTextView` for now.
## Rendering Challenges
The current spike uses line-level styling, not full Markdown rendering.
Validated:
- Headings can be styled by line.
- Bold, italic, and inline code can be styled inline.
- Unsupported constructs can remain plain source text without breaking the editor model.
- Active source line can remain monospaced and visibly distinct.
Not validated:
- True delimiter removal without cursor side effects.
- Embedded rendered blocks inside the text layout.
- Images, tables, code block widgets, Mermaid, LaTeX, and attachments.
- Precise layout matching between rendered inactive lines and raw active lines.
## Hybrid Editing Feasibility
Hybrid editing is achievable for Milestone 2 when the first implementation is scoped to line-level attributed styling.
The safest path is:
1. Keep the Markdown source as the only editable text buffer.
2. Detect the active line from the native selection range.
3. Apply source styling to the active line.
4. Apply rendered-style attributes to inactive lines.
5. Delay true line replacement and overlay rendering until selection behavior is better understood.
This does not produce perfect rendered Markdown, but it validates the writing feel and cursor behavior before Sapling invests in a custom editor engine.
## Recommended Path Forward
Continue with `NSTextView` for the next editor milestone on macOS and keep the iOS `UITextView` adapter available behind the same editor abstraction.
Immediate next steps:
- Capture native edited ranges so source updates can avoid rebuilding the full line model.
- Manually test undo/redo, IME input, and keyboard navigation with the stress prototype files.
- Add visible AppKit scroll and typing benchmarks for the 1,000/5,000/10,000-line documents.
- Explore delimiter hiding with temporary attributes only after selection mapping is robust.
Future investigation:
- Use TextKit layout fragments or overlay views for rich block rendering.
- Prototype image and code block rendering outside the text buffer.
- Reevaluate whether a custom editor engine is required after Milestone 2.
## Decision
`NSTextView` can continue to support Sapling's roadmap through the next phase.
Evidence:
- The active-line source and inactive-line presentation model works while preserving a single source buffer.
- Cursor and selection ranges remain native source ranges because rendering does not replace text.
- Scroll position can be preserved around rendering updates.
- Redundant render passes are avoided when source text and active line are unchanged.
- Dirty-line invalidation avoids full attributed-string rebuilds for user edits and active-line transitions.
- The 1,000/5,000/10,000-line stress documents are tractable in automated model-level validation, with 10,000-line open proxy time measured at 17.642 ms.
Sapling should not begin a custom editor engine now. The evidence supports continuing with native text systems while measuring state-line rebuilding and richer overlay experiments.
A future custom editor engine may still be required for high-fidelity block rendering, especially images, tables, Mermaid, LaTeX, attachments, and fully hidden delimiters. That decision should be made only after overlay rendering and native edited-range state updates have been measured against real documents.
The architecture should continue with a SwiftUI shell, a Sapling editor abstraction, and native platform text views hidden behind replaceable adapters.
Reference in a new issue View git blame Copy permalink