Sapling/Docs/editor-investigation.md

# Editor Investigation

Date: 2026-05

Status: Milestone 2 validation

Related decisions:

- D-008: Hybrid Markdown Editing Is a Core Feature
- D-009: The Editor Is the Highest-Priority System
- D-010: Git Access Must Be Abstracted
- D-012: Platform Focus
- D-013: Editor Technology Selection

## Question

Can Sapling support a high-quality hybrid Markdown editing experience where the active line shows Markdown source and inactive lines show rendered Markdown?

## Prototype Summary

The Milestone 1 prototype uses a SwiftUI application shell with a native text view bridge:

- macOS: `NSTextView` through `NSViewRepresentable`
- iOS path: `UITextView` through `UIViewRepresentable`
- shared editor state in `SaplingEditor`
- document loading and saving through `HybridMarkdownEditorViewModel`
- line tracking through `EditorState`, `EditorLine`, and `EditorSelection`
- in-place line styling through `NSTextStorage`

The prototype validates that native text views can support the first version of Sapling's editor without coupling the app directly to AppKit.

## Current Architecture

```mermaid
flowchart TD
    App["SwiftUI app shell"] --> Editor["HybridMarkdownEditor"]
    Editor --> Bridge["Native text view bridge"]
    Bridge --> Mac["NSTextView on macOS"]
    Bridge --> IOS["UITextView on iOS"]
    Editor --> Coordinator["EditorCoordinator"]
    Coordinator --> State["EditorState"]
    State --> Document["EditorDocument"]
    State --> Lines["EditorLine array"]
    State --> Selection["EditorSelection"]
    Bridge --> TextKit["Text storage and selection APIs"]
```

The important boundary is the Sapling editor abstraction. Application code talks to `HybridMarkdownEditorViewModel` and editor state, not to `NSTextView`.

## Architectures Explored

### SwiftUI TextEditor

SwiftUI `TextEditor` was already present in the early prototype and worked for simple active-line editing.

Benefits:

- Minimal SwiftUI integration.
- Fast to prototype.
- Good enough for plain text entry.

Limitations:

- Does not expose reliable low-level selection and cursor APIs.
- Does not provide direct TextKit access.
- Makes line-level source/render switching difficult.
- Offers limited control over attributed rendering and layout.

Conclusion: `TextEditor` is not suitable as Sapling's primary editor implementation.

### NSTextView With Attributed Styling

The implemented spike keeps Markdown source as the actual text buffer and applies visual styling to inactive lines with `NSTextStorage`.

Benefits:

- Native cursor movement remains intact.
- Native selection ranges remain tied to the real source buffer.
- Undo, find, spell checking, and keyboard navigation come from AppKit.
- Active line detection can be derived from selection location.
- Inactive lines can be styled differently from the active source line.

Limitations:

- Styling alone cannot remove Markdown delimiters without affecting layout.
- It cannot render block elements such as images, tables, or Mermaid diagrams as real embedded views.
- Changing font sizes per line affects line height and may create visual jumps.
- Reapplying attributes on every selection change must be optimized for large documents.

Conclusion: This is sufficient for Milestone 1 and likely sufficient for an early Milestone 2 implementation of headings, emphasis, links, lists, and task lists.

### Line Replacement

Line replacement would swap inactive Markdown source with rendered text while preserving the source elsewhere.

Benefits:

- More closely matches the desired hybrid model.
- Can hide Markdown delimiters from inactive lines.

Risks:

- Cursor offsets no longer map directly to source offsets.
- Selection crossing active and inactive lines becomes complex.
- Undo behavior may become fragile.
- TextKit may fight source/render buffer divergence.

Conclusion: Worth exploring after the attributed-string path, but risky as the primary approach.

### Overlay Rendering

Overlay rendering would keep source text in the editor but draw rendered content above inactive lines.

Benefits:

- Source buffer remains canonical.
- Rendered views could support richer blocks.
- Cursor math can still use the real text buffer.

Risks:

- Requires precise layout synchronization with TextKit.
- Hit testing and selection feedback are difficult.
- Accessibility must be designed carefully.

Conclusion: Promising for richer Milestone 3 rendering, but heavier than needed for Milestone 2.

### Mixed Text and Render Layers

A custom layout could combine text editing lines with rendered SwiftUI/AppKit views.

Benefits:

- Maximum control over rendering.
- Can support images, diagrams, tables, and custom blocks.

Risks:

- Requires building editor infrastructure Sapling does not yet have.
- Cursor movement, selection, IME, undo, accessibility, and text input become product-critical systems.

Conclusion: This may be necessary in the long term, but it should not be the first Milestone 2 implementation.

## TextKit Findings

`NSTextView` gives Sapling the hooks needed for the prototype:

- `selectedRange()` gives the cursor and selection range.
- `NSTextStorage` allows line-level attributes without replacing source text.
- `NSTextViewDelegate` reports text and selection changes.
- Native editing behavior remains available while the app tracks editor state.

The key finding is that Sapling can preserve a single source buffer and still present different active and inactive line states. This reduces risk compared with maintaining separate source and render buffers.

## Finding #1 — Selection Feedback Loop

Root cause:

The first `NSTextView` bridge treated every selection notification as user intent, even when the selection change was caused by Sapling itself while restoring selection after an attribute pass.

The recursive path was:

```text
textViewDidChangeSelection()
-> selection binding update
-> EditorState update
-> @Published state refresh
-> SwiftUI updateNSView()
-> applyHybridAttributes()
-> setSelectedRange()
-> textViewDidChangeSelection()
```

This created an unbounded feedback loop and could crash immediately after launch while the hybrid styling pass and SwiftUI refresh cycle bounced selection updates back and forth.

Architecture impact:

- Selection is not just editor state; it is also a native text system side effect.
- Attribute rendering can produce selection notifications even when the user's logical selection did not change.
- SwiftUI refreshes must not be allowed to feed native programmatic changes back into `EditorState` as if they were user edits.
- Hybrid editing remains viable, but the platform adapter must own reentrancy control.

Chosen solution:

- Programmatic text, selection, and attribute changes are wrapped in a coordinator transaction.
- Selection delegate callbacks are ignored while that transaction is active.
- `EditorCoordinator.updateSelection` and `updateSource` ignore no-op updates before publishing.
- Attribute passes restore selection only if the text view actually changed it.
- The bridge skips redundant restyling when both source text and active line are unchanged.

This keeps the canonical source and selection model in `EditorState`, while making the native adapter responsible for distinguishing user-originated changes from TextKit side effects.

Alternative solutions considered:

- Temporary delegate removal: workable, but broader than needed and easy to forget when adding new native callbacks.
- Debounced rendering: useful for future performance work, but it would only delay the recursion instead of fixing the feedback path.
- Never restoring selection after attributes: avoids this crash, but risks cursor drift if TextKit changes selection during editing.
- Moving selection outside published editor state: reduces SwiftUI refresh pressure, but weakens the architecture because active-line rendering depends on selection.

Result:

The app now survives launch, typing, arrow-key movement, and select-all smoke testing without re-entering the selection loop.

## Finding #2 — Writing Comfort Matters More Than Features

Root cause:

After the selection recursion fix, the editor was technically functional but still uncomfortable. The three-column layout left the editor fighting for space, the inspector panel kept Git-oriented information visible during writing, and the text column was too close to the window edges.

Milestone 1 needs to validate whether a person wants to keep writing in Sapling. That requires layout and typography work even before advanced Markdown behavior exists.

Layout experiments:

- The three-column split view made the editor feel secondary.
- Removing the inspector from the default writing surface immediately improved focus.
- A two-column sidebar/detail split gives the editor roughly 70-80% of the default window.
- The sidebar remains collapsible for focused writing.
- The editor should be the detail column, not the middle column in a sidebar/content/inspector arrangement.

Sidebar impact:

The workspace sidebar is useful for opening documents, but it should not dominate Milestone 1. Git status and project inspection are distractions during editor validation and should stay hidden until later milestones.

Cursor behavior:

Native `NSTextView` keyboard behavior remains the right default. Arrow keys, Shift-selection, Option-arrow word navigation, Home, End, Page Up, and Page Down should be allowed to flow through AppKit instead of being reimplemented in SwiftUI.

Smoke testing after the layout and typography changes covered:

- Basic typing.
- Left and right arrow movement.
- Home and End.
- Page Up and Page Down.
- Option + Arrow movement.
- Shift + Arrow selection.

No selection recursion or immediate cursor instability appeared during the smoke pass.

Typography observations:

- A 14-point editor made the prototype feel cramped.
- A 16-point base font with more line and paragraph spacing is more comfortable for writing.
- A readable-width text column is more important than using all available horizontal space.
- Generous document padding makes the editor feel like a writing surface instead of a debug widget.
- The active-line highlight should stay subtle; it is there to orient the writer, not decorate the page.

Status bar observations:

The status bar should remain quiet. Line number, column number, total line count, and saved/modified state are enough for Milestone 1. Showing rendered previews in the status bar added visual noise without improving writing flow.

Architecture impact:

- Editor usability depends on app layout, not only text view internals.
- The native text view bridge can support comfortable writing if it controls text insets, readable width, and cursor visibility.
- Keyboard navigation should be validated through native behavior first.
- More features should wait until the writing surface itself feels calm.

Result:

The prototype now opens into a focused writing layout with a larger text area, collapsible sidebar, readable line width, generous padding, a subtle current-line treatment, and a realistic sample document for testing.

## Finding #3 — Hybrid Editing Architecture

Milestone 2 keeps one canonical Markdown source buffer inside the native text view. The editor does not maintain a second rendered document, and it does not replace inactive lines with different strings. Active-line source mode and inactive-line rendered mode are presentation states derived from the same source.

```mermaid
flowchart TD
    TextView["NSTextView / UITextView source buffer"] --> Delegate["Native delegate callbacks"]
    Delegate --> Coordinator["Bridge coordinator"]
    Coordinator --> ViewModel["HybridMarkdownEditorViewModel"]
    ViewModel --> State["EditorState"]
    State --> Tracker["EditorActiveLineTracker"]
    Tracker --> Lines["EditorLine ranges and modes"]
    Lines --> Styler["MarkdownTextStyler"]
    Styler --> TextStorage["NSTextStorage attributes"]
    TextStorage --> TextView
    Coordinator --> Metrics["EditorInstrumentationSnapshot"]
```

The active-line model now has a single source of truth:

- `EditorState.selection` stores the current source-buffer selection.
- `EditorActiveLineTracker` maps the selection location to one line index.
- `EditorState.activeLineIndex` is derived from that mapping after every source or selection update.
- `EditorLine.mode` is rebuilt from the active line and source ranges.
- The native bridge only applies attributes from that state; it does not decide which line is active.

This avoids recursive update loops because the native adapter still owns programmatic-update suppression. Text, selection, and attribute changes caused by Sapling are wrapped in coordinator transactions and ignored by delegate callbacks. User-originated changes flow back to the view model only when the source or selection actually changed.

```mermaid
sequenceDiagram
    participant User
    participant Native as Native text view
    participant Bridge as Coordinator
    participant State as EditorState
    participant Render as Attribute styler

    User->>Native: Move cursor or edit text
    Native->>Bridge: Delegate notification
    Bridge->>State: Update source or selection
    State->>State: Derive activeLineIndex
    State->>Render: Apply attributes for active/inactive lines
    Render->>Native: Preserve selection and visible origin
```

Implementation scope is intentionally minimal:

- Headings are visually emphasized on inactive lines.
- Bold, italic, and inline code receive inline attributes.
- Markdown delimiters are deemphasized, not hidden.
- Links, tasks, lists, code blocks, tables, images, Mermaid, LaTeX, and attachments are not promoted in the hybrid renderer for this phase.

Cursor and selection behavior remain stable because the visible text length is still the source text length. Delimiters are styled but not removed, so source offsets, native selection ranges, undo grouping, and TextKit cursor movement continue to share the same coordinate system.

## Finding #4 — Rendering Strategy Evaluation

Milestone 2 compared four rendering strategies against the current product risk: editing correctness is more important than rendering quality.

| Approach | Complexity | Performance | Cursor stability | Maintainability | Finding |
| --- | --- | --- | --- | --- | --- |
| A. Attributed-string rendering | Low to medium | O(n) per full attribute pass today; optimizable by dirty ranges | Strong because source offsets are unchanged | Strong while rendering is inline and line-level | Chosen for Milestone 2 |
| B. Overlay rendering | High | Potentially good with cached layout, but expensive to synchronize | Medium risk from hit testing, selection painting, and accessibility | Medium; requires layout synchronization code | Defer until rich block rendering |
| C. Line replacement | High | Could be efficient per line, but requires source/render mapping | High risk because cursor offsets diverge from visible text | Fragile around undo, IME, and cross-line selection | Not appropriate for current milestone |
| D. Mixed editor/render layers | Very high | Potentially best long term with a custom layout engine | Unknown until Sapling owns text input, IME, selection, undo, accessibility | Expensive; effectively a custom editor engine | Long-term fallback only |

Approach A was selected because it validates Sapling's core hybrid interaction while preserving native text editing. It does not prove that full rendered blocks can live inside `NSTextView`, but it does prove that active-line source editing and inactive-line presentation can coexist without destabilizing cursor and selection behavior.

```mermaid
quadrantChart
    title Rendering Strategy Risk
    x-axis Low cursor stability --> High cursor stability
    y-axis Low implementation complexity --> High implementation complexity
    quadrant-1 "Good validation path"
    quadrant-2 "Powerful but expensive"
    quadrant-3 "Avoid"
    quadrant-4 "Defer"
    "Attributed attributes": [0.82, 0.28]
    "Overlay rendering": [0.55, 0.72]
    "Line replacement": [0.28, 0.68]
    "Mixed custom layers": [0.45, 0.92]
```

The important tradeoff is honest: attributed rendering cannot hide Markdown syntax without leaving visible gaps or breaking cursor math. For Milestone 2, that is acceptable. It is better to keep the editor believable and predictable than to make inactive lines look fully rendered at the cost of offset instability.

## Finding #5 — Performance Characteristics

Instrumentation was added at the editor boundary:

- Source changes are counted in `EditorInstrumentationSnapshot.sourceChangeCount`.
- Selection changes are counted in `selectionChangeCount`.
- Active-line transitions are counted in `activeLineChangeCount`.
- Native attribute passes are counted in `renderPassCount`.
- Each render pass records reason, duration, source character count, line count, and active line index.

The counters are intentionally not `@Published`. They can be inspected by tests, logs, or future debug UI without causing every render pass to trigger another SwiftUI update.

Automated validation added in `SaplingEditorTests` covers:

- Cursor-derived active-line transitions.
- Selection clamping after deletion.
- Trailing blank-line range tracking.
- Heading, bold, italic, and inline-code render plans.
- Links and tasks remaining unsupported in the Milestone 2 renderer.
- A 2,100-line prototype document shape.

Current measured automated results on May 29, 2026:

- `swift test`: 13 tests passed.
- Large render-plan test: 2,100 lines processed in roughly 0.04 to 0.06 seconds during the XCTest run.
- Full suite runtime: roughly 0.05 to 0.07 seconds after build.

Prototype documents were added for repeatable manual validation:

- `Docs/EditorPrototypes/hybrid-small-50.md`
- `Docs/EditorPrototypes/hybrid-medium-500.md`
- `Docs/EditorPrototypes/hybrid-large-2100.md`

Scroll stability work:

- Attribute passes restore the pre-render visible origin.
- Selection restoration after styling does not call scroll-to-visible.
- Explicit programmatic selection changes still scroll the selected range into view.

This means rendering updates should not yank the viewport while the user scrolls or while inactive-line attributes refresh. Native cursor movement still gets to scroll naturally when the insertion point moves outside the visible area.

Known performance limitation:

The current styler still applies a full-buffer attribute pass when the source changes or the active line changes. The pass is guarded by a text/active-line cache, so redundant SwiftUI updates do not restyle, but active-line navigation through a large document is still O(n). This is acceptable for architecture validation but should become dirty-line invalidation before larger Markdown features are added.

Recommended next optimization:

Track the previous active line and new active line, then restyle only those two lines when the source is unchanged. Source edits can be narrowed to changed line ranges after text-diffing or native edited-range callbacks are introduced.

## AttributedString and NSAttributedString

Swift `AttributedString` is useful for renderer-facing APIs and SwiftUI previews.

`NSAttributedString` and `NSTextStorage` are still required for the actual native editor bridge because the text view operates on TextKit types.

Recommended split:

- Use `AttributedString` in cross-platform renderer interfaces.
- Use `NSAttributedString` and `NSTextStorage` inside platform adapters.
- Keep conversion and styling code contained in `SaplingEditor`.

## Cursor Challenges

The prototype validates basic active-line detection:

- The selected range location maps to a line index.
- The active line switches as the cursor moves.
- The active line can be styled as source while other lines are styled as rendered.

Open challenges:

- Multi-line selection across source and rendered lines.
- Selection after visually hidden or replaced Markdown delimiters.
- Cursor placement if future phases hide links, list markers, task markers, or other delimiters.
- IME composition and marked text handling.
- Very large documents where full restyling on each selection change is too expensive.

Cursor correctness is the main reason to keep the canonical source text inside `NSTextView` for now.

## Rendering Challenges

The current spike uses line-level styling, not full Markdown rendering.

Validated:

- Headings can be styled by line.
- Bold, italic, and inline code can be styled inline.
- Unsupported constructs can remain plain source text without breaking the editor model.
- Active source line can remain monospaced and visibly distinct.

Not validated:

- True delimiter removal without cursor side effects.
- Embedded rendered blocks inside the text layout.
- Images, tables, code block widgets, Mermaid, LaTeX, and attachments.
- Precise layout matching between rendered inactive lines and raw active lines.

## Hybrid Editing Feasibility

Hybrid editing is achievable for Milestone 2 when the first implementation is scoped to line-level attributed styling.

The safest path is:

1. Keep the Markdown source as the only editable text buffer.
2. Detect the active line from the native selection range.
3. Apply source styling to the active line.
4. Apply rendered-style attributes to inactive lines.
5. Delay true line replacement and overlay rendering until selection behavior is better understood.

This does not produce perfect rendered Markdown, but it validates the writing feel and cursor behavior before Sapling invests in a custom editor engine.

## Recommended Path Forward

Continue with `NSTextView` for the next editor milestone on macOS and keep the iOS `UITextView` adapter available behind the same editor abstraction.

Immediate next steps:

- Add incremental line invalidation instead of restyling the whole buffer.
- Add automated coverage for multi-line selections and edited-range invalidation.
- Manually test long documents, undo/redo, IME input, and keyboard navigation with the prototype files.
- Explore delimiter hiding with temporary attributes only after selection mapping is robust.

Future investigation:

- Use TextKit layout fragments or overlay views for rich block rendering.
- Prototype image and code block rendering outside the text buffer.
- Reevaluate whether a custom editor engine is required after Milestone 2.

## Decision

`NSTextView` can continue to support Sapling's roadmap through the next phase.

Evidence:

- The active-line source and inactive-line presentation model works while preserving a single source buffer.
- Cursor and selection ranges remain native source ranges because rendering does not replace text.
- Scroll position can be preserved around rendering updates.
- Redundant render passes are avoided when source text and active line are unchanged.
- The large 2,100-line prototype shape is tractable for render planning, though the native attribute pass still needs dirty-line invalidation.

Sapling should not begin a custom editor engine now. The evidence supports continuing with native text systems while moving toward incremental invalidation and richer overlay experiments.

A future custom editor engine may still be required for high-fidelity block rendering, especially images, tables, Mermaid, LaTeX, attachments, and fully hidden delimiters. That decision should be made only after overlay rendering and dirty-line attributed rendering have been measured against real documents.

The architecture should continue with a SwiftUI shell, a Sapling editor abstraction, and native platform text views hidden behind replaceable adapters.