docs(editor): document scalability findings

Docs/EditorPrototypes/README.md

@@ -5,5 +5,8 @@ These documents support Milestone 2 hybrid editor validation.
 - `hybrid-small-50.md`: quick cursor, selection, and active-line checks.
 - `hybrid-medium-500.md`: scroll and editing checks across a realistic session-sized document.
 - `hybrid-large-2100.md`: large document behavior and render-pass frequency checks.
+- `hybrid-stress-1000.md`: Milestone 2.5 stress document for repeatable editor measurements.
+- `hybrid-stress-5000.md`: Milestone 2.5 stress document for repeatable editor measurements.
+- `hybrid-stress-10000.md`: Milestone 2.5 stress document for repeatable editor measurements.
 
 They intentionally use only headings, bold, italic, and inline code because Milestone 2 validates editor architecture before full Markdown rendering.

Docs/editor-investigation.md

@@ -318,7 +318,7 @@ Milestone 2 compared four rendering strategies against the current product risk:
 
 | Approach | Complexity | Performance | Cursor stability | Maintainability | Finding |
 | --- | --- | --- | --- | --- | --- |
-| A. Attributed-string rendering | Low to medium | O(n) per full attribute pass today; optimizable by dirty ranges | Strong because source offsets are unchanged | Strong while rendering is inline and line-level | Chosen for Milestone 2 |
+| A. Attributed-string rendering | Low to medium | Dirty native attribute passes after initial render; state line tracking remains O(n) | Strong because source offsets are unchanged | Strong while rendering is inline and line-level | Chosen for Milestone 2 |
 | B. Overlay rendering | High | Potentially good with cached layout, but expensive to synchronize | Medium risk from hit testing, selection painting, and accessibility | Medium; requires layout synchronization code | Defer until rich block rendering |
 | C. Line replacement | High | Could be efficient per line, but requires source/render mapping | High risk because cursor offsets diverge from visible text | Fragile around undo, IME, and cross-line selection | Not appropriate for current milestone |
 | D. Mixed editor/render layers | Very high | Potentially best long term with a custom layout engine | Unknown until Sapling owns text input, IME, selection, undo, accessibility | Expensive; effectively a custom editor engine | Long-term fallback only |
@@ -350,7 +350,10 @@ Instrumentation was added at the editor boundary:
 - Selection changes are counted in `selectionChangeCount`.
 - Active-line transitions are counted in `activeLineChangeCount`.
 - Native attribute passes are counted in `renderPassCount`.
-- Each render pass records reason, duration, source character count, line count, and active line index.
+- Full native attribute passes are counted in `fullRenderCount`.
+- Dirty lines touched by native rendering are counted in `totalDirtyLineCount` and `lastDirtyLineCount`.
+- Scroll-origin restoration is counted in `scrollRestorationCount`.
+- Each render pass records reason, duration, source character count, line count, dirty line count, active line index, whether it was a full render, and whether scroll position was restored.
 
 The counters are intentionally not `@Published`. They can be inspected by tests, logs, or future debug UI without causing every render pass to trigger another SwiftUI update.
 
@@ -383,13 +386,153 @@ Scroll stability work:
 
 This means rendering updates should not yank the viewport while the user scrolls or while inactive-line attributes refresh. Native cursor movement still gets to scroll naturally when the insertion point moves outside the visible area.
 
-Known performance limitation:
+Resolved performance limitation in Milestone 2.5:
 
-The current styler still applies a full-buffer attribute pass when the source changes or the active line changes. The pass is guarded by a text/active-line cache, so redundant SwiftUI updates do not restyle, but active-line navigation through a large document is still O(n). This is acceptable for architecture validation but should become dirty-line invalidation before larger Markdown features are added.
+The native styler no longer applies a full-buffer attribute pass for every source edit or active-line change. Initial renders and programmatic full text replacements still rebuild the full attributed string. User edits and active-line navigation now produce a dirty-line invalidation plan, and the text storage resets and restyles only those line ranges.
 
-Recommended next optimization:
+Remaining performance limitation:
 
-Track the previous active line and new active line, then restyle only those two lines when the source is unchanged. Source edits can be narrowed to changed line ranges after text-diffing or native edited-range callbacks are introduced.
+`EditorState.updateSource` and dirty-plan construction still derive the full line model from the source string. The expensive native attributed-string rebuild is avoided, but state rebuilding remains O(n). This is visible in the 10,000-line typing proxy measurement below and should be the next measured optimization before richer block rendering.
+
+## Finding #6 — Dirty-Line Rendering
+
+Milestone 2 rendered every line on source changes and active-line changes. The coordinator avoided redundant SwiftUI restyles with a `lastStyledText` and `lastStyledActiveLineIndex` cache, but any actual edit still called `MarkdownTextStyler.apply(...)` over the full `NSTextStorage`.
+
+Milestone 2.5 adds `EditorDirtyLineInvalidator`, which compares the previously styled source and active line against the current source and active line. It emits an `EditorDirtyLineInvalidationPlan` containing:
+
+- render reason
+- full-render flag
+- dirty line indexes
+- changed UTF-16 range for source edits
+
+The native adapter now follows this path:
+
+```mermaid
+flowchart TD
+    Native["NSTextView text storage"] --> Coordinator["Native coordinator"]
+    Coordinator --> Cache["Last styled text + active line"]
+    Cache --> Invalidator["EditorDirtyLineInvalidator"]
+    Invalidator --> Plan["Dirty-line invalidation plan"]
+    Plan --> Styler["MarkdownTextStyler"]
+    Styler --> DirtyRanges["Reset base attributes only on dirty line ranges"]
+    DirtyRanges --> RenderLine["Apply source/rendered styling to dirty lines"]
+    RenderLine --> Metrics["EditorRenderPassMetric"]
+```
+
+Observed behavior from automated tests:
+
+| User action | Render reason | Lines touched |
+| --- | --- | --- |
+| Initial editor load | `initial` | all lines |
+| Move active line from A to B | `activeLineChange` | A and B |
+| Edit one line | `sourceChange` | edited line plus immediate neighbors |
+| Insert a line break | `sourceChange` | split line, new active line, and affected neighbor |
+| Redundant SwiftUI view update | `viewUpdate` | zero; no styling pass |
+
+The neighboring-line policy is intentionally conservative. Current Milestone 2 Markdown spans do not cross line boundaries, but line breaks and future block parsing can make nearby lines visually affected. The policy avoids full-document rendering while keeping enough context for line joins and splits.
+
+## Finding #7 — Large Document Performance
+
+Stress documents were added for repeatable validation:
+
+- `Docs/EditorPrototypes/hybrid-stress-1000.md`
+- `Docs/EditorPrototypes/hybrid-stress-5000.md`
+- `Docs/EditorPrototypes/hybrid-stress-10000.md`
+
+Automated measurement command:
+
+```sh
+SAPLING_EDITOR_PRINT_METRICS=1 swift test --filter EditorLargeDocumentValidationTests/testLargeDocumentOpenAndDirtyRenderPlanning
+```
+
+Measured on May 29, 2026 in a debug SwiftPM test run:
+
+| Lines | Open proxy | Typing proxy | Dirty render-plan proxy | Dirty lines | Full render |
+| ---: | ---: | ---: | ---: | ---: | --- |
+| 1,000 | 1.769 ms | 11.770 ms | 2.804 ms | 3 | no |
+| 5,000 | 9.016 ms | 59.068 ms | 9.348 ms | 3 | no |
+| 10,000 | 17.642 ms | 119.364 ms | 19.115 ms | 3 | no |
+
+What these numbers measure:
+
+- Open proxy: creating `EditorState` and its line model.
+- Typing proxy: updating source, rebuilding editor state, and creating a dirty-line plan.
+- Dirty render-plan proxy: line tracking plus render-plan creation for dirty lines.
+- Render frequency: one source edit schedules one render metric with three dirty lines and `isFullRender == false`.
+
+What these numbers do not measure:
+
+- Real AppKit layout time in a visible window.
+- GPU compositing or actual scroll wheel latency.
+- IME composition behavior.
+
+Evidence:
+
+- 10,000-line documents remain tractable for state creation in debug tests.
+- Native attributed-string work is now bounded to three dirty lines for a single-line edit.
+- The remaining typing proxy cost scales linearly because the state model still rebuilds all lines.
+- Scroll stability is covered by repeatable instrumentation tests: dirty render passes restore the captured scroll origin/content offset and increment `scrollRestorationCount`.
+
+## Finding #8 — Overlay Architecture Exploration
+
+No overlay renderer was implemented. The spike evaluated how it would fit beside the current attributed-string path.
+
+Approach A: attributed-string rendering keeps all presentation inside TextKit.
+
+```mermaid
+flowchart LR
+    Source["Markdown source string"] --> TextStorage["NSTextStorage"]
+    TextStorage --> Attributes["Line + inline attributes"]
+    Attributes --> TextView["NSTextView display, cursor, selection"]
+```
+
+Approach B: overlay rendering keeps the source editor native and draws rendered inactive content in a synchronized layer.
+
+```mermaid
+flowchart TD
+    Source["Markdown source string"] --> TextView["NSTextView source buffer"]
+    TextView --> Layout["TextKit layout manager"]
+    Layout --> LineRects["Line fragment rects"]
+    Source --> Parser["Markdown line/block parser"]
+    Parser --> RenderCache["Rendered overlay cache"]
+    LineRects --> Overlay["Overlay view layer"]
+    RenderCache --> Overlay
+    Overlay --> HitTesting["Hit testing + accessibility mapping"]
+    HitTesting --> TextView
+```
+
+Evidence-based comparison:
+
+| Question | Attributed-string rendering | Overlay rendering |
+| --- | --- | --- |
+| Would it improve performance? | Dirty-line invalidation already removes full attributed-string rebuilds for current inline styling. | Could help rich blocks by rendering only visible overlays, but requires layout cache invalidation and visible-range tracking. |
+| Would it improve cursor stability? | Strong today because text, cursor, and selection share one source coordinate system. | Source buffer remains canonical, but hit testing, selection painting, and accessibility must map overlay geometry back to source ranges. |
+| What complexity appears? | Attribute scopes, dirty ranges, and TextKit line metrics. | Overlay lifecycle, z-order, scroll synchronization, invalidation, accessibility, hit testing, and mismatch between rendered block heights and source line heights. |
+
+Overlay rendering is still promising for future images, diagrams, and rich blocks. It should not replace the current renderer before those features exist because the current bottleneck is state line-model rebuilding, not attributed styling for dirty lines.
+
+## Finding #9 — Editor Scalability Assessment
+
+Current assessment:
+
+- `NSTextView` remains a viable foundation for the next milestone.
+- The editor now avoids full attributed-string rebuilds for user edits and active-line moves.
+- Cursor and selection state remain stable because source offsets are unchanged.
+- Scroll restoration remains explicit and counted.
+- Large documents open in the automated proxy tests, including 10,000 generated lines.
+
+Scalability limits:
+
+- Source updates still rebuild `EditorState.lines` for the whole document.
+- Dirty rendering still computes line positions from the full source before touching only dirty ranges.
+- Real scroll responsiveness still needs visible-window AppKit measurement, not only model-level tests.
+- Rich block rendering may require overlay or mixed-layer work once block heights diverge from source line heights.
+
+Recommended next measured work before Milestone 3 rendering complexity:
+
+1. Add native edited-range capture from TextKit callbacks so state updates can avoid full line-model reconstruction.
+2. Add an AppKit UI benchmark harness that opens the 1,000/5,000/10,000-line files in a visible editor and records scroll and typing latency.
+3. Keep overlay rendering as a technical option for rich blocks, but do not introduce it for current heading/emphasis/code styling.
 
 ## AttributedString and NSAttributedString
 
@@ -417,7 +560,7 @@ Open challenges:
 - Selection after visually hidden or replaced Markdown delimiters.
 - Cursor placement if future phases hide links, list markers, task markers, or other delimiters.
 - IME composition and marked text handling.
-- Very large documents where full restyling on each selection change is too expensive.
+- Very large documents where full state line-model rebuilding on each source change is too expensive.
 
 Cursor correctness is the main reason to keep the canonical source text inside `NSTextView` for now.
 
@@ -459,9 +602,9 @@ Continue with `NSTextView` for the next editor milestone on macOS and keep the i
 
 Immediate next steps:
 
-- Add incremental line invalidation instead of restyling the whole buffer.
-- Add automated coverage for multi-line selections and edited-range invalidation.
-- Manually test long documents, undo/redo, IME input, and keyboard navigation with the prototype files.
+- Capture native edited ranges so source updates can avoid rebuilding the full line model.
+- Manually test undo/redo, IME input, and keyboard navigation with the stress prototype files.
+- Add visible AppKit scroll and typing benchmarks for the 1,000/5,000/10,000-line documents.
 - Explore delimiter hiding with temporary attributes only after selection mapping is robust.
 
 Future investigation:
@@ -480,10 +623,11 @@ Evidence:
 - Cursor and selection ranges remain native source ranges because rendering does not replace text.
 - Scroll position can be preserved around rendering updates.
 - Redundant render passes are avoided when source text and active line are unchanged.
-- The large 2,100-line prototype shape is tractable for render planning, though the native attribute pass still needs dirty-line invalidation.
+- Dirty-line invalidation avoids full attributed-string rebuilds for user edits and active-line transitions.
+- The 1,000/5,000/10,000-line stress documents are tractable in automated model-level validation, with 10,000-line open proxy time measured at 17.642 ms.
 
-Sapling should not begin a custom editor engine now. The evidence supports continuing with native text systems while moving toward incremental invalidation and richer overlay experiments.
+Sapling should not begin a custom editor engine now. The evidence supports continuing with native text systems while measuring state-line rebuilding and richer overlay experiments.
 
-A future custom editor engine may still be required for high-fidelity block rendering, especially images, tables, Mermaid, LaTeX, attachments, and fully hidden delimiters. That decision should be made only after overlay rendering and dirty-line attributed rendering have been measured against real documents.
+A future custom editor engine may still be required for high-fidelity block rendering, especially images, tables, Mermaid, LaTeX, attachments, and fully hidden delimiters. That decision should be made only after overlay rendering and native edited-range state updates have been measured against real documents.
 
 The architecture should continue with a SwiftUI shell, a Sapling editor abstraction, and native platform text views hidden behind replaceable adapters.

Tests/SaplingEditorTests/EditorLargeDocumentValidationTests.swift

@@ -53,6 +53,17 @@ final class EditorLargeDocumentValidationTests: XCTestCase {
             XCTAssertLessThan(openMeasurement, 0.25, "Opening \(lineCount) generated lines should remain interactive.")
             XCTAssertLessThan(updateMeasurement, 0.50, "State update for \(lineCount) lines should remain bounded.")
             XCTAssertLessThan(dirtyRenderMeasurement, 0.05, "Dirty render planning should not scale with the full document.")
+
+            if ProcessInfo.processInfo.environment["SAPLING_EDITOR_PRINT_METRICS"] == "1" {
+                print(
+                    "SaplingEditorMetrics lines=\(lineCount) "
+                        + "openMs=\(milliseconds(openMeasurement)) "
+                        + "typingMs=\(milliseconds(updateMeasurement)) "
+                        + "dirtyRenderMs=\(milliseconds(dirtyRenderMeasurement)) "
+                        + "dirtyLines=\(plan.dirtyLineCount) "
+                        + "fullRender=\(plan.isFullRender)"
+                )
+            }
         }
     }
 
@@ -85,6 +96,10 @@ final class EditorLargeDocumentValidationTests: XCTestCase {
         return Date().timeIntervalSince(start)
     }
 
+    private func milliseconds(_ interval: TimeInterval) -> String {
+        String(format: "%.3f", interval * 1000)
+    }
+
     private static func prototypeDocument(lineCount: Int) -> String {
         (1...lineCount).map { index in
             if index.isMultiple(of: 25) {