Sapling/Docs/performance-report-5mb.md

5 MB Editor Performance Report

Date: 2026-05-30

Milestone: 2.6 Large Document Profiling

Status: Investigation updated after Milestone 2.7 CRLF correctness fix; no editor optimization or redesign was performed.

Executive Summary

The 5 MB document becomes unusable primarily because Sapling's current line tracker does not split CRLF line endings. The benchmark file has 51,481 CRLF pairs. Swift treats "\r\n" as one newline grapheme cluster, and EditorActiveLineTracker.lines(from:) searches the String for Character("\n"). The result is one giant EditorLine containing the entire 5.5M UTF-16-unit document.

Evidence:

• Asset profile: 51,482 physical lines, 51,481 CR characters, 51,481 LF characters.
• Swift check: "a\r\nb".firstIndex(of: "\n") == nil; "a\r\nb".split(separator: "\n").count == 1.
• Benchmark rendered-mode trace for Docs/Benchmarks/5mb.md: 0 rendered lines, 1 source line.
• The initial styling pass reports 1 styled line, not 51,482 styled lines.

This explains the visible source-only behavior: the editor believes the whole document is the active line. Dirty-line invalidation is technically working, but the dirty line is the entire file.

Milestone 2.7 corrected this by introducing a UTF-16 based DocumentLineIndex with explicit LF, CRLF, and CR handling. The corrected benchmark now reports 51,482 editor lines, 51,481 rendered lines, and one active source line.

Benchmark Asset

The benchmark file is now stored at:

Docs/Benchmarks/5mb.md

Recorded characteristics:

Property	Value
File size	5,526,168 bytes
Characters	5,474,683
UTF-16 units	5,526,164
Physical lines	51,482
CR characters	51,481
LF characters	51,481
Max line UTF-16 length	658
Average line UTF-16 length	105.342

Markdown structure:

Construct	Count
ATX headings	2
Unordered list items	2
Ordered list items	0
Blockquotes	0
Fenced code fences	0
Lines with inline code	0
Lines with bold markers	0
Lines with italic markers	0
Inline links	0
Reference-style link markers	51,156
Images	0

The document mostly consists of repeated reference-style link text. Milestone 2 rendering does not promote reference-style links, so even after correct line splitting most lines would remain visually close to source text unless future rendering adds support for that construct.

Benchmark Harness

Reusable benchmark command:

swift run -c release SaplingEditorBenchmark

Single-file command:

swift run -c release SaplingEditorBenchmark Docs/Benchmarks/5mb.md

The harness measures:

• file read time
• document model creation
• editor state parse and line-model creation
• render-plan generation
• initial attributed-string generation
• active-line lookup
• selection update
• dirty-line invalidation for click and typing paths
• dirty render update for typing
• NSTextStorage updates
• NSLayoutManager glyph and layout generation
• viewport glyph-range lookup as a scroll proxy

Percentages below are percentages of measured benchmark stage time, not Instruments Time Profiler samples.

Scenario Summary

Original Milestone 2.6 release benchmark run on May 30, 2026:

Scenario	Lines	Rendered trace	Measured total
Sample document	54	53 rendered, 1 source	7.722 ms
2,100-line prototype	2,101	2,100 rendered, 1 source	348.580 ms
5 MB benchmark	51,482 physical, 1 editor line	0 rendered, 1 source	4,376.271 ms

Corrected Milestone 2.7 release benchmark run on May 30, 2026:

Scenario	Lines	Rendered trace	Measured total
Sample document	54	53 rendered, 1 source	7.367 ms
2,100-line prototype	2,101	2,100 rendered, 1 source	269.568 ms
5 MB benchmark	51,482 editor lines	51,481 rendered, 1 source	5,674.222 ms

Memory measurements from /usr/bin/time -l after the CRLF fix:

Scenario	Maximum resident set size
50-line prototype	22,396,928 bytes
2,100-line prototype	30,998,528 bytes
5 MB benchmark	197,378,048 bytes

These memory values are benchmark-process measurements, not full Sapling app memory measurements.

Top 20 Operations

5 MB release benchmark before the CRLF fix:

Rank	Operation	Category	Time	Percent	Evidence
1	attributed_string_generation_initial	rendering	903.494 ms	20.645%	1 styled line
2	render_update_typing_dirty	interaction	887.213 ms	20.273%	1 styled line
3	layout_generation_full_document	layout	766.331 ms	17.511%	NSLayoutManager.ensureLayout
4	dirty_line_invalidation_typing	editor update	566.852 ms	12.953%	1 dirty line
5	layout_after_typing	interaction	490.160 ms	11.200%	layout for inserted character
6	glyph_generation_full_document	layout	157.546 ms	3.600%	full character range
7	typing_state_update	interaction	136.721 ms	3.124%	EditorState.updateSource
8	selection_update	editor update	136.422 ms	3.117%	EditorState.updateSelection
9	active_line_lookup	editor update	73.509 ms	1.680%	midpoint cursor lookup
10	line_model_generation	editor update	63.505 ms	1.451%	EditorActiveLineTracker.lines
11	document_parse_editor_state_init	document load	63.444 ms	1.450%	initial EditorState
12	dirty_line_invalidation_click	editor update	63.043 ms	1.441%	0 dirty lines
13	render_plan_generation_all_lines	rendering	51.323 ms	1.173%	one render plan
14	text_storage_initial_update	layout	10.957 ms	0.250%	full source replace
15	file_read	document load	4.954 ms	0.113%	disk read
16	text_storage_typing_insert	interaction	0.711 ms	0.016%	one-character insertion
17	document_model_init	document load	0.044 ms	0.001%	MarkdownDocument
18	scroll_latency_middle_viewport	interaction	0.013 ms	0.000%	cached glyph-range lookup
19	scroll_latency_deep_viewport	interaction	0.012 ms	0.000%	cached glyph-range lookup
20	scroll_latency_top_viewport	interaction	0.011 ms	0.000%	cached glyph-range lookup

5 MB release benchmark after the CRLF fix:

Rank	Operation	Category	Time	Percent	Evidence
1	layout_generation_full_document	layout	1,320.499 ms	23.272%	NSLayoutManager.ensureLayout
2	dirty_line_invalidation_typing	editor update	980.651 ms	17.283%	3 dirty lines
3	attributed_string_generation_initial	rendering	773.474 ms	13.631%	51,482 styled lines
4	render_plan_generation_all_lines	rendering	637.268 ms	11.231%	51,482 render plans
5	selection_update	editor update	394.957 ms	6.961%	EditorState.updateSelection
6	typing_state_update	interaction	382.159 ms	6.735%	EditorState.updateSource
7	glyph_generation_full_document	layout	279.739 ms	4.930%	full character range
8	line_model_generation	editor update	217.457 ms	3.832%	EditorActiveLineTracker.lines
9	document_parse_editor_state_init	document load	208.638 ms	3.677%	initial EditorState
10	dirty_line_invalidation_click	editor update	204.958 ms	3.612%	0 dirty lines
11	active_line_lookup	editor update	198.833 ms	3.504%	midpoint cursor lookup
12	render_update_typing_dirty	interaction	55.805 ms	0.983%	3 styled lines
13	text_storage_initial_update	layout	14.411 ms	0.254%	full source replace
14	file_read	document load	3.651 ms	0.064%	disk read
15	layout_after_typing	interaction	0.903 ms	0.016%	layout for inserted character
16	text_storage_typing_insert	interaction	0.737 ms	0.013%	one-character insertion
17	document_model_init	document load	0.047 ms	0.001%	MarkdownDocument
18	scroll_latency_top_viewport	interaction	0.009 ms	0.000%	cached glyph-range lookup
19	scroll_latency_middle_viewport	interaction	0.009 ms	0.000%	cached glyph-range lookup
20	scroll_latency_deep_viewport	interaction	0.008 ms	0.000%	cached glyph-range lookup

CRLF Fix Impact

Direct before/after comparison for the 5 MB benchmark:

Measurement	Before CRLF fix	After CRLF fix	Change
Editor lines	1	51,482	correctness restored
Rendered/source trace	0 rendered, 1 source	51,481 rendered, 1 source	hybrid mode restored
Measured total	4,376.271 ms	5,674.222 ms	+29.704%
Initial attributed styling	903.494 ms	773.474 ms	-14.390%
Dirty typing render	887.213 ms	55.805 ms	-93.710%
Full TextKit layout	766.331 ms	1,320.499 ms	+72.052%
Dirty invalidation typing	566.852 ms	980.651 ms	+72.997%
Layout after typing	490.160 ms	0.903 ms	-99.816%
Selection update	136.422 ms	394.957 ms	+189.516%
Typing state update	136.721 ms	382.159 ms	+179.516%
Render plan generation	51.323 ms	637.268 ms	+1,141.680%

What improved:

• The source-only rendering failure disappeared.
• Dirty-line rendering now behaves as intended: the typing render touched 3 styled lines instead of one giant document-sized line.
• Post-typing TextKit layout dropped from 490.160 ms to 0.903 ms.
• Dirty typing render dropped from 887.213 ms to 55.805 ms.

What became more expensive:

• The benchmark now performs real per-line work for 51,482 editor lines. Render-plan generation, state updates, selection updates, and dirty invalidation are now measuring the actual line count instead of one accidental giant line.
• Full TextKit layout increased from 766.331 ms to 1,320.499 ms after the document became many real layout lines.

What bottlenecks remain:

• EditorState.updateSelection and EditorState.updateSource still rebuild the full line model.
• EditorDirtyLineInvalidator.plan still scans full previous/current strings and rebuilds current line boundaries.
• MarkdownTextStyler.apply still recomputes the full line list before applying dirty-line styling.
• Full-document render-plan generation is expensive when run across 51,482 lines.

Conclusion:

The CRLF bug caused the source-only display and pathological dirty render/layout behavior. Fixing it restored correctness and removed the giant-line typing/rendering failure. It also exposed the real remaining scalability issue: editor updates still perform document-wide line indexing and render planning work.

What Consumes the Most CPU and Time?

Measured stage totals indicate the dominant costs are:

Area	Representative operations	Time
Rendering/styling one giant line	initial attributed styling, dirty typing styling	1,790.707 ms
TextKit layout	full layout, glyph generation, layout after typing	1,414.037 ms
Whole-document editor scans	dirty invalidation, selection/source updates, active-line lookup, line generation	1,103.496 ms
File/model load	file read, model init, state init	68.442 ms

Before the CRLF fix, the largest single cost was initial attributed styling of one giant line. After the fix, the largest single cost is full TextKit layout over the correctly segmented document.

After the CRLF fix, measured stage groups are:

Area	Representative operations	Time
TextKit layout	full layout, glyph generation, layout after typing	1,601.141 ms
Whole-document editor scans	dirty invalidation, selection/source updates, active-line lookup, line generation	2,379.015 ms
Rendering/render planning	initial attributed styling, full render-plan generation, dirty typing styling	1,466.547 ms
File/model load	file read, model init, state init	212.336 ms

Interaction Paths

Clicking

Observed measured path:

sequenceDiagram
    participant User
    participant TextView as NSTextView
    participant VM as HybridMarkdownEditorViewModel
    participant State as EditorState
    participant Dirty as EditorDirtyLineInvalidator

    User->>TextView: Click
    TextView->>VM: updateSelection
    VM->>State: updateSelection
    State->>State: lineIndex + lines over full source
    TextView->>Dirty: plan same source/active line
    Dirty-->>TextView: 0 dirty lines, render skipped

Measured operations for the 5 MB file after the CRLF fix:

• selection_update: 394.957 ms
• dirty_line_invalidation_click: 204.958 ms
• render update: skipped because no dirty lines were scheduled

Clicking is not dominated by Markdown rendering in this benchmark. It is dominated by whole-document source scans and line-model rebuilding.

Typing

Observed measured path:

sequenceDiagram
    participant User
    participant TextStorage as NSTextStorage
    participant VM as ViewModel
    participant State as EditorState
    participant Dirty as Dirty invalidator
    participant Styler as MarkdownTextStyler
    participant Layout as NSLayoutManager

    User->>TextStorage: Insert one character
    TextStorage->>VM: updateSource
    VM->>State: updateSource
    State->>State: lineIndex + lines over full source
    VM->>Dirty: plan previous/current source
    Dirty->>Dirty: prefix/suffix diff over full source
    Dirty->>Styler: 1 dirty line
    Styler->>Styler: restyle the one giant line
    Styler->>Layout: invalidate layout
    Layout->>Layout: layout inserted character

Measured operations for the 5 MB file after the CRLF fix:

• typing_state_update: 382.159 ms
• dirty_line_invalidation_typing: 980.651 ms
• render_update_typing_dirty: 55.805 ms
• text_storage_typing_insert: 0.737 ms
• layout_after_typing: 0.903 ms

Typing is no longer dominated by dirty rendering of one giant line. After the fix, typing is dominated by dirty invalidation and state updates that still scan/rebuild full-document line data.

Scrolling

After the CRLF fix, the harness measured cached viewport glyph-range lookup after full layout:

• top viewport: 0.009 ms
• middle viewport: 0.009 ms
• deep viewport: 0.008 ms

This does not reproduce the user's slow visible scrolling by itself. The measured expensive TextKit operations are:

• layout_generation_full_document: 1,320.499 ms
• glyph_generation_full_document: 279.739 ms
• layout_after_typing: 0.903 ms

Evidence supports this conclusion: cached glyph-range lookup is cheap after layout, but full layout generation remains expensive. A visible AppKit scroll-wheel trace is still needed to prove whether slow scrolling comes from repeated layout invalidation, drawing, or accessibility/selection work during live scrolling.

Rendering Investigation

Rendered mode appeared absent before Milestone 2.7 because the line tracker emitted one editor line for the entire CRLF document.

Actual execution path:

flowchart TD
    File["5mb.md with CRLF endings"] --> SwiftString["Swift String"]
    SwiftString --> Tracker["EditorActiveLineTracker.lines"]
    Tracker --> Search["firstIndex(of: Character newline)"]
    Search --> OneLine["No Character(\\n) match inside CRLF grapheme clusters"]
    OneLine --> State["EditorState.lines = 1"]
    State --> Active["activeLineIndex = 0"]
    Active --> Modes["1 source line, 0 rendered lines"]
    Modes --> Styler["MarkdownTextStyler styles active source line"]

Evidence:

CR=51481 LF=51481
"a\r\nb".firstIndex(of: "\n") == nil
"a\r\nb".split(separator: "\n").count == 1
(NSString("a\r\nb")).components(separatedBy: "\n").count == 2

There is no evidence of an explicit safety fallback, disabled rendering threshold, performance guard, or render-plan failure. The rendered mode was absent because the editor's line model contained no inactive lines.

After Milestone 2.7:

• DocumentLineIndex scans UTF-16 code units and treats LF, CRLF, and CR as explicit line endings.
• EditorActiveLineTracker.lines(from:) returns 51,482 editor lines for the benchmark document.
• The corrected rendered-mode trace is 51,481 rendered lines and 1 source line.

Scalability Audit

Operation	Trigger	Frequency	Complexity	Evidence / justification
String(contentsOf:)	document open	once	O(n) bytes	4.954 ms for 5.5 MB
EditorState.init	document open	once	O(n)	builds EditorState.lines; 208.638 ms after fix
EditorActiveLineTracker.lines	open, source update, selection update, dirty plan, styler	repeated	O(n)	scans source to create ranges; 217.457 ms standalone after fix
EditorActiveLineTracker.lineIndex	cursor/selection/source updates	repeated	O(n)	scans line boundaries; 198.833 ms midpoint lookup after fix
EditorState.updateSelection	click/cursor movement	every selection change	O(n)	rebuilds lines; 394.957 ms after fix
EditorState.updateSource	typing	every source edit	O(n)	rebuilds lines; 382.159 ms after fix
Dirty diff prefix/suffix scan	source edit	every source edit	O(n)	dirty_line_invalidation_typing 980.651 ms after fix
Dirty line selection	source edit/click	every dirty plan	O(n) today	dirty plan first rebuilds current lines
Full render-plan generation	benchmark / future full render	on demand	O(line count + inline regex work)	637.268 ms after fix
Initial attributed styling	initial render	once per view text load	O(n + line count)	773.474 ms after fix
Dirty attributed styling	typing/active line change	per dirty render	O(n + dirty content) today	55.805 ms after fix because styler still rebuilds line list before styling 3 lines
NSTextStorage.replaceCharacters full	text load	once	O(n)	10.957 ms
NSTextStorage.replaceCharacters insert	typing	every edit	local TextKit edit	0.711 ms
NSLayoutManager.ensureGlyphs full	initial layout	once / after invalidation	O(n)	279.739 ms after fix
NSLayoutManager.ensureLayout full	initial layout	once / after invalidation	O(n layout fragments)	1,320.499 ms after fix
ensureLayout for inserted char	typing	every edit	TextKit-dependent	0.903 ms after fix
Viewport glyph range	scroll proxy after layout	per query	near O(1) after cached layout	0.008-0.009 ms in harness

No O(n^2) operation was proven by this benchmark. After the CRLF fix, the measured failure is multiple O(n) scans over 51,482 real editor lines plus full TextKit layout.

Complexity Analysis

Workflow	Current complexity	Evidence
Document opening	O(n) read + O(n) editor line model + O(n) initial styling + O(n) layout	3.651 ms read, 208.638 ms state init, 773.474 ms styling, 1,320.499 ms layout
Typing	O(n) state update + O(n) dirty diff + O(n) dirty styling setup + local TextKit layout	382.159 ms + 980.651 ms + 55.805 ms + 0.903 ms
Clicking / cursor movement	O(n) selection update + O(n) dirty plan	394.957 ms + 204.958 ms
Active-line changes	O(n) line lookup + O(n) line model rebuild + dirty render	measured through selection and dirty planning
Scrolling after cached layout	not proven O(n) by harness	viewport glyph-range lookup was near zero after full layout
Scrolling during invalidated layout	likely TextKit layout-bound, not isolated by this harness	full layout and post-typing layout are expensive; live scroll trace still required

Any operation that calls EditorActiveLineTracker.lines, EditorActiveLineTracker.lineIndex, or EditorDirtyLineInvalidator.plan currently grows with total document size.

Viewport Analysis

Sapling currently operates on the entire document for editor state and rendering inputs. It does not operate on visible lines plus a buffer.

flowchart TD
    Source["Full Markdown source"] --> State["EditorState.lines for full source"]
    Source --> Dirty["Dirty invalidator scans full previous/current source"]
    Source --> Styler["MarkdownTextStyler recomputes full line list"]
    Styler --> TextStorage["NSTextStorage dirty attributes"]
    TextStorage --> Layout["NSLayoutManager layout/glyph cache"]
    Layout --> Viewport["Visible viewport"]

Subsystem	Entire document or viewport?	Evidence
Active-line update	Entire document	lineIndex(containing:in:) scans line index; 198.833 ms
Selection update	Entire document	updateSelection rebuilds lines; 394.957 ms
Source update	Entire document	updateSource rebuilds lines; 382.159 ms
Dirty invalidation	Entire document	compares previous/current strings and rebuilds current lines; 980.651 ms
Render planning	Entire document in full benchmark; dirty line only for actual render plans after valid dirty plan	full render-plan generation measured separately
Attributed styling	Dirty line set, but line list recomputed from entire document	55.805 ms for 3 styled lines
TextKit layout	TextKit-managed full layout/glyph cache	full layout 1,320.499 ms
Scroll lookup after layout	Viewport query against cached layout	0.008-0.009 ms

Bottleneck Assessment

The bottleneck is not one subsystem alone.

Evidence ranks after the CRLF fix:

1. TextKit full layout is the largest single measured stage: 1,320.499 ms.
2. Dirty-line invalidation still does document-wide work: 980.651 ms.
3. Initial attributed styling across 51,482 lines remains expensive: 773.474 ms.
4. Full render-plan generation across 51,482 lines is expensive: 637.268 ms.
5. Active-line, selection, source update, and dirty invalidation still perform whole-document scans.

File I/O is not the bottleneck. file_read was 3.651 ms in the corrected benchmark.

Recommendations For Future Work

These are recommendations from evidence, not implemented changes:

1. Remove full-document line-model rebuilding from selection changes. Cursor movement should not rebuild every EditorLine.
2. Change dirty invalidation to use native edited ranges instead of prefix/suffix diffing full strings on every edit.
3. Change MarkdownTextStyler.apply so dirty rendering does not recompute the full line list before styling dirty ranges.
4. Add a visible AppKit scroll benchmark or Time Profiler trace to isolate live scroll-wheel latency after correct CRLF handling.
5. Keep DocumentLineIndex as the single source for editor line segmentation; future editor systems should not search raw Swift String values for newline characters.

Reproduction Commands

perl -0ne '$cr = tr/\r//; $lf = tr/\n//; print "CR=$cr LF=$lf\n"' Docs/Benchmarks/5mb.md
swift -e 'import Foundation; let s = "a\r\nb"; print(s.firstIndex(of: "\n") as Any); print(s.split(separator: "\n", omittingEmptySubsequences: false).count); print((s as NSString).components(separatedBy: "\n").count)'
swift run -c release SaplingEditorBenchmark Docs/Benchmarks/5mb.md