limits.md gets before and after numbers for each, a Formula depth section, the serve sessions at 120 fps, and recommendations 4, 5, 7 and 8 marked done.
- commit
- bfe10f8f1c68a4de4ccb17d3d8580b597106f8ac
- parent
- 7ea2199c090dbda7f2c6ad485096a67869e94cdb
- author
- Dami <dami.etoile@gmail.com>
| M | docs/ | 7 | +- |
| M | docs/ | 170 | ++++++++++++++++++++++-------- |
@@ -123,8 +123,11 @@ move without touching callers:
123123 aggregates read ranges.124124 - **Files.** A streaming or columnar `.012` format replaces `file.go`'s125125 whole-document JSON; the format is already separate from the store.126-- **Depth limits.** A cap on nesting goes in `internal/formula`'s parser,127- so a pathological formula fails to parse before anything evaluates it.126+- **Depth limits.** `internal/formula`'s parser caps nesting at127+ `formula.MaxDepth` (1024 levels), so a pathological formula fails to128+ parse before anything evaluates it; evaluation puts off cells past129+ 65,536 levels of a chain rather than recursing further130+ (`evaluate.go`).128131 - **Functions.** The function library (values, evaluation, `FuncDef` and129132 the `functions_*.go` tables, about a third of `sheet`) is the next130133 boundary: it needs cells only through `lookup`. It stays in `sheet`@@ -23,14 +23,14 @@ it lags, and past a second it stalls.
2323 | Incremental recalc | A change that makes formulas read under about 500,000 cells in total (fan-out, chains and volatiles of 8192 cells: 1 to 2 ms) | 1000 SUMs over a full column (1000 x 8192 reads): 170 ms per edit | 8192 running totals (`=SUM($A$1:An)`, 33 M reads): 0.8 s per edit | About 20 ns per cell read: two map lookups |2424 | Full recalc | Any sheet without heavy range fan-in: under 40 ms for 213 k numbers | 1000 full-column SUMs: 170 ms | 8192 running totals: 0.7 s | Same as above |2525 | Rendering | Any sheet at up to 200 x 60: 1 ms a frame; 400 x 120: 4.5 ms | 20 charts at 400 x 120: 7 ms | | View building styled strings, then Bubble Tea parsing and diffing them |26-| Selection statistics | Any selection, once computed (cached) | Extending a selection over 2 M cells: 30 ms per key | | Summing 2 M map entries per change |26+| Selection statistics | Any selection: extending one over all 2 M cells costs 0.5 ms a key | | | Per column, a block of 64 rows at a time, plus the rows at the selection's ends |2727 | Imports | CSV, SQLite, Parquet: 1 to 3 M cells/s (a full 2.1 M-cell sheet in about 1 s) | XLSX with formulas: 200 k cells/s | Data past the limits (dropped, with a note) | Building cells one at a time; XLSX formula translation |2828 | Undo | One step of any size: undo costs what the edit cost | History capped at 100 steps and 256 MB of before-images: 100 whole-column steps hold 250 MB | | Whole-cell before-images, about 300 B per cell per step |29-| JEV | Up to about 1000 JEV cells: 0.3 ms of CPU per answer | 4000 JEV cells: 1.3 ms per answer, 5 s of CPU to answer them all | | Every answer recalculates every JEV cell (they're volatile) |30-| Formula depth | 10,000 nested parentheses or IFs: under 5 ms | | No explicit limit; recursion grows the stack | Recursive parser and evaluator |29+| JEV | 4000 JEV cells: 4 us of CPU per answer, 15 ms to answer them all | | | An answer recalculates the cells that asked it; answers within a frame recalculate together |30+| Formula depth | 1000 nested parentheses or IFs: under 0.5 ms; chains of formulas through every cell of a sheet | | More than 1024 levels of nesting in one formula: a parse error | Recursive parser; evaluation puts off cells past 65,536 levels |3131 | Macros | Replaying 1000 recorded actions: 2.2 ms, one undo step; a script's call to the sheet: about 1.4 us | | Scripts past 10 M Starlark steps: stopped, with the line | One message per call to the sheet, served in batches on the UI goroutine |3232 | Find, filter, sort, fill | Filter or sort 8191 rows: 1 to 32 ms; find over 213 k cells: 38 ms; fill 8192 rows: 4 ms | Replace all over 213 k cells: 280 ms | | Per-cell string conversion and regexp |33-| SSH sessions (`012 serve`) | 50 sessions typing at once: frames within one frame interval (p95 16.6 ms), 1.3 MiB per session plus its sheets (upper bound) | | More than `--max-sessions` (8 by default): turned away | Bubble Tea's 60 fps pacing; per session, the terminal's cell buffers |33+| SSH sessions (`012 serve`) | 50 sessions typing at once: frames within one frame interval (p95 8.7 ms), 1.3 MiB per session plus its sheets (upper bound) | | More than `--max-sessions` (8 by default): turned away | Bubble Tea's 120 fps pacing; per session, the terminal's cell buffers |3434 | Pivot tables | A pivot over 8191 rows recomputes after an edit to its source in 1 to 4 ms | | Results past 8192 x 256 (the pivot shows #REF!) | Reading each source cell of its fields: a map lookup each |3535 3636 ## Sheet size@@ -72,11 +72,15 @@ Every volatile formula is recomputed on every change, as in Sheets, so
7272 8192 volatile cells add about 1.2 ms to every edit anywhere.7373 7474 Through a real terminal (`make stress-e2e`: 200 x 60, 8192 x 26 imported,75-200 SUMs over column A): arrow keys p50 16.7 ms, p95 17.5 ms; an entry76-that recalculates the 200 SUMs p50 57 ms, p95 72 ms. The arrow floor is77-Bubble Tea's renderer, which draws at most 60 frames a second, so a key78-waits up to 16 ms for the next frame; 012's own share of an arrow key is79-0.2 to 1 ms.75+200 SUMs over column A): arrow keys p50 8.3 ms, p95 10.2 ms; an entry76+that recalculates the 200 SUMs p50 62 to 70 ms, p95 67 to 75 ms. The77+arrow floor is Bubble Tea's renderer, which draws at most 120 frames a78+second (`ui.FrameRate`, its maximum; the default is 60), so a key waits79+up to 8 ms for the next frame; 012's own share of an arrow key is 0.280+to 1 ms. At 60 frames a second the same arrows took p50 16.6 ms, p9581+18.4 ms. The renderer writes a frame only when the view changed, so an82+idle 012 costs the same at either rate: 0.6 to 2.2 ms of CPU a second83+on an empty sheet over 30 s, within the noise of measuring it.8084 8185 ## Rendering8286 @@ -105,9 +109,29 @@ when its data, size or theme changes: 25 to 260 us at 24 x 10 cells and
105109 is 0.21 ms at 80 x 24 and 1.06 ms at 200 x 60; Page Down 1.26 ms.106110 107111 With the whole of an 8192 x 256 sheet selected, the status line's Sum,108-Avg and Count cover 2.1 M cells: 29 ms to compute, then cached until a109-cell changes, so frames with a standing selection cost 0.3 ms; each key110-that changes the selection pays the 29 ms again.112+Avg and Count cover 2.1 M cells. On a sheet of more than 16,384 cells,113+a selection of more than 4096 is summed from an index of each column's114+sum, count and count of numbers in blocks of 64 rows (`internal/sheet/stats.go`):115+the whole blocks it covers, plus the rows at its ends a cell at a time.116+The index is built on first use, in one pass over the cells (the 30 ms117+the whole sum used to take every time), and a stored, deleted or118+recalculated cell marks its block to be summed again when a selection119+next covers it. Sums are recomputed, never adjusted, so they don't120+drift. The result is still cached until a cell changes. Each key that121+changes a selection over the whole sheet, through to its frame122+(`BenchmarkKeystroke`, 3 runs each, same machine):123+124+| Key, 8192 x 256 numbers | 80 x 24 before | after | 200 x 60 before | after |125+|---|---|---|---|---|126+| Shift+Up and Shift+Down with the data selected (`extend-data`) | 31 to 44 ms | 0.53 ms | 32 to 46 ms | 2.1 ms |127+| Ctrl+Shift+Down and Ctrl+Shift+Up over the first row (`extend-edge`) | 16 to 23 ms | 0.43 ms | 18 to 24 ms | 1.7 ms |128+| Ctrl+A, the selection standing (`select-data`) | 0.30 ms | 0.30 ms | 1.8 ms | 1.8 ms |129+130+A CPU profile of `extend-data` before put 54% of the time in131+`RangeStats` walking the map; the rest is building the benchmark's sheet.132+The index costs 0.5 MB per sheet that has one (32,768 blocks of 16 B),133+nothing on sheets that never had a large selection, and a nil check per134+stored or recalculated cell.111135 112136 ## Imports113137 @@ -160,20 +184,65 @@ BigUndo's 3%; everything else is unchanged.
160184 ## JEV161185 162186 Each JEV formula is volatile: every recalculation looks its question up163-in the cache, and every answer that arrives recalculates all JEV164-cells. With a fake client answering instantly (`BenchmarkJEV`), the CPU165-per answer grows with the number of JEV cells:166-167-| JEV cells | Per answer | All answers |168-|---|---|---|169-| 100 | 26 us | 2.7 ms |170-| 1000 | 261 us | 0.26 s |171-| 4000 | 1.3 ms | 5.1 s |187+in the cache. A formula shown Loading… is noted against the question it188+waits for (by `RemoteCall.Key`), and an answer recalculates only the189+formulas waiting for it and what reads them190+(`Workbook.RecalcAnswered`). Answers are stored as they arrive; the191+first since the last recalculation schedules one a frame later192+(`ui.FrameInterval`), which covers every answer stored meanwhile, so193+answers arriving together recalculate once. With a fake client194+answering instantly (`BenchmarkJEV`, 3 runs each, same machine; the195+benchmark doesn't wait the frame, so answers queued together, up to 8,196+recalculate together):197+198+| JEV cells | Per answer before | after | All answers before | after |199+|---|---|---|---|---|200+| 100 | 29 us | 3.5 us | 2.9 ms | 0.35 ms |201+| 1000 | 270 us | 3.6 us | 0.27 s | 3.6 ms |202+| 4000 | 1.2 ms | 3.8 us | 4.9 s | 15 ms |172203 173204 Against the real service the network dominates: at most 8 questions are174-in flight, so 4000 questions at 300 ms each take 2.5 minutes, of which175-5 s is recalculation spread over that time. Each answer blocks the UI176-for its recalculation (1.3 ms at 4000 cells).205+in flight, so 4000 questions at 300 ms each take 2.5 minutes, and the206+recalculation is now a few microseconds per answer whatever the number207+of JEV cells. Other changes still recalculate every JEV cell, as every208+volatile formula, looking the answers up in the cache.209+210+## Formula depth211+212+The parser (`internal/formula`) counts nesting as it recurses:213+parentheses, function calls and prefix operators each open a level. Past214+`formula.MaxDepth`, 1024 levels, a formula fails to parse with "Formula215+is nested too deeply (more than 1024 levels)", shown on the context line216+with the caret at the level past the cap, as other parse errors are.217+Excel allows 64 nested functions; 1024 is far beyond a formula written218+by hand, and bounds the recursion of everything that walks a formula219+(the printer, reference rewriting, the evaluator within one formula). A220+`.012` file holding a deeper formula (none could have been written by221+hand) fails to open with that message and the cell; imports keep such a222+formula's text or cached value, as for any formula they can't read.223+224+Evaluation recurses once per link of a chain of formulas, which no225+parser cap bounds: a chain through all 2.1 M cells of a sheet used to226+overflow the goroutine's 1 GB stack and kill the program. Evaluation now227+counts its depth in cells and operators; a cell reached past 65,536228+levels is put off: the evaluation in progress is abandoned (its cells go229+back to dirty), the cell is evaluated from the top of the stack, and the230+abandoned evaluation is retried and finds it done. Values are exactly as231+before, a cycle longer than the limit is still a cycle, and the stack232+stays under 32 MB (`TestFullSheetChain`, run with233+`SHEET_FULL_CHAIN=1`, checks it under a 64 MB limit: 4 s for 2.1 M234+cells, building included).235+236+Same machine, 8 runs each with before and after interleaved, other237+work running (differences under 10% are noise):238+239+| Benchmark | Before | After |240+|---|---|---|241+| Parse (`BenchmarkParse`) | 2.36 us | 2.41 us |242+| 1000 nested parentheses / IFs, parse and evaluate | 195 / 425 us | 203 / 438 us |243+| 10,000 nested parentheses / IFs | 1.8 / 4.4 ms, accepted | 1.1 / 2.0 ms to refuse |244+| RecalcAll chain of 8192 / dense 8192 x 26 | 1.33 / 50 ms | 1.28 to 1.38 / 41 to 51 ms |245+| Edit chain of 8192 / 8192 volatiles | 2.36 / 1.68 ms | 2.34 / 1.72 ms |177246 178247 ## Several sheets179248 @@ -216,7 +285,7 @@ session press an arrow key at once, 200 times, timing each key from the
216285 client's write to the first bytes of its frame arriving. The clients217286 run in the same process, so the heap and CPU figures include their side218287 of the connections and are upper bounds for the server. Medians of 3219-runs, `-benchtime 200x`:288+runs, `-benchtime 200x`, at 60 frames a second:220289 221290 | Sessions, sheet | Heap per session | Key to frame p50 / p95 / max | CPU per frame |222291 |---|---|---|---|@@ -225,8 +294,18 @@ runs, `-benchtime 200x`:
225294 | 50, new sheet | 1.30 MiB | 9.2 / 16.3 / 19.2 ms | 0.53 ms |226295 | 50, 1000 x 26 numbers | 8.5 MiB | 9.4 / 16.7 / 20.5 ms | 0.69 ms |227296 297+Sessions now ask for 120 frames a second, as the local app does. One298+run each, back to back, before and after:299+300+| Sessions, sheet | p50 / p95 / max at 60 fps | at 120 fps | CPU per frame, 60 / 120 fps |301+|---|---|---|---|302+| 10, new sheet | 10.8 / 16.6 / 16.8 ms | 5.1 / 8.3 / 9.6 ms | 0.78 / 0.67 ms |303+| 10, 1000 x 26 numbers | 9.7 / 16.6 / 20.7 ms | 4.3 / 8.3 / 8.5 ms | 1.41 / 0.71 ms |304+| 50, new sheet | 9.3 / 16.3 / 18.5 ms | 5.0 / 8.7 / 17.8 ms | 0.58 / 0.53 ms |305+| 50, 1000 x 26 numbers | 9.5 / 16.7 / 19.7 ms | 5.1 / 8.9 / 12.8 ms | 0.71 / 0.66 ms |306+228307 Latency doesn't move from 10 to 50 sessions: it's Bubble Tea's frame229-pacing (at most one frame every 16.7 ms, so a key waits half a frame on308+pacing (at most one frame every 8.3 ms, so a key waits half a frame on230309 average), not load. A heap profile with 50 sessions open231310 (`SERVE_HEAP_PROFILE=file`) puts three quarters of a session's heap in232311 Bubble Tea's render buffers (about 1 MiB at 120 x 40, growing with the@@ -236,7 +315,9 @@ top come about 300 B per cell of the sheets a session has open;
236315 sessions opening the same file each hold a copy. CPU per frame covers both ends of the connection (the237316 encryption twice, the client reading the frame); at 0.7 ms, 50 sessions238317 typing continuously at 60 frames a second would keep about two cores239-busy.318+busy, and at 120 about four: frames are drawn only when a session's319+screen changes, so the rate is the most keys can cause, not a cost of320+being connected.240321 241322 ## Pivot tables242323 @@ -293,14 +374,17 @@ Each with its benchmark before and after, in the commit that fixed it.
293374 | `Names()` upper-cased both names in every sort comparison, every frame | 21 k allocations per View with 1000 names | View/names 80 x 24: 662 us | 197 us |294375 | Status line summed a whole-sheet selection with 2 M map lookups, every frame | CPU profile: `RangeStats` 87% of a frame | 91 ms per frame | 0.3 ms (cached), 29 ms when the selection changes |295376 | JEV cache keys encoded with `encoding/json` | CPU profile of answers: JSON 60% of recalc | JEV/1000 575 ms, JEV/4000 9.3 s | 263 ms, 5.1 s |377+| Changing a selection summed every cell in it | CPU profile of Shift+Up over the whole sheet: `RangeStats` 54% | extend-data 80 x 24: 31 to 44 ms | 0.53 ms |378+| Every JEV answer recalculated every JEV cell | Per-answer cost grew with the number of JEV cells | JEV/4000 4.9 s | 15 ms |379+| Bubble Tea drew at most 60 frames a second | Arrow keys through a terminal waited for the next frame | p50 16.6 ms, p95 18.4 ms | 8.3 ms, 10.2 ms |380+| Lazy evaluation recursed once per link of a chain | A chain through all 2 M cells of a sheet overflowed the 1 GB stack and killed the program | crash | 4 s to build and evaluate, under 32 MB of stack |296381 297382 ## What would raise the bounds298383 299384 In order of value for effort. The first three are what raising the300385 limits toward Google Sheets' 10 M cells (or Excel's 1,048,576 x 16,384)301386 would take; without them a 10 M-cell sheet would need about 3 GB of302-heap, take 10 s to open and 140 ms per key to extend a selection over303-it.387+heap and take 10 s to open.304388 305389 1. **Compact cell storage** (L, 2 to 3 weeks). Cells live in a306390 `map[Addr]*Cell` behind `cellStore` (`internal/sheet/store.go`), 300 B@@ -330,26 +414,20 @@ it.
330414 allocation tenfold; a columnar or gzip-compressed variant would cut331415 the size about fourfold. The format is `internal/sheet/file.go`, apart332416 from the cell store.333-4. **Incremental selection statistics** (S, 2 days). Keep per-column334- sums and counts, updated in `place` and after recalc, and compute a335- selection's Sum and Count from column totals minus the rows outside336- it; extending a whole-sheet selection would drop from 29 ms to well337- under 1 ms.338-5. **JEV recalc by question** (S to M, 3 days). Index JEV cells by339- question key and recalculate only the cells whose question was340- answered (and their dependents), and coalesce answers that arrive in341- the same frame: answering 4000 questions would cost 4000 small342- recalcs instead of 4000 full ones.417+4. **Incremental selection statistics.** Done: per-column block418+ aggregates, marked stale as cells change (see [Rendering](#rendering));419+ extending a whole-sheet selection went from 31 to 44 ms to 0.53 ms.420+5. **JEV recalc by question.** Done: an answer recalculates the cells421+ waiting for it, and answers within a frame recalculate together (see422+ [JEV](#jev)); answering 4000 questions went from 4.9 s of CPU to423+ 15 ms.343424 6. **Smaller undo steps** (S, 1 day). The history is capped by the344425 memory its before-images hold; storing formatting-only changes as345426 diffs rather than whole cells would let it keep more of them.346-7. **Frame rate** (S, hours). Bubble Tea draws at most 60 frames a347- second; asking for 120 would halve the key-to-screen floor from about348- 16 ms to 8 ms at the cost of more redraws.349-8. **Recursion limits** (S, hours). The parser (`internal/formula`) and350- the evaluator recurse without a limit; a depth cap (Excel allows 64351- nested functions) would turn a pathological file into an error rather352- than a deep stack.427+7. **Frame rate.** Done: 120 frames a second, Bubble Tea's maximum,428+ locally and over SSH; arrow keys through a terminal went from p50429+ 16.6 ms to 8.3 ms, with no more CPU when idle.430+8. **Recursion limits.** Done: see [Formula depth](#formula-depth).353431 354432 ## Measuring355433