Each step counts its before-images as they are recorded, the history keeps a running total, and the oldest steps are dropped once it holds more than 256 MB (the newest is always kept). BenchmarkHistoryWide shows the cap; limits.md has the numbers before and after.
- commit
- 344feb4ccacc405c304e8bf589cf4e2e70a78eef
- parent
- 9a70f51870c8a0b2c3fc005c4c3d9d0d9c341482
- author
- Dami <dami.etoile@gmail.com>
| M | docs/ | 2 | + |
| M | docs/ | 1 | - |
| M | docs/ | 34 | ++++++-- |
| M | internal/ | 73 | +++++++++++------ |
| A | internal/ | 106 | ++++++++++++++++++++++++ |
| A | internal/ | 132 | ++++++++++++++++++++++++++++++ |
| M | internal/ | 20 | +++++ |
@@ -85,6 +85,8 @@ style.
8585 the cells, widths, names, charts, view state and sheet list they change,8686 on any sheet, so a step can be reversed exactly; multi-cell operations8787 are one step. A deleted sheet keeps its cells, so undo brings it back.88+ The history keeps at most 100 steps and about 256 MB of before-images,89+ counted as they're recorded, dropping the oldest steps first.8890 - **JEV.** The engine never touches the network. JEV functions describe a8991 question and look up the answer in the workbook's `RemoteSource`, set9092 with `SetRemote`; `internal/jev` answers from a cache and queues new@@ -114,7 +114,6 @@ a golden screen reviewed in the gallery for anything visible (see
114114 115115 Where the code doesn't follow the patterns yet:116116 117-- Undo history is capped by step count, not bytes (pattern 8).118117 - Components in `internal/ui` are handed the whole `*Model` rather than a119118 narrower interface, so they stay in package `ui` (pattern 2).120119 - Movement keys, typing, F4 in formulas, Alt+letter menus and direct mouse@@ -25,7 +25,7 @@ it lags, and past a second it stalls.
2525 | Rendering | Any sheet at up to 200 x 60: 1 ms a frame; 400 x 120: 4.5 ms | 20 charts at 400 x 120: 7 ms | | View building styled strings, then Bubble Tea parsing and diffing them |2626 | Selection statistics | Any selection, once computed (cached) | Extending a selection over 2 M cells: 30 ms per key | | Summing 2 M map entries per change |2727 | Imports | CSV, SQLite, Parquet: 1 to 3 M cells/s (a full 2.1 M-cell sheet in about 1 s) | XLSX with formulas: 200 k cells/s | Data past the limits (dropped, with a note) | Building cells one at a time; XLSX formula translation |28-| Undo | One step of any size: undo costs what the edit cost | 100 steps that each rewrite a whole column: 217 MB held | | Whole-cell before-images, 264 B per cell per step |28+| Undo | One step of any size: undo costs what the edit cost | History capped at 100 steps and 256 MB of before-images: 100 whole-column steps hold 250 MB | | Whole-cell before-images, about 300 B per cell per step |2929 | JEV | Up to about 1000 JEV cells: 0.3 ms of CPU per answer | 4000 JEV cells: 1.3 ms per answer, 5 s of CPU to answer them all | | Every answer recalculates every JEV cell (they're volatile) |3030 | Formula depth | 10,000 nested parentheses or IFs: under 5 ms | | No explicit limit; recursion grows the stack | Recursive parser and evaluator |3131 | Find, filter, sort, fill | Filter or sort 8191 rows: 1 to 32 ms; find over 213 k cells: 38 ms; fill 8192 rows: 4 ms | Replace all over 213 k cells: 280 ms | | Per-cell string conversion and regexp |@@ -131,11 +131,28 @@ XLSX worksheet in memory while its rows are read.
131131 132132 ## Undo133133 134-Each step keeps a copy of every cell it changed (`MaxUndo = 100`135-steps). Undoing costs about what the change cost, since it recalculates136-the same cells. Clearing 213 k cells and undoing it takes 158 ms. The137-history is bounded by steps, not bytes: 100 steps that each rewrite a138-whole column of 8192 cells hold 217 MB.134+Each step keeps a copy of every cell it changed. Undoing costs about135+what the change cost, since it recalculates the same cells. Clearing136+213 k cells and undoing it takes 158 ms.137+138+The history is bounded by steps (`MaxUndo = 100`) and by an estimate of139+the memory its before-images hold (`MaxUndoBytes = 256 MB`); past either,140+the oldest steps are dropped first, and the newest step is kept however141+large it is. Each cell is counted as it is recorded (a fixed cost per142+cell plus its text, more for a formula, whose parsed tree stays alive),143+so the budget costs nothing to check. The estimate follows the measured144+heap within about 10% (`BenchmarkHistoryFull`, `BenchmarkHistoryWide`,145+3 runs each, same machine):146+147+| Benchmark | Before | After |148+|---|---|---|149+| 100 edits rewriting a whole column (819 k before-images) | 250 MB held, all 100 kept | 250 MB held (239 MB estimated), all 100 kept |150+| 12 edits rewriting all of 8192 x 26 (2.6 M before-images) | 705 MB held, all 12 kept | 230 MB held (248 MB estimated), the last 4 kept |151+| BigUndo: clear 8192 x 26, undo | 154 ms, 430 k allocs | 158 ms, 430 k allocs |152+| Undo + redo of one cell (dense, names) | 0.98 us, 106 us | 1.0 us, 106 us |153+154+Counting cells as they're recorded costs about 20 ns a cell, which is155+BigUndo's 3%; everything else is unchanged.139156 140157 ## JEV141158 @@ -247,8 +264,9 @@ it.
247264 answered (and their dependents), and coalesce answers that arrive in248265 the same frame: answering 4000 questions would cost 4000 small249266 recalcs instead of 4000 full ones.250-6. **Undo bounded by bytes** (S, 1 day). Cap the history by the memory251- its before-images hold, and store formatting-only changes as diffs.267+6. **Smaller undo steps** (S, 1 day). The history is capped by the268+ memory its before-images hold; storing formatting-only changes as269+ diffs rather than whole cells would let it keep more of them.252270 7. **Frame rate** (S, hours). Bubble Tea draws at most 60 frames a253271 second; asking for 120 would halve the key-to-screen floor from about254272 16 ms to 8 ms at the cost of more redraws.@@ -7,7 +7,8 @@ import (
77 "github.com/FelineStateMachine/012/internal/formula"88 )99 10-// MaxUndo is how many steps of undo history a workbook keeps.10+// MaxUndo is how many steps of undo history a workbook keeps, at most;11+// MaxUndoBytes also bounds it.1112 const MaxUndo = 1001213 1314 // History is a command log of before-images, shared by every sheet of a@@ -27,6 +28,7 @@ type history struct {
2728 depth int // nesting of change calls2829 dirty []loc2930 lastID int31+ bytes int64 // estimated heap held by the undo steps; see historysize.go3032 // mergeWidths lets the next width-only step join the top one, so a3133 // live preview of a column width and its final value (or its3234 // cancellation) are one step. Seal ends the run.@@ -34,14 +36,18 @@ type history struct {
3436 }3537 3638 type step struct {37- id int38- label string // what the step did, e.g. "clear B3:B5"39- sheet *Sheet // the sheet the UI shows when the step is undone or redone40- focus Rect // what the UI selects there41- cells map[loc]*Cell // before the step; nil for blank42- widths map[colKey]int // before the step; 0 for the default width43- names map[string]*Name // before the step, by key; nil for undefined44- views map[*Sheet]*viewState39+ id int40+ bytes int64 // estimated heap held, set when pushed on the undo stack41+ // cellBytes is the estimated heap held by cells, kept as they are42+ // recorded so sizing a step doesn't walk them again.43+ cellBytes int6444+ label string // what the step did, e.g. "clear B3:B5"45+ sheet *Sheet // the sheet the UI shows when the step is undone or redone46+ focus Rect // what the UI selects there47+ cells map[loc]*Cell // before the step; nil for blank48+ widths map[colKey]int // before the step; 0 for the default width49+ names map[string]*Name // before the step, by key; nil for undefined50+ views map[*Sheet]*viewState4551 // charts holds each touched sheet's charts before the step.4652 charts map[*Sheet][]Chart4753 // sheets is the sheet list before the step, when it changed.@@ -145,7 +151,7 @@ func (s *Sheet) record(a Addr) {
145151 }146152 l := loc{s, a}147153 if _, seen := st.cells[l]; !seen {148- st.cells[l] = s.cells.get(a).clone()154+ st.keep(l, s.cells.get(a).clone())149155 }150156 s.wb.hist.dirty = append(s.wb.hist.dirty, l)151157 }@@ -252,10 +258,7 @@ func (w *Workbook) push(st *step) {
252258 return253259 }254260 st.id = h.lastID255- h.undo = append(h.undo, st)256- if len(h.undo) > MaxUndo {257- h.undo = h.undo[len(h.undo)-MaxUndo:]258- }261+ w.pushUndo(st)259262 h.mergeWidths = widthOnly260263 }261264 @@ -269,6 +272,7 @@ func (w *Workbook) dropUnchanged(st *step) {
269272 for l, c := range st.cells {270273 if c == nil && l.s.cells.get(l.a) == nil {271274 delete(st.cells, l)275+ st.cellBytes -= entryBytes272276 }273277 }274278 for k, n := range st.names {@@ -318,8 +322,11 @@ func (h *history) joinWidths(top, st *step) {
318322 }319323 }320324 top.id, top.focus = h.lastID, union(top.focus, st.focus)325+ h.bytes -= top.bytes326+ top.bytes = top.size()327+ h.bytes += top.bytes321328 if top.empty() {322- h.undo = h.undo[:len(h.undo)-1]329+ h.popUndo()323330 }324331 }325332 @@ -335,10 +342,10 @@ func (h *history) top() *step {
335342 func (w *Workbook) Seal() { w.hist.mergeWidths = false }336343 337344 // Undo reverts the last step and describes it.338-func (w *Workbook) Undo() (Change, bool) { return w.swap(&w.hist.undo, &w.hist.redo) }345+func (w *Workbook) Undo() (Change, bool) { return w.swap(true) }339346 340347 // Redo reapplies the last undone step and describes it.341-func (w *Workbook) Redo() (Change, bool) { return w.swap(&w.hist.redo, &w.hist.undo) }348+func (w *Workbook) Redo() (Change, bool) { return w.swap(false) }342349 343350 // CanUndo and CanRedo report whether there is a step to undo or redo.344351 func (w *Workbook) CanUndo() bool { return len(w.hist.undo) > 0 }@@ -368,14 +375,24 @@ func (s *Sheet) CanRedo() bool { return s.wb.CanRedo() }
368375 func (s *Sheet) StateID() int { return s.wb.StateID() }369376 func (s *Sheet) ClearHistory() { s.wb.ClearHistory() }370377 371-// swap pops a step from one stack, restores its before-image, and pushes372-// the state it replaced onto the other stack.373-func (w *Workbook) swap(from, to *[]*step) (Change, bool) {374- if len(*from) == 0 || w.hist.open != nil {378+// swap pops a step from the undo stack (or the redo stack), restores its379+// before-image, and pushes the state it replaced onto the other stack.380+func (w *Workbook) swap(undo bool) (Change, bool) {381+ h := &w.hist382+ from := &h.redo383+ if undo {384+ from = &h.undo385+ }386+ if len(*from) == 0 || h.open != nil {375387 return Change{}, false376388 }377- st := (*from)[len(*from)-1]378- *from = (*from)[:len(*from)-1]389+ var st *step390+ if undo {391+ st = h.popUndo()392+ } else {393+ st = h.redo[len(h.redo)-1]394+ h.redo = h.redo[:len(h.redo)-1]395+ }379396 // The inverse keeps the step's ID: on the redo stack, an ID names the380397 // state the step leads back to.381398 inv := newStep(st.label, st.sheet, st.focus)@@ -387,7 +404,7 @@ func (w *Workbook) swap(from, to *[]*step) (Change, bool) {
387404 }388405 changed := make([]loc, 0, len(st.cells))389406 for l, c := range st.cells {390- inv.cells[l] = l.s.cells.get(l.a).clone()407+ inv.keep(l, l.s.cells.get(l.a).clone())391408 l.s.place(l.a, c.clone())392409 changed = append(changed, l)393410 }@@ -420,8 +437,12 @@ func (w *Workbook) swap(from, to *[]*step) (Change, bool) {
420437 } else {421438 w.recalc(changed)422439 }423- *to = append(*to, inv)424- w.hist.mergeWidths = false440+ if undo {441+ h.redo = append(h.redo, inv)442+ } else {443+ w.pushUndo(inv)444+ }445+ h.mergeWidths = false425446 return Change{st.label, st.focus, st.sheet, st.sheets != nil}, true426447 }427448 @@ -0,0 +1,106 @@
1+package sheet2+3+import "slices"4+5+// MaxUndoBytes caps the memory the undo history's before-images hold, as6+// estimated by step.size. When a new step takes the history past it, the7+// oldest steps are dropped first; the newest step is always kept, however8+// large, so any single change can be undone. 256 MB keeps MaxUndo steps9+// that each rewrite a whole column (about 250 MB, see docs/limits.md).10+const MaxUndoBytes = 256 << 2011+12+// undoBudget is MaxUndoBytes; tests lower it.13+var undoBudget int64 = MaxUndoBytes14+15+// Estimated heap held by the parts of a step. A before-image is a shallow16+// copy of a Cell (216 B) plus its map entry and boxed literal; a formula17+// also keeps its parsed tree and reference lists alive, which grow with18+// its text. The constants match the heap measured by the stress19+// benchmarks (BenchmarkHistoryFull) within about 10%.20+const (21+ stepBytes = 512 // the step and its empty maps22+ entryBytes = 64 // a map entry: the key and a pointer23+ cellBytes = 240 // a copied Cell and its boxed literal24+ formulaBytes = 256 // a parsed formula's tree and references, plus perFormulaByte per byte of text25+ perFormulaByte = 1626+ chartBytes = 12827+)28+29+// keep records c as the before-image of the cell at l and counts it.30+func (st *step) keep(l loc, c *Cell) {31+ st.cells[l] = c32+ st.cellBytes += entryBytes + cellSize(c)33+}34+35+// size estimates the heap a finished step holds: its cells, counted as36+// they were recorded, and the rest, which is small.37+func (st *step) size() int64 {38+ n := stepBytes + st.cellBytes39+ n += int64(len(st.widths)+len(st.names)+len(st.views)) * entryBytes40+ for _, charts := range st.charts {41+ n += entryBytes + int64(len(charts))*chartBytes42+ }43+ if st.sheets != nil {44+ n += int64(len(st.sheets.order)) * 2 * entryBytes45+ }46+ return n47+}48+49+// cellSize estimates the heap one before-image holds; nil stands for a50+// blank cell.51+func cellSize(c *Cell) int64 {52+ if c == nil {53+ return 054+ }55+ n := int64(cellBytes + len(c.Input))56+ if c.IsFormula() {57+ n += formulaBytes + perFormulaByte*int64(len(c.Input))58+ }59+ return n60+}61+62+// deletedSize estimates the cells held by sheets a step's sheet list63+// has that the workbook no longer does: a deleted sheet keeps its cells64+// so undo can bring it back.65+func (w *Workbook) deletedSize(st *step) int64 {66+ if st.sheets == nil {67+ return 068+ }69+ var n int6470+ for _, s := range st.sheets.order {71+ if !slices.Contains(w.sheets, s) {72+ n += int64(s.Len()) * (entryBytes + cellBytes)73+ }74+ }75+ return n76+}77+78+// pushUndo adds st to the undo stack with its size, then drops the79+// oldest steps while the history holds more than MaxUndo steps or more80+// than the byte budget.81+func (w *Workbook) pushUndo(st *step) {82+ h := &w.hist83+ st.bytes = st.size() + w.deletedSize(st)84+ h.undo = append(h.undo, st)85+ h.bytes += st.bytes86+ drop := 087+ for len(h.undo)-drop > MaxUndo || h.bytes > undoBudget && len(h.undo)-drop > 1 {88+ h.bytes -= h.undo[drop].bytes89+ drop++90+ }91+ if drop > 0 {92+ clear(h.undo[:drop])93+ h.undo = h.undo[drop:]94+ }95+}96+97+// popUndo removes and returns the top of the undo stack.98+func (h *history) popUndo() *step {99+ st := h.undo[len(h.undo)-1]100+ h.undo = h.undo[:len(h.undo)-1]101+ h.bytes -= st.bytes102+ return st103+}104+105+// HistoryBytes estimates the memory the undo steps hold.106+func (w *Workbook) HistoryBytes() int64 { return w.hist.bytes }@@ -0,0 +1,132 @@
1+package sheet2+3+import (4+ "strconv"5+ "testing"6+)7+8+// fillColumn writes v into rows 0..n-1 of column A as one step.9+func fillColumn(t *testing.T, s *Sheet, n int, v string) {10+ t.Helper()11+ col := NewRect(Addr{}, Addr{Row: n - 1})12+ if err := s.FillEntry(col, col.From, v); err != nil {13+ t.Fatal(err)14+ }15+}16+17+// The history drops its oldest steps once it holds more than the byte18+// budget, keeps a running total that matches its steps, and always keeps19+// the newest step.20+func TestUndoByteBudget(t *testing.T) {21+ s := New()22+ fillColumn(t, s, 1000, "x")23+ w := s.Book()24+ w.ClearHistory()25+ fillColumn(t, s, 1000, "0") // before-images of 1000 cells26+ one := w.HistoryBytes()27+ if one < 1000*cellBytes/2 || one > 1000*(entryBytes+cellBytes+64) {28+ t.Fatalf("a 1000-cell step is estimated at %d bytes", one)29+ }30+ defer func(b int64) { undoBudget = b }(undoBudget)31+ undoBudget = 5 * one32+33+ for i := 1; i < 20; i++ {34+ fillColumn(t, s, 1000, strconv.Itoa(i))35+ }36+ if w.HistoryBytes() > undoBudget {37+ t.Errorf("history holds %d bytes, over the budget of %d", w.HistoryBytes(), undoBudget)38+ }39+ if got := historySum(w); got != w.HistoryBytes() {40+ t.Errorf("running total %d, steps add up to %d", w.HistoryBytes(), got)41+ }42+ n := 043+ for s.CanUndo() {44+ s.Undo()45+ n++46+ }47+ if n < 3 || n > 5 {48+ t.Errorf("kept %d steps under a budget of about 5", n)49+ }50+ // The oldest steps went first: undoing everything kept lands on the51+ // value written n steps before the last.52+ if want := strconv.Itoa(19 - n); inputs(s)["A1"] != want {53+ t.Errorf("after undoing all: A1 %q, want %q", inputs(s)["A1"], want)54+ }55+ if w.HistoryBytes() != 0 {56+ t.Errorf("empty undo stack holds %d bytes", w.HistoryBytes())57+ }58+ // Redoing brings the steps back onto the undo stack, counted again.59+ for s.CanRedo() {60+ s.Redo()61+ }62+ if got := historySum(w); got != w.HistoryBytes() || got == 0 || got > undoBudget {63+ t.Errorf("after redo: running total %d, steps %d", w.HistoryBytes(), got)64+ }65+66+ // A single step bigger than the budget is still kept.67+ undoBudget = one / 268+ fillColumn(t, s, 1000, "big")69+ if !s.CanUndo() {70+ t.Fatal("the newest step was dropped")71+ }72+ s.Undo()73+ if s.CanUndo() {74+ t.Error("older steps survived a budget smaller than one step")75+ }76+ if inputs(s)["A1"] != "19" {77+ t.Errorf("undo big: A1 %q", inputs(s)["A1"])78+ }79+}80+81+// Formulas are estimated larger than numbers: their parsed trees stay82+// alive in the before-image.83+func TestUndoBytesFormulas(t *testing.T) {84+ nums, forms := New(), New()85+ fillColumn(t, nums, 100, "1")86+ fillColumn(t, forms, 100, "=SUM(B1:B10)*2")87+ nums.Book().ClearHistory()88+ forms.Book().ClearHistory()89+ fillColumn(t, nums, 100, "")90+ fillColumn(t, forms, 100, "")91+ if n, f := nums.Book().HistoryBytes(), forms.Book().HistoryBytes(); f < n+100*formulaBytes {92+ t.Errorf("clearing 100 numbers %d bytes, 100 formulas %d", n, f)93+ }94+}95+96+// A deleted sheet keeps its cells for undo, and the step that deleted it97+// counts them.98+func TestUndoBytesCountDeletedSheet(t *testing.T) {99+ s := New()100+ w := s.Book()101+ data, err := w.AddSheet("Data", 1)102+ if err != nil {103+ t.Fatal(err)104+ }105+ for i := range 500 {106+ data.Set(Addr{Row: i}, "x")107+ }108+ w.ClearHistory()109+ if err := w.DeleteSheet(data); err != nil {110+ t.Fatal(err)111+ }112+ if got := w.HistoryBytes(); got < 500*cellBytes {113+ t.Errorf("deleting a 500-cell sheet is estimated at %d bytes", got)114+ }115+}116+117+// historySum adds up the undo steps' sizes, checking each step's118+// running count of its cells against a walk over them.119+func historySum(w *Workbook) int64 {120+ var n int64121+ for _, st := range w.hist.undo {122+ var cells int64123+ for _, c := range st.cells {124+ cells += entryBytes + cellSize(c)125+ }126+ if cells != st.cellBytes {127+ return -1128+ }129+ n += st.bytes130+ }131+ return n132+}@@ -94,6 +94,26 @@ func BenchmarkHistoryFull(b *testing.B) {
9494 }9595 }9696 b.ReportMetric(float64(heap()-before)/(1<<20), "MB-history")97+ b.ReportMetric(float64(s.Book().HistoryBytes())/(1<<20), "MB-estimate")98+ runtime.KeepAlive(s)99+ }100+}101+102+// BenchmarkHistoryWide makes 12 edits that each rewrite a whole103+// 8192x26 block, more than the history's byte budget holds, and reports104+// the heap the history keeps.105+func BenchmarkHistoryWide(b *testing.B) {106+ all := sheet.NewRect(sheet.Addr{}, sheet.Addr{Col: 25, Row: sheet.MaxRows - 1})107+ for b.Loop() {108+ s := stress.Dense(sheet.MaxRows, 26)109+ before := heap()110+ for i := range 12 {111+ if err := s.FillEntry(all, all.From, fmt.Sprint(i)); err != nil {112+ b.Fatal(err)113+ }114+ }115+ b.ReportMetric(float64(heap()-before)/(1<<20), "MB-history")116+ b.ReportMetric(float64(s.Book().HistoryBytes())/(1<<20), "MB-estimate")97117 runtime.KeepAlive(s)98118 }99119 }