# Conflicts: # ROADMAP.md
- commit
- 6328ad1a0791ef9a0cc8d0ea1a24518e87253eea
- parent
- 4074c28f70d6b62647d7870d08880f0fb9896d11
- parent
- bd5d0bbd57e9f2e735e742b61c9d5d8dc9663161
- author
- Dami <dami.etoile@gmail.com>
@@ -29,7 +29,8 @@ rest: building from a clone, importing files, `012 serve` and settings.
2929 mouse, named ranges, references between sheets, arrays that spill3030 (FILTER, SORT, UNIQUE, LAMBDA), optional decimal3131 arithmetic for money, and undo for everything32- ([formulas](docs/formulas/README.md)).32+ ([formulas](docs/formulas/README.md)); numbers, dates and formulas33+ typed and shown in a [locale](docs/sheets/locale.md) per file.3334 - **Data tools**: freeze, sort, filter, find and replace, conditional3435 formatting, dropdowns and checkboxes, notes, protected ranges, and live3536 pivot tables ([working with data](docs/sheets/README.md)); wrapped@@ -56,19 +57,16 @@ rest: building from a clone, importing files, `012 serve` and settings.
5657 | Menus and the command palette (Ctrl+K) | Charts that float over the grid and follow their data |5758 |  |  |5859 | Conditional formatting and data validation | Pivot tables, live |59-|  |  |60-| Freeze, sort and filter | Find and replace |61-|  |  |62-| JEV functions in formulas | Formats detected as you type |63-|  |  |64-| Arrays that spill | Macros |65-|  |  |66-| Wrapped text, borders and merged cells | A locale per file |60+|  |  |61+| Arrays that spill | Wrapped text, borders and merged cells |62+|  |  |63+| JEV functions in formulas | Macros |6764 68-The recordings are [VHS](https://github.com/charmbracelet/vhs) tapes in69-[`demos/`](demos), rendered by `make demos` in the Catppuccin Mocha70-palette; 012 draws in your terminal's own colors by default71-([themes](docs/terminal/themes.md)). Charts show as text here.65+Each guide shows its own recordings too. They are66+[VHS](https://github.com/charmbracelet/vhs) tapes in [`demos/`](demos),67+rendered by `make demos` in the Catppuccin Mocha palette; 012 draws in68+your terminal's own colors by default ([themes](docs/terminal/themes.md)).69+Charts show as text here.7270 7371 ## More7472 @@ -18,7 +18,6 @@ Sizes: S (a day or two), M (about a week), L (weeks).
1818 1919 | Item | Why | Size |2020 |---|---|---|21-| Open 012's XLSX output in Excel and LibreOffice before releases, as was done for Google Sheets | The writer is otherwise checked by excelize and 012's reader | S |2221 | Conditional formatting and validation, what's left: data bars and icon sets, "top 10", "duplicate values", date periods (this week, last month), rules moving with cells cut to another sheet, a dropdown's chip look, and custom checkbox values | Excel's other rule types come in as notes | M |2322 | Remaining overlays get narrow hosts: filter picker, sort and choice bars, chart editor, shortcuts, named ranges, cell entry and prompts | Components testable without the model ([Architecture](docs/contributing/architecture.md#the-ui)) | M |2423 @@ -36,7 +35,6 @@ rises in measured steps; see [Bounds of support](docs/contributing/limits.md#wha
3635 3736 | Item | Result | Size |3837 |---|---|---|39-| Compact spilled and pivot cells: derived cells in the compact slot form instead of whole `Cell`s | Large spills and pivots cost what plain cells cost ([Bounds of support](docs/contributing/limits.md#what-would-raise-the-bounds)) | M |4038 | Linked, paged read-only ranges over Parquet and SQLite that feed pivots and formulas by streaming | Sources too big for any grid | L |4139 4240 ### 4. Macros, keys and the terminal@@ -44,7 +42,6 @@ rises in measured steps; see [Bounds of support](docs/contributing/limits.md#wha
4442 | Item | Size |4543 |---|---|4644 | Record dialog choices (sort bar, filter picker, find and replace, chart editor) and chart drags; let scripts run commands that open dialogs, with answers | M |47-| Sixel chart images, redrawn on resize, for terminals without kitty graphics | M |4845 4946 ## Later: sharing a live sheet (shelved)5047 @@ -74,9 +71,9 @@ by a gateway dialing the iroh ticket.
7471 - Dynamic arrays that spill (FILTER, SORT, UNIQUE, SEQUENCE and more, ARRAYFORMULA), LET and LAMBDA, SPLIT and the REGEX functions: [Arrays and spills](docs/formulas/arrays.md)7572 - Opt-in decimal arithmetic for money: [Decimal arithmetic](docs/formulas/decimal.md)7673 - A locale per file, as Sheets' File > Settings > Locale: decimal commas, date order, currency and `;` in formulas, typed and shown while files store en-US's form: [Locale](docs/sheets/locale.md)77-- An Excel-sized grid with a `max-cells` budget; operations cost the data, not the grid: [Bounds of support](docs/contributing/limits.md#sheet-size)78-- Compact column storage and a ten-million-cell `max-cells` budget: [Bounds of support](docs/contributing/limits.md#sheet-size)74+- An Excel-sized grid in compact column storage, with a ten-million-cell `max-cells` budget; operations cost the data, not the grid: [Bounds of support](docs/contributing/limits.md#sheet-size)7975 - Undo steps in the compact form: clearing a full ten-million-cell sheet holds about what the sheet does, and a step past 1 GB asks first: [Bounds of support](docs/contributing/limits.md#undo)76+- Spilled cells and pivot results in the compact form, about 20 B each: [Bounds of support](docs/contributing/limits.md#sheet-size)8077 8178 **Finding and using features**8279 @@ -105,6 +102,7 @@ by a gateway dialing the iroh ticket.
105102 - A high-contrast theme at WCAG AAA, and every state readable without color: [Themes](docs/terminal/themes.md#high-contrast), [UX](docs/contributing/ux.md#reading-without-color)106103 - `012 serve` over SSH, with files on the ssh command line and recovery of unsaved work: [Serving over SSH](docs/terminal/ssh.md)107104 - Shift+Enter, Ctrl+I and keys held to preview, with the kitty keyboard protocol: [Keys and mouse](docs/reference/keys.md#keys-the-terminal-has-to-tell-apart)105+- Sixel chart images on terminals without kitty graphics, drawn after the frame and redrawn as the screen moves: [Charts](docs/sheets/charts.md)108106 109107 **Files**110108 @@ -114,6 +112,6 @@ by a gateway dialing the iroh ticket.
114112 **Upkeep**115113 116114 - `make check` before every push; demo tapes (`make demos`); annotated version tags with release notes (v0.2.0 onward; v0.1.0 remains on the Go module proxy): [Testing](docs/contributing/testing.md)117-- A release checklist and `make dist`, release archives cross-compiled locally with SHA256SUMS: [Releasing](docs/contributing/releasing.md)115+- A release checklist (XLSX output opened in Excel, LibreOffice and Google Sheets) and `make dist`, release archives cross-compiled locally with SHA256SUMS: [Releasing](docs/contributing/releasing.md)118116 - `make stress-report` flags regressions against the last release's run, allowing for noise: [Observability](docs/contributing/observability.md#regressions-against-the-last-release)119117 - Grafana: recent traces and a trace view of the nested spans: [Observability](docs/contributing/observability.md#the-stack)@@ -118,7 +118,9 @@ style.
118118 its slot, with its text in a table of strings kept once each, its119119 formatting in a table of looks, and its input only when that isn't120120 the value's own text ("1.50" is kept as 1.5 printed with two121- decimals). Formulas, notes, and what pivots and spills write are whole121+ decimals). What pivots and spills write is a slot too, marked as122+ derived, holding the value and, in its look, a spill's inferred123+ format; its entry is the value's text. Formulas and notes are whole122124 `Cell`s in a side table, which recalculation updates in place. `get`123125 hands out a plain cell as a `Cell` made for the caller, a copy whose124126 changes reach nothing; `set` is the one way to change a cell. What@@ -323,7 +325,7 @@ move without touching callers:
323325 the places functions read ranges.324326 - **Files.** The `.012` reader and writer stream (`fileread.go`,325327 `filescan.go`) and meet the store only through its methods, so another326- encoding would sit beside them; [The .012 format](../files/format.md#reading-and-writing)328+ encoding would sit beside them; [Bounds of support](limits.md#the-012-file)327329 says why there is one.328330 - **Depth limits.** `internal/formula`'s parser caps nesting at329331 `formula.MaxDepth` (1024 levels), so a pathological formula fails to@@ -500,8 +502,12 @@ the user's editor (`AllowEditor`), since that starts a program.
500502 `internal/chart` draws a chart in two layers that share one layout: text501503 for any terminal (block elements for bars, braille for lines, half blocks502504 for pies, eighths filled column by column for areas) and, on terminals503-with kitty graphics, an image of the plot area with the axes and legend504-still terminal text. Each chart type is a layout in the `types` table,505+with kitty or sixel graphics, an image of the plot area with the axes and506+legend still terminal text. A kitty image is placed by placeholder507+characters, which the renderer treats as text; a sixel image508+(`chart.Sixel`) is pixels Bubble Tea's renderer doesn't know of, so509+`internal/ui/sixel.go` keeps blank cells for it, draws it once the frame510+has settled, and clears the screen when it moves. Each chart type is a layout in the `types` table,505511 returning a plan that draws it both ways, and the series its legend506512 lists; `Draw` places the legend and gives the layout the room left. The507513 type constants and their names, and the options of each chart@@ -38,7 +38,7 @@ it lags, and past a second it stalls.
3838 | Find, filter, sort, fill | Filter or sort 8191 rows: 1 to 32 ms; find over 213 k cells: 38 ms; fill 8192 rows: 4 ms | Replace all over 213 k cells: 280 ms | | Per-cell string conversion and regexp |3939 | SSH sessions (`012 serve`) | 50 sessions typing at once: frames within one frame interval (p95 8.7 ms), 1.3 MiB per session plus its sheets (upper bound) | | More than `--max-sessions` (8 by default): turned away | Bubble Tea's 120 fps pacing; per session, the terminal's cell buffers |4040 | Pivot tables | A pivot over 8191 rows recomputes after an edit to its source in 0.8 to 2.5 ms | | Results past the grid (the pivot shows #REF!) | Grouping the source rows |41-| Arrays and spills | FILTER, SORT or UNIQUE over 8192 rows: about 0.1 ms each per edit; 1000 of them spilling 516 k cells, an edit recomputing 500: 52 ms | Full recalculation of those 1000: 217 ms | An array past `maxArray` values (2,097,152) stored: `#VALUE!`; a spill past the sheet's edge or `max-cells` cells: `#REF!` | Computing each array's values; writing only the spilled cells that changed |41+| Arrays and spills | FILTER, SORT or UNIQUE over 8192 rows: about 0.1 ms each per edit; 1000 of them spilling 516 k cells, an edit recomputing 500: 52 ms | Full recalculation of those 1000: 106 ms | An array past `maxArray` values (2,097,152) stored: `#VALUE!`; a spill past the sheet's edge or `max-cells` cells: `#REF!` | Computing each array's values; writing only the spilled cells that changed |4242 4343 ## Sheet size4444 @@ -48,9 +48,8 @@ ten million cells by default): imports keep whole rows up to it, pastes
4848 and fills write at most that many cells at once. WK1 files keep their own4949 8192 x 256. The costs below grow with the cells; the grid itself costs5050 nothing (see [The grid](#the-grid)). A `.012` file is read and written51-as a stream, cells straight into and out of the store ([The .01252-format](../files/format.md#reading-and-writing)), so opening costs about53-the memory the workbook then holds.51+as a stream, so opening costs about the memory the workbook then holds52+([The .012 file](#the-012-file)).5453 5554 | Measure | 8192 x 26 (213 k cells) | 8192 x 256 (2.1 M cells) | 1,000,000 x 10 (10 M cells) |5655 |---|---|---|---|@@ -71,13 +70,32 @@ B (the parsed tree and reference lists); imported CSV, SQLite and
7170 Parquet 21 to 63 B; text about 60 B plus its length, once per distinct string7271 (a column of 90 texts repeated takes 27 B a cell). A plain cell (a7372 number, boolean or text as typed, with a format and style) is a 16-byte74-slot in its column's block of 1024 rows; formulas, notes, and the75-results pivots and spills write are whole `Cell`s, about 300 B each with76-their entry and input (the `Cell` itself is 216 B). How the store lays73+slot in its column's block of 1024 rows, and so is a derived cell, what74+a pivot or a spill writes: a pivot's results take 20.6 B a cell and75+spilled numbers 20.5 B, plus the 32 B a value of the array the anchor76+keeps to compare with the next one (`BenchmarkMemoryDerived`). Formulas77+and notes are whole `Cell`s, about 300 B each with their entry and78+input (the `Cell` itself is 216 B). How the store lays7779 them out is in [Architecture](architecture.md#the-engine). Reading a7880 cell's value is an index into its column's block: 2.8 ns7981 (`BenchmarkRead`), 8.6 ns a cell read by a SUM in a full recalculation.8082 83+### The .012 file84+85+The reader and writer stream (`internal/sheet/fileread.go`,86+`filescan.go`): each cell goes into its sheet as its line is read, a87+number typed plainly straight into its 16-byte slot, and the other88+fields are small and decoded whole. Saving writes the cells in89+row-major order as it reads them from the sheet, straight into the90+file.91+92+There is one format. A binary one that stored the slots themselves93+would open ten million numbers in about 0.3 s at best (storing ten94+million slots alone takes 0.17 s) where the JSON takes 1.2 s: a second95+saved on sheets near `max-cells`, and nothing a smaller sheet would96+notice, for the loss of what the JSON gives, diffs, merges, and a file97+anyone can read and fix.98+8199 ## The grid82100 83101 Nothing costs what the grid costs. The cells' occupancy@@ -210,7 +228,9 @@ every cell on screen a shade), at 200 x 60 (`BenchmarkFrame`,
210228 | Typing a number and Enter, through to the frame | 0.89 ms | 2.3 ms (the scale's percentile over 213 k numbers) |211229 212230 Shades and rule colors keep their escape codes, so a plain cell on one213-costs a string concatenation, not a style render.231+costs a string concatenation, not a style render. A custom formula is232+evaluated for each cell drawn, once per recalculation, with its233+references moved for the cell.214234 215235 Wrapped text, borders, row heights and merged cells cost what the216236 screen shows too. A sheet with none of them is drawn a line per row@@ -226,16 +246,17 @@ top (`BenchmarkFrame`, `BenchmarkKeystroke`):
226246 |---|---|---|---|227247 | 8192 x 26 numbers, a frame | 0.18 ms | 0.90 ms | 3.1 ms |228248 | The same laid out, a frame | 0.24 ms | 1.28 ms | 4.3 ms |229-| The same laid out, an arrow key through to its frame | 0.25 ms | 1.25 ms | | A custom formula is230-evaluated for each cell drawn, once per recalculation, with its231-references moved for the cell.249+| The same laid out, an arrow key through to its frame | 0.25 ms | 1.25 ms | |232250 233251 A chart drawn as text costs 4 to 50 us at 24 x 10 to 120 x 40 cells,234252 whatever its data: only the categories that fit are drawn (a pie of 8192235253 slices, 0.44 ms, is the worst). An image for kitty graphics is redrawn236254 when its data, size or theme changes: 25 to 260 us at 24 x 10 cells and237255 1 to 1.5 ms at 120 x 40, a pie 8.6 ms (`internal/chart`,238-`BenchmarkDraw`, `BenchmarkImage`). Key presses add little: an arrow key256+`BenchmarkDraw`, `BenchmarkImage`). A sixel image is encoded then too,257+in 0.07 to 0.3 ms at 24 x 10 cells and 3 to 7 ms at 120 x 40 (10 x 20258+pixel cells), 0.6 to 67 KB to send (`BenchmarkSixel`); it is sent again,259+without encoding, whenever the screen under it is redrawn. Key presses add little: an arrow key239260 through to its frame is 0.19 ms at 80 x 24 and 0.9 ms at 200 x 60;240261 Page Down 1.1 ms.241262 @@ -319,7 +340,7 @@ entities, so entity bombs fail as unknown entities.
319340 Each step keeps the cells it changed as they were, in the form the320341 sheet keeps them (`historyimage.go`): a plain cell as its 16-byte slot321342 in columns of blocks, its text and formatting in the step's own tables,322-and a whole `Cell` only for a formula, a note or a derived cell. A step343+and a whole `Cell` only for a formula or a note. A step323344 of one cell or a few keeps them in a list. Undoing costs about what the324345 change cost, since it recalculates the same cells. Clearing 213 k cells325346 and undoing it takes 86 ms; clearing all ten million cells of a full@@ -498,10 +519,10 @@ FILTERs, 63 of whose arrays change:
498519 499520 | Measure | Time | Allocated |500521 |---|---|---|501-| Edit a number | 52 ms | 161 MB, 7.9 k allocations |502-| Undo and redo it | 103 ms | |503-| Full recalculation | 217 ms | |504-| Build (load and recalculate) | 298 ms | |522+| Edit a number | 52 ms | 161 MB, 7.8 k allocations |523+| Undo and redo it | 106 ms | |524+| Full recalculation | 106 ms | |525+| Build (load and recalculate) | 181 ms | |505526 506527 Most of an edit is the FILTERs' own work: comparing 8192 categories each507528 (`B1:B8192="alpha"`, about 20 ns a value) and picking the rows that@@ -535,10 +556,10 @@ Sizes and scheduling are in the [roadmap](../../ROADMAP.md#3-scale).
535556 `max-cells` stands at ten million, as Google Sheets' limit does; the536557 sheet itself would hold several times that in a few GB, and opening,537558 saving and undoing cost about what the cells do ([Sheet size](#sheet-size),538-[Undo](#undo)). What stands in the way is derived cells: what arrays539-and pivots write is kept as whole `Cell`s (516 k of them in the arrays540-stress shape), where their values and inferred formats would fit in541-slots.559+[Undo](#undo)). Derived cells cost what plain cells cost, a slot each560+([Sheet size](#sheet-size)); what stays large is a formula, about 750 B561+with its parsed tree and reference lists, and the array a spilling562+formula keeps, 32 B a value.542563 543564 ## Measuring544565 @@ -39,26 +39,16 @@ only for their formatting and notes.
3939 4040 ## Reading and writing4141 42-012 reads and writes the file as a stream (`internal/sheet/fileread.go`,43-`filescan.go`). Each cell goes into its sheet as its line is read, a44-number typed plainly straight into the 16-byte slot the sheet keeps it45-in, so opening a file takes about the memory of the workbook, however46-large the file; the other fields are small and decoded whole. Saving47-writes the cells in row-major order as it reads them from the sheet,48-straight into the file. Ten million numbers (236 MB of JSON) open in49-1.2 s and save in 0.5 s ([Bounds of support](../contributing/limits.md#sheet-size)).50-5142 Any JSON with the same fields reads the same: the order of keys,5243 spacing, and the case of field names don't matter. A cell given twice5344 takes its last entry, though an invalid earlier one is refused; the5445 `version` or the `sheets` list given twice around cells is refused.5546 56-There is one format. A binary one that stored the slots themselves57-would open ten million numbers in about 0.3 s at best (storing ten58-million slots alone takes 0.17 s) where the JSON takes 1.2 s: a second59-saved on sheets near `max-cells`, and nothing a smaller sheet would60-notice, for the loss of what the JSON gives, diffs, merges, and a file61-anyone can read and fix.47+012 reads and writes the file as a stream, each cell straight into or48+out of its sheet, so opening a file takes about the memory of the49+workbook however large the file; what that costs, and why there is no50+binary format beside this one, is in51+[Bounds of support](../contributing/limits.md#the-012-file).6252 6353 ## Versions6454 @@ -5,6 +5,8 @@ sidebar_position: 7
55 66 # JEV functions77 8+9+810 With a TypeSafe API key, four functions ask the hosted JEV model about a911 value (a cell, a range or text). They follow Sheets' argument style: the1012 value, the question, then what the answers mean.@@ -22,8 +24,6 @@ line shows the confidence for the selected cell. Data > Ask JEV again
2224 re-asks the selection.2325 2426 Without a key the functions show `#N/A` and say how to add one.25-`JEV_LIVE_TEST=1 go test ./internal/jev -run TestLive` checks the real26-service.2727 2828 ## Setup2929 @@ -104,7 +104,7 @@ Colors. `terminal` uses the terminal's own 16-color palette. `high-contrast` dra
104104 105105 #### `chart-images`106106 107-Draw charts as images on terminals with kitty graphics (kitty, Ghostty, WezTerm). When false, charts are always text.107+Draw charts as images on terminals with kitty graphics (kitty, Ghostty, WezTerm) or, outside tmux, sixel graphics (foot, xterm, mlterm, Windows Terminal). When false, charts are always text.108108 109109 | | |110110 |---|---|@@ -5,6 +5,8 @@ sidebar_position: 10
55 66 # Charts77 8+9+810 Insert > Chart charts the selection, or the table around the active cell,911 guessing the header row, the label column and a title. The chart floats over1012 the grid, pinned to a cell, and redraws when its data changes.@@ -43,6 +45,13 @@ columns, and save with the sheet.
4345 **Images or text.** At startup 012 asks the terminal whether it supports the4446 kitty graphics protocol. Where it does (kitty, Ghostty, WezTerm) the plot is4547 a real image, drawn in the terminal's own palette and placed with Unicode46-placeholders, so redraws never erase it. Elsewhere, and usually inside tmux,47-the plot is drawn with block and braille characters. Axis labels, the48-legend and the frame are text either way.48+placeholders, so redraws never erase it. Otherwise, where the terminal49+lists sixel graphics among its device attributes (foot, xterm, mlterm,50+Windows Terminal), the plot is a sixel image in the terminal's palette,51+in as many colors as it has color registers. A sixel image is drawn over52+the finished screen, so it shows only where its whole plot area does: a53+chart scrolled partly out of the grid, or under another chart, a menu or54+a dialog, is text until it shows whole again, and the image is drawn55+again after scrolling, resizing or an edit to its data. Elsewhere, and56+inside tmux, the plot is drawn with block and braille characters. Axis57+labels, the legend and the frame are text either way.@@ -5,6 +5,8 @@ sidebar_position: 6
55 66 # Find and replace77 8+9+810 Ctrl+F opens a find bar on the context line: matches highlight as you type,911 the active cell follows the current one, and Enter and Shift+Enter step1012 through them. Ctrl+H adds a replacement field; Enter replaces and moves on,@@ -13,6 +13,8 @@ whole columns or rows, then pick a format from the Format menu, a key
1313 1414 ## Number formats1515 16+17+1618 Format > Number, as Sheets names them:1719 1820 | Format | Shows |@@ -53,16 +53,6 @@ results. They follow inserted and deleted rows and columns, and a rule
5353 whose cells are all deleted goes. Format > Clear conditional formats5454 takes the selection out of every rule.5555 56-Rules, conditional formats and validation alike, move with cut and57-paste on a sheet, as in Sheets: the cut cells take their rules to where58-they land, and the cells they land on lose theirs. A rule whose cells59-partly move keeps both parts; with a custom formula it becomes two60-rules, each with the formula written for its own first cell, so the61-cells that stayed read what they read and the moved ones read the cells62-beside their new places. Formulas in rules follow cells that63-move. Cut on one sheet and pasted on another, the cells leave their64-rules behind and meet the rules where they land.65-6656 ## Data validation6757 6858 Data > Data validation opens the sheet's validation rules in the same@@ -96,19 +86,36 @@ own words there. A formula is checked by the value it computes. Blanks
9686 are always valid.9787 9888 Each cell has one rule: a rule added over cells that had another takes99-them from it. Validation checks what's typed, picked from a dropdown and100-entered by macros. Pastes (Ctrl+V, paste values, text pasted from101-another program) and fills (Ctrl+D, Ctrl+R, Ctrl+Enter, the fill102-handle) are checked after they're written, as in Sheets: when a cell's103-rule rejects what landed there, the whole paste or fill is undone and104-ERROR mode says which cell and why ("Paste undone: Invalid entry in105-B2: ..."; Ctrl+Enter keeps the entry open instead); when only rules that106-warn fail, it stays, the cells are marked, and the context line says how107-many ("2 cells invalid, first B2: ..."). Imports aren't checked, and what108-they leave invalid is marked. Validation can't go on a pivot table's109-results, which take no entries. Data > Remove data validation takes the110-selection out of every rule.89+them from it. Validation can't go on a pivot table's results, which take90+no entries. Data > Remove data validation takes the selection out of91+every rule.92+93+What validation checks:94+95+- **Typing, a dropdown's pick and macros' entries**, as they're96+ entered, rejected or warned as above.97+- **Pastes and fills** (Ctrl+V, paste values, text pasted from another98+ program; Ctrl+D, Ctrl+R, Ctrl+Enter, the fill handle), after they're99+ written, as in Sheets. When a cell's rule rejects what landed there,100+ the whole paste or fill is undone and ERROR mode says which cell and101+ why ("Paste undone: Invalid entry in B2: ..."); Ctrl+Enter keeps the102+ entry open instead. When only rules that warn fail, it stays, the103+ cells are marked, and the context line says how many ("2 cells104+ invalid, first B2: ...").105+- **Imports** aren't checked; what they leave invalid is marked.111106 112107 Adding a checkbox or a dropdown changes no cells, so on a protected range113108 it doesn't ask first (on a protected sheet it does); checking a box or114109 picking an item asks as typing does ([protection](notes-protection.md#protected-sheets-and-ranges)).110+111+## Moving cells112+113+Conditional formats and validation rules move with cut and paste on a114+sheet, as in Sheets: the cut cells take their rules to where115+they land, and the cells they land on lose theirs. A rule whose cells116+partly move keeps both parts; with a custom formula it becomes two117+rules, each with the formula written for its own first cell, so the118+cells that stayed read what they read and the moved ones read the cells119+beside their new places. Formulas in rules follow cells that120+move. Cut on one sheet and pasted on another, the cells leave their121+rules behind and meet the rules where they land.@@ -22,6 +22,7 @@ colors that follow light and dark.
2222 | Mouse, all motion | hover highlights, drag, resize | most terminals |2323 | Pointer shape (OSC 22) | resize and text cursors over the grid | Ghostty, kitty, foot, xterm |2424 | Kitty graphics | chart images | kitty, Ghostty, WezTerm |25+| Sixel graphics (DA1 attribute 4, XTSMGRAPHICS) | chart images where kitty graphics aren't, outside tmux: [Charts](../sheets/charts.md) | foot, xterm, mlterm, Windows Terminal |2526 | Hyperlinks (OSC 8) | links in cells | most modern terminals |2627 | Clipboard (OSC 52) | copying ranges | most terminals; tmux needs `set-clipboard on` |2728 | Progress (OSC 9;4) | JEV and import progress in the tab | Ghostty, Windows Terminal, iTerm2 |@@ -139,7 +139,7 @@ Every session is a separate 012 with nothing shared but the process:
139139 its own workbook, undo history, clipboard, theme, JEV cache and140140 terminal state. What it learns about the terminal comes from the141141 client: the window size and its changes, the colors (from the `TERM`142-the client sends), the light or dark background, kitty graphics, and142+the client sends), the light or dark background, kitty or sixel graphics, and143143 the clipboard, which Ctrl+C sets on the client's machine through OSC 52.144144 145145 [Macros](../sheets/macros.md) can be recorded, run, renamed and deleted in a@@ -66,9 +66,8 @@ muted text included, and 4.5:1 for lines, borders, chart axes and
6666 unavailable menu items. The pointer is black on cyan, the selection6767 white on blue, errors a light red on black (a dark red on white).6868 69-States never rest on color alone in any theme: the pointer and the70-selection are in reverse video, errors have a curly underline, invalid71-entries a dotted one, spilled values are italic, and so on69+States never rest on color alone in any theme: each also shows as70+text, a glyph or an attribute such as reverse video or an underline7271 ([Reading without color](../contributing/ux.md#reading-without-color)).7372 7473 ## Your own schemes@@ -0,0 +1,219 @@
1+package chart2+3+import (4+ "image"5+ "image/color"6+ "slices"7+ "strconv"8+ "strings"9+)10+11+// Sixel is the other way images reach a terminal, for terminals without12+// kitty graphics (foot, xterm, mlterm, Windows Terminal and others). A13+// sixel image is pixels drawn at the cursor, not text: the UI draws it14+// after the frame over blank cells it keeps for it (see the ui package's15+// sixel.go). The image is sent whole each time, in at most as many16+// colors as the terminal has color registers, run-length encoded.17+18+// Sixel returns the DCS sequence that draws img at the cursor: pixels19+// with any coverage composited over bg, fully transparent ones left20+// unset so the cell background shows, in at most registers colors21+// (at least 2, at most 256).22+func Sixel(img *image.RGBA, bg color.RGBA, registers int) string {23+ registers = min(max(registers, 2), 256)24+ b := img.Bounds()25+ w, h := b.Dx(), b.Dy()26+ if w <= 0 || h <= 0 {27+ return ""28+ }29+ idx, pal := quantize(img, bg, registers)30+ var out strings.Builder31+ // P2 = 1: pixels not drawn keep what's under them. The raster32+ // attributes set square pixels and the image's size.33+ out.WriteString("\x1bP0;1;0q\"1;1;")34+ out.WriteString(strconv.Itoa(w) + ";" + strconv.Itoa(h))35+ for i, c := range pal {36+ out.WriteString("#" + strconv.Itoa(i) + ";2;" + percent(c.R) + ";" + percent(c.G) + ";" + percent(c.B))37+ }38+ bands := sixelBands(idx, w, h, len(pal))39+ out.WriteString(bands)40+ out.WriteString("\x1b\\")41+ return out.String()42+}43+44+// percent is a color channel as sixel's 0 to 100.45+func percent(v uint8) string { return strconv.Itoa((int(v)*100 + 127) / 255) }46+47+// sixelBands encodes the image of register indexes (-1 for unset), six48+// rows at a time: for each color in the band, its row of sixels,49+// run-length encoded, then "$" to go back for the next color, and "-"50+// between bands.51+func sixelBands(idx []int16, w, h, colors int) string {52+ var out strings.Builder53+ b := bands{w: w, rows: make([][]byte, colors), band: make([]int, colors)}54+ for y0 := 0; y0 < h; y0 += 6 {55+ if y0 > 0 {56+ out.WriteByte('-')57+ }58+ b.gather(idx[y0*w:min(y0+6, h)*w], y0)59+ for i, c := range b.used {60+ if i > 0 {61+ out.WriteByte('$')62+ }63+ out.WriteString("#" + strconv.Itoa(int(c)))64+ writeRuns(&out, b.rows[c])65+ }66+ }67+ return out.String()68+}69+70+// bands holds the rows of sixels of one band of six pixel rows.71+type bands struct {72+ w int73+ rows [][]byte // each color's row of sixels, made when first used74+ band []int // the band each color was last used in, plus one75+ used []int16 // the colors of the band, in order76+}77+78+// gather makes the rows of the band at y0 from its pixels' registers.79+func (b *bands) gather(px []int16, y0 int) {80+ b.used = b.used[:0]81+ for i, c := range px {82+ if c < 0 {83+ continue84+ }85+ if b.band[c] != y0+1 {86+ b.band[c] = y0 + 187+ b.used = append(b.used, c)88+ if b.rows[c] == nil {89+ b.rows[c] = make([]byte, b.w)90+ }91+ for x := range b.rows[c] {92+ b.rows[c][x] = '?'93+ }94+ }95+ b.rows[c][i%b.w] += 1 << (i / b.w) // each pixel has one color, so each bit is added once96+ }97+ slices.Sort(b.used)98+}99+100+// writeRuns writes a row of sixels, runs of four or more as "!n", and101+// leaves out the empty sixels at its end.102+func writeRuns(out *strings.Builder, row []byte) {103+ end := len(row)104+ for end > 0 && row[end-1] == '?' {105+ end--106+ }107+ for i := 0; i < end; {108+ j := i + 1109+ for j < end && row[j] == row[i] {110+ j++111+ }112+ if n := j - i; n >= 4 {113+ out.WriteString("!" + strconv.Itoa(n))114+ out.WriteByte(row[i])115+ } else {116+ for range n {117+ out.WriteByte(row[i])118+ }119+ }120+ i = j121+ }122+}123+124+// quantize maps each pixel of img, composited over bg, to a register,125+// -1 for a fully transparent pixel, and returns the registers' colors.126+// Colors are kept exactly while there are at most registers of them;127+// past that, their low bits are dropped until they fit, and each128+// register is the average of the colors it stands for. Registers are129+// numbered in the order their colors first appear, so the same image130+// always encodes the same way.131+func quantize(img *image.RGBA, bg color.RGBA, registers int) ([]int16, []color.RGBA) {132+ b := img.Bounds()133+ w, h := b.Dx(), b.Dy()134+ px := make([]uint32, w*h) // 0 for transparent, else 1<<24 | RGB135+ for y := range h {136+ for x := range w {137+ if c := over(img, b.Min.X+x, b.Min.Y+y, bg); c.A != 0 {138+ px[y*w+x] = 1<<24 | uint32(c.R)<<16 | uint32(c.G)<<8 | uint32(c.B)139+ }140+ }141+ }142+ shift := uint(0)143+ for ; shift < 8; shift++ {144+ if distinct(px, shift, registers) <= registers {145+ break146+ }147+ }148+ reg := map[uint32]int16{}149+ var sums [][4]int // R, G, B sums and count per register150+ idx := make([]int16, w*h)151+ last, lastReg := uint32(0), int16(-1) // runs of one color are common: look each up once152+ for i, p := range px {153+ if p == 0 {154+ idx[i] = -1155+ continue156+ }157+ r := lastReg158+ if p != last {159+ k := bucket(p, shift)160+ var ok bool161+ if r, ok = reg[k]; !ok {162+ r = int16(len(sums))163+ reg[k] = r164+ sums = append(sums, [4]int{})165+ }166+ last, lastReg = p, r167+ }168+ idx[i] = r169+ s := &sums[r]170+ s[0] += int(p >> 16 & 0xff)171+ s[1] += int(p >> 8 & 0xff)172+ s[2] += int(p & 0xff)173+ s[3]++174+ }175+ pal := make([]color.RGBA, len(sums))176+ for i, s := range sums {177+ pal[i] = color.RGBA{uint8(s[0] / s[3]), uint8(s[1] / s[3]), uint8(s[2] / s[3]), 255}178+ }179+ return idx, pal180+}181+182+// bucket is the key of pixel p with shift low bits of each channel183+// dropped.184+func bucket(p uint32, shift uint) uint32 {185+ m := uint32(0xff>>shift<<shift) * 0x010101186+ return p & (1<<24 | m)187+}188+189+// distinct counts the buckets of px at shift, stopping past limit.190+func distinct(px []uint32, shift uint, limit int) int {191+ seen := map[uint32]struct{}{}192+ last := uint32(0)193+ for _, p := range px {194+ if p == 0 || p == last {195+ continue196+ }197+ last = p198+ seen[bucket(p, shift)] = struct{}{}199+ if len(seen) > limit {200+ break201+ }202+ }203+ return len(seen)204+}205+206+// over is the pixel at x, y of the premultiplied img composited over207+// bg, opaque, or transparent where img has no coverage.208+func over(img *image.RGBA, x, y int, bg color.RGBA) color.RGBA {209+ i := img.PixOffset(x, y)210+ p := img.Pix[i : i+4 : i+4]211+ a := int(p[3])212+ if a == 0 {213+ return color.RGBA{}214+ }215+ return color.RGBA{mix(p[0], bg.R, a), mix(p[1], bg.G, a), mix(p[2], bg.B, a), 255}216+}217+218+// mix is the premultiplied channel c of coverage a over under.219+func mix(c, under uint8, a int) uint8 { return uint8(min(int(c)+int(under)*(255-a)/255, 255)) }@@ -0,0 +1,177 @@
1+package chart2+3+import (4+ "image"5+ "image/color"6+ "strings"7+ "testing"8+9+ "github.com/FelineStateMachine/012/internal/sheet"10+)11+12+// The encoder's bytes for a small image: a register per color in the13+// order they appear, runs of four or more as "!n", trailing empty14+// sixels left out, "$" between the colors of a band and "-" between15+// bands, a pixel with half coverage composited over the background and16+// a transparent one left unset.17+func TestSixelGolden(t *testing.T) {18+ img := image.NewRGBA(image.Rect(0, 0, 4, 7))19+ red := color.RGBA{255, 0, 0, 255}20+ for x := range 4 {21+ img.SetRGBA(x, 0, red)22+ }23+ img.SetRGBA(1, 1, color.RGBA{0, 128, 0, 128}) // premultiplied: half green24+ img.SetRGBA(0, 2, red)25+ img.SetRGBA(1, 2, red)26+ img.SetRGBA(2, 2, red)27+ img.SetRGBA(0, 6, color.RGBA{0, 0, 255, 255})28+ got := Sixel(img, color.RGBA{0, 0, 0, 255}, 256)29+ want := "\x1bP0;1;0q\"1;1;4;7" +30+ "#0;2;100;0;0#1;2;0;50;0#2;2;0;0;100" +31+ "#0DDD@$#1?A" + // red: rows 0 and 2 (1+4) three times, then row 0 alone32+ "-#2@" +33+ "\x1b\\"34+ if got != want {35+ t.Errorf("Sixel =\n%q\nwant\n%q", got, want)36+ }37+ img.SetRGBA(3, 2, red)38+ if got := Sixel(img, color.RGBA{0, 0, 0, 255}, 256); !strings.Contains(got, "#0!4D$") {39+ t.Errorf("a run of four isn't run-length encoded: %q", got)40+ }41+}42+43+// Past the terminal's registers, colors are merged until they fit, and44+// every pixel still names a register that exists.45+func TestSixelQuantize(t *testing.T) {46+ img := image.NewRGBA(image.Rect(0, 0, 40, 30))47+ for y := range 30 {48+ for x := range 40 {49+ img.SetRGBA(x, y, color.RGBA{uint8(x * 6), uint8(y * 8), uint8((x + y) * 3), 255})50+ }51+ }52+ for _, regs := range []int{16, 256} {53+ idx, pal := quantize(img, color.RGBA{}, regs)54+ if len(pal) > regs {55+ t.Errorf("%d registers: %d colors", regs, len(pal))56+ }57+ for _, i := range idx {58+ if i < 0 || int(i) >= len(pal) {59+ t.Fatalf("%d registers: pixel names register %d of %d", regs, i, len(pal))60+ }61+ }62+ }63+ if s := Sixel(img, color.RGBA{}, 16); strings.Count(s, ";2;") > 16 {64+ t.Errorf("more than 16 colors defined")65+ }66+}67+68+// A chart's image encodes, and an empty image draws nothing.69+func TestSixelChart(t *testing.T) {70+ img := Image(sheet.ChartColumn, single, 40, 12, Options{CellW: 8, CellH: 16}, testPalette)71+ if img == nil {72+ t.Fatal("no image")73+ }74+ s := Sixel(img, color.RGBA{0, 0, 0, 255}, 256)75+ if !strings.HasPrefix(s, "\x1bP0;1;0q\"1;1;") || !strings.HasSuffix(s, "\x1b\\") {76+ t.Errorf("not a sixel sequence: %.40q", s)77+ }78+ if Sixel(image.NewRGBA(image.Rect(0, 0, 0, 0)), color.RGBA{}, 256) != "" {79+ t.Error("an empty image drew something")80+ }81+}82+83+// A chart's sixel image decodes to the image composited over the84+// background, each channel within sixel's percent steps, and pixels85+// without coverage unset.86+func TestSixelDecodes(t *testing.T) {87+ bg := color.RGBA{250, 250, 245, 255}88+ for _, ct := range []sheet.ChartType{sheet.ChartLine, sheet.ChartPie, sheet.ChartArea} {89+ img := Image(ct, budget, 40, 12, Options{CellW: 7, CellH: 15}, testPalette)90+ got := decodeSixel(t, Sixel(img, bg, 256))91+ b := img.Bounds()92+ for y := range b.Dy() {93+ for x := range b.Dx() {94+ want := over(img, x, y, bg)95+ c, set := got[image.Pt(x, y)]96+ if set != (want.A != 0) || set && (diff(c.R, want.R) > 3 || diff(c.G, want.G) > 3 || diff(c.B, want.B) > 3) {97+ t.Fatalf("%v at %d,%d: %v %v, want %v", ct, x, y, c, set, want)98+ }99+ }100+ }101+ }102+}103+104+func diff(a, b uint8) int { return max(int(a)-int(b), int(b)-int(a)) }105+106+// decodeSixel reads what Sixel writes: color definitions, "#n"107+// selections, sixels and their runs, "$" and "-".108+func decodeSixel(t *testing.T, s string) map[image.Point]color.RGBA {109+ t.Helper()110+ body, ok := strings.CutPrefix(s, "\x1bP0;1;0q")111+ body, ok2 := strings.CutSuffix(body, "\x1b\\")112+ if !ok || !ok2 {113+ t.Fatalf("not a sixel sequence: %.30q", s)114+ }115+ d := sixelDecoder{body: body, pal: map[int]color.RGBA{}, out: map[image.Point]color.RGBA{}}116+ for d.i < len(body) {117+ d.step()118+ }119+ return d.out120+}121+122+type sixelDecoder struct {123+ body string124+ i int125+ x, y, cur int126+ pal map[int]color.RGBA127+ out map[image.Point]color.RGBA128+}129+130+func (d *sixelDecoder) num() int {131+ n := 0132+ for ; d.i < len(d.body) && d.body[d.i] >= '0' && d.body[d.i] <= '9'; d.i++ {133+ n = n*10 + int(d.body[d.i]-'0')134+ }135+ return n136+}137+138+func (d *sixelDecoder) step() {139+ ch := d.body[d.i]140+ d.i++141+ switch ch {142+ case '"':143+ for d.i < len(d.body) && (d.body[d.i] >= '0' && d.body[d.i] <= '9' || d.body[d.i] == ';') {144+ d.i++145+ }146+ case '#':147+ d.cur = d.num()148+ if d.i < len(d.body) && d.body[d.i] == ';' {149+ var c [4]int150+ for k := range c {151+ d.i++152+ c[k] = d.num()153+ }154+ pc := func(v int) uint8 { return uint8((v*255 + 50) / 100) }155+ d.pal[d.cur] = color.RGBA{pc(c[1]), pc(c[2]), pc(c[3]), 255}156+ }157+ case '$':158+ d.x = 0159+ case '-':160+ d.x, d.y = 0, d.y+6161+ default:162+ run := 1163+ if ch == '!' {164+ run = d.num()165+ ch = d.body[d.i]166+ d.i++167+ }168+ for range run {169+ for dy := range 6 {170+ if (ch-'?')&(1<<dy) != 0 {171+ d.out[image.Pt(d.x, d.y+dy)] = d.pal[d.cur]172+ }173+ }174+ d.x++175+ }176+ }177+}@@ -76,3 +76,22 @@ func BenchmarkImage(b *testing.B) {
7676 }7777 }7878 }79+80+// BenchmarkSixel is a chart's image encoded as sixel, in 256 colors, as81+// it is whenever its data, size or colors change.82+func BenchmarkSixel(b *testing.B) {83+ pal := Palette{Series: [Colors]color.RGBA{{205, 49, 49, 255}, {13, 188, 121, 255}, {36, 114, 200, 255}}, Grid: color.RGBA{128, 128, 128, 80}}84+ d := stressData(12)85+ for _, ct := range stressTypes {86+ for _, sz := range stressSizes {87+ img := Image(ct.t, d, sz.w, sz.h, Options{CellW: 10, CellH: 20}, pal)88+ b.Run(fmt.Sprintf("%s/%dx%d", ct.name, sz.w, sz.h), func(b *testing.B) {89+ n := 090+ for b.Loop() {91+ n = len(Sixel(img, color.RGBA{0, 0, 0, 255}, 256))92+ }93+ b.ReportMetric(float64(n), "bytes")94+ })95+ }96+ }97+}@@ -93,8 +93,8 @@ var Options = []Option{
9393 "`light:NAME,dark:NAME` picks one by the terminal's background and follows it when it changes.",9494 Check: checkTheme},9595 {Name: "chart-images", Kind: Bool, Group: GroupAppearance, Default: "true", Env: []string{"O12_CHART_IMAGES"}, Live: true,96- Desc: "Draw charts as images on terminals with kitty graphics (kitty, Ghostty, WezTerm). " +97- "When false, charts are always text."},96+ Desc: "Draw charts as images on terminals with kitty graphics (kitty, Ghostty, WezTerm) or, " +97+ "outside tmux, sixel graphics (foot, xterm, mlterm, Windows Terminal). When false, charts are always text."},9898 {Name: "notifications", Kind: Bool, Group: GroupAppearance, Default: "true", Env: []string{"O12_NOTIFICATIONS"}, Live: true,9999 Desc: "Send a desktop notification (OSC 9) when JEV answers or an import finishes while the " +100100 "terminal window is in the background."},@@ -143,7 +143,7 @@ func decodeBorders(fb *fileBorders) (Borders, error) {
143143 // its slot, without making a Cell.144144 func (st *cellStore) appendSaved(buf []byte, a Addr) ([]byte, bool, error) {145145 b, i := st.find(a)146- if sl := b.vals[i]; sl.kind != slotRich && sl.look == 0 {146+ if sl := b.vals[i]; sl.kind != slotRich && sl.kind&slotDerived == 0 && sl.look == 0 {147147 return st.appendInput(buf, sl), true, nil148148 }149149 c := st.get(a).saved()@@ -24,10 +24,7 @@ func (s *Sheet) DisplayFormat(a Addr) Format {
2424 return f2525 }2626 }27- if c := s.cells.richAt(a); c != nil { // only formulas infer formats28- return c.auto29- }30- return Format{}27+ return s.cells.autoAt(a) // formulas and spilled cells infer formats3128 }3229 3330 // SetFormat gives every cell in r the number format f, including blank@@ -0,0 +1,50 @@
1+//go:build stress2+3+package sheet_test4+5+import (6+ "runtime"7+ "testing"8+9+ "github.com/FelineStateMachine/012/internal/sheet"10+ "github.com/FelineStateMachine/012/internal/stress"11+)12+13+// BenchmarkMemoryDerived is the heap a derived cell holds: the cells a14+// formula spills (SEQUENCE over 8192 x 64, the array the anchor keeps15+// to compare with the next one reported apart) and a pivot's results16+// (a row per id of a 8191-row table, with a SUM and a COUNTA).17+func BenchmarkMemoryDerived(b *testing.B) {18+ b.Run("spilled-8192x64", func(b *testing.B) {19+ for b.Loop() {20+ before := heap()21+ s := sheet.New()22+ if err := s.Set(sheet.Addr{}, "=SEQUENCE(8192, 64)"); err != nil {23+ b.Fatal(err)24+ }25+ after := heap()26+ n := float64(s.Len())27+ const valueBytes = 32 // a Value in the array the anchor keeps28+ b.ReportMetric(float64(after-before)/n, "B/cell")29+ b.ReportMetric(float64(after-before)/n-valueBytes, "B/cell-stored")30+ runtime.KeepAlive(s)31+ }32+ })33+ b.Run("pivot-8191", func(b *testing.B) {34+ for b.Loop() {35+ src := stress.Table(stress.Rows-1, 8)36+ r := sheet.NewRect(sheet.Addr{}, sheet.Addr{Col: 7, Row: stress.Rows - 1})37+ p := sheet.NewPivot(src, r)38+ p.Rows = []sheet.PivotGroup{{Col: 0}}39+ p.Values = []sheet.PivotValue{{Col: 2, Summarize: sheet.SumBy}, {Col: 3, Summarize: sheet.CountABy}}40+ before := heap()41+ pv, err := src.Book().CreatePivot(src, r, "", p)42+ if err != nil {43+ b.Fatal(err)44+ }45+ after := heap()46+ b.ReportMetric(float64(after-before)/float64(pv.Len()), "B/cell")47+ runtime.KeepAlive(pv)48+ }49+ })50+}@@ -0,0 +1,131 @@
1+package sheet2+3+import (4+ "bytes"5+ "strings"6+ "testing"7+)8+9+// A derived cell of every value, format, inferred format and mark comes10+// back from the store as it went in, kept in a slot unless it has a11+// note, and the store counts the pivot's results it holds.12+func TestSlotDerived(t *testing.T) {13+ st := newCellStore()14+ a := Addr{Col: 2, Row: 700}15+ values := []Value{num(1.5), num(-0.25), boolean(true), boolean(false), {Kind: Text, Str: "north"},16+ {Kind: Text, Str: "=not a formula"}, {Kind: Text, Str: "12"}, ErrRef, ErrDiv0, {Kind: Text}}17+ for _, v := range values {18+ for _, pivot := range []bool{true, false} {19+ for _, note := range []string{"", "a note"} {20+ c := &Cell{Input: derivedInput(v), Value: v, Format: Preset(FmtCurrency), Note: note, derived: pivot, spilled: !pivot}21+ if !pivot {22+ c.auto = Preset(FmtPercent)23+ }24+ c.Style.Bold = pivot25+ checkDerived(t, &st, a, c)26+ }27+ }28+ }29+ st.delete(a)30+ if st.len() != 0 || st.pivots != 0 || len(st.strs.index) != 0 || len(st.rich) != len(st.richFree) {31+ t.Errorf("left: %d stored, %d pivot cells, %d strings, %d rich", st.len(), st.pivots, len(st.strs.index), len(st.rich)-len(st.richFree))32+ }33+}34+35+// checkDerived stores the derived cell c at a and reads it back.36+func checkDerived(t *testing.T, st *cellStore, a Addr, c *Cell) {37+ t.Helper()38+ want := c.clone()39+ st.set(a, c)40+ checkSame(t, st.get(a), want)41+ if rich := st.richAt(a) != nil; rich != (want.Note != "") {42+ t.Errorf("%+v: rich %v", want, rich)43+ }44+ if got := st.filledAt(a); got != !want.Blank() {45+ t.Errorf("%+v: filled %v", want, got)46+ }47+ if !st.holds(a, st.get(a)) {48+ t.Errorf("%+v: doesn't hold its own view", want)49+ }50+ if n := len(collect(st.pivotKeys())); n != st.pivots || (n == 1) != want.derived {51+ t.Errorf("%+v: %d pivot cells found, %d counted", want, n, st.pivots)52+ }53+}54+55+func collect(seq func(func(Addr) bool)) []Addr {56+ var out []Addr57+ seq(func(a Addr) bool { out = append(out, a); return true })58+ return out59+}60+61+// Spilled cells are slots, and read as they did as Cells: values of each62+// kind, the anchor's inferred format, values when copied, skipped by63+// Replace and left out of the file.64+func TestSpillCompact(t *testing.T) {65+ s := sheetOf(t, map[string]string{66+ "A1": "$5", "A2": "$6", "A3": "x",67+ "B1": "=ARRAYFORMULA(A1:A2*2)",68+ "C1": `=ARRAYFORMULA(IF(SEQUENCE(3)=2, "none", SEQUENCE(3)))`,69+ "D1": "=SEQUENCE(2)>1",70+ "E1": "=1/(SEQUENCE(2)-2)",71+ })72+ wantShown(t, s, map[string]string{"B2": "12", "C2": "none", "C3": "3", "D2": "TRUE", "E2": "#DIV/0!"})73+ for _, a := range []string{"B2", "C2", "C3", "D2", "E2"} {74+ if s.cells.richAt(at(a)) != nil {75+ t.Errorf("%s is a whole Cell", a)76+ }77+ if c := s.Cell(at(a)); !c.Spilled() || c.Input != derivedInput(c.Value) {78+ t.Errorf("%s = %+v", a, c)79+ }80+ }81+ if f := s.DisplayFormat(at("B2")); f.Kind != FmtCurrency {82+ t.Errorf("B2's format %v, want the anchor's currency", f)83+ }84+ found, err := s.Find("none", FindOptions{})85+ if err != nil || len(found) != 1 || found[0] != at("C2") {86+ t.Errorf("Find found %v %v, want the spilled C2", found, err)87+ }88+ if changed, err := s.Replace(at("C2"), "none", "x", FindOptions{}); changed || err != nil {89+ t.Errorf("Replace changed a spilled cell: %v %v", changed, err)90+ }91+ clip := s.Copy(rect("E1:E2"))92+ if _, err := s.Paste(clip, rect("G1:G1"), false); err != nil {93+ t.Fatal(err)94+ }95+ if c := s.Cell(at("G2")); c.Spilled() || c.Value.String() != "#DIV/0!" {96+ t.Errorf("pasted spilled error = %+v", c)97+ }98+ var buf bytes.Buffer99+ if err := s.Book().Write(&buf); err != nil {100+ t.Fatal(err)101+ }102+ if strings.Contains(buf.String(), `"B2"`) || strings.Contains(buf.String(), `"D2"`) {103+ t.Errorf("spilled cells saved:\n%s", buf.String())104+ }105+}106+107+// A pivot's results are slots, found again when they shrink.108+func TestPivotCompact(t *testing.T) {109+ src := sheetOf(t, map[string]string{"A1": "Region", "B1": "Sales", "A2": "north", "B2": "10", "A3": "south", "B3": "5", "A4": "east", "B4": "7"})110+ r := rect("A1:B4")111+ p := NewPivot(src, r)112+ p.Rows = []PivotGroup{{Col: 0}}113+ p.Values = []PivotValue{{Col: 1, Summarize: SumBy}}114+ pv, err := src.Book().CreatePivot(src, r, "", p)115+ if err != nil {116+ t.Fatal(err)117+ }118+ n := pv.cells.pivots119+ if n == 0 || len(pv.cells.rich) != len(pv.cells.richFree) {120+ t.Fatalf("%d pivot cells, %d rich", n, len(pv.cells.rich)-len(pv.cells.richFree))121+ }122+ if err := src.Set(at("A4"), "north"); err != nil {123+ t.Fatal(err)124+ }125+ if pv.cells.pivots >= n {126+ t.Errorf("%d pivot cells after a group went, had %d", pv.cells.pivots, n)127+ }128+ if got := len(collect(pv.cells.pivotKeys())); got != pv.cells.pivots {129+ t.Errorf("%d pivot cells found, %d counted", got, pv.cells.pivots)130+ }131+}@@ -385,7 +385,7 @@ func (s *Sheet) loadNum(a Addr, input []byte, f Format, st Style) bool {
385385 if err != nil {386386 return false387387 }388- lk, ok := s.cells.lookID(f, st)388+ lk, ok := s.cells.lookID(look{f: f, st: st})389389 if !ok {390390 return false391391 }@@ -6,7 +6,7 @@ import "slices"
66 // the form the sheet keeps them (store.go, slot.go): columns of blocks77 // of 16-byte slots, the text in a table of strings kept once each, the88 // formatting in a table of looks, and whole Cells only for rich cells9-// (formulas, notes, what pivots and spills write). A step that clears9+// (formulas, notes). A step that clears1010 // or rewrites a whole sheet so holds about what the sheet does, 20 B1111 // per number, rather than a Cell per cell. A cell that held nothing1212 // before the step is a bit in blank.@@ -178,7 +178,7 @@ func (img *image) look(src *cellStore, id uint16) (uint16, bool) {
178178 return img.looks[id] - 1, true179179 }180180 l := src.looks[id]181- lk, ok := img.store().lookID(l.f, l.st)181+ lk, ok := img.store().lookID(l)182182 if !ok {183183 return 0, false184184 }@@ -284,7 +284,7 @@ func (w *Workbook) refreshPivot(s *Sheet) []loc {
284284 s.pivot.blocked = extent(want)285285 s.pivot.err = err.Error()286286 want = []pivotCell{{v: ErrRef, bold: true}}287- if c := s.cells.get(Addr{}); c != nil && !c.derived && !c.Blank() {287+ if s.typedAt(Addr{}) {288288 want = nil // never write over what the user typed289289 }290290 }@@ -353,7 +353,7 @@ func (w *Workbook) computePivot(s *Sheet) ([]pivotCell, *pivotCalc, error) {
353353 return nil, c, fmt.Errorf("The pivot table doesn't fit on a sheet")354354 }355355 for _, pc := range l.cells {356- if old := s.cells.get(pc.a); old != nil && !old.derived && !old.Blank() {356+ if s.typedAt(pc.a) {357357 return l.cells, c, fmt.Errorf("The pivot table's results would overwrite data in %s", pc.a)358358 }359359 }@@ -387,16 +387,19 @@ func (s *Sheet) writeDerived(want []pivotCell) []loc {
387387 next[pc.a] = c388388 }389389 var changed []loc390- for a, c := range s.cells.richCells() {391- if c.derived && next[a] == nil {392- s.setDerived(a, nil)393- changed = append(changed, loc{s, a})390+ var gone []Addr391+ for a := range s.cells.pivotKeys() {392+ if next[a] == nil {393+ gone = append(gone, a)394394 }395395 }396+ for _, a := range gone {397+ s.setDerived(a, nil)398+ changed = append(changed, loc{s, a})399+ }396400 for a, c := range next {397- if old := s.cells.get(a); old != nil && old.derived && old.Input == c.Input && old.Value == c.Value &&398- old.Format == c.Format && old.Style == c.Style {399- continue401+ if v, l, kind := s.cells.derivedOf(a); kind == slotPivot && v == c.Value && l.f == c.Format && l.st == c.Style {402+ continue // its input is its value's400403 }401404 s.setDerived(a, c)402405 changed = append(changed, loc{s, a})@@ -404,6 +407,15 @@ func (s *Sheet) writeDerived(want []pivotCell) []loc {
404407 return changed405408 }406409 410+// typedAt reports whether the cell at a has contents that aren't the411+// pivot's results: what the pivot mustn't write over.412+func (s *Sheet) typedAt(a Addr) bool {413+ if !s.cells.filledAt(a) {414+ return false415+ }416+ return s.cells.derivedAt(a) != slotPivot417+}418+407419 // setDerived stores or removes a derived cell without recording it for408420 // undo. Derived cells have no formula, so there are no references to409421 // index.@@ -13,16 +13,21 @@ import (
1313 // number, a boolean or text, with at most a format and a style; their1414 // slot holds the value itself, the text in the store's string table, the1515 // format and style in its table of looks, and the input only when it16-// isn't the value's own text. Everything else (formulas, notes, what17-// pivots and spills write, text typed with a leading ') is rich: the18-// slot points at a whole Cell in the store's side table.16+// isn't the value's own text. What pivots and spills write (derived17+// cells) is a slot too, marked by its kind: the value alone, its entry18+// being the value's text (derivedInput), and a spill's inferred format19+// in its look. Everything else (formulas, notes, text typed with a20+// leading ') is rich: the slot points at a whole Cell in the store's21+// side table.1922 type slot struct {2023 num float64 // slotNum's value; slotBool's, 1 or 02124 // ref is slotText's text in strs; slotNum's or slotBool's input in2225 // strs when it isn't the value's text (0 when it is); slotRich's cell2326 // in rich.2427 ref uint3225- look uint16 // the format and style in looks; 0 for neither28+ look uint16 // the format, style and inferred format in looks; 0 for none29+ // kind is one of the kinds below, with slotPivot or slotSpill added30+ // for a derived cell.2631 kind uint82732 // dec is how a slotNum without ref prints its input: the value with2833 // dec-1 decimals, or its shortest text when 0.@@ -36,17 +41,30 @@ const (
3641 slotBool3742 slotText3843 slotRich44+ slotErr // a derived cell's error value, its code in strs3945 )4046 47+// Marks of a derived cell's slot, added to its kind: a pivot's result48+// (Cell.derived) or part of a spilled array (Cell.spilled).49+const (50+ slotPivot uint8 = 1 << 651+ slotSpill uint8 = 1 << 752+ slotDerived = slotPivot | slotSpill53+)54+55+// base is sl's kind without the marks of a derived cell.56+func (sl slot) base() uint8 { return sl.kind &^ slotDerived }57+4158 // maxDec bounds the decimals a number's input may have to be kept as a4259 // count rather than as text.4360 const maxDec = 404461 45-// look is a cell's format and style, kept once per store however many46-// cells share it.62+// look is a cell's format and style, and a spilled cell's inferred63+// format, kept once per store however many cells share it.4764 type look struct {48- f Format49- st Style65+ f Format66+ st Style67+ auto Format5068 }5169 5270 // strTable holds the text of plain cells, each distinct string once,@@ -97,8 +115,7 @@ func (t *strTable) release(i uint32) {
97115 98116 // lookID is the entry of l in the table of looks, added if new, and99117 // false when the table is full.100-func (st *cellStore) lookID(f Format, sty Style) (uint16, bool) {101- l := look{f, sty}118+func (st *cellStore) lookID(l look) (uint16, bool) {102119 if l == (look{}) {103120 return 0, true104121 }@@ -118,21 +135,29 @@ func (st *cellStore) lookID(f Format, sty Style) (uint16, bool) {
118135 return i, true119136 }120137 121-// lookOf is sl's format and style.122-func (st *cellStore) lookOf(sl slot) look {138+// lookOf is sl's look, which the caller mustn't change.139+func (st *cellStore) lookOf(sl slot) *look {123140 if sl.look == 0 {124- return look{}141+ return &noLook125142 }126- return st.looks[sl.look]143+ return &st.looks[sl.look]127144 }128145 146+// noLook is the look of a slot without one, never written.147+var noLook look148+129149 // plainSlot is c as a plain slot, or false if c is rich. It adds the130150 // strings it uses to the table.131151 func (st *cellStore) plainSlot(c *Cell) (slot, bool) {132- if c.Note != "" || c.derived || c.spilled || !c.auto.IsZero() || strings.HasPrefix(c.Input, "=") {152+ switch {153+ case c.Note != "":154+ return slot{}, false155+ case c.derived || c.spilled:156+ return st.derivedSlot(c)157+ case !c.auto.IsZero() || strings.HasPrefix(c.Input, "="):133158 return slot{}, false134159 }135- lk, ok := st.lookID(c.Format, c.Style)160+ lk, ok := st.lookID(look{f: c.Format, st: c.Style})136161 if !ok {137162 return slot{}, false138163 }@@ -171,6 +196,44 @@ func (st *cellStore) plainSlot(c *Cell) (slot, bool) {
171196 return sl, true172197 }173198 199+// derivedSlot is plainSlot for a derived cell without a note: its value,200+// marked with what derived it, its entry being the value's text.201+func (st *cellStore) derivedSlot(c *Cell) (slot, bool) {202+ if c.expr != nil {203+ return slot{}, false204+ }205+ lk, ok := st.lookID(look{c.Format, c.Style, c.auto})206+ if !ok {207+ return slot{}, false208+ }209+ return st.derivedValue(c.Value, lk, c.derived), true210+}211+212+// derivedValue is the slot of a derived cell showing v with look lk: a213+// pivot's result if pivot, else a spilled cell. It adds the string v214+// uses to the table.215+func (st *cellStore) derivedValue(v Value, lk uint16, pivot bool) slot {216+ sl := slot{look: lk, kind: slotSpill}217+ if pivot {218+ sl.kind = slotPivot219+ }220+ switch v.Kind {221+ case Number:222+ sl.kind |= slotNum223+ sl.num = v.Num224+ case Bool:225+ sl.kind |= slotBool226+ sl.num = v.Num227+ case Text:228+ sl.kind |= slotText229+ sl.ref = st.strs.add(v.Str)230+ case Error:231+ sl.kind |= slotErr232+ sl.ref = st.strs.add(v.Str)233+ }234+ return sl235+}236+174237 // numDec is how input prints v, as slot.dec, or false if it isn't v175238 // printed plainly.176239 func numDec(v float64, input string) (uint8, bool) {@@ -253,14 +316,16 @@ func boolText(b bool) string {
253316 return "FALSE"254317 }255318 256-// input is the entry of a plain slot.319+// input is the entry of a plain slot. A derived slot's is its value's320+// text, as derivedInput writes it: a number's shortest text, TRUE or321+// FALSE, the text or the error code.257322 func (st *cellStore) input(sl slot) string {258323 switch {259- case sl.kind == slotBlank:324+ case sl.base() == slotBlank:260325 return ""261326 case sl.ref != 0:262327 return st.strs.strs[sl.ref]263- case sl.kind == slotBool:328+ case sl.base() == slotBool:264329 return boolText(sl.num != 0)265330 }266331 return strconv.FormatFloat(sl.num, 'f', int(sl.dec)-1, 64)@@ -268,13 +333,15 @@ func (st *cellStore) input(sl slot) string {
268333 269334 // slotValue is the value of a plain slot.270335 func (st *cellStore) slotValue(sl slot) Value {271- switch sl.kind {336+ switch sl.base() {272337 case slotNum:273338 return Value{Kind: Number, Num: sl.num}274339 case slotBool:275340 return Value{Kind: Bool, Num: sl.num}276341 case slotText:277342 return Value{Kind: Text, Str: st.strs.strs[sl.ref]}343+ case slotErr:344+ return Value{Kind: Error, Str: st.strs.strs[sl.ref]}278345 }279346 return Value{}280347 }@@ -284,6 +351,10 @@ func (st *cellStore) slotValue(sl slot) Value {
284351 func (st *cellStore) view(sl slot) *Cell {285352 lk := st.lookOf(sl)286353 c := &Cell{Input: st.input(sl), Value: st.slotValue(sl), Format: lk.f, Style: lk.st}354+ if sl.kind&slotDerived != 0 { // a derived cell has no expression355+ c.auto, c.derived, c.spilled = lk.auto, sl.kind&slotPivot != 0, sl.kind&slotSpill != 0356+ return c357+ }287358 switch sl.kind {288359 case slotNum:289360 c.expr = formula.Num{V: sl.num}@@ -295,11 +366,11 @@ func (st *cellStore) view(sl slot) *Cell {
295366 296367 // releaseSlot counts one fewer use of what sl refers to.297368 func (st *cellStore) releaseSlot(sl slot) {298- switch sl.kind {369+ switch sl.base() {299370 case slotRich:300371 st.rich[sl.ref] = richCell{}301372 st.richFree = append(st.richFree, sl.ref)302- case slotNum, slotBool, slotText:373+ case slotNum, slotBool, slotText, slotErr:303374 st.strs.release(sl.ref)304375 }305376 }@@ -176,8 +176,11 @@ func (s *Sheet) spillArea(a Addr, arr *functions.Array) (Rect, string) {
176176 return area, "Array result was not expanded because it would overwrite merged cells in " + ms[0].String()177177 }178178 old := s.spills[a]179- for at, c := range s.cells.anyInRange(area) {180- if at == a || c.Blank() || c.spilled && old != nil && old.why == "" && old.area.Contains(at) {179+ for at := range s.cells.anyKeysIn(area) {180+ if at == a || !s.cells.filledAt(at) {181+ continue182+ }183+ if s.cells.derivedAt(at) == slotSpill && old != nil && old.why == "" && old.area.Contains(at) {181184 continue182185 }183186 return area, "Array result was not expanded because it would overwrite data in " + at.String()@@ -188,20 +191,25 @@ func (s *Sheet) spillArea(a Addr, arr *functions.Array) (Rect, string) {
188191 // writeSpilled makes the cell at a show v as a spilled cell, keeping its189192 // formatting and note, and reports whether it changed.190193 func (s *Sheet) writeSpilled(a Addr, v Value, auto Format) bool {191- old := s.cells.get(a)194+ was, l, kind := s.cells.derivedOf(a)195+ spilled := kind == slotSpill192196 if v.Kind == Empty {193- if old == nil || !old.spilled {197+ if !spilled {194198 return false195199 }196- s.setDerived(a, old.leftover())200+ s.setDerived(a, s.cells.get(a).leftover())197201 return true198202 }199- if old != nil && old.spilled && old.Value == v && old.auto == auto {203+ if spilled && was == v && l.auto == auto {200204 return false // its input is its value's201205 }202- c := &Cell{Input: derivedInput(v), Value: v, auto: auto, spilled: true}203- if old != nil {204- c.Format, c.Style, c.Note = old.Format, old.Style, old.Note206+ if s.cells.setSpilled(a, v, auto) { // formatting kept, no formula to unlink207+ s.version++208+ return true209+ }210+ c := &Cell{Input: derivedInput(v), Value: v, Format: l.f, Style: l.st, auto: auto, spilled: true}211+ if old := s.cells.richAt(a); old != nil {212+ c.Note = old.Note205213 }206214 s.setDerived(a, c)207215 return true@@ -212,8 +220,8 @@ func (s *Sheet) writeSpilled(a Addr, v Value, auto Format) bool {
212220 func (s *Sheet) clearSpilled(area, keep Rect) []loc {213221 var changed []loc214222 var gone []Addr215- for at, c := range s.cells.anyInRange(area) {216- if c.spilled && !keep.Contains(at) {223+ for at := range s.cells.anyKeysIn(area) {224+ if s.cells.derivedAt(at) == slotSpill && !keep.Contains(at) {217225 gone = append(gone, at)218226 }219227 }@@ -18,9 +18,10 @@ import (
1818 // occupancy index (occupancy.go) has a bitmap per block of the rows1919 // holding a cell, and each block keeps their slots (slot.go) in row2020 // order, so a cell is found by indexing a column, a block and a popcount,21-// without hashing. A plain number costs its 16-byte slot. Rich cells22-// (formulas, notes, pivots' and spills' results) are whole Cells in a23-// side table the slot points at.21+// without hashing. A plain number costs its 16-byte slot, and so does a22+// pivot's or a spill's result (a derived cell, marked in its slot). Rich23+// cells (formulas, notes) are whole Cells in a side table the slot24+// points at.2425 //2526 // get returns a rich cell's own Cell, which recalculation updates in2627 // place, as ever, and a plain cell as a Cell made for the caller (a@@ -47,6 +48,12 @@ type cellStore struct {
4748 lookIdx map[look]uint16 // looks' entries4849 rich []richCell // rich cells, by slot.ref4950 richFree []uint32 // entries of rich not in use51+52+ // pivots counts the pivot's results stored (Cell.derived), and53+ // pivotArea holds them all while there are any, so they are found54+ // without visiting every cell.55+ pivots int56+ pivotArea Rect5057 }5158 5259 func newCellStore() cellStore { return cellStore{} }@@ -111,8 +118,8 @@ func (st *cellStore) has(a Addr) bool { return st.stored.has(a) }
111118 // filledAt reports whether the cell at a has contents: !get(a).Blank().112119 func (st *cellStore) filledAt(a Addr) bool { return st.filled.has(a) }113120 114-// richAt returns the cell at a if it is rich (a formula, a note, a115-// pivot's or a spill's result), or nil.121+// richAt returns the cell at a if it is rich (a formula or a note, or a122+// cell a slot can't hold), or nil.116123 func (st *cellStore) richAt(a Addr) *Cell {117124 b, i := st.find(a)118125 if b == nil || b.vals[i].kind != slotRich {@@ -148,9 +155,15 @@ func (st *cellStore) holds(a Addr, c *Cell) bool {
148155 if sl.kind == slotRich {149156 return st.rich[sl.ref].c == c150157 }151- if l := st.lookOf(sl); c.Note != "" || c.derived || c.spilled || !c.auto.IsZero() || c.Format != l.f || c.Style != l.st {158+ if l := st.lookOf(sl); c.Note != "" || c.auto != l.auto || c.Format != l.f || c.Style != l.st {159+ return false160+ }161+ if c.derived != (sl.kind&slotPivot != 0) || c.spilled != (sl.kind&slotSpill != 0) {152162 return false153163 }164+ if sl.kind&slotDerived != 0 {165+ return c.expr == nil && c.Value == st.slotValue(sl)166+ }154167 switch sl.kind {155168 case slotText:156169 return c.expr == nil && c.Input == st.strs.strs[sl.ref]@@ -201,6 +214,7 @@ func (st *cellStore) set(a Addr, c *Cell) {
201214 b.vals = slices.Insert(b.vals, i, slot{kind: slotBlank})202215 }203216 old := b.vals[i]217+ st.countPivot(a, st.isPivot(old), c.derived)204218 sl, ok := st.plainSlot(c) // before releasing old, which may share its strings205219 switch {206220 case ok:@@ -226,6 +240,7 @@ func (st *cellStore) setSlot(a Addr, sl slot) {
226240 i, _ = b.index(a.Row & (blockRows - 1))227241 b.vals = slices.Insert(b.vals, i, slot{kind: slotBlank})228242 }243+ st.countPivot(a, st.isPivot(b.vals[i]), false)229244 st.releaseSlot(b.vals[i])230245 b.vals[i] = sl231246 st.filledAs(a, st.filled.has(a), sl.kind != slotBlank)@@ -251,6 +266,7 @@ func (st *cellStore) delete(a Addr) {
251266 }252267 st.touch(a)253268 old := b.vals[i]269+ st.countPivot(a, st.isPivot(old), false)254270 if st.filled.has(a) {255271 st.filled.unmark(a)256272 }@@ -0,0 +1,143 @@
1+package sheet2+3+import (4+ "iter"5+ "slices"6+)7+8+// The store's side of derived cells (slot.go): finding the pivot's9+// results, reading what a derived cell shows without making a Cell, and10+// writing a spilled value straight into its slot.11+12+// isPivot reports whether sl holds a pivot's result.13+func (st *cellStore) isPivot(sl slot) bool {14+ if sl.kind == slotRich {15+ return st.rich[sl.ref].c.derived16+ }17+ return sl.kind&slotPivot != 018+}19+20+// countPivot counts the cell at a, which was a pivot's result or not and21+// becomes one or not.22+func (st *cellStore) countPivot(a Addr, was, is bool) {23+ switch {24+ case is && !was:25+ if st.pivots++; st.pivots == 1 {26+ st.pivotArea = Rect{From: a, To: a}27+ } else {28+ st.pivotArea = union(st.pivotArea, Rect{From: a, To: a})29+ }30+ case was && !is:31+ st.pivots--32+ }33+}34+35+// pivotKeys yields the addresses of the pivot's results stored.36+func (st *cellStore) pivotKeys() iter.Seq[Addr] {37+ return func(yield func(Addr) bool) {38+ if st.pivots == 0 {39+ return40+ }41+ for a := range st.anyKeysIn(st.pivotArea) {42+ if b, i := st.find(a); b != nil && st.isPivot(b.vals[i]) && !yield(a) {43+ return44+ }45+ }46+ }47+}48+49+// derivedOf returns what the cell at a holds as a derived cell would:50+// its value, format, style and inferred format, and slotPivot or51+// slotSpill for a pivot's result or a spilled cell, 0 for any other52+// cell (none included). It makes no Cell, so the derived cells being53+// rewritten are compared cheaply; the look mustn't be changed.54+func (st *cellStore) derivedOf(a Addr) (v Value, l *look, kind uint8) {55+ b, i := st.find(a)56+ if b == nil {57+ return Value{}, &noLook, 058+ }59+ sl := b.vals[i]60+ if sl.kind != slotRich {61+ return st.slotValue(sl), st.lookOf(sl), sl.kind & slotDerived62+ }63+ c := st.rich[sl.ref].c64+ return c.Value, &look{c.Format, c.Style, c.auto}, richDerived(c)65+}66+67+// richDerived is derivedOf's kind for a rich cell.68+func richDerived(c *Cell) uint8 {69+ switch {70+ case c.derived:71+ return slotPivot72+ case c.spilled:73+ return slotSpill74+ }75+ return 076+}77+78+// derivedAt is derivedOf's kind alone.79+func (st *cellStore) derivedAt(a Addr) uint8 {80+ b, i := st.find(a)81+ switch {82+ case b == nil:83+ return 084+ case b.vals[i].kind == slotRich:85+ return richDerived(st.rich[b.vals[i].ref].c)86+ }87+ return b.vals[i].kind & slotDerived88+}89+90+// autoAt is the format the cell at a infers: a formula's or a spilled91+// cell's.92+func (st *cellStore) autoAt(a Addr) Format {93+ b, i := st.find(a)94+ if b == nil {95+ return Format{}96+ }97+ if sl := b.vals[i]; sl.kind == slotRich {98+ return st.rich[sl.ref].c.auto99+ } else {100+ return st.lookOf(sl).auto101+ }102+}103+104+// setSpilled makes the cell at a, blank, formatting only or spilled, a105+// spilled cell showing v with the inferred format auto, keeping its106+// formatting, straight in its slot. It reports false, changing nothing,107+// for a cell a slot can't hold (one with a note), which the caller108+// stores as a Cell.109+func (st *cellStore) setSpilled(a Addr, v Value, auto Format) bool {110+ b, i := st.find(a)111+ var old slot112+ if b != nil {113+ old = b.vals[i]114+ }115+ if old.kind == slotRich {116+ return false117+ }118+ lk := old.look119+ if l := st.lookOf(old); l.auto != auto {120+ var ok bool121+ if lk, ok = st.lookID(look{l.f, l.st, auto}); !ok {122+ return false123+ }124+ }125+ if b == nil {126+ b = st.stored.mark(a)127+ i, _ = b.index(a.Row & (blockRows - 1))128+ b.vals = slices.Insert(b.vals, i, slot{kind: slotBlank})129+ }130+ st.countPivot(a, st.isPivot(old), false)131+ sl := st.derivedValue(v, lk, false) // before releasing old, which may share its string132+ st.releaseSlot(old)133+ b.vals[i] = sl134+ st.filledAs(a, st.filled.has(a), !derivedBlank(v))135+ st.touch(a)136+ return true137+}138+139+// derivedBlank reports whether a derived cell showing v is blank: its140+// entry, the value's text, is empty.141+func derivedBlank(v Value) bool {142+ return v.Kind == Empty || v.Kind == Text && v.Str == ""143+}@@ -228,6 +228,9 @@ func (m *Model) drawChart(i int, c sheet.Chart, selected bool) []string {
228228 w, h := chartInner(c)229229 o := m.term.chartOptions()230230 o.Chart = c.ChartOptions231+ if m.term.sixelOn() {232+ o.Image = m.term.six.placed(i) // see sixel.go233+ }231234 if i >= maxImages {232235 o.Image = false233236 }@@ -246,7 +249,7 @@ func (m *Model) drawChart(i int, c sheet.Chart, selected bool) []string {
246249 lines = append(lines, border.Render("┌─")+m.th.Title.Render(title)+247250 border.Render(strings.Repeat("─", max(c.W-3-ansi.StringWidth(title), 0))+"┐"))248251 for y := range h {249- lines = append(lines, border.Render("│")+" "+chartRow(&m.th, g, y, firstImageID+i, o.Image)+" "+border.Render("│"))252+ lines = append(lines, border.Render("│")+" "+chartRow(&m.th, g, y, firstImageID+i, o.Image, m.term.kitty)+" "+border.Render("│"))250253 }251254 corner := "┘"252255 if selected {@@ -262,8 +265,9 @@ func (m *Model) drawChart(i int, c sheet.Chart, selected bool) []string {
262265 }263266 264267 // chartRow renders row y of a drawn chart with the theme; with images,265-// the plot area is the image's placeholders.266-func chartRow(th *theme.Theme, g *chart.Grid, y, imageID int, image bool) string {268+// the plot area is the image's kitty placeholders, or blank cells for a269+// sixel image to cover.270+func chartRow(th *theme.Theme, g *chart.Grid, y, imageID int, image, kitty bool) string {267271 var b, run strings.Builder268272 var style lipgloss.Style269273 styled := false@@ -283,8 +287,10 @@ func chartRow(th *theme.Theme, g *chart.Grid, y, imageID int, image bool) string
283287 c := g.At(x, y)284288 key, st, isStyled := 0, lipgloss.Style{}, true285289 text := c.Text286- switch {287- case image && y >= g.Plot.Min.Y && y < g.Plot.Max.Y && x >= g.Plot.Min.X && x < g.Plot.Max.X:290+ switch inPlot := y >= g.Plot.Min.Y && y < g.Plot.Max.Y && x >= g.Plot.Min.X && x < g.Plot.Max.X; {291+ case image && inPlot && !kitty:292+ key, isStyled, text = 2, false, " "293+ case image && inPlot:288294 key, st = 1, th.ImageID(imageID)289295 text = chart.Placeholder(y-g.Plot.Min.Y, x-g.Plot.Min.X)290296 case c.Fg == chart.None && c.Bg == chart.None:@@ -4,6 +4,7 @@ import (
44 "fmt"55 "image/color"66 "log/slog"7+ "slices"78 "strconv"89 "strings"910 @@ -21,24 +22,30 @@ import (
2122 // replies to queries sent at startup. It survives File New and Open.2223 //2324 // Charts become images when the terminal answers a kitty graphics query24-// (kitty, Ghostty, WezTerm and others); everywhere else they stay text.25-// Images use the terminal's own palette, asked for with OSC 4 once26-// graphics are known to work, so bars match the text legend.25+// (kitty, Ghostty, WezTerm and others), or else when its primary device26+// attributes (DA1) list sixel graphics (foot, xterm, mlterm, Windows27+// Terminal; see sixel.go); everywhere else they stay text. Images use28+// the terminal's own palette, asked for with OSC 4 at startup, so bars29+// match the text legend.2730 type terminal struct {2831 noImages bool // chart-images = false in the config2932 noNotify bool // notifications = false in the config3033 kitty bool // answers kitty graphics queries34+ sixel bool // lists sixel in DA1, without kitty graphics, outside tmux35+ registers int // sixel color registers, as XTSMGRAPHICS reports them3136 tmux bool // inside tmux: sequences for the outer terminal need passthrough3237 cellW, cellH int // cell size in pixels, 0 until reported3338 palette map[int]color.RGBA39+ bg color.Color // the terminal's background, nil until reported3440 sent map[int]string // image id -> what was sent, to send only changes41+ six sixelState // the sixel images drawn and to draw3542 blurred bool // the terminal window doesn't have focus3643 }3744 3845 // newTerminal starts with what getenv, the terminal's environment, says:3946 // the process's own locally, the client's when served over SSH.4047 func newTerminal(getenv func(string) string) terminal {41- return terminal{tmux: getenv("TMUX") != "", palette: map[int]color.RGBA{}, sent: map[int]string{}}48+ return terminal{tmux: getenv("TMUX") != "", palette: map[int]color.RGBA{}, sent: map[int]string{}, registers: 256}4249 }4350 4451 // wrap prepares a sequence for the outer terminal.@@ -50,10 +57,13 @@ func (t *terminal) wrap(seq string) string {
5057 }5158 5259 // probes are the startup queries: kitty graphics support, live light53-// and dark changes (mode 2031), and the 16 palette colors (OSC 4), which54-// chart images and color scales draw with.60+// and dark changes (mode 2031), the 16 palette colors (OSC 4), which61+// chart images and color scales draw with, the sixel color registers62+// (XTSMGRAPHICS) and, last, the primary device attributes. Terminals63+// answer in order, so the kitty and XTSMGRAPHICS replies, if any, come64+// before DA1's, which then decides between kitty images, sixel and text.5565 func (t *terminal) probes() tea.Cmd {56- return tea.Raw(t.wrap(chart.Query()) + ansi.SetModeLightDark + paletteQuery())66+ return tea.Raw(t.wrap(chart.Query()) + ansi.SetModeLightDark + paletteQuery() + sixelRegistersQuery + ansi.RequestPrimaryDeviceAttributes)5767 }5868 5969 // paletteQuery asks for the 16 ANSI colors; terminals that don't answer@@ -84,6 +94,15 @@ func (t *terminal) handle(msg tea.Msg) tea.Cmd {
8494 // was asked for at startup.8595 return tea.Raw(ansi.WindowOp(16)) // 16: report the cell size in pixels8696 }97+ case uv.PrimaryDeviceAttributesEvent:98+ if slices.Contains(msg, 4) && !t.kitty && !t.tmux && !t.sixel {99+ t.sixel = true100+ return tea.Raw(ansi.WindowOp(16)) // sixel images are drawn at the cell size101+ }102+ case uv.UnknownCsiEvent:103+ if n, ok := parseRegisters(string(msg)); ok {104+ t.registers = n105+ }87106 case uv.CellSizeEvent:88107 if msg.Width > 0 && msg.Height > 0 {89108 t.cellW, t.cellH = msg.Width, msg.Height@@ -156,10 +175,12 @@ func (t *terminal) chartPalette(th *theme.Theme) chart.Palette {
156175 return p157176 }158177 159-// images reports whether charts are drawn as images.178+// images reports whether charts are drawn as kitty images.160179 func (t *terminal) images() bool { return t.kitty && !t.noImages }161180 162-// chartOptions are the drawing options for charts on this terminal.181+// chartOptions are the drawing options for charts on this terminal. For182+// sixel, whether a chart is an image depends on where it is: see183+// Model.drawChart.163184 func (t *terminal) chartOptions() chart.Options {164185 return chart.Options{Image: t.images(), CellW: t.cellW, CellH: t.cellH}165186 }@@ -238,7 +238,7 @@ func (m *Model) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
238238 m.clampView()239239 // Any edit may have queued JEV questions.240240 // Chart images follow any change, see graphics.go.241- return m, tea.Batch(cmd, m.jev.send(m.spans.Parent()), m.term.syncImages(m.sheet, m.displayCharts, &m.th, m.spans))241+ return m, tea.Batch(cmd, m.jev.send(m.spans.Parent()), m.term.syncImages(m.sheet, m.displayCharts, &m.th, m.spans), m.syncSixel(msg))242242 }243243 244244 // beginUpdate prepares for an input event and returns the sheet's state@@ -219,6 +219,7 @@ func (m *Model) handlePrefs(msg tea.Msg) bool {
219219 switch msg := msg.(type) {220220 case tea.BackgroundColorMsg:221221 m.prefs.light = !msg.IsDark()222+ m.term.bg = msg.Color222223 if p := m.applyTheme(); p != "" && m.note == "" {223224 m.note = m.th.Warning.Render(p)224225 }@@ -0,0 +1,293 @@
1+package ui2+3+import (4+ "fmt"5+ "hash/fnv"6+ "image"7+ "image/color"8+ "log/slog"9+ "slices"10+ "strconv"11+ "strings"12+ "time"13+14+ tea "charm.land/bubbletea/v2"15+ "github.com/charmbracelet/x/ansi"16+17+ "github.com/FelineStateMachine/012/internal/chart"18+)19+20+// Sixel chart images, for terminals without kitty graphics. A sixel21+// image is pixels the terminal draws at the cursor, which Bubble Tea's22+// renderer knows nothing of: it keeps drawing cells, and a cell it23+// writes over the image punches a hole in it. So:24+//25+// - A chart is an image only where the whole of its plot area shows: in26+// the grid, under no chart drawn later and no floating box (a menu,27+// a dialog, formula suggestions, a note). Elsewhere it is text. The28+// frame keeps blank cells where an image goes, so the renderer has29+// nothing to write there.30+// - Images are drawn after the frame: a sixelDrawMsg comes once the31+// frame has been still for sixelSettle, longer than a frame, and32+// draws each image with the cursor saved and restored around it.33+// - When an image moves, changes or goes (scrolling, resizing, an edit34+// of its data, an overlay opening over it), the screen is cleared and35+// redrawn whole, which erases the images, and they are drawn again.36+// When a row an image crosses changes, the images are drawn again.37+// - Never inside tmux: tmux answers DA1 for itself, and an image passed38+// through to the outer terminal lands where tmux doesn't know, so its39+// redraws would erase it.40+41+// sixelRegistersQuery asks for the number of sixel color registers42+// (XTSMGRAPHICS, read).43+const sixelRegistersQuery = "\x1b[?1;1;0S"44+45+// parseRegisters parses the XTSMGRAPHICS reply "CSI ? 1 ; 0 ; n S".46+func parseRegisters(s string) (int, bool) {47+ s, ok := strings.CutPrefix(s, "\x1b[?1;0;")48+ if !ok {49+ return 0, false50+ }51+ s, ok = strings.CutSuffix(s, "S")52+ n, err := strconv.Atoi(s)53+ if !ok || err != nil || n < 2 {54+ return 0, false55+ }56+ return min(n, 256), true57+}58+59+// sixelSettle is how long the frame stays still before images are drawn60+// over it: more than a frame at 60 fps, so the frame is out first.61+const sixelSettle = 25 * time.Millisecond62+63+// sixelState is what the sixel images are and where.64+type sixelState struct {65+ cache map[int]sixelImage // by chart66+ plan []sixelPlace // the images the frame keeps room for67+ drawn []sixelPlace // the images on screen68+ frame uint64 // the rows of the latest frame the plan crosses, hashed69+ drawnFrame uint64 // frame when the images were drawn70+ changed time.Time // when frame last changed71+ pending bool // a sixelDrawMsg is on its way72+ now func() time.Time // the clock; time.Now when nil73+ tick func() tea.Cmd // what sends sixelDrawMsg; sixelTick when nil74+}75+76+// sixelImage is a chart's image encoded, and where its plot area is in77+// the chart's contents.78+type sixelImage struct {79+ key string80+ plot image.Rectangle81+ seq string // "" when there is nothing to plot82+}83+84+// sixelPlace is chart's image at screen cells at.85+type sixelPlace struct {86+ chart int87+ at image.Rectangle88+ key string89+}90+91+// sixelDrawMsg draws the images once the frame has settled.92+type sixelDrawMsg struct{}93+94+func sixelTick() tea.Cmd {95+ return tea.Tick(sixelSettle, func(time.Time) tea.Msg { return sixelDrawMsg{} })96+}97+98+// later asks for a sixelDrawMsg.99+func (s *sixelState) later() tea.Cmd {100+ s.pending = true101+ if s.tick != nil {102+ return s.tick()103+ }104+ return sixelTick()105+}106+107+func (s *sixelState) clock() time.Time {108+ if s.now != nil {109+ return s.now()110+ }111+ return time.Now()112+}113+114+// sixelOn reports whether charts are drawn as sixel images: the115+// terminal has sixel and has said its cell size, and chart-images isn't116+// off.117+func (t *terminal) sixelOn() bool { return t.sixel && !t.noImages && t.cellW > 0 && t.cellH > 0 }118+119+// placed reports whether chart i is an image in the current frame.120+func (s *sixelState) placed(i int) bool {121+ return slices.ContainsFunc(s.plan, func(p sixelPlace) bool { return p.chart == i })122+}123+124+// syncSixel follows msg, after Update has handled it: it encodes the125+// charts that changed, works out where images go, clears the screen when126+// one drawn moved, changed or went, and asks for the images to be drawn127+// once the frame settles.128+func (m *Model) syncSixel(msg tea.Msg) tea.Cmd {129+ t := &m.term130+ if _, ok := msg.(sixelDrawMsg); ok {131+ return m.drawSixels()132+ }133+ var cmds []tea.Cmd134+ if _, ok := msg.(tea.WindowSizeMsg); ok && t.sixel {135+ cmds = append(cmds, tea.Raw(ansi.WindowOp(16))) // the font may have changed size too136+ }137+ if !t.sixelOn() {138+ t.six.plan, t.six.cache = nil, nil139+ if len(t.six.drawn) > 0 {140+ t.six.drawn = nil141+ cmds = append(cmds, tea.ClearScreen)142+ }143+ return tea.Batch(cmds...)144+ }145+ m.encodeSixels()146+ t.six.plan = m.sixelPlan()147+ if t.six.gone() {148+ t.six.drawn = nil149+ cmds = append(cmds, tea.ClearScreen)150+ }151+ if len(t.six.plan) > 0 && !t.six.pending {152+ cmds = append(cmds, t.six.later())153+ }154+ return tea.Batch(cmds...)155+}156+157+// gone reports whether an image drawn isn't in the plan as it is.158+func (s *sixelState) gone() bool {159+ for _, d := range s.drawn {160+ if !slices.Contains(s.plan, d) {161+ return true162+ }163+ }164+ return false165+}166+167+// drawSixels draws the planned images over the settled frame, unless168+// they are drawn already over the same rows.169+func (m *Model) drawSixels() tea.Cmd {170+ s := &m.term.six171+ s.pending = false172+ if !m.term.sixelOn() || len(s.plan) == 0 || s.frame == s.drawnFrame && slices.Equal(s.drawn, s.plan) {173+ return nil174+ }175+ if s.clock().Sub(s.changed) < sixelSettle {176+ return s.later()177+ }178+ var b strings.Builder179+ for _, p := range s.plan {180+ b.WriteString(ansi.SaveCursor + ansi.CursorPosition(p.at.Min.X+1, p.at.Min.Y+1))181+ b.WriteString(s.cache[p.chart].seq)182+ b.WriteString(ansi.RestoreCursor)183+ }184+ s.drawn, s.drawnFrame = slices.Clone(s.plan), s.frame185+ return tea.Raw(b.String())186+}187+188+// noteFrame records the rows of the frame content that the planned189+// images cross, so images are drawn again when those rows change.190+func (s *sixelState) noteFrame(content string) {191+ if len(s.plan) == 0 {192+ s.frame = 0193+ return194+ }195+ h := fnv.New64a()196+ lines := strings.Split(content, "\n")197+ for _, p := range s.plan {198+ fmt.Fprintf(h, "%d %v %s\n", p.chart, p.at, p.key)199+ for y := p.at.Min.Y; y < p.at.Max.Y && y < len(lines); y++ {200+ h.Write([]byte(lines[y]))201+ }202+ }203+ if f := h.Sum64(); f != s.frame {204+ s.frame, s.changed = f, s.clock()205+ }206+}207+208+// encodeSixels encodes the image of each chart whose data, size, colors209+// or cell size changed.210+func (m *Model) encodeSixels() {211+ t := &m.term212+ if t.six.cache == nil {213+ t.six.cache = map[int]sixelImage{}214+ }215+ o := t.chartOptions()216+ o.Image = true217+ pal, bg := t.chartPalette(&m.th), m.sixelBackground()218+ charts := m.displayCharts()219+ for i, c := range charts {220+ if i >= maxImages {221+ break222+ }223+ w, h := chartInner(c)224+ d := m.sheet.ChartData(c)225+ o.Chart = c.ChartOptions226+ key := fmt.Sprintf("%v %v %d %d %v %v %v %d", c.Type, d, w, h, o, pal, bg, t.registers)227+ if t.six.cache[i].key == key {228+ continue229+ }230+ span := m.spans.Start("chart.sixel", slog.String("type", c.Type.String()), slog.Int("w", w), slog.Int("h", h))231+ g := chart.Draw(c.Type, d, w, h, o)232+ si := sixelImage{key: key, plot: g.Plot}233+ if img := chart.Image(c.Type, d, w, h, o, pal); img != nil && !g.Plot.Empty() {234+ si.seq = chart.Sixel(img, bg, t.registers)235+ }236+ span.End(slog.Int("bytes", len(si.seq)))237+ t.six.cache[i] = si238+ }239+ for i := range t.six.cache {240+ if i >= len(charts) || i >= maxImages {241+ delete(t.six.cache, i)242+ }243+ }244+}245+246+// sixelBackground is the color images are composited over: the theme's247+// background, or the terminal's, or black.248+func (m *Model) sixelBackground() color.RGBA {249+ c := m.term.bg250+ if m.th.Palette != nil && m.th.Palette.Background != nil {251+ c = m.th.Palette.Background252+ }253+ if c == nil {254+ return color.RGBA{0, 0, 0, 255}255+ }256+ r, g, b, _ := c.RGBA()257+ return color.RGBA{uint8(r >> 8), uint8(g >> 8), uint8(b >> 8), 255}258+}259+260+// sixelPlan is where images go: the charts whose plot area shows whole261+// in the grid, under no chart drawn later and no floating box.262+func (m *Model) sixelPlan() []sixelPlace {263+ var over []image.Rectangle264+ for _, b := range m.floating() {265+ if len(b.Lines) > 0 {266+ over = append(over, image.Rect(b.X, b.Y, b.X+b.Width(), b.Y+b.Height()))267+ }268+ }269+ charts := m.displayCharts()270+ boxes := make([]image.Rectangle, len(charts))271+ for i, c := range charts {272+ x, y := m.chartScreen(c)273+ boxes[i] = image.Rect(x, y, x+c.W, y+c.H)274+ }275+ grid := image.Rect(m.hdrW(), gridTop, m.width, gridTop+m.visibleRows())276+ var plan []sixelPlace277+ for i := range charts {278+ img, ok := m.term.six.cache[i]279+ if !ok || img.seq == "" {280+ continue281+ }282+ at := img.plot.Add(boxes[i].Min.Add(image.Pt(2, 1))) // the border and a column of padding283+ if !at.In(grid) || overlapsAny(at, over) || overlapsAny(at, boxes[i+1:]) {284+ continue285+ }286+ plan = append(plan, sixelPlace{i, at, img.key})287+ }288+ return plan289+}290+291+func overlapsAny(r image.Rectangle, rs []image.Rectangle) bool {292+ return slices.ContainsFunc(rs, r.Overlaps)293+}@@ -0,0 +1,159 @@
1+package ui2+3+import (4+ "fmt"5+ "strings"6+ "testing"7+ "time"8+9+ tea "charm.land/bubbletea/v2"10+ uv "github.com/charmbracelet/ultraviolet"11+ "github.com/charmbracelet/x/ansi/kitty"12+)13+14+// ticked stands for a sixelDrawMsg asked for: tests deliver15+// sixelDrawMsg themselves, with the clock where they want it.16+type ticked struct{}17+18+// sixelModel is a wide model with a chart, on a terminal that has said19+// what a sixel terminal says: sixel in DA1, 16 color registers and its20+// cell size. Its clock stands still unless the test moves it.21+func sixelModel(t *testing.T) (*Model, *time.Time) {22+ t.Helper()23+ m := wideModel()24+ m.term.tmux = false // wherever the tests run25+ clock := time.Unix(1000, 0)26+ m.term.six.now = func() time.Time { return clock }27+ m.term.six.tick = func() tea.Cmd { return func() tea.Msg { return ticked{} } }28+ chartTable(t, m)29+ m.runCommand("insert.chart")30+ press(t, m, "<enter>", "<esc>")31+ if !strings.Contains(screen(m), "█") {32+ t.Fatalf("no bars in the text chart:\n%s", screen(m))33+ }34+ send(m, uv.UnknownCsiEvent("\x1b[?1;0;16S"))35+ if got := sixelOut(m, uv.PrimaryDeviceAttributesEvent{62, 4, 22}); !m.term.sixel || got != "\x1b[16t" {36+ t.Fatalf("sixel not detected, or no cell size asked for: %q", got)37+ }38+ if m.term.registers != 16 {39+ t.Errorf("%d registers, want 16", m.term.registers)40+ }41+ if got := sixelOut(m, uv.CellSizeEvent{Width: 8, Height: 16}); got != "<tick>" {42+ t.Fatalf("after the cell size: %q, want images asked for", got)43+ }44+ return m, &clock45+}46+47+// sixelOut updates m with msg and returns what the commands write to the48+// terminal, with "<clear>" for a screen cleared and "<tick>" for images49+// asked for.50+func sixelOut(m *Model, msg tea.Msg) string {51+ _, cmd := m.Update(msg)52+ m.View()53+ var b strings.Builder54+ for _, msg := range flatten(cmd) {55+ switch msg := msg.(type) {56+ case tea.RawMsg:57+ b.WriteString(msg.Msg.(string))58+ case ticked:59+ b.WriteString("<tick>")60+ default:61+ if fmt.Sprintf("%T", msg) == "tea.clearScreenMsg" {62+ b.WriteString("<clear>")63+ }64+ }65+ }66+ return b.String()67+}68+69+func TestSixelImages(t *testing.T) {70+ m, clock := sixelModel(t)71+ if len(m.term.six.plan) != 1 || strings.Contains(screen(m), "█") {72+ t.Fatalf("plan %v; the plot area isn't blank:\n%s", m.term.six.plan, screen(m))73+ }74+ if !strings.Contains(screen(m), "1,500") {75+ t.Error("the axes aren't text around the image")76+ }77+ // Not drawn until the frame has been still for sixelSettle.78+ if got := sixelOut(m, sixelDrawMsg{}); got != "<tick>" {79+ t.Errorf("drawn over a frame still changing: %q", got)80+ }81+ *clock = clock.Add(sixelSettle)82+ p := m.term.six.plan[0]83+ got := sixelOut(m, sixelDrawMsg{})84+ at := fmt.Sprintf("\x1b7\x1b[%d;%dH\x1bP0;1;0q", p.at.Min.Y+1, p.at.Min.X+1)85+ if !strings.HasPrefix(got, at) || !strings.HasSuffix(got, "\x1b\\\x1b8") {86+ t.Fatalf("image drawn as %.60q, want it at %q with the cursor saved", got, at)87+ }88+ if n := strings.Count(got, ";2;"); n == 0 || n > 16 {89+ t.Errorf("%d colors defined for 16 registers", n)90+ }91+ // Drawn once: a frame that doesn't change draws nothing.92+ if got := sixelOut(m, sixelDrawMsg{}); got != "" {93+ t.Errorf("drawn again over the same frame: %.40q", got)94+ }95+ // A row the image crosses changes: drawn again.96+ press(t, m, "<f5>", fmt.Sprintf("A%d", p.at.Min.Y-gridTop+1), "<enter>")97+ m.View() // as Bubble Tea does after every update98+ *clock = clock.Add(time.Second)99+ if got := sixelOut(m, sixelDrawMsg{}); !strings.HasPrefix(got, at) {100+ t.Errorf("not drawn again after its rows changed: %.40q", got)101+ }102+}103+104+// Scrolling moves the image: the screen is cleared, erasing it, and it is105+// drawn where it went. The command palette over it makes it text.106+func TestSixelMovesAndHides(t *testing.T) {107+ m, clock := sixelModel(t)108+ *clock = clock.Add(time.Second)109+ sixelOut(m, sixelDrawMsg{})110+ was := m.term.six.plan[0].at111+ if got := sixelOut(m, tea.MouseWheelMsg{X: 50, Y: gridTop + 2, Button: tea.MouseWheelDown}); !strings.Contains(got, "<clear>") {112+ t.Fatalf("scrolled without clearing: %q", got)113+ }114+ if len(m.term.six.plan) == 1 && m.term.six.plan[0].at == was {115+ t.Fatalf("the image didn't move: %v", m.term.six.plan)116+ }117+ *clock = clock.Add(time.Second)118+ if got := sixelOut(m, sixelDrawMsg{}); len(m.term.six.plan) == 1 && !strings.Contains(got, "\x1bP") {119+ t.Error("not drawn where it went")120+ }121+ sixelOut(m, tea.MouseWheelMsg{X: 50, Y: gridTop + 2, Button: tea.MouseWheelUp})122+ *clock = clock.Add(time.Second)123+ sixelOut(m, sixelDrawMsg{})124+ if got := sixelOut(m, tea.KeyPressMsg{Code: 'k', Mod: tea.ModCtrl}); len(m.term.six.plan) != 0 || !strings.Contains(got, "<clear>") {125+ t.Errorf("palette open: plan %v, %q", m.term.six.plan, got)126+ }127+}128+129+// Sixel is used only when kitty graphics aren't, never inside tmux, and130+// not with chart-images off.131+func TestSixelDetection(t *testing.T) {132+ m := wideModel()133+ m.term.tmux = false134+ m.Update(uv.PrimaryDeviceAttributesEvent{62, 22})135+ if m.term.sixel {136+ t.Error("sixel without attribute 4")137+ }138+ m.Update(uv.KittyGraphicsEvent{Options: kitty.Options{ID: 31}, Payload: []byte("OK")})139+ m.Update(uv.PrimaryDeviceAttributesEvent{62, 4})140+ if m.term.sixel {141+ t.Error("sixel on a terminal with kitty graphics")142+ }143+ m = wideModel()144+ m.term.tmux = true145+ m.Update(uv.PrimaryDeviceAttributesEvent{62, 4})146+ if m.term.sixel {147+ t.Error("sixel inside tmux")148+ }149+ m, _ = sixelModel(t)150+ m.term.noImages = true151+ if got := sixelOut(m, nil); m.term.sixelOn() || len(m.term.six.plan) != 0 || got != "" {152+ t.Errorf("sixel images with chart-images = false: %q", got)153+ }154+ for in, want := range map[string]int{"\x1b[?1;0;256S": 256, "\x1b[?1;0;1024S": 256, "\x1b[?1;3;0S": 0, "\x1b[?2;0;4S": 0} {155+ if n, _ := parseRegisters(in); n != want {156+ t.Errorf("%q: %d registers, want %d", in, n, want)157+ }158+ }159+}