The finished phase plans and the Done notes growing on Ahead rows go; Ahead keeps only open work (including what was left of partly done rows: Excel and LibreOffice checks, prebuilt binaries, hold-to-preview, sixel), and Shipped lists each feature once with the doc that describes it. TEXTJOIN leaves the text functions row: it exists.
- commit
- acea83599ea1e672bb5a6bc6bef4d35adb50a5d0
- parent
- 6cc3303840dc83eb61772c9b2c1b3748a17d16ad
- author
- Dami <dami.etoile@gmail.com>
| M | ROADMAP.md | 213 | ++++++++++++------------------ |
@@ -1,172 +1,123 @@
1-# Roadmap: past 1-2-31+# Roadmap22 33 Principles: inside the grid 012 behaves like Google Sheets; 1-2-3 is the4-visual identity around it. The main binary stays pure Go (`CGO_ENABLED=0`), and every feature ships with unit5-tests, a libghostty e2e test and reviewed golden screens. Visual and UX6-quality keep pace with features: see [docs/UX.md](docs/UX.md).4+visual identity around it. The main binary stays pure Go5+(`CGO_ENABLED=0`), and every feature ships with unit tests, a libghostty6+e2e test and reviewed golden screens. Visual and UX quality keep pace7+with features: see [docs/UX.md](docs/UX.md).78 8-## Goals9-10-| Phase | Goal | Done when |11-|---|---|---|12-| 1 | The engine does what Sheets users reach for daily, and every change can be undone | Undo/redo (Ctrl+Z/Ctrl+Y) covers every mutation; copy, cut and paste (Ctrl+C/X/V, paste values) and insert/delete rows and columns adjust relative and `$absolute` references; number formats (Format menu, auto-detected `$` and `%` on entry) and named ranges work and persist; Sheets' everyday functions, dates and finance functions match excelize on a differential test suite |13-| 2 | Finding and using features is faster than in any terminal spreadsheet | A Sheets-structured menu bar (File Edit View Insert Format Data Help) and a searchable command palette reach every command with its key shown; formulas autocomplete with signature hints; precedents and dependents are visible and jumpable; rows and columns freeze; find/replace and filter work; fill down (Ctrl+D) and series fill work; an optional vim keymap exists |14-| 3 | 012 uses the terminal it's running in to the fullest, and degrades gracefully | Copy and paste ranges through the system clipboard; charts render as images in Ghostty/kitty and as text elsewhere; URLs are clickable; error cells are marked beyond color; cursor shape reflects the mode; the theme follows light and dark |15-| 4 | Data moves in and out of 012 without friction or cgo | CSV/TSV, XLSX, SQLite, Parquet and Lotus .wk1 import, with export where it makes sense; large imports show progress; the binary still builds with `CGO_ENABLED=0` |16-| 5 | 012 goes where 1-2-3 couldn't | Multiple sheets with cross-sheet references; derived frequency and pivot sheets; recorded and Starlark macros; an SSH server mode; opt-in decimal arithmetic; VHS demo tapes |17-18-## Phase 1: engine foundations (unblocks everything else)19-20-| Item | Why | Size |21-|---|---|---|22-| Command log: every mutation is a command with an inverse | Multi-level undo/redo, macro recording, later sync | M |23-| /Copy and /Move with relative and `$absolute` ref adjustment | 1-2-3 parity; `$` is parsed today but ignored | M |24-| Insert/Delete rows and columns with reference rewriting | Parity | M |25-| /Range Format: Fixed, Currency, Percent, Comma, Date | Parity, and the biggest visual gap today | S |26-| Range names (`@SUM(SALES)`) | Parity | S |27-| @DATE, @NOW, @PMT, @NPV, @IRR, @VLOOKUP | Lotus serial dates (with the 1900 leap-day quirk) done in-house | M |28-| Differential tests against `excelize` `CalcCellValue` | Test oracle: translate our formulas to Excel syntax, compare results | S |29-30-## Phase 2: modern UX31-32-| Item | Source of the idea | Building block | Size |33-|---|---|---|---|34-| Command palette (Ctrl+P), shows each command's key | VisiData, Quadratic, MS Edit | lipgloss v2 `Compositor` layers, `sahilm/fuzzy` | S-M |35-| Formula autocomplete and signature hint while typing `@SU` | Excel, Sheets | third panel line, function table | M |36-| Live SUM/AVG/COUNT of the pointed range in the status line | Excel | engine `aggregate` | S |37-| Trace precedents/dependents: highlight, `[` and `]` to jump | Excel | dependency graph already exists | M |38-| One-key frozen titles | csvlens, sc-im, 1-2-3 /Worksheet Titles | viewport split | S-M |39-| Search (`n`/`N`) and non-destructive row filter | csvlens | | M |40-| Fill down / fill series (1,2,3, Jan, Feb) | Excel, Sheets | needs Copy ref adjustment | S-M |41-| Optional vim keymap (`hjkl`, counts, `:` commands). Done: File > Settings > Vim keys, [keys.md](docs/keys.md#vim-keys) | sc-im, csvlens | tables of command bindings, the command registry for `:` | M |42-| Frequency table of a column as a derived sheet | VisiData Shift+F | a preset pivot table | M, done: Data > Frequency table, Alt+Shift+F |43-44-## Phase 3: terminal-native features45-46-| Item | API | Notes |47-|---|---|---|48-| Copy range as TSV to the system clipboard | `tea.SetClipboard` (OSC 52) | Works over SSH |49-| Paste TSV/CSV into a range | `tea.PasteMsg` (bracketed paste, already on) | |50-| Text charts in a popup (/Graph) | `ntcharts/v2` | Works everywhere |51-| Real image charts | x/ansi `kitty` Unicode placeholders + `tea.Raw` | Ghostty/kitty; survives the diff renderer and tmux. Sixel fallback redrawn on resize |52-| Clickable URLs in cells | lipgloss `Style.Hyperlink` (OSC 8) | |53-| Red undercurl on ERR cells | `UnderlineStyle(UnderlineCurly).UnderlineColor` | |54-| Bar cursor in EDIT, block elsewhere | `View.Cursor.Shape` | |55-| Light/dark palette | `tea.RequestBackgroundColor` | |56-| Progress bar and notification for long imports | `View.ProgressBar`, `ansi.Notify` | |57-| Hold-to-preview, shift+enter | `View.KeyboardEnhancements` | kitty protocol terminals |58-59-Bubble Tea v2 already turns on synchronized output (2026) and grapheme width60-(2027) when the terminal supports them.61-62-## Phase 4: data in and out (all pure Go)63-64-| Format | Library |65-|---|---|66-| CSV/TSV | stdlib `encoding/csv` plus a small dialect sniffer |67-| XLSX | `github.com/xuri/excelize/v2` |68-| SQLite | `modernc.org/sqlite` |69-| Parquet | `github.com/parquet-go/parquet-go` |70-| Lotus .wk1 | in-house (no Go reader exists), from the LibreOffice and Gnumeric filters |71-72-Avoid in the main binary: DuckDB and automerge (both need cgo).73-74-## Phase 5: beyond75-76-- Multiple sheets with references between them, in Sheets' style (`Sheet2!A1`) rather than 1-2-3 R3's `B:A1`, and sheet tabs. Done.77-- Pivot tables as derived sheets. Done: Data > Pivot table and its editor, live results the engine owns, saved as their definition (file version 5), values in XLSX; see [docs/data.md](docs/data.md#pivot-tables).78-- Macros: replay the command log; Starlark (`go.starlark.net`) for scripts, with step limits. Done: Data > Macros records (absolute or relative references), saves, runs (Ctrl+Alt+Shift+digit, the palette) and manages macros, stored as Starlark in the file; see [docs/macros.md](docs/macros.md).79-- Serve over SSH with `charm.land/wish/v2`. Done: `012 serve [dir]`, a 012 per session confined to the directory, public-key auth only ([docs/ssh.md](docs/ssh.md)). Not shared editing: see below.80-- Decimal mode for currency (`cockroachdb/apd/v3`), opt-in. Done: File > Settings > Decimal arithmetic, boundary in the README.81-- Demos with VHS tapes. Done: `make demos` renders `demos/` locally and the README shows them. Not run in CI: the GitHub repo is hosting only.82-83-## Next: vast data84-85-The grid is Excel's, 1,048,576 x 16,384, since step A; memory is bounded86-by the `max-cells` budget rather than the grid. Raise that budget in87-measured steps (see [docs/limits.md](docs/limits.md)):88-89-| Step | What | Result |90-|---|---|---|91-| A | Make every operation cost what the data costs, not the grid: clip whole-column and whole-row ranges to the used area, sparse range reads for aggregates, column and row formats instead of per-cell formatting, a range index sized to the used columns, three-letter columns (to XFD), wider row headers. Then raise the grid to 1,048,576 x 16,384 with a `max-cells` budget in the config file, and a stress test that fails when any command on an empty full-grid selection costs more than its data. **Done**: occupancy indexes beside the cell map, sparse and clipped range reads, running aggregates shared per recalculation and interval trees of range users (1000 full-column SUMs: 174 ms to 0.5 ms per edit; 8192 running totals: 0.73 s to 2.6 ms), line formats for columns, rows and the sheet, `A:C` and `2:5` references, `max-cells` for imports and pastes, and `TestCommandsCostTheDataNotTheGrid` | Excel-sized grids, memory still bounded |92-| B | Compact column storage behind `cellStore`: typed value blocks, formulas and formats in side tables (about 20 to 40 B per cell instead of 300) | A budget of 10 M+ cells in a few hundred MB |93-| C | A streaming or binary file format next to the readable JSON one | Open and save scale with the data |94-| Later | Linked, paged read-only ranges over Parquet and SQLite that feed pivots and formulas by streaming | Sources too big for any grid |9+This page is what's ahead. What has shipped is listed once, at the end,10+with the doc that describes it; when an item ships, it moves there as11+one line.9512 9613 ## Ahead9714 98-Collected after phase 5 from the docs, the limits, and what each piece of99-work left open. Sizes: S (a day or two), M (about a week), L (weeks).15+Sizes: S (a day or two), M (about a week), L (weeks).10016 101-### 1. Finish and harden what shipped17+### 1. Harden what shipped10218 10319 | Item | Why | Size |10420 |---|---|---|105-| Open 012's XLSX output in a real spreadsheet app before releases. Done for Google Sheets: sheets, hidden sheets, named ranges, implicit intersection, column and cell formats, widths, frozen panes, filters, links and missing-sheet values all match; repeat with Excel and LibreOffice when available | The writer is otherwise checked by excelize and 012's reader | S |106-| Column and row formats travel with copy, cut and move; formulas reading blanks of a newly formatted column re-infer at once. Done: pastes and moves carry what cells show, whole lines their line formats; see [data.md](docs/data.md#copy-paste-and-fill) | Left open by step A | S |107-| Save as onto another existing file asks before replacing (local and `012 serve`). Done: Save as, `:w name` and `:wq name` ask on the context line ([files.md](docs/files.md)) | Replacing a file by mistake loses it | S |108-| XLSX sheet names Excel can't take: formulas and named ranges that name a renamed sheet use the written name. Done ([files.md](docs/files.md)) | They'd otherwise go out as values | S |109-| Replace current sheet keeps its charts; Insert new sheet(s) goes after the current tab. Done: charts of a whole table are re-pointed to the new one when it has as many series, and the note says which; see [files.md](docs/files.md) | Import location details | S |110-| Precedent tracing explains when every precedent is on a hidden sheet. Done, for dependents too | A generic "no formula" hides why nothing shows | S |111-| Settings > JEV API key checks the key with a test call. Done: one fixed question, "Key saved and checked" or why the check failed ([jev.md](docs/jev.md#setup)) | A mistyped key otherwise shows only as `#ERROR!` in cells | S |112-| `012 serve`: open a file from the ssh command line, and save unsaved work to a recovery file on shutdown or idle timeout. Done: `ssh -t host file.012` (a one-word exec request taken as a file name inside the served directory, never run) and `.012-recovery/`, offered back on the next open ([ssh.md](docs/ssh.md#unsaved-work)) | Sessions reach a file in one step, and stopping the server or idling out keeps work | M |113-| Vim `p`/`P` after `yy` carries the rows' line formats, as a whole-row paste does. Done: `yy` and `dd` copy whole rows ([data.md](docs/data.md#copy-paste-and-fill)) | Only cell formats go with it | S |114-| Cutting a block clears the source cells to plain, as Sheets does. Done, over formatted columns, rows and the sheet ([data.md](docs/data.md#copy-paste-and-fill)) | They keep showing their column or row format | S |115-| Replace current sheet removes charts whose range no longer fits the new data, or asks. Done: removed, listed first in the note, and back on undo ([files.md](docs/files.md)) | They stay, pointing at empty cells, with a note | S |21+| Open 012's XLSX output in Excel and LibreOffice before releases, as was done for Google Sheets | The writer is otherwise checked by excelize and 012's reader | S |22+| Conditional formatting and validation, what's left: data bars and icon sets, "top 10", "duplicate values", date periods (this week, last month), validation on pastes and fills, rules moving with cut and paste, a dropdown's chip look, and custom checkbox values | Excel's other rule types come in as notes | M |23+| Remaining overlays get narrow hosts: filter picker, sort and choice bars, chart editor, shortcuts, named ranges, cell entry and prompts | Components testable without the model ([architecture.md](docs/architecture.md#the-ui)) | M |11624 11725 ### 2. Spreadsheet features Sheets users reach for11826 11927 | Item | Notes | Size |12028 |---|---|---|12129 | Dynamic arrays: FILTER, SORT, UNIQUE, SEQUENCE, spill ranges, `#SPILL!`; then LET and LAMBDA | The biggest gap in the function library; spills need engine support for ranges a formula owns | L |122-| Text and regex: TEXTJOIN, SPLIT, REGEXMATCH, REGEXEXTRACT, REGEXREPLACE | Sheets staples | S |123-| Conditional formatting (color scales, rules on values and formulas), drawn in theme roles. Done: single-color rules on Sheets' conditions and custom formulas, 2 and 3 point color scales, in named colors on the ANSI slots with readable ink checked under every scheme, a rules panel, and XLSX both ways; see [data.md](docs/data.md#conditional-formatting) | Visual; must read under all 349 schemes | M |124-| Data validation: dropdown lists (a picker in the cell), number and date rules, checkboxes. Done: dropdowns from items or a range (Alt+Down or a click), checkboxes (Space or a click), number, date and text length bounds, custom formulas, reject or warn with Sheets' help text, and XLSX both ways; see [data.md](docs/data.md#data-validation) | Pairs with the filter picker | M |125-| Conditional formatting and validation, what's left: data bars and icon sets, "top 10", "duplicate values", date periods (this week, last month), validation on pastes and fills, rules moving with cut and paste, a dropdown's chip look, and custom checkbox values | Excel's other rule types come in as notes today | M |30+| Text and regex: SPLIT, REGEXMATCH, REGEXEXTRACT, REGEXREPLACE | Sheets staples | S |12631 | Wrap text, row heights, borders, merged cells | Layout changes in the grid renderer | M to L |127-| Notes on cells (shown on hover and in the context line). Done: Insert > Note (Shift+F2), a corner mark, the context line and a hover box, saved and in XLSX as Excel's notes; see [data.md](docs/data.md#notes) | | S |12832 | Locale: decimal comma, date order, list separator in formulas | Sheets' File > Settings > Locale | M |129-| Charts: scatter, area, stacked columns and bars, axis options, a legend position. Done: area and scatter (with trend lines) types, stacked and 100% stacked columns, bars and areas, value axis minimum, maximum and log scale, gridlines on and off, the legend at the bottom, right or none; see [charts.md](docs/charts.md) | Registry entries in `internal/chart` | M |130-| Pivot tables: column subtotals, renaming value columns, check the "(blank)" label against Sheets. Done: a subtotal column per outer column group, R renames a value in the editor, and blank groups have a blank label as in Sheets ([data.md](docs/data.md#pivot-tables)) | Left open by pivots | S |131-| Protected ranges and sheets (warn on edit). Done: Data > Protect sheets and ranges, edits ask on the context line; an XLSX sheet's protection is noted on import; see [data.md](docs/data.md#protected-sheets-and-ranges) | | S |13233 133-### 3. Scale (see "Next: vast data" above and docs/limits.md)34+### 3. Scale35+36+The grid is Excel's, 1,048,576 x 16,384, and memory is bounded by the37+`max-cells` budget rather than the grid. The budget rises in measured38+steps; see [docs/limits.md](docs/limits.md#what-would-raise-the-bounds).13439 13540 | Item | Result | Size |13641 |---|---|---|137-| Step B: compact column storage behind `cellStore` | 10 M+ cells in a few hundred MB, cell reads 2 to 3 ns | L |138-| Step C: streaming and compact `.012` format | Open and save scale with the data | M |42+| Compact column storage behind `cellStore`: typed value blocks, formulas and formats in side tables (about 20 to 40 B per cell instead of 300) | A budget of 10 M+ cells in a few hundred MB, cell reads 2 to 3 ns | L |43+| A streaming or binary `.012` format next to the readable JSON one | Open and save scale with the data | M |13944 | Smaller undo steps: formatting changes as diffs | More history in the same memory | S |140-| Linked, paged read-only ranges over Parquet and SQLite feeding pivots and formulas | Sources too big for any grid | L |45+| Linked, paged read-only ranges over Parquet and SQLite that feed pivots and formulas by streaming | Sources too big for any grid | L |14146 142-### 4. Macros and keys47+### 4. Macros, keys and the terminal14348 14449 | Item | Size |14550 |---|---|14651 | Record dialog choices (sort bar, filter picker, find and replace, chart editor) and chart drags; let scripts run commands that open dialogs, with answers | M |147-| Vim: `.` repeat, registers, marks, `cc`/`s`, command-line history, `:w!`, `:wq file` quitting | M |148-| Remaining overlays get narrow hosts: filter picker, sort and choice bars, chart editor, shortcuts, named ranges, cell entry and prompts | M |52+| Vim: `.` repeat, registers, marks, `cc`/`s`, command-line history, `:w!` | M |53+| Hold-to-preview and Shift+Enter on terminals with the kitty keyboard protocol (`View.KeyboardEnhancements`) | S |54+| Sixel chart images, redrawn on resize, for terminals without kitty graphics | M |14955 15056 ### 5. Distribution and upkeep15157 15258 | Item | Size |15359 |---|---|154-| Version tags and release notes; prebuilt binaries attached to tags (built locally, since the repo is hosting only). Tags done from v0.1.0, with notes in the tag message | S |155-| `make check`: vet, lint, tests, oracle, e2e in one target, run before every push. Done | S |60+| Prebuilt binaries attached to version tags, built locally (the GitHub repo is hosting only); tags carry release notes in their message | S |15661 | `make stress-report` thresholds that flag regressions over a set percentage against the last release | S |15762 | Grafana: a trace panel for the nested spans | S |158-| Accessibility: a high-contrast theme, and every state readable without color (already a UX rule; audit it) | S |63+| Accessibility: a high-contrast theme, and an audit that every state reads without color (a [UX rule](docs/UX.md#visual-rules)) | S |15964 16065 ## Later: sharing a live sheet (shelved)16166 16267 Explored, not scheduled. `012 serve` gives each SSH session its own16368 spreadsheet; saving over a file another session saved asks first, and164-that is all the sessions know of each other. One session host that runs a Bubble Tea program165-for any byte stream with window-size events, fed by SSH166-(`charm.land/wish/v2`), iroh tickets (the Go transport in69+that is all the sessions know of each other. One session host that runs70+a Bubble Tea program for any byte stream with window-size events, fed by71+SSH (`charm.land/wish/v2`), iroh tickets (the Go transport in16772 `FelineStateMachine/allons` `local/transport/iroh`, which needs cgo and a168-prebuilt iroh-ffi archive, so it would sit behind a build tag), and the web169-(`NimbleMarkets/go-booba` serves Bubble Tea over WebSocket/WebTransport with170-ghostty-web; Bubble Tea v2 support unverified). Open questions: per-user171-sessions on one sheet versus mirroring one session, and whether the web page172-is served by the host or by a gateway dialing the iroh ticket.73+prebuilt iroh-ffi archive, so it would sit behind a build tag), and the74+web (`NimbleMarkets/go-booba` serves Bubble Tea over75+WebSocket/WebTransport with ghostty-web; Bubble Tea v2 support76+unverified). Open questions: per-user sessions on one sheet versus77+mirroring one session, and whether the web page is served by the host or78+by a gateway dialing the iroh ticket.79+80+## Shipped81+82+**Engine**83+84+- Undo and redo for every change, across sheets: [data.md](docs/data.md#undo)85+- Copy, cut, paste and fill with relative and `$absolute` references; insert and delete rows and columns: [data.md](docs/data.md#copy-paste-and-fill)86+- Entries detected as typed (currency, percent, dates, times) and number formats: [formulas.md](docs/formulas.md#what-you-type)87+- Named ranges; several sheets with references between them, hidden sheets: [formulas.md](docs/formulas.md#references)88+- Sheets' everyday, math, text, lookup, date and finance functions, checked against excelize: [functions.md](docs/functions.md), [testing.md](docs/testing.md#the-excelize-oracle)89+- Opt-in decimal arithmetic for money: [formulas.md](docs/formulas.md#decimal-arithmetic)90+- An Excel-sized grid with a `max-cells` budget; operations cost the data, not the grid: [limits.md](docs/limits.md#sheet-size)91+92+**Finding and using features**93+94+- Sheets-structured menus, a command palette and a key for every command: [keys.md](docs/keys.md)95+- Formula suggestions, argument hints and pointing at cells and sheets: [formulas.md](docs/formulas.md#building-formulas)96+- Tracing precedents and dependents, hidden sheets explained: [formulas.md](docs/formulas.md#building-formulas)97+- An optional vim keymap with a `:` command line: [keys.md](docs/keys.md#vim-keys)98+99+**Data tools**100+101+- Freeze, multi-column sort, filters with value pickers and conditions: [data.md](docs/data.md#freeze)102+- Find and replace, with regular expressions, across sheets: [data.md](docs/data.md#find-and-replace)103+- Conditional formatting (single-color rules, color scales) and data validation (dropdowns, checkboxes, bounds): [data.md](docs/data.md#conditional-formatting)104+- Pivot tables and frequency tables, live, with subtotals and renamed values: [data.md](docs/data.md#pivot-tables)105+- Notes on cells: [data.md](docs/data.md#notes)106+- Protected sheets and ranges that warn on edit: [data.md](docs/data.md#protected-sheets-and-ranges)107+- Charts (column, bar, line, area, pie, scatter; stacking, trend lines, axis and legend options), as images or text: [charts.md](docs/charts.md)108+- Macros, recorded or written in Starlark: [macros.md](docs/macros.md)109+- JEV functions, with the API key in the OS keychain: [jev.md](docs/jev.md)110+111+**The terminal**112+113+- System clipboard, hyperlinks, curly error underlines, cursor shapes, progress and notifications, light and dark: [charts.md](docs/charts.md#terminal-features-012-uses)114+- Color schemes and a config file: [themes.md](docs/themes.md), [config.md](docs/config.md)115+- `012 serve` over SSH, with files on the ssh command line and recovery of unsaved work: [ssh.md](docs/ssh.md)116+117+**Files**118+119+- Import CSV, TSV, XLSX, SQLite, Parquet and Lotus `.wk1`; export CSV, TSV, XLSX and SQLite; import locations; save-as and overwrite checks: [files.md](docs/files.md)120+121+**Upkeep**122+123+- `make check` before every push; demo tapes (`make demos`); version tags from v0.1.0: [testing.md](docs/testing.md)