A linked region is an anchor whose rows come from outside the workbook: LiveOps (the change stream) write its cells as derived cells, outside the undo history, with a window keeping the last rows and max-cells bounding the rest; formulas, pivots and spills over it recalculate. Regions are saved as their source (links, linkOrigin), follow inserted lines, undo with their definitions and are read again after. fileio gains Tail, reading CSV, TSV, JSON and NUON as they arrive in pieces, with partial records waiting, and Rows for sources read whole.
- commit
- ce682befb2d3a2f43ae732bb6073f40a89738b09
- parent
- 211999286af526b35f28bcf4f6a3c08c2cbfa826
- author
- Dami <dami.etoile@gmail.com>
- kind
- feat risk score 2.4 of 3, Jev's estimate
| M | docs/ | 26 | ++- |
| M | internal/ | 54 | +++- |
| M | internal/ | 46 | +++- |
| A | internal/ | 292 | ++++++++++++++++++++++++ |
| A | internal/ | 180 | +++++++++++++++ |
| M | internal/ | 17 | +- |
| M | internal/ | 1 | + |
| M | internal/ | 5 | +- |
| M | internal/ | 13 | +- |
| M | internal/ | 5 | + |
| A | internal/ | 368 | ++++++++++++++++++++++++++++++ |
| A | internal/ | 77 | ++++++ |
| A | internal/ | 341 | ++++++++++++++++++++++++++++ |
| A | internal/ | 275 | ++++++++++++++++++++++ |
| M | internal/ | 1 | + |
| M | internal/ | 16 | + |
| M | internal/ | 7 | +- |
| M | internal/ | 4 | + |
@@ -37,7 +37,8 @@ a line in a [color](../sheets/formatting.md#border-colors):
3737 Only what's typed is saved. What 012 computes is computed again when the3838 file opens: formula results, [pivot tables](#pivot-tables)' results, and3939 the arrays formulas [spill](../formulas/arrays.md), whose cells are kept40-only for their formatting and notes.40+only for their formatting and notes, and the rows of41+[linked files](following.md), read from their files again.4142 4243 ## Reading and writing4344 @@ -84,8 +85,10 @@ save), so they raise no version:
8485 | `charts` | a sheet | One chart per line: `type` (`column`, `bar`, `line`, `pie`, `area`, `scatter`), `data`, `at`, `width`, `height`, `byRow`, `header`, `labels`, `title`, and options left out at their defaults: `stack` (`stacked`, `percent`), `trend`, `min`, `max`, `log`, `gridlines` (only when off), `legend` (`right`, `none`). A build that charts but lacks a chart's type refuses the file |8586 | `protected` | a sheet | [Protected ranges](../sheets/notes-protection.md#protected-sheets-and-ranges): `{"range":"B2:C9","description":"Totals"}`, or `{"sheet":true}` |8687 | `conditionalFormats`, `validations` | a sheet | [Rules](#conditional-formats-and-data-validation), one per line |88+| `links` | a sheet | [Linked files](#linked-files), one per line; older builds show their cells empty |8789 | `arithmetic` | the workbook | `decimal` for [decimal arithmetic](../formulas/decimal.md) |8890 | `locale` | the workbook | The [locale](../sheets/locale.md) it's typed and shown in (`"de-DE"`), when File > Settings > Locale chose one; without it, the file follows the reader's `locale` setting. Cells are stored the same way in every locale: `input` is always as typed in en-US (`1,234.5`, `9/26/2026`, `=ROUND(A1,2)`) |91+| `linkOrigin` | the workbook | The computer its linked files were linked or trusted on: see [Following files](following.md#files-from-elsewhere) |8992 | `macros`, `macroOrigin` | the workbook | Macros as Starlark scripts, and the computer they were made or trusted on: see [Macro scripting API](../reference/macro-api.md#in-the-file). Opening a file never runs them |9093 9194 ## Column and row formats@@ -157,6 +160,27 @@ with its data.
157160 `condition` with its `value`.158161 - `rowTotals` and `columnTotals` are the grand total row and column.159162 163+## Linked files164+165+A sheet's [linked files](following.md) are a list after its cells,166+charts and rules, one per line, holding where each region starts and167+what it reads, never its rows:168+169+```json170+ "links": [171+ {"at":"A1","path":"logs/app.csv","window":500},172+ {"at":"F1","path":"../shop.db","format":"SQLite","query":"SELECT * FROM orders"}173+ ]174+```175+176+- `at` is the region's top-left cell.177+- `path` is the file, relative to the workbook's folder when it's saved178+ there or below, and otherwise as it was linked.179+- `format` names the file's format (`CSV`, `TSV`, `JSON`, `NUON`,180+ `XLSX`, `SQLite`, `Parquet`) when it isn't told by the extension.181+- `table` or `query` say what to read from a SQLite database.182+- `window` keeps the last rows under the header, when set.183+160184 ## Conditional formats and data validation161185 162186 A sheet's rules are two lists after its cells and charts, one rule per@@ -103,25 +103,38 @@ func fieldCounts(sample []byte, d byte, truncated bool) []int {
103103 return counts104104 }105105 106-// decode returns a reader of f's text as UTF-8 and the encoding found: a107-// UTF-8 or UTF-16 byte order mark decides, and text that isn't valid108-// UTF-8 is read as Windows-1252, what Excel writes on Windows.106+// decode returns a reader of f's text as UTF-8 and the encoding found107+// (see encodingOf).109108 func decode(r *bufio.Reader) (io.Reader, string, error) {110109 head, err := r.Peek(sniffSize)111110 if err != nil && !errors.Is(err, io.EOF) && !errors.Is(err, bufio.ErrBufferFull) {112111 return nil, "", err113112 }113+ enc, bom := encodingOf(head)114+ r.Discard(bom)115+ return decodeAs(r, enc), enc, nil116+}117+118+// Encodings of text files, as notes name them.119+const (120+ encUTF8 = "UTF-8"121+ encUTF16LE = "UTF-16"122+ encUTF16BE = "UTF-16 big-endian"123+ encWin1252 = "Windows-1252"124+)125+126+// encodingOf tells the encoding of text starting with head, and the127+// length of its byte order mark: a UTF-8 or UTF-16 mark decides, and128+// text that isn't valid UTF-8 is read as Windows-1252, what Excel129+// writes on Windows.130+func encodingOf(head []byte) (enc string, bom int) {114131 switch {115132 case bytes.HasPrefix(head, []byte{0xEF, 0xBB, 0xBF}):116- r.Discard(3)117- return r, "UTF-8", nil118- case bytes.HasPrefix(head, []byte{0xFF, 0xFE}), bytes.HasPrefix(head, []byte{0xFE, 0xFF}):119- order := unicode.LittleEndian120- if head[0] == 0xFE {121- order = unicode.BigEndian122- }123- r.Discard(2)124- return unicode.UTF16(order, unicode.IgnoreBOM).NewDecoder().Reader(r), "UTF-16", nil133+ return encUTF8, 3134+ case bytes.HasPrefix(head, []byte{0xFF, 0xFE}):135+ return encUTF16LE, 2136+ case bytes.HasPrefix(head, []byte{0xFE, 0xFF}):137+ return encUTF16BE, 2125138 }126139 // A sample cut mid-character is still UTF-8.127140 check := head@@ -129,9 +142,22 @@ func decode(r *bufio.Reader) (io.Reader, string, error) {
129142 check = check[:len(check)-1]130143 }131144 if len(head) > 0 && !utf8.Valid(check) {132- return charmap.Windows1252.NewDecoder().Reader(r), "Windows-1252", nil145+ return encWin1252, 0146+ }147+ return encUTF8, 0148+}149+150+// decodeAs reads r, text in enc past its byte order mark, as UTF-8.151+func decodeAs(r io.Reader, enc string) io.Reader {152+ switch enc {153+ case encUTF16LE:154+ return unicode.UTF16(unicode.LittleEndian, unicode.IgnoreBOM).NewDecoder().Reader(r)155+ case encUTF16BE:156+ return unicode.UTF16(unicode.BigEndian, unicode.IgnoreBOM).NewDecoder().Reader(r)157+ case encWin1252:158+ return charmap.Windows1252.NewDecoder().Reader(r)133159 }134- return r, "UTF-8", nil160+ return r135161 }136162 137163 // countingReader counts the bytes read through it.@@ -11,6 +11,7 @@ import (
1111 "github.com/FelineStateMachine/012/internal/numfmt"1212 "github.com/FelineStateMachine/012/internal/nuon"1313 "github.com/FelineStateMachine/012/internal/sheet"14+ "github.com/FelineStateMachine/012/internal/value"1415 )1516 1617 // NUON and JSON tables become cells with their types (see@@ -137,35 +138,50 @@ const nsPerDay = 86400e9
137138 // value stores one value as a cell.138139 func (t *tableCells) value(a sheet.Addr, v nuon.Value) {139140 b, none := t.b, sheet.Style{}141+ c := nuonCell(v, t.zone)142+ switch c.V.Kind {143+ case sheet.Empty:144+ case sheet.Bool:145+ b.boolean(a, c.V.Num != 0, none)146+ case sheet.Number:147+ b.number(a, c.V.Num, c.F, none)148+ default:149+ b.text(a, c.V.Str, c.F, none)150+ }151+ if v.Kind == nuon.Date && !t.dated[a.Col] && a.Valid() {152+ t.dated[a.Col] = true153+ b.s.LoadColWidth(a.Col, dateTimeWidth)154+ }155+}156+157+// nuonCell is what a cell shows for a NUON value: its value, text as158+// the value's own, and its format.159+func nuonCell(v nuon.Value, zone *time.Location) sheet.LiveCell {160+ num := func(n float64, f sheet.Format) sheet.LiveCell { return sheet.LiveCell{V: value.Num(n), F: f} }140161 switch v.Kind {141162 case nuon.Null:163+ return sheet.LiveCell{}142164 case nuon.Bool:143- b.boolean(a, v.Bool, none)165+ return sheet.LiveCell{V: value.Boolean(v.Bool)}144166 case nuon.Int:145- b.number(a, float64(v.Int), sheet.Format{}, none)167+ return num(float64(v.Int), sheet.Format{})146168 case nuon.Float:147169 if math.IsNaN(v.Float) || math.IsInf(v.Float, 0) {148- b.text(a, v.String(), sheet.Format{}, none)149- return170+ return sheet.LiveCell{V: value.Str(v.String())}150171 }151- b.number(a, v.Float, sheet.Format{}, none)172+ return num(v.Float, sheet.Format{})152173 case nuon.String:153- b.text(a, v.Str, sheet.Format{}, none)174+ return sheet.LiveCell{V: value.Str(v.Str)}154175 case nuon.Filesize:155- b.number(a, float64(v.Int), sheet.Preset(sheet.FmtSize), none)176+ return num(float64(v.Int), sheet.Preset(sheet.FmtSize))156177 case nuon.Duration:157178 f := durationFormat158179 if v.Int%1e9 != 0 {159180 f = durationFormatFrac160181 }161- b.number(a, float64(v.Int)/nsPerDay, f, none)182+ return num(float64(v.Int)/nsPerDay, f)162183 case nuon.Date:163- b.number(a, numfmt.SerialOf(v.Time.In(t.zone)), sheet.Preset(sheet.FmtDateTime), none)164- if !t.dated[a.Col] && a.Valid() {165- t.dated[a.Col] = true166- b.s.LoadColWidth(a.Col, dateTimeWidth)167- }168- default: // binary, lists and records169- b.text(a, v.String(), sheet.Format{}, none)184+ return num(numfmt.SerialOf(v.Time.In(zone)), sheet.Preset(sheet.FmtDateTime))170185 }186+ return sheet.LiveCell{V: value.Str(v.String())} // binary, lists and records171187 }@@ -0,0 +1,292 @@
1+package fileio2+3+import (4+ "encoding/csv"5+ "errors"6+ "fmt"7+ "io"8+ "runtime"9+ "strings"10+ "sync"11+12+ "github.com/FelineStateMachine/012/internal/locale"13+ "github.com/FelineStateMachine/012/internal/nuon"14+ "github.com/FelineStateMachine/012/internal/sheet"15+)16+17+// Tails: a text table read as it arrives in pieces, as a file followed18+// grows. The text is parsed by the importers' own readers (encoding/csv19+// and nuon.Reader) on a goroutine of the tail's, reading from a pipe20+// that waits for the next piece: a record or value cut off at the end of21+// one piece waits there for the rest, so a partial last line is never22+// read as a row. Rows come out typed as the importer types them: CSV and23+// TSV fields as if typed in the locale, NUON and JSON values with their24+// types (nuonCell).25+26+// Grows reports whether a file of kind k can be followed as it grows,27+// its new rows read without reading it again: text tables, which28+// Tail reads.29+func (k Kind) Grows() bool { return k == CSV || k == TSV || k == JSON || k == NUON }30+31+// TailRows are the rows a piece of text completed.32+type TailRows struct {33+ // Header is the table's first row when it's new or changed: a CSV34+ // file's first record, or NUON's column names as they're seen.35+ Header sheet.LiveRow36+ Rows []sheet.LiveRow37+}38+39+// A Tail reads a table of kind k from text that arrives in pieces.40+type Tail struct {41+ p *tailPipe42+ out *tailOut43+}44+45+// tailPipe hands pieces of text to the tail's goroutine and tells when46+// it has read them all: it has then parsed every row they complete.47+type tailPipe struct {48+ in chan []byte49+ starved chan struct{} // the goroutine wants more50+ done chan struct{} // the goroutine ended51+ buf []byte52+ eof bool53+ once sync.Once54+}55+56+// tailOut is what the goroutine parsed since the last piece; the Tail57+// reads it only while the goroutine waits.58+type tailOut struct {59+ header sheet.LiveRow60+ changed bool // header changed61+ rows []sheet.LiveRow62+ err error63+}64+65+// Read gives the goroutine's reader the rest of the piece, or waits for66+// the next one.67+func (p *tailPipe) Read(b []byte) (int, error) {68+ for len(p.buf) == 0 {69+ if p.eof {70+ return 0, io.EOF71+ }72+ p.starved <- struct{}{}73+ chunk, ok := <-p.in74+ if !ok {75+ p.eof = true76+ return 0, io.EOF77+ }78+ p.buf = chunk79+ }80+ n := copy(b, p.buf)81+ p.buf = p.buf[n:]82+ return n, nil83+}84+85+func (p *tailPipe) close() { p.once.Do(func() { close(p.in) }) }86+87+// NewTail starts reading a table of kind k (see Kind.Grows). head is the88+// start of the text, what it's sniffed by (its encoding, and a CSV89+// file's delimiter and decimal separator), in the locale loc; the text90+// itself comes through Feed, head included.91+func NewTail(k Kind, head []byte, loc *locale.Locale) (*Tail, error) {92+ if !k.Grows() {93+ return nil, fmt.Errorf("can't follow %s files as they grow", k)94+ }95+ p := &tailPipe{in: make(chan []byte), starved: make(chan struct{}), done: make(chan struct{})}96+ out := &tailOut{}97+ var parse func()98+ switch k {99+ case CSV, TSV:100+ parse = delimitedTail(p, out, k, head, loc)101+ default:102+ parse = tableTail(p, out)103+ }104+ go func() {105+ defer close(p.done)106+ parse()107+ }()108+ t := &Tail{p: p, out: out}109+ // A tail dropped without Close ends its goroutine too.110+ runtime.AddCleanup(t, func(p *tailPipe) { p.close() }, p)111+ select {112+ case <-p.starved:113+ case <-p.done:114+ }115+ return t, nil116+}117+118+// errTailClosed is Feed's error once the tail is closed or failed.119+var errTailClosed = errors.New("the tail is closed")120+121+// Feed reads the next piece of text and returns the rows it completed.122+// After an error the tail reads nothing more.123+func (t *Tail) Feed(piece []byte) (TailRows, error) {124+ select {125+ case <-t.p.done:126+ return TailRows{}, t.failure()127+ default:128+ }129+ if len(piece) > 0 {130+ t.p.in <- piece131+ select {132+ case <-t.p.starved:133+ case <-t.p.done:134+ }135+ }136+ out := t.out137+ res := TailRows{Rows: out.rows}138+ if out.changed {139+ res.Header = out.header140+ }141+ out.rows, out.changed = nil, false142+ if out.err != nil {143+ return res, out.err144+ }145+ return res, nil146+}147+148+// failure is why the goroutine ended.149+func (t *Tail) failure() error {150+ if t.out.err != nil {151+ return t.out.err152+ }153+ return errTailClosed154+}155+156+// Close ends the tail's goroutine.157+func (t *Tail) Close() { t.p.close() }158+159+// delimitedTail parses CSV or TSV: fields as if typed, the delimiter and160+// the numbers' decimal separator sniffed from head as an import does.161+func delimitedTail(p *tailPipe, out *tailOut, k Kind, head []byte, loc *locale.Locale) func() {162+ enc, bom := encodingOf(head)163+ sample := head[bom:]164+ if enc != encUTF8 {165+ b, _ := io.ReadAll(decodeAs(strings.NewReader(string(sample)), enc))166+ sample = b167+ }168+ comma := '\t'169+ if k == CSV {170+ comma = sniff(sample, len(head) >= sniffSize, csvComma(loc))171+ }172+ loc, _ = numberLocale(sample, comma, loc)173+ return func() {174+ skip := &skipReader{r: p, n: bom}175+ cr := csv.NewReader(decodeAs(skip, enc))176+ cr.Comma, cr.LazyQuotes, cr.FieldsPerRecord = comma, true, -1177+ first := true178+ for {179+ rec, err := cr.Read()180+ if err != nil {181+ if !errors.Is(err, io.EOF) {182+ out.err = err183+ }184+ return185+ }186+ row := make(sheet.LiveRow, len(rec))187+ for i, field := range rec {188+ if field != "" {189+ v, f := sheet.EntryValue(entryInput(field, loc))190+ row[i] = sheet.LiveCell{V: v, F: f}191+ }192+ }193+ if first {194+ first, out.header, out.changed = false, row, true195+ continue196+ }197+ out.rows = append(out.rows, row)198+ }199+ }200+}201+202+// skipReader leaves out the first n bytes of r: a byte order mark.203+type skipReader struct {204+ r io.Reader205+ n int206+}207+208+func (s *skipReader) Read(b []byte) (int, error) {209+ for s.n > 0 {210+ var one [4]byte211+ m, err := s.r.Read(one[:min(s.n, len(one))])212+ s.n -= m213+ if err != nil {214+ return 0, err215+ }216+ }217+ return s.r.Read(b)218+}219+220+// tableTail parses NUON or JSON: the columns named as rows name them,221+// the header growing as new names appear.222+func tableTail(p *tailPipe, out *tailOut) func() {223+ return func() {224+ r := nuon.NewReader(p)225+ cols := map[string]int{}226+ zone := zone()227+ col := func(name string) int {228+ i, ok := cols[name]229+ if !ok {230+ i = len(cols)231+ cols[name] = i232+ out.header = append(out.header, textCell(name))233+ out.changed = true234+ }235+ return i236+ }237+ for {238+ fields, err := r.Next()239+ if err != nil {240+ if !errors.Is(err, io.EOF) {241+ out.err = err242+ }243+ return244+ }245+ for _, c := range r.Header() {246+ col(c)247+ }248+ row := make(sheet.LiveRow, len(cols))249+ for _, f := range fields {250+ i := col(f.Key)251+ if i >= len(row) {252+ row = append(row, make(sheet.LiveRow, i+1-len(row))...)253+ }254+ row[i] = nuonCell(f.Value, zone)255+ }256+ out.rows = append(out.rows, row)257+ }258+ }259+}260+261+// Rows reads a sheet an import made as a linked region's rows: its first262+// row as the header and the rest under it, from column A, values with263+// the formats they show in. A file that is read again whole when it264+// changes (XLSX, SQLite, Parquet) is read this way.265+func Rows(s *sheet.Sheet) TailRows {266+ used, ok := s.UsedRange()267+ if !ok {268+ return TailRows{}269+ }270+ var res TailRows271+ for r := 0; r <= used.To.Row; r++ {272+ row := make(sheet.LiveRow, used.To.Col+1)273+ for c := range row {274+ a := sheet.Addr{Col: c, Row: r}275+ if s.Filled(a) {276+ row[c] = sheet.LiveCell{V: s.Value(a), F: s.DisplayFormat(a)}277+ }278+ }279+ if r == 0 {280+ res.Header = row281+ continue282+ }283+ res.Rows = append(res.Rows, row)284+ }285+ return res286+}287+288+// textCell is a cell showing s as text, as the builder stores it.289+func textCell(s string) sheet.LiveCell {290+ v, _ := sheet.EntryValue("'" + flatten(s))291+ return sheet.LiveCell{V: v}292+}@@ -0,0 +1,180 @@
1+package fileio2+3+import (4+ "context"5+ "os"6+ "path/filepath"7+ "strings"8+ "testing"9+ "time"10+11+ "github.com/FelineStateMachine/012/internal/locale"12+ "github.com/FelineStateMachine/012/internal/sheet"13+)14+15+// liveText writes rows as a test reads them: cells' values by commas, rows16+// by bars.17+func liveText(rows ...sheet.LiveRow) string {18+ var out []string19+ for _, r := range rows {20+ var cells []string21+ for _, c := range r {22+ cells = append(cells, c.V.String())23+ }24+ out = append(out, strings.Join(cells, ","))25+ }26+ return strings.Join(out, "|")27+}28+29+// feed feeds pieces to a tail, returning what each completed.30+func feed(t *testing.T, tl *Tail, pieces ...string) []TailRows {31+ t.Helper()32+ var out []TailRows33+ for _, p := range pieces {34+ got, err := tl.Feed([]byte(p))35+ if err != nil {36+ t.Fatalf("feeding %q: %v", p, err)37+ }38+ out = append(out, got)39+ }40+ return out41+}42+43+func TestTailCSV(t *testing.T) {44+ head := "when,level,msg\n9/29/2026,3,\"two\nlines\"\n"45+ tl, err := NewTail(CSV, []byte(head), nil)46+ if err != nil {47+ t.Fatal(err)48+ }49+ defer tl.Close()50+ got := feed(t, tl, "when,level,m", "sg\n9/29/2026,3,\"two\n", "lines\"\n10/1/2026,4,partial", "\n")51+ if liveText(got[0].Header) != "" || len(got[0].Rows) != 0 {52+ t.Fatalf("a partial header was read: %+v", got[0])53+ }54+ if liveText(got[1].Header) != "when,level,msg" || len(got[1].Rows) != 0 {55+ t.Fatalf("header: %+v", got[1])56+ }57+ if len(got[2].Rows) != 1 || got[2].Rows[0][2].V.Str != "two lines" && got[2].Rows[0][2].V.Str != "two\nlines" {58+ t.Fatalf("a quoted field across pieces: %+v", got[2].Rows)59+ }60+ if got[2].Rows[0][0].F.IsZero() || got[2].Rows[0][1].V.Num != 3 {61+ t.Fatalf("fields weren't typed: %+v", got[2].Rows[0])62+ }63+ if len(got[3].Rows) != 1 || got[3].Rows[0][2].V.Str != "partial" {64+ t.Fatalf("the last line: %+v", got[3])65+ }66+}67+68+func TestTailEncodingsAndDelimiters(t *testing.T) {69+ // A UTF-16 file split mid-character, with a byte order mark.70+ text := "a\tb\n1\tç\n"71+ var utf16 []byte72+ utf16 = append(utf16, 0xFF, 0xFE)73+ for _, r := range text {74+ utf16 = append(utf16, byte(r), byte(r>>8))75+ }76+ tl, err := NewTail(TSV, utf16, nil)77+ if err != nil {78+ t.Fatal(err)79+ }80+ defer tl.Close()81+ var rows []sheet.LiveRow82+ var header sheet.LiveRow83+ for i := 0; i < len(utf16); i += 3 {84+ got, err := tl.Feed(utf16[i:min(i+3, len(utf16))])85+ if err != nil {86+ t.Fatal(err)87+ }88+ if got.Header != nil {89+ header = got.Header90+ }91+ rows = append(rows, got.Rows...)92+ }93+ if liveText(header) != "a,b" || liveText(rows...) != "1,ç" {94+ t.Fatalf("UTF-16: %q %q", liveText(header), liveText(rows...))95+ }96+ // Semicolons sniffed, in a locale with decimal commas.97+ head := "x;y\n1,5;2\n"98+ tl, _ = NewTail(CSV, []byte(head), deLocale(t))99+ defer tl.Close()100+ got := feed(t, tl, head)101+ if len(got[0].Rows) != 1 || got[0].Rows[0][0].V.Num != 1.5 {102+ t.Fatalf("semicolons: %+v", got[0])103+ }104+}105+106+func TestTailNDJSONAndNUON(t *testing.T) {107+ tl, err := NewTail(JSON, nil, nil)108+ if err != nil {109+ t.Fatal(err)110+ }111+ defer tl.Close()112+ got := feed(t, tl, `{"a": 1, "b": "x"}`+"\n"+`{"a": 2`, `, "c": true}`+"\n")113+ if liveText(got[0].Header) != "a,b" || liveText(got[0].Rows...) != "1,x" {114+ t.Fatalf("first piece: %q %q", liveText(got[0].Header), liveText(got[0].Rows...))115+ }116+ if liveText(got[1].Header) != "a,b,c" || liveText(got[1].Rows...) != "2,,TRUE" {117+ t.Fatalf("a new column: %q %q", liveText(got[1].Header), liveText(got[1].Rows...))118+ }119+ // NUON's table form, streaming, with types.120+ tl, _ = NewTail(NUON, nil, nil)121+ defer tl.Close()122+ got = feed(t, tl, "[[size, took]; [1kb, 2sec],\n", "[3b, 1min]]\n")123+ if liveText(got[0].Header) != "size,took" || len(got[0].Rows) != 1 || got[0].Rows[0][0].V.Num != 1000 || got[0].Rows[0][0].F.IsZero() {124+ t.Fatalf("table form: %+v", got[0])125+ }126+ if len(got[1].Rows) != 1 || got[1].Header != nil {127+ t.Fatalf("second piece: %+v", got[1])128+ }129+ // A syntax error ends the tail.130+ tl, _ = NewTail(NUON, nil, nil)131+ defer tl.Close()132+ if _, err := tl.Feed([]byte("{a: }}\n")); err == nil {133+ t.Fatal("no error for bad NUON")134+ }135+ if _, err := tl.Feed([]byte("{a: 1}\n")); err == nil {136+ t.Fatal("a failed tail read on")137+ }138+}139+140+func TestTailClose(t *testing.T) {141+ tl, _ := NewTail(CSV, nil, nil)142+ feed(t, tl, "a,b\n1,")143+ tl.Close()144+ select {145+ case <-tl.p.done:146+ case <-time.After(5 * time.Second):147+ t.Fatal("the tail's goroutine didn't end")148+ }149+ if _, err := tl.Feed([]byte("2\n")); err == nil {150+ t.Fatal("a closed tail read on")151+ }152+ if _, err := NewTail(XLSX, nil, nil); err == nil {153+ t.Fatal("a tail of XLSX")154+ }155+}156+157+func deLocale(t *testing.T) *locale.Locale {158+ t.Helper()159+ de, ok := locale.Lookup("de-DE")160+ if !ok {161+ t.Fatal("no de-DE")162+ }163+ return de164+}165+166+func TestRowsOfImport(t *testing.T) {167+ name := filepath.Join(t.TempDir(), "t.csv")168+ os.WriteFile(name, []byte("a,b\n1,$2\n,x\n"), 0o644)169+ res, err := Import(context.Background(), name, Options{})170+ if err != nil {171+ t.Fatal(err)172+ }173+ got := Rows(res.Sheet)174+ if liveText(got.Header) != "a,b" || liveText(got.Rows...) != "1,2|,x" || got.Rows[0][1].F.IsZero() {175+ t.Fatalf("%q %q", liveText(got.Header), liveText(got.Rows...))176+ }177+ if got := Rows(sheet.New()); got.Header != nil || got.Rows != nil {178+ t.Fatalf("an empty sheet: %+v", got)179+ }180+}@@ -58,8 +58,11 @@ type fileFormat struct {
5858 // earlier builds ignore them, and the sheets read the same.5959 MacroOrigin string `json:"macroOrigin,omitempty"`6060 Macros []fileMacro `json:"macros,omitempty"`61- fileSheet // versions 1 to 3: the only sheet62- Sheets []fileSheet `json:"sheets,omitempty"` // version 461+ // LinkOrigin, like MacroOrigin, is where the linked regions were62+ // made or trusted (linkfile.go).63+ LinkOrigin string `json:"linkOrigin,omitempty"`64+ fileSheet // versions 1 to 3: the only sheet65+ Sheets []fileSheet `json:"sheets,omitempty"` // version 46366 }6467 6568 // fileSheet is one sheet of a file.@@ -78,6 +81,8 @@ type fileSheet struct {
7881 // Rules need no version: see rulefile.go.7982 CondFormats []fileCondFormat `json:"conditionalFormats,omitempty"`8083 Validations []fileValidation `json:"validations,omitempty"`84+ // Links need no version: see linkfile.go.85+ Links []fileLink `json:"links,omitempty"`8186 }8287 8388 // Write saves the workbook the sheet belongs to; see Workbook.Write.@@ -165,6 +170,9 @@ func (w *Workbook) headLines() string {
165170 }166171 lines = append(lines, w.macrosLines())167172 }173+ if w.linkOrigin != "" && len(w.LinkedRegions()) > 0 {174+ lines = append(lines, `"linkOrigin": `+jsonString(w.linkOrigin))175+ }168176 return strings.Join(lines, ",\n ")169177 }170178 @@ -266,7 +274,7 @@ func (s *Sheet) writeObjects(b *bufio.Writer, indent string) error {
266274 fmt.Fprintf(b, ",\n%s\"pivot\": %s", indent, raw)267275 }268276 s.writeRules(b, indent)269- return nil277+ return s.writeLinks(b, indent)270278 }271279 272280 // Read loads a file written by Write, of this or an earlier version, and@@ -354,5 +362,8 @@ func (s *Sheet) read(f fileSheet) error {
354362 if err := s.readRules(f.CondFormats, f.Validations); err != nil {355363 return err356364 }365+ if err := s.readLinks(f.Links); err != nil {366+ return err367+ }357368 return s.readView(f.fileView)358369 }@@ -109,6 +109,7 @@ func readBook(r io.Reader, trace any) (*Workbook, error) {
109109 return nil, err110110 }111111 w.macroOrigin = f.MacroOrigin112+ w.linkOrigin = f.LinkOrigin112113 w.RecalcAll()113114 return w, nil114115 }@@ -63,7 +63,7 @@ func readWhole(data []byte) (w *Workbook, ambiguous bool, err error) {
6363 if err := w.readMacros(f.Macros); err != nil {6464 return nil, false, err6565 }66- w.macroOrigin = f.MacroOrigin66+ w.macroOrigin, w.linkOrigin = f.MacroOrigin, f.LinkOrigin6767 w.RecalcAll()6868 return w, ambiguous, nil6969 }@@ -233,6 +233,9 @@ func streamSeeds(t testing.TB) [][]byte {
233233 []byte(`{"version": 2, "conditionalFormats": [{"ranges":"A1","iconSet":{"icons":"symbols","points":[{"type":"percent","value":"50"}]}}]}`),234234 []byte(`{"version": 2, "validations": [{"ranges":"A1","criteria":"list","items":["a"],"display":"bubbles"}]}`),235235 []byte(`{"version": 2, "heights": {"0": 2}, "merges": ["A1:"], "cells": {}}`),236+ []byte(`{"version": 2, "linkOrigin": "m", "cells": {"C1": "x"}, "links": [{"at": "A1", "path": "logs/app.csv", "window": 3}, {"at": "E5", "path": "/tmp/t.db", "format": "SQLite", "query": "select 1"}]}`),237+ []byte(`{"version": 4, "sheets": [{"name": "Log", "cells": {}, "links": [{"at": "B2", "path": "a.nuon"}]}, {"name": "S", "cells": {"A1": "=SUM(Log!B:B)"}}]}`),238+ []byte(`{"version": 2, "cells": {}, "links": [{"at": "A1", "path": "a"}, {"at": "A1", "path": "b"}, {"at": "", "window": -2}]}`),236239 []byte(`null`), []byte(`[]`), []byte(`{"version": 2, "cells": {"A1": "1",}}`), []byte(`{"version": 2, "cells": {"A1"`),237240 }238241 }@@ -59,6 +59,8 @@ type step struct {
5959 // rules holds each touched sheet's conditional formats and data6060 // validation before the step.6161 rules map[*Sheet]rulesState62+ // links holds each touched sheet's linked regions before the step.63+ links map[*Sheet][]*linked6264 // sheets is the sheet list before the step, when it changed.6365 sheets *sheetList6466 // settings are the workbook's settings before the step, when they@@ -76,7 +78,7 @@ type colKey struct {
7678 7779 func newStep(label string, s *Sheet, focus Rect) *step {7880 return &step{label: label, sheet: s, focus: focus, cells: map[*Sheet]*image{}, widths: map[colKey]int{}, lines: map[lineKey]lineFmt{},79- names: map[string]*Name{}, views: map[*Sheet]*viewState{}, charts: map[*Sheet][]Chart{}, pivots: map[*Sheet]*Pivot{}, rules: map[*Sheet]rulesState{}}81+ names: map[string]*Name{}, views: map[*Sheet]*viewState{}, charts: map[*Sheet][]Chart{}, pivots: map[*Sheet]*Pivot{}, rules: map[*Sheet]rulesState{}, links: map[*Sheet][]*linked{}}8082 }8183 8284 func (st *step) empty() bool {@@ -86,7 +88,7 @@ func (st *step) empty() bool {
8688 // widthOnly reports whether the step changed nothing but column widths8789 // and row heights.8890 func (st *step) widthOnly() bool {89- return len(st.cells) == 0 && len(st.lines) == 0 && len(st.names) == 0 && len(st.views) == 0 && len(st.charts) == 0 && len(st.pivots) == 0 && len(st.rules) == 0 && st.sheets == nil && st.settings == nil &&91+ return len(st.cells) == 0 && len(st.lines) == 0 && len(st.names) == 0 && len(st.views) == 0 && len(st.charts) == 0 && len(st.pivots) == 0 && len(st.rules) == 0 && len(st.links) == 0 && st.sheets == nil && st.settings == nil &&9092 st.macros == nil9193 }9294 @@ -227,6 +229,11 @@ func (w *Workbook) dropUnchanged(st *step) {
227229 delete(st.pivots, s)228230 }229231 }232+ for s, l := range st.links {233+ if sameLinks(l, s.links) {234+ delete(st.links, s)235+ }236+ }230237 if st.sheets != nil && st.sheets.equal(w.sheetList()) {231238 st.sheets = nil232239 }@@ -381,6 +388,6 @@ func (w *Workbook) swap(undo bool) (Change, bool) {
381388 w.pushUndo(inv)382389 }383390 h.mergeWidths = false384- macrosOnly := st.macros != nil && len(st.cells) == 0 && len(st.lines) == 0 && len(st.names) == 0 && len(st.charts) == 0 && len(st.pivots) == 0 && len(st.rules) == 0 && st.sheets == nil391+ macrosOnly := st.macros != nil && len(st.cells) == 0 && len(st.lines) == 0 && len(st.names) == 0 && len(st.charts) == 0 && len(st.pivots) == 0 && len(st.rules) == 0 && len(st.links) == 0 && st.sheets == nil385392 return Change{Label: st.label, Focus: st.focus, Sheet: st.sheet, Tabs: st.sheets != nil, Macros: macrosOnly}, true386393 }@@ -40,6 +40,10 @@ func (w *Workbook) restore(st *step) (inv *step, changed []loc) {
4040 changed = append(changed, w.putName(k, n)...)4141 }4242 w.restoreSheetParts(st, inv)43+ for s, links := range st.links {44+ inv.links[s] = s.links45+ changed = append(changed, s.restoreLinks(links)...)46+ }4347 return inv, changed4448 }4549 @@ -147,6 +151,7 @@ func (st *step) join(child *step) {
147151 joinMap(st.charts, child.charts)148152 joinMap(st.pivots, child.pivots)149153 joinMap(st.rules, child.rules)154+ joinMap(st.links, child.links)150155 st.sheets = cmpOrPtr(st.sheets, child.sheets)151156 st.settings = cmpOrPtr(st.settings, child.settings)152157 st.macros = cmpOrPtr(st.macros, child.macros)@@ -0,0 +1,368 @@
1+package sheet2+3+import (4+ "errors"5+ "path/filepath"6+ "slices"7+ "time"8+9+ "github.com/FelineStateMachine/012/internal/formula"10+)11+12+// Linked regions: a range of a sheet whose cells come from outside the13+// workbook, such as a file followed as it grows or is rewritten. A14+// region is an anchor (its top-left cell) whose rows arrive as live15+// operations (LiveOp, live.go) from whatever reads the source; the16+// engine owns its cells as it owns a spill's: they are derived cells,17+// written outside the undo history, refused by Set, constants to the18+// formulas reading them, values to copies and exports, and never saved.19+// The file keeps the region's source instead (linkfile.go), and the UI20+// reads it again on opening.21+//22+// The first row is the source's header and stays; the rows under it are23+// the data, all of them up to max-cells, or the last Window of them. A24+// region grows right and down as rows arrive; one that would write over25+// cells holding something stops, showing why, until it's read again.26+27+// LinkSource is where a linked region's rows come from, as the file28+// keeps it.29+type LinkSource struct {30+ // Path is the file, as the UI resolves it: relative to the31+ // workbook's folder when it can be (see RebaseLinks).32+ Path string33+ // Format is the name of the file's format ("CSV", "NUON"), or "" to34+ // tell it by the extension.35+ Format string36+ // Table is a SQLite table to read, or Query a query to run.37+ Table, Query string38+ // Window keeps the last Window rows under the header, dropping older39+ // ones, as tail -f does; 0 keeps every row, up to max-cells.40+ Window int41+}42+43+// LinkedRegion is a linked region as the UI shows it: where it is, what44+// it reads, and how the reading goes.45+type LinkedRegion struct {46+ // ID names the region in the workbook while it's open, in LiveOps47+ // and to the UI.48+ ID int49+ Sheet *Sheet50+ Anchor Addr51+ // Area is the cells it covers, the anchor alone while it has none.52+ Area Rect53+ Source LinkSource54+ // Rows counts the data rows it shows, under the header; Dropped the55+ // older rows the window let go.56+ Rows, Dropped int57+ // Updated is when the last rows arrived, zero until they do.58+ Updated time.Time59+ // Err says why the source can't be read, or the rows can't be60+ // written, "" when all is well; Note says what was left out.61+ Err, Note string62+ // Paused is set while the region isn't following its source.63+ Paused bool64+ // Stale is set when its cells may no longer be the source's (an undo65+ // took them back, or the source changed): the source is to be read66+ // again, whole.67+ Stale bool68+}69+70+// ErrLinkedEdit is what typing into a linked region says.71+var ErrLinkedEdit = errors.New("That cell shows part of a linked file: unlink it (Data > Linked file > Unlink) to edit it")72+73+// linked is a region's state: its definition, replaced whole when it74+// changes so undo steps keep the old one, and what the rows written75+// have made it.76+type linked struct {77+ id int78+ anchor Addr79+ src LinkSource80+ *liveState81+}82+83+// liveState is what the source's rows have made of a region, shared by84+// the copies of its definition that undo steps keep.85+type liveState struct {86+ cols, rows int // the region's size, header included; 0 by 0 when empty87+ data int // data rows written, before the window88+ dropped int89+ updated time.Time90+ err, note string91+ paused bool92+ stale bool93+ fitted bool // its columns were widened to its text once94+}95+96+// area is the cells the region covers, the anchor alone when empty.97+func (l *linked) area() Rect {98+ return Rect{From: l.anchor, To: Addr{Col: l.anchor.Col + max(l.cols, 1) - 1, Row: l.anchor.Row + max(l.rows, 1) - 1}}99+}100+101+// info describes l on s.102+func (l *linked) info(s *Sheet) LinkedRegion {103+ return LinkedRegion{ID: l.id, Sheet: s, Anchor: l.anchor, Area: l.area(), Source: l.src,104+ Rows: max(l.rows-1, 0), Dropped: l.dropped, Updated: l.updated, Err: l.err, Note: l.note,105+ Paused: l.paused, Stale: l.stale}106+}107+108+// linkedAt is the region covering a, or nil.109+func (s *Sheet) linkedAt(a Addr) *linked {110+ for _, l := range s.links {111+ if l.area().Contains(a) {112+ return l113+ }114+ }115+ return nil116+}117+118+// LinkedAt returns the linked region covering the cell at a.119+func (s *Sheet) LinkedAt(a Addr) (LinkedRegion, bool) {120+ if l := s.linkedAt(a); l != nil {121+ return l.info(s), true122+ }123+ return LinkedRegion{}, false124+}125+126+// InLinked returns a cell of r in a linked region, which an edit of r127+// can't change.128+func (s *Sheet) InLinked(r Rect) (Addr, bool) {129+ for _, l := range s.links {130+ if l.rows == 0 && l.err == "" {131+ continue // nothing written yet132+ }133+ if overlap, ok := intersectRect(l.area(), r); ok {134+ return overlap.From, true135+ }136+ }137+ return Addr{}, false138+}139+140+// HasLinked reports whether the sheet holds a linked region.141+func (s *Sheet) HasLinked() bool { return len(s.links) > 0 }142+143+// LinkedRegions returns the sheet's linked regions, in the order they144+// were made.145+func (s *Sheet) LinkedRegions() []LinkedRegion {146+ out := make([]LinkedRegion, 0, len(s.links))147+ for _, l := range s.links {148+ out = append(out, l.info(s))149+ }150+ return out151+}152+153+// LinkedRegions returns every live sheet's linked regions, sheet by154+// sheet.155+func (w *Workbook) LinkedRegions() []LinkedRegion {156+ var out []LinkedRegion157+ for _, s := range w.sheets {158+ out = append(out, s.LinkedRegions()...)159+ }160+ return out161+}162+163+// LinkedRegion returns the region id names, on a live sheet.164+func (w *Workbook) LinkedRegion(id int) (LinkedRegion, bool) {165+ if s, l := w.findLinked(id); l != nil {166+ return l.info(s), true167+ }168+ return LinkedRegion{}, false169+}170+171+// findLinked is the region id names and its sheet, or nil.172+func (w *Workbook) findLinked(id int) (*Sheet, *linked) {173+ for _, s := range w.sheets {174+ for _, l := range s.links {175+ if l.id == id {176+ return s, l177+ }178+ }179+ }180+ return nil, nil181+}182+183+// errLinkOver is why a region can't be made where it's asked.184+var errLinkOver = errors.New("A linked file needs an empty cell to start in")185+186+// AddLinked makes a linked region at a reading src, as one undo step,187+// and returns its ID. It starts empty and stale: the UI reads the188+// source and sends its rows.189+func (s *Sheet) AddLinked(a Addr, src LinkSource) (int, error) {190+ switch {191+ case !a.Valid():192+ return 0, errLinkOver193+ case s.cells.filledAt(a) || s.linkedAt(a) != nil || s.InPivot(Rect{From: a, To: a}):194+ return 0, errLinkOver195+ }196+ s.wb.linkSeq++197+ l := &linked{id: s.wb.linkSeq, anchor: a, src: src, liveState: &liveState{stale: true}}198+ s.change("link "+filepath.Base(src.Path), Rect{From: a, To: a}, func() {199+ s.recordLinks()200+ s.links = append(slices.Clone(s.links), l)201+ })202+ return l.id, nil203+}204+205+// SetLinkSource changes what a region reads (its window, say), as one206+// undo step; the region is read again.207+func (w *Workbook) SetLinkSource(id int, src LinkSource) error {208+ s, l := w.findLinked(id)209+ if l == nil {210+ return ErrNoLinked211+ }212+ s.change("change link "+filepath.Base(src.Path), l.area(), func() {213+ s.recordLinks()214+ nl := *l215+ nl.src = src216+ nl.stale = true217+ s.links = slices.Clone(s.links)218+ s.links[slices.Index(s.links, l)] = &nl219+ })220+ return nil221+}222+223+// ErrNoLinked is returned for a region that no longer exists: unlinked,224+// or on a sheet deleted.225+var ErrNoLinked = errors.New("That linked file is no longer linked")226+227+// Unlink turns a region into the values it shows, as one undo step:228+// they stay as ordinary cells, keeping their formatting and notes, and229+// the source is no longer read.230+func (w *Workbook) Unlink(id int) error {231+ s, l := w.findLinked(id)232+ if l == nil {233+ return ErrNoLinked234+ }235+ area := l.area()236+ var cells []*Cell237+ var at []Addr238+ for a := range s.cells.anyKeysIn(area) {239+ if c := s.cells.get(a); c.Spilled() {240+ cells, at = append(cells, c.plain().withNote(c.Note)), append(at, a)241+ }242+ }243+ s.change("unlink "+filepath.Base(l.src.Path), area, func() {244+ s.recordLinks()245+ s.links = slices.DeleteFunc(slices.Clone(s.links), func(x *linked) bool { return x == l })246+ for i, a := range at {247+ s.place(a, cells[i])248+ }249+ })250+ return nil251+}252+253+// PauseLinked stops a region following its source, or has it follow254+// again. It isn't an edit: what follows the source reads it.255+func (w *Workbook) PauseLinked(id int, paused bool) {256+ if _, l := w.findLinked(id); l != nil {257+ l.paused = paused258+ }259+}260+261+// ReloadLinked marks a region stale, so the UI reads its source again,262+// whole.263+func (w *Workbook) ReloadLinked(id int) {264+ if _, l := w.findLinked(id); l != nil {265+ l.stale = true266+ }267+}268+269+// RebaseLinks rewrites the path of every region with fn, as the UI does270+// when the workbook is saved in another folder. It isn't an edit: the271+// regions read the same files.272+func (w *Workbook) RebaseLinks(fn func(string) string) {273+ for _, s := range w.sheets {274+ for _, l := range s.links {275+ l.src.Path = fn(l.src.Path)276+ }277+ }278+}279+280+// recordLinks saves the sheet's regions before their first change in281+// the open step.282+func (s *Sheet) recordLinks() {283+ if st := s.wb.hist.open; st != nil {284+ if _, seen := st.links[s]; !seen {285+ st.links[s] = s.links286+ }287+ }288+}289+290+// restoreLinks puts back a sheet's regions from an undo step. The291+// regions there now are emptied first, and those put back read their292+// sources again, as the cells undo put back may not be theirs.293+func (s *Sheet) restoreLinks(links []*linked) []loc {294+ changed := s.emptyLinked()295+ for _, l := range links {296+ l.rows, l.cols, l.data, l.stale = 0, 0, 0, true297+ }298+ s.links = links299+ return changed300+}301+302+// noRect contains no cell.303+var noRect = Rect{From: Addr{Col: -1, Row: -1}, To: Addr{Col: -1, Row: -1}}304+305+// emptyLinked clears every region's cells (see emptyRegion).306+func (s *Sheet) emptyLinked() []loc {307+ var changed []loc308+ for _, l := range s.links {309+ changed = append(changed, s.emptyRegion(l)...)310+ }311+ return changed312+}313+314+// emptyRegion clears a region's cells, outside the undo history, leaving315+// it empty and stale, and returns the cells that changed.316+func (s *Sheet) emptyRegion(l *linked) []loc {317+ changed := s.clearSpilled(l.area(), noRect)318+ l.rows, l.cols, l.data, l.stale = 0, 0, 0, true319+ return changed320+}321+322+// shiftLinked moves the regions' anchors with inserted or deleted rows323+// or columns (at sp, a span of rows when rows is set, mapped by cell); a324+// region whose anchor is deleted goes. A region the lines reach is325+// emptied first, so cells moving where it was aren't mistaken for its326+// own, and read again, whole, at its new place.327+func (s *Sheet) shiftLinked(rows bool, sp formula.Span, cell func(Addr) (Addr, bool)) {328+ if len(s.links) == 0 {329+ return330+ }331+ s.recordLinks()332+ var next []*linked333+ for _, l := range s.links {334+ end := l.area().To.Col335+ if rows {336+ end = l.area().To.Row337+ }338+ if end < sp.At {339+ next = append(next, l) // before the lines: untouched340+ continue341+ }342+ for _, c := range s.emptyRegion(l) {343+ s.wb.markDirty(c)344+ }345+ to, ok := cell(l.anchor)346+ if !ok {347+ continue348+ }349+ nl := *l350+ nl.anchor = to351+ next = append(next, &nl)352+ }353+ s.links = next354+}355+356+// copyLinks gives cp, a copy of s, regions of its own reading the same357+// sources, to be read again.358+func (s *Sheet) copyLinks(cp *Sheet) {359+ for _, l := range s.links {360+ s.wb.linkSeq++361+ cp.links = append(cp.links, &linked{id: s.wb.linkSeq, anchor: l.anchor, src: l.src, liveState: &liveState{stale: true}})362+ }363+}364+365+// sameLinks reports whether two lists of regions define the same ones.366+func sameLinks(a, b []*linked) bool {367+ return slices.EqualFunc(a, b, func(x, y *linked) bool { return x.id == y.id && x.anchor == y.anchor && x.src == y.src })368+}@@ -0,0 +1,77 @@
1+package sheet2+3+import (4+ "bufio"5+ "encoding/json"6+ "fmt"7+)8+9+// fileLink is a linked region in the file: its anchor and its source,10+// one per line after the charts, never its rows, which are read from the11+// source again on opening. Older builds ignore the field and show the12+// region's cells empty, so it needs no version:13+//14+// "links": [15+// {"at": "A1", "path": "logs/app.csv", "window": 500}16+// ]17+type fileLink struct {18+ At string `json:"at"`19+ Path string `json:"path"`20+ Format string `json:"format,omitempty"`21+ Table string `json:"table,omitempty"`22+ Query string `json:"query,omitempty"`23+ Window int `json:"window,omitempty"`24+}25+26+// writeLinks writes the sheet's linked regions.27+func (s *Sheet) writeLinks(b *bufio.Writer, indent string) error {28+ if len(s.links) == 0 {29+ return nil30+ }31+ b.WriteString(",\n" + indent + `"links": [`)32+ for i, l := range s.links {33+ raw, err := json.Marshal(fileLink{At: l.anchor.String(), Path: l.src.Path, Format: l.src.Format,34+ Table: l.src.Table, Query: l.src.Query, Window: l.src.Window})35+ if err != nil {36+ return err37+ }38+ if i > 0 {39+ b.WriteByte(',')40+ }41+ fmt.Fprintf(b, "\n%s %s", indent, raw)42+ }43+ b.WriteString("\n" + indent + "]")44+ return nil45+}46+47+// readLinks makes the file's linked regions, empty and stale for the UI48+// to read.49+func (s *Sheet) readLinks(links []fileLink) error {50+ for i, fl := range links {51+ a, ok := ParseAddr(fl.At)52+ switch {53+ case !ok:54+ return fmt.Errorf("link %d: invalid cell %q", i+1, fl.At)55+ case fl.Path == "":56+ return fmt.Errorf("link %d: no path", i+1)57+ case fl.Window < 0:58+ return fmt.Errorf("link %d: invalid window %d", i+1, fl.Window)59+ case s.linkedAt(a) != nil:60+ return fmt.Errorf("link %d: %s is linked twice", i+1, fl.At)61+ }62+ s.wb.linkSeq++63+ s.links = append(s.links, &linked{id: s.wb.linkSeq, anchor: a, liveState: &liveState{stale: true},64+ src: LinkSource{Path: fl.Path, Format: fl.Format, Table: fl.Table, Query: fl.Query, Window: fl.Window}})65+ }66+ return nil67+}68+69+// LinkOrigin identifies the computer the workbook's linked regions were70+// made or trusted on, as saved in the file; "" when unknown. The UI asks71+// before following files outside the workbook's folder for a workbook72+// made elsewhere, as it does before running its macros.73+func (w *Workbook) LinkOrigin() string { return w.linkOrigin }74+75+// SetLinkOrigin records where the linked regions were made or trusted.76+// It isn't an edit, and is saved with the next save.77+func (w *Workbook) SetLinkOrigin(origin string) { w.linkOrigin = origin }@@ -0,0 +1,341 @@
1+package sheet2+3+import (4+ "fmt"5+ "strings"6+ "time"7+ "unicode"8+9+ "github.com/FelineStateMachine/012/internal/value"10+)11+12+// The change stream: what reads a linked region's source sends the13+// workbook as LiveOps, each applied at once through ApplyLive, outside14+// the undo history, as a spill is written. An op names the region by15+// ID and carries whole rows of values with the formats they arrived in,16+// so it depends on nothing but the region: the same op applied to the17+// same workbook elsewhere makes the same cells, which is what a session18+// following another's would be sent.19+20+// LiveCell is a value as a source gave it, with its format (a date's,21+// a file size's), Automatic for none.22+type LiveCell struct {23+ V Value24+ F Format25+}26+27+// LiveRow is a row of a linked region, from its first column.28+type LiveRow []LiveCell29+30+// LiveOp is one change a source makes to its linked region.31+type LiveOp struct {32+ // Link is the region's ID (LinkedRegion.ID).33+ Link int34+ // At is when the source changed.35+ At time.Time36+ // Reset has Rows replace every row under the header; otherwise they37+ // follow the last.38+ Reset bool39+ // Header is the region's first row, when it is new or changed: the40+ // source's column names.41+ Header LiveRow42+ // Rows are data rows, under the header.43+ Rows []LiveRow44+ // Err says why the source can't be read, "" when it can. An op with45+ // an error changes no cells: the region keeps what it shows.46+ Err string47+ // Note says what the source left out, if anything.48+ Note string49+}50+51+// LiveInfo describes one op applied, for telemetry. It holds counts52+// only, never contents.53+type LiveInfo struct {54+ Rows int // rows the op brought55+ Cells int // cells that changed56+ Reset bool57+ Duration time.Duration // writing the cells and recalculating what reads them58+}59+60+// OnLive, when set, is called after every op applied, like OnRecalc61+// (see OnBegin, whose op is "live").62+var OnLive func(trace any, i LiveInfo)63+64+// ApplyLive applies op to its region: its rows are written, the window65+// drops the oldest, and what reads the cells that changed recalculates.66+// It returns the region as it now is; ErrNoLinked when it's gone.67+func (w *Workbook) ApplyLive(op LiveOp) (LinkedRegion, error) {68+ s, l := w.findLinked(op.Link)69+ if l == nil {70+ return LinkedRegion{}, ErrNoLinked71+ }72+ var start time.Time73+ if OnLive != nil {74+ if OnBegin != nil {75+ OnBegin(w.trace, "live")76+ }77+ start = time.Now()78+ }79+ changed := s.applyLive(l, op)80+ if len(changed) > 0 {81+ if w.hist.depth > 0 {82+ w.hist.dirty = append(w.hist.dirty, changed...) // recalculated as the open step ends83+ } else {84+ w.recalc(changed)85+ }86+ }87+ if OnLive != nil {88+ OnLive(w.trace, LiveInfo{Rows: len(op.Rows), Cells: len(changed), Reset: op.Reset, Duration: time.Since(start)})89+ }90+ return l.info(s), nil91+}92+93+// applyLive writes op's rows into l and returns the cells that changed.94+func (s *Sheet) applyLive(l *linked, op LiveOp) []loc {95+ if !op.At.IsZero() {96+ l.updated = op.At97+ }98+ if op.Reset {99+ l.stale = false100+ }101+ if op.Err != "" {102+ l.err = op.Err103+ if l.rows == 0 {104+ return s.writeLinkedError(l)105+ }106+ return nil107+ }108+ l.err, l.note = "", op.Note109+ rows, header := s.liveRows(l, op)110+ return s.writeLinkedRows(l, header, rows, op.Reset)111+}112+113+// liveRows is the data rows the region is to show after op, with the114+// header: when rows are dropped (by the window, or when the region is115+// full) the rows kept are read back from the region's own cells.116+// Rows that already show and stay are nil, so only new ones are written.117+func (s *Sheet) liveRows(l *linked, op LiveOp) (rows []LiveRow, header LiveRow) {118+ header = op.Header119+ if header == nil && l.rows > 0 {120+ header = s.readLinkedRow(l, 0)121+ }122+ cols := max(len(header), l.cols)123+ for _, r := range op.Rows {124+ cols = max(cols, len(r))125+ }126+ old := max(l.rows-1, 0)127+ if op.Reset {128+ old = 0129+ }130+ total := old + len(op.Rows)131+ keep := total132+ if l.src.Window > 0 {133+ keep = min(keep, l.src.Window)134+ }135+ // Whole rows, as many as fit in max-cells.136+ if fit := MaxCells()/max(cols, 1) - 1; keep > fit {137+ keep = max(fit, 0)138+ l.note = fmt.Sprintf("only the last %d rows fit in max-cells (%d cells)", keep, MaxCells())139+ if l.src.Window == 0 {140+ l.note = fmt.Sprintf("only the first %d rows fit in max-cells (%d cells)", keep, MaxCells())141+ }142+ }143+ if op.Reset {144+ l.dropped = 0145+ l.data = 0146+ }147+ l.data += len(op.Rows)148+ if keep < total && l.src.Window == 0 {149+ // Keep everything up to the budget: the newest rows are left out.150+ rows = make([]LiveRow, keep)151+ copy(rows[min(old, keep):], op.Rows[:max(keep-old, 0)])152+ return rows[:keep], header153+ }154+ drop := total - keep155+ l.dropped += drop156+ rows = make([]LiveRow, keep)157+ for i := range keep {158+ switch j := i + drop; {159+ case j >= old:160+ rows[i] = op.Rows[j-old]161+ case drop > 0:162+ rows[i] = s.readLinkedRow(l, j+1)163+ }164+ }165+ return rows, header166+}167+168+// readLinkedRow reads row r of the region (0 for the header) back from169+// its cells.170+func (s *Sheet) readLinkedRow(l *linked, r int) LiveRow {171+ row := make(LiveRow, l.cols)172+ for c := range l.cols {173+ v, lk, kind := s.cells.derivedOf(Addr{Col: l.anchor.Col + c, Row: l.anchor.Row + r})174+ if kind == slotSpill {175+ row[c] = LiveCell{V: v, F: lk.auto}176+ }177+ }178+ return row179+}180+181+// writeLinkedRows makes the region show header and rows, writing the182+// rows that aren't nil (nil ones show already), and returns the cells183+// that changed. When a cell it needs holds something of its own, it184+// writes nothing and says so.185+func (s *Sheet) writeLinkedRows(l *linked, header LiveRow, rows []LiveRow, reset bool) []loc {186+ cols := len(header)187+ for _, r := range rows {188+ cols = max(cols, len(r))189+ }190+ if !reset {191+ cols = max(cols, l.cols)192+ }193+ nrows := 0194+ if cols > 0 {195+ nrows = 1 + len(rows)196+ }197+ area := Rect{From: l.anchor, To: Addr{Col: l.anchor.Col + max(cols, 1) - 1, Row: l.anchor.Row + max(nrows, 1) - 1}}198+ if at, ok := s.linkedBlocked(l, area); ok {199+ l.err = "The linked rows would overwrite data in " + at.String()200+ if area.To.Row >= MaxRows || area.To.Col >= MaxCols {201+ l.err = "The linked rows would go past the edge of the sheet"202+ }203+ return nil204+ }205+ keep := area206+ if nrows == 0 {207+ keep = noRect208+ }209+ changed := s.clearSpilled(l.area(), keep)210+ write := func(r int, row LiveRow, all bool) {211+ for c := range cols {212+ var lc LiveCell213+ if c < len(row) {214+ lc = row[c]215+ }216+ if lc.V.Kind == Empty && !all {217+ continue218+ }219+ a := Addr{Col: l.anchor.Col + c, Row: l.anchor.Row + r}220+ if s.writeSpilled(a, lc.V, lc.F) {221+ changed = append(changed, loc{s, a})222+ }223+ }224+ }225+ if cols > 0 {226+ write(0, header, true)227+ }228+ for i, row := range rows {229+ // A row written over one that showed clears what it doesn't230+ // fill; one below the old region has nothing to clear.231+ if row != nil {232+ write(i+1, row, i+1 < l.rows || reset)233+ }234+ }235+ l.cols, l.rows = cols, nrows236+ if !l.fitted && nrows > 1 {237+ l.fitted = true238+ s.fitLinked(l)239+ }240+ for _, c := range changed {241+ s.freedFor(c.a)242+ }243+ return changed244+}245+246+// linkedBlocked returns a cell of area the region can't write: one247+// holding something that isn't the region's, or past the sheet's edge.248+func (s *Sheet) linkedBlocked(l *linked, area Rect) (Addr, bool) {249+ if area.To.Row >= MaxRows || area.To.Col >= MaxCols {250+ return area.To, true251+ }252+ own := l.area()253+ for a := range s.cells.anyKeysIn(area) {254+ if !s.cells.filledAt(a) || own.Contains(a) && s.cells.derivedAt(a) == slotSpill {255+ continue256+ }257+ return a, true258+ }259+ if ms := s.MergesIn(area); len(ms) > 0 {260+ return ms[0].From, true261+ }262+ return Addr{}, false263+}264+265+// writeLinkedError shows the region's error in its anchor, as #REF!,266+// while it has nothing else to show.267+func (s *Sheet) writeLinkedError(l *linked) []loc {268+ if s.cells.filledAt(l.anchor) && s.cells.derivedAt(l.anchor) != slotSpill {269+ return nil // never over what the user typed270+ }271+ l.cols, l.rows = 0, 0272+ if s.writeSpilled(l.anchor, ErrRef, Format{}) {273+ return []loc{{s, l.anchor}}274+ }275+ return nil276+}277+278+// freedFor has a spill waiting for the cell at a try again, now that the279+// region wrote or cleared it.280+func (s *Sheet) freedFor(a Addr) {281+ if s.spills == nil {282+ return283+ }284+ s.spillAt.readers(a, func(anchor Addr) {285+ if sp := s.spills[anchor]; sp != nil && sp.why != "" {286+ sp.stale = true287+ s.wb.markDirty(loc{s, anchor})288+ }289+ })290+}291+292+// maxLinkedWidth caps how wide fitting makes a region's column.293+const maxLinkedWidth = 30294+295+// fitLinked widens the region's columns, the first time rows arrive, to296+// their widest text in the first rows, never below the default width,297+// outside the undo history as a pivot's are.298+func (s *Sheet) fitLinked(l *linked) {299+ const sample = 200300+ for c := range l.cols {301+ col := l.anchor.Col + c302+ if s.widths[col] != 0 {303+ continue // set by hand, or by the file304+ }305+ widest := 0306+ for r := range min(l.rows, sample) {307+ widest = max(widest, len([]rune(s.ShownText(Addr{Col: col, Row: l.anchor.Row + r}))))308+ }309+ if widest+2 > DefaultWidth {310+ s.setWidth(col, min(widest+2, maxLinkedWidth))311+ }312+ }313+}314+315+// EntryValue is the value and the format an entry implies ("$5", a316+// date), as typing it into a cell would store them, for a source whose317+// rows arrive as text. Formulas stay text, as a data file's should.318+func EntryValue(input string) (Value, Format) {319+ input = strings.Map(func(r rune) rune {320+ if unicode.IsControl(r) {321+ return -1322+ }323+ return r324+ }, input)325+ if input == "" {326+ return Value{}, Format{}327+ }328+ if rest, ok := strings.CutPrefix(input, "'"); ok {329+ return value.Str(rest), Format{}330+ }331+ if n, f, ok := ParseValue(input); ok {332+ return value.Num(n), f333+ }334+ switch strings.ToUpper(input) {335+ case "TRUE":336+ return value.Boolean(true), Format{}337+ case "FALSE":338+ return value.Boolean(false), Format{}339+ }340+ return value.Str(input), Format{}341+}@@ -0,0 +1,275 @@
1+package sheet2+3+import (4+ "bytes"5+ "errors"6+ "strings"7+ "testing"8+ "time"9+10+ "github.com/FelineStateMachine/012/internal/value"11+)12+13+// liveRow makes a row of entries, as a CSV line's fields.14+func liveRow(fields ...string) LiveRow {15+ row := make(LiveRow, len(fields))16+ for i, f := range fields {17+ v, fm := EntryValue(f)18+ row[i] = LiveCell{V: v, F: fm}19+ }20+ return row21+}22+23+// linkedSheet is a sheet with a region at A1 showing a header and n rows24+// of numbers 1..n in column B.25+func linkedSheet(t *testing.T, window int) (*Sheet, int) {26+ t.Helper()27+ s := New()28+ id, err := s.AddLinked(Addr{}, LinkSource{Path: "log.csv", Window: window})29+ if err != nil {30+ t.Fatal(err)31+ }32+ return s, id33+}34+35+func apply(t *testing.T, s *Sheet, op LiveOp) LinkedRegion {36+ t.Helper()37+ r, err := s.Book().ApplyLive(op)38+ if err != nil {39+ t.Fatal(err)40+ }41+ return r42+}43+44+func TestLiveResetAndAppend(t *testing.T) {45+ s, id := linkedSheet(t, 0)46+ s.Set(Addr{Col: 4}, "=SUM(B:B)")47+ s.Set(Addr{Col: 5}, "=COUNTA(A2:A100)")48+ r := apply(t, s, LiveOp{Link: id, Reset: true, Header: liveRow("name", "n"), Rows: []LiveRow{liveRow("a", "1"), liveRow("b", "2")}})49+ if r.Rows != 2 || r.Area != NewRect(Addr{}, Addr{Col: 1, Row: 2}) || r.Stale {50+ t.Fatalf("after reset: %+v", r)51+ }52+ if got := s.Value(Addr{Col: 4}).Num; got != 3 {53+ t.Fatalf("SUM = %v, want 3", got)54+ }55+ at := time.Date(2026, 9, 29, 12, 0, 0, 0, time.UTC)56+ r = apply(t, s, LiveOp{Link: id, At: at, Rows: []LiveRow{liveRow("c", "3", "extra")}})57+ if r.Rows != 3 || r.Area.To != (Addr{Col: 2, Row: 3}) || !r.Updated.Equal(at) {58+ t.Fatalf("after append: %+v", r)59+ }60+ if got := s.Value(Addr{Col: 4}).Num; got != 6 {61+ t.Fatalf("SUM = %v, want 6", got)62+ }63+ if got := s.Value(Addr{Col: 5}).Num; got != 3 {64+ t.Fatalf("COUNTA = %v, want 3", got)65+ }66+ if !s.Cell(Addr{Row: 3}).Spilled() || s.Value(Addr{Col: 2, Row: 3}).Str != "extra" {67+ t.Fatalf("row 4: %+v", s.Cell(Addr{Row: 3}))68+ }69+ if s.CanUndo() && s.Book().UndoLabel() != "edit F1" {70+ t.Fatalf("rows arriving made an undo step: %q", s.Book().UndoLabel())71+ }72+ // A reset with fewer rows clears the rest.73+ apply(t, s, LiveOp{Link: id, Reset: true, Header: liveRow("name", "n"), Rows: []LiveRow{liveRow("z", "10")}})74+ if s.Filled(Addr{Row: 2}) || s.Filled(Addr{Col: 2, Row: 3}) {75+ t.Fatal("a reset left old rows")76+ }77+ if got := s.Value(Addr{Col: 4}).Num; got != 10 {78+ t.Fatalf("SUM = %v, want 10", got)79+ }80+}81+82+func TestLiveWindow(t *testing.T) {83+ s, id := linkedSheet(t, 3)84+ s.Set(Addr{Col: 4}, "=SUM(B2:B10)")85+ var rows []LiveRow86+ for _, n := range []string{"1", "2", "3", "4", "5"} {87+ rows = append(rows, liveRow("r"+n, n))88+ }89+ r := apply(t, s, LiveOp{Link: id, Reset: true, Header: liveRow("name", "n"), Rows: rows[:2]})90+ if r.Rows != 2 || r.Dropped != 0 {91+ t.Fatalf("%+v", r)92+ }93+ r = apply(t, s, LiveOp{Link: id, Rows: rows[2:]})94+ if r.Rows != 3 || r.Dropped != 2 {95+ t.Fatalf("after the window filled: %+v", r)96+ }97+ var got []string98+ for row := 1; row <= 3; row++ {99+ got = append(got, s.Value(Addr{Row: row}).Str)100+ }101+ if strings.Join(got, ",") != "r3,r4,r5" || s.Value(Addr{Row: 0}).Str != "name" {102+ t.Fatalf("rows %v, header %v", got, s.Value(Addr{}))103+ }104+ if s.Filled(Addr{Row: 4}) {105+ t.Fatal("the window left a row below")106+ }107+ if v := s.Value(Addr{Col: 4}).Num; v != 12 {108+ t.Fatalf("SUM = %v, want 12", v)109+ }110+ // One more row drops one more.111+ r = apply(t, s, LiveOp{Link: id, Rows: []LiveRow{liveRow("r6", "6")}})112+ if r.Dropped != 3 || s.Value(Addr{Row: 1}).Str != "r4" || s.Value(Addr{Row: 3}).Str != "r6" {113+ t.Fatalf("%+v: %v..%v", r, s.Value(Addr{Row: 1}), s.Value(Addr{Row: 3}))114+ }115+}116+117+func TestLiveMaxCells(t *testing.T) {118+ SetMaxCells(8) // a header and three rows of two119+ defer SetMaxCells(0)120+ s, id := linkedSheet(t, 0)121+ var rows []LiveRow122+ for range 5 {123+ rows = append(rows, liveRow("x", "1"))124+ }125+ r := apply(t, s, LiveOp{Link: id, Reset: true, Header: liveRow("a", "b"), Rows: rows})126+ if r.Rows != 3 || !strings.Contains(r.Note, "only the first 3 rows") {127+ t.Fatalf("%+v", r)128+ }129+}130+131+func TestLiveRefusesEditsAndBlocks(t *testing.T) {132+ s, id := linkedSheet(t, 0)133+ apply(t, s, LiveOp{Link: id, Reset: true, Header: liveRow("a"), Rows: []LiveRow{liveRow("1")}})134+ if err := s.Set(Addr{Row: 1}, "5"); !errors.Is(err, ErrLinkedEdit) {135+ t.Fatalf("Set in a region: %v", err)136+ }137+ if _, ok := s.InLinked(NewRect(Addr{Row: 1}, Addr{Col: 3, Row: 5})); !ok {138+ t.Fatal("InLinked missed the region")139+ }140+ // Rows that would overwrite a typed cell stop the region, saying so.141+ s.Set(Addr{Row: 3}, "mine")142+ r := apply(t, s, LiveOp{Link: id, Rows: []LiveRow{liveRow("2"), liveRow("3")}})143+ if !strings.Contains(r.Err, "overwrite data in A4") || r.Rows != 1 || s.Value(Addr{Row: 3}).Str != "mine" {144+ t.Fatalf("%+v", r)145+ }146+ // Formatting a linked cell keeps its value.147+ s.SetStyle(NewRect(Addr{Row: 1}, Addr{Row: 1}), func(st *Style) { st.Bold = true })148+ if c := s.Cell(Addr{Row: 1}); c.Value.Num != 1 || !c.Style.Bold || !c.Spilled() {149+ t.Fatalf("formatted linked cell: %+v", c)150+ }151+}152+153+func TestLiveErrorShowsInAnchor(t *testing.T) {154+ s, id := linkedSheet(t, 0)155+ r := apply(t, s, LiveOp{Link: id, Err: "no such file"})156+ if r.Err != "no such file" || s.Value(Addr{}) != value.ErrRef {157+ t.Fatalf("%+v, anchor %v", r, s.Value(Addr{}))158+ }159+ r = apply(t, s, LiveOp{Link: id, Reset: true, Header: liveRow("a"), Rows: []LiveRow{liveRow("1")}})160+ if r.Err != "" || s.Value(Addr{}).Str != "a" {161+ t.Fatalf("%+v, anchor %v", r, s.Value(Addr{}))162+ }163+ // An error once rows show keeps them.164+ r = apply(t, s, LiveOp{Link: id, Err: "gone"})165+ if r.Err != "gone" || s.Value(Addr{Row: 1}).Num != 1 {166+ t.Fatalf("%+v", r)167+ }168+}169+170+func TestUnlinkAndUndo(t *testing.T) {171+ s, id := linkedSheet(t, 0)172+ apply(t, s, LiveOp{Link: id, Reset: true, Header: liveRow("a"), Rows: []LiveRow{liveRow("1")}})173+ if err := s.Book().Unlink(id); err != nil {174+ t.Fatal(err)175+ }176+ if s.HasLinked() || s.Cell(Addr{Row: 1}).Spilled() || s.Value(Addr{Row: 1}).Num != 1 {177+ t.Fatalf("unlinked: %+v", s.Cell(Addr{Row: 1}))178+ }179+ if err := s.Set(Addr{Row: 1}, "7"); err != nil {180+ t.Fatal(err)181+ }182+ s.Undo()183+ s.Undo() // the unlink184+ r, ok := s.Book().LinkedRegion(id)185+ if !ok || !r.Stale || s.Filled(Addr{Row: 1}) {186+ t.Fatalf("undo of unlink: %+v %v, A2 %v", r, ok, s.Value(Addr{Row: 1}))187+ }188+ apply(t, s, LiveOp{Link: id, Reset: true, Header: liveRow("a"), Rows: []LiveRow{liveRow("2")}})189+ s.Undo() // making the region190+ if s.HasLinked() || s.Filled(Addr{}) || s.Filled(Addr{Row: 1}) {191+ t.Fatal("undoing the region left its cells")192+ }193+ if _, err := s.Book().ApplyLive(LiveOp{Link: id}); !errors.Is(err, ErrNoLinked) {194+ t.Fatalf("op for a region undone: %v", err)195+ }196+ s.Redo()197+ if r, ok := s.Book().LinkedRegion(id); !ok || !r.Stale {198+ t.Fatalf("redo: %+v", r)199+ }200+}201+202+func TestLinkedShiftsWithRows(t *testing.T) {203+ s := New()204+ s.Set(Addr{Row: 0}, "above")205+ id, err := s.AddLinked(Addr{Row: 2}, LinkSource{Path: "log.csv"})206+ if err != nil {207+ t.Fatal(err)208+ }209+ apply(t, s, LiveOp{Link: id, Reset: true, Header: liveRow("a"), Rows: []LiveRow{liveRow("1")}})210+ s.Set(Addr{Col: 3, Row: 1}, "x")211+ if err := s.InsertRows(1, 2); err != nil {212+ t.Fatal(err)213+ }214+ r, _ := s.Book().LinkedRegion(id)215+ if r.Anchor != (Addr{Row: 4}) || !r.Stale || s.Filled(Addr{Row: 2}) || s.Filled(Addr{Row: 4}) {216+ t.Fatalf("%+v", r)217+ }218+ if s.Value(Addr{Col: 3, Row: 3}).Str != "x" {219+ t.Fatal("a cell moving past the region was lost")220+ }221+ apply(t, s, LiveOp{Link: id, Reset: true, Header: liveRow("a"), Rows: []LiveRow{liveRow("1")}})222+ if s.Value(Addr{Row: 5}).Num != 1 {223+ t.Fatal("the region wasn't written at its new place")224+ }225+ // Rows inserted below it leave it be.226+ s.InsertRows(20, 1)227+ if r, _ := s.Book().LinkedRegion(id); r.Stale {228+ t.Fatal("rows below the region made it stale")229+ }230+ s.DeleteRows(4, 1) // its anchor's row231+ if s.HasLinked() {232+ t.Fatal("deleting the anchor's row kept the region")233+ }234+}235+236+func TestLinkedFileRoundTrip(t *testing.T) {237+ s, id := linkedSheet(t, 500)238+ s.Book().SetLinkOrigin("here")239+ apply(t, s, LiveOp{Link: id, Reset: true, Header: liveRow("a"), Rows: []LiveRow{liveRow("1")}})240+ var b bytes.Buffer241+ if err := s.Write(&b); err != nil {242+ t.Fatal(err)243+ }244+ text := b.String()245+ if !strings.Contains(text, `"links": [`) || !strings.Contains(text, `{"at":"A1","path":"log.csv","window":500}`) ||246+ !strings.Contains(text, `"linkOrigin": "here"`) || strings.Contains(text, `"A2"`) {247+ t.Fatalf("written:\n%s", text)248+ }249+ got, err := Read(strings.NewReader(text))250+ if err != nil {251+ t.Fatal(err)252+ }253+ rs := got.LinkedRegions()254+ if len(rs) != 1 || rs[0].Source != (LinkSource{Path: "log.csv", Window: 500}) || !rs[0].Stale || got.Book().LinkOrigin() != "here" {255+ t.Fatalf("read: %+v", rs)256+ }257+ for _, bad := range []string{`[{"at": "ZZZZ9", "path": "a"}]`, `[{"at": "A1"}]`, `[{"at": "A1", "path": "a", "window": -1}]`,258+ `[{"at": "A1", "path": "a"}, {"at": "A1", "path": "b"}]`} {259+ if _, err := Read(strings.NewReader(`{"version": 2, "cells": {}, "links": ` + bad + `}`)); err == nil {260+ t.Errorf("read links %s", bad)261+ }262+ }263+}264+265+func TestEntryValue(t *testing.T) {266+ for in, want := range map[string]Value{"12": value.Num(12), "$5": value.Num(5), "true": value.Boolean(true),267+ "'12": value.Str("12"), "=A1": value.Str("=A1"), "a\x1bb": value.Str("ab"), "": {}} {268+ if got, _ := EntryValue(in); got != want {269+ t.Errorf("EntryValue(%q) = %v, want %v", in, got, want)270+ }271+ }272+ if _, f := EntryValue("9/26/2026"); f.IsZero() {273+ t.Error("a date implied no format")274+ }275+}@@ -138,6 +138,7 @@ func (s *Sheet) restructure(rows bool, sp formula.Span) {
138138 label += "s"139139 }140140 s.change(label, focus, func() {141+ s.shiftLinked(rows, sp, cell) // first, so remap doesn't take cells moving into regions for theirs141142 s.remap(label, focus, cell, rng)142143 s.remapNames(rng)143144 s.shiftCharts(cell, rng)@@ -105,6 +105,9 @@ type Sheet struct {
105105 // spillAt the anchors by the cells they cover; see spill.go.106106 spills map[Addr]*spill107107 spillAt rangeIndex108+ // links are the linked regions, in the order they were made; see109+ // linked.go. The list is replaced whole on every change.110+ links []*linked108111 109112 view viewState // frozen panes, the filter and merges, see view.go110113 mergeIdx mergeIndex // the merges by the cells they cover, see merge.go@@ -234,6 +237,9 @@ func (s *Sheet) Set(a Addr, input string) error {
234237 if _, ok := s.SpillAnchor(a); ok {235238 return ErrSpillEdit236239 }240+ if s.linkedAt(a) != nil {241+ return ErrLinkedEdit242+ }237243 var err error238244 s.change("edit "+a.String(), Rect{From: a, To: a}, func() { err = s.put(a, input) })239245 return err@@ -354,6 +360,16 @@ func (s *Sheet) place(a Addr, c *Cell) {
354360 if c.Spilled() {355361 c = c.leftover()356362 }363+ if len(s.links) > 0 {364+ if v, lk, kind := s.cells.derivedOf(a); kind == slotSpill && s.linkedAt(a) != nil {365+ // A linked region's cell keeps its value: only its366+ // formatting and note are placed.367+ defer s.writeSpilled(a, v, lk.auto)368+ if c != nil {369+ c = c.leftover()370+ }371+ }372+ }357373 s.trackShape(a, c)358374 s.spillTouched(a, c)359375 if c == nil {@@ -331,9 +331,10 @@ func (s *Sheet) SpillAnchor(a Addr) (Addr, bool) {
331331 return found, ok332332 }333333 334-// HasSpills reports whether a formula on the sheet spills or would, so335-// what draws the sheet can skip looking for spilled cells.336-func (s *Sheet) HasSpills() bool { return len(s.spills) > 0 }334+// HasSpills reports whether a formula on the sheet spills or would, or a335+// linked region shows rows (whose cells are spilled cells too), so what336+// draws the sheet can skip looking for spilled cells.337+func (s *Sheet) HasSpills() bool { return len(s.spills) > 0 || len(s.links) > 0 }337338 338339 // SpillArea returns the cells the formula at a spills into, the anchor339340 // first, when it spills.@@ -56,6 +56,7 @@ type Workbook struct {
5656 5757 macros []Macro // see macros.go5858 macroOrigin string59+ linkOrigin string // see linkfile.go5960 6061 // structural is set when sheets were added, deleted or renamed during6162 // the open change, which then recalculates everything: references by@@ -72,6 +73,8 @@ type Workbook struct {
7273 // spillWork is the arrays an evaluation pass computed, to spill once7374 // it's done; see spill.go.7475 spillWork spillWork76+ // linkSeq is the last linked region's ID; see linked.go.77+ linkSeq int7578 }7679 7780 // loc is a cell on a particular sheet.@@ -234,6 +237,7 @@ func (w *Workbook) DuplicateSheet(s *Sheet) (*Sheet, error) {
234237 cp.charts = slices.Clone(s.charts)235238 cp.rules = s.rules // never changed in place236239 cp.pivot = pivotState{def: s.pivot.def.clone(), stale: s.pivot.def != nil, out: s.pivot.out, fit: s.pivot.fit}240+ s.copyLinks(cp)237241 w.change(cp, "duplicate "+s.name, Rect{}, func() {238242 w.recordSheets()239243 w.insert(cp, w.Index(s)+1)