DA1 attribute 4, after no kitty graphics reply and outside tmux, turns on sixel; XTSMGRAPHICS gives the color registers and CSI 16 t the cell size. chart.Sixel composites the plot over the background, keeps at most the terminal's registers of colors (low bits dropped until they fit, each register the average of its colors) and run-length encodes each band. The UI keeps blank cells where a chart's whole plot area shows (in the grid, under no later chart or floating box), draws the images with the cursor saved once the frame has been still for 25 ms, draws them again when a row they cross changes, and clears the screen when one moves, changes or goes. BenchmarkSixel: 0.07 to 0.3 ms at 24x10 cells, 3 to 7 ms at 120x40.
- commit
- 2f52c8217d9679a45966d945de6539ec453c9eb7
- parent
- 5fe016216a7cb163a80d67d4ce8c6d800d8c451d
- author
- Dami <dami.etoile@gmail.com>
- kind
- feat risk score 1.5 of 3, Jev's estimate
| M | ROADMAP.md | 2 | +- |
| M | docs/ | 8 | +- |
| M | docs/ | 5 | +- |
| M | docs/ | 2 | +- |
| M | docs/ | 13 | +- |
| M | docs/ | 1 | + |
| M | docs/ | 2 | +- |
| A | internal/ | 219 | ++++++++++++++++++++++ |
| A | internal/ | 81 | ++++++++ |
| M | internal/ | 19 | ++ |
| M | internal/ | 4 | +- |
| M | internal/ | 16 | +- |
| M | internal/ | 39 | +++- |
| M | internal/ | 2 | +- |
| M | internal/ | 1 | + |
| M | internal/ | 1 | + |
| A | internal/ | 293 | ++++++++++++++++++++++++++++++ |
| A | internal/ | 159 | ++++++++++++++++ |
@@ -45,7 +45,6 @@ rises in measured steps; see [Bounds of support](docs/contributing/limits.md#wha
4545 | Record dialog choices (sort bar, filter picker, find and replace, chart editor) and chart drags; let scripts run commands that open dialogs, with answers | M |4646 | Vim: `.` repeat, registers, marks, `cc`/`s`, command-line history, `:w!` | M |4747 | Hold-to-preview and Shift+Enter on terminals with the kitty keyboard protocol (`View.KeyboardEnhancements`) | S |48-| Sixel chart images, redrawn on resize, for terminals without kitty graphics | M |4948 5049 ## Later: sharing a live sheet (shelved)5150 @@ -105,6 +104,7 @@ by a gateway dialing the iroh ticket.
105104 - Color schemes and a config file: [Themes](docs/terminal/themes.md), [Configuration](docs/reference/config.md)106105 - A high-contrast theme at WCAG AAA, and every state readable without color: [Themes](docs/terminal/themes.md#high-contrast), [UX](docs/contributing/ux.md#reading-without-color)107106 - `012 serve` over SSH, with files on the ssh command line and recovery of unsaved work: [Serving over SSH](docs/terminal/ssh.md)107+- Sixel chart images on terminals without kitty graphics, drawn after the frame and redrawn as the screen moves: [Charts](docs/sheets/charts.md)108108 109109 **Files**110110 @@ -502,8 +502,12 @@ the user's editor (`AllowEditor`), since that starts a program.
502502 `internal/chart` draws a chart in two layers that share one layout: text503503 for any terminal (block elements for bars, braille for lines, half blocks504504 for pies, eighths filled column by column for areas) and, on terminals505-with kitty graphics, an image of the plot area with the axes and legend506-still terminal text. Each chart type is a layout in the `types` table,505+with kitty or sixel graphics, an image of the plot area with the axes and506+legend still terminal text. A kitty image is placed by placeholder507+characters, which the renderer treats as text; a sixel image508+(`chart.Sixel`) is pixels Bubble Tea's renderer doesn't know of, so509+`internal/ui/sixel.go` keeps blank cells for it, draws it once the frame510+has settled, and clears the screen when it moves. Each chart type is a layout in the `types` table,507511 returning a plan that draws it both ways, and the series its legend508512 lists; `Draw` places the legend and gives the layout the room left. The509513 type constants and their names, and the options of each chart@@ -238,7 +238,10 @@ whatever its data: only the categories that fit are drawn (a pie of 8192
238238 slices, 0.44 ms, is the worst). An image for kitty graphics is redrawn239239 when its data, size or theme changes: 25 to 260 us at 24 x 10 cells and240240 1 to 1.5 ms at 120 x 40, a pie 8.6 ms (`internal/chart`,241-`BenchmarkDraw`, `BenchmarkImage`). Key presses add little: an arrow key241+`BenchmarkDraw`, `BenchmarkImage`). A sixel image is encoded then too,242+in 0.07 to 0.3 ms at 24 x 10 cells and 3 to 7 ms at 120 x 40 (10 x 20243+pixel cells), 0.6 to 67 KB to send (`BenchmarkSixel`); it is sent again,244+without encoding, whenever the screen under it is redrawn. Key presses add little: an arrow key242245 through to its frame is 0.19 ms at 80 x 24 and 0.9 ms at 200 x 60;243246 Page Down 1.1 ms.244247 @@ -104,7 +104,7 @@ Colors. `terminal` uses the terminal's own 16-color palette. `high-contrast` dra
104104 105105 #### `chart-images`106106 107-Draw charts as images on terminals with kitty graphics (kitty, Ghostty, WezTerm). When false, charts are always text.107+Draw charts as images on terminals with kitty graphics (kitty, Ghostty, WezTerm) or, outside tmux, sixel graphics (foot, xterm, mlterm, Windows Terminal). When false, charts are always text.108108 109109 | | |110110 |---|---|@@ -40,6 +40,13 @@ columns, and save with the sheet.
4040 **Images or text.** At startup 012 asks the terminal whether it supports the4141 kitty graphics protocol. Where it does (kitty, Ghostty, WezTerm) the plot is4242 a real image, drawn in the terminal's own palette and placed with Unicode43-placeholders, so redraws never erase it. Elsewhere, and usually inside tmux,44-the plot is drawn with block and braille characters. Axis labels, the45-legend and the frame are text either way.43+placeholders, so redraws never erase it. Otherwise, where the terminal44+lists sixel graphics among its device attributes (foot, xterm, mlterm,45+Windows Terminal), the plot is a sixel image in the terminal's palette,46+in as many colors as it has color registers. A sixel image is drawn over47+the finished screen, so it shows only where its whole plot area does: a48+chart scrolled partly out of the grid, or under another chart, a menu or49+a dialog, is text until it shows whole again, and the image is drawn50+again after scrolling, resizing or an edit to its data. Elsewhere, and51+inside tmux, the plot is drawn with block and braille characters. Axis52+labels, the legend and the frame are text either way.@@ -22,6 +22,7 @@ colors that follow light and dark.
2222 | Mouse, all motion | hover highlights, drag, resize | most terminals |2323 | Pointer shape (OSC 22) | resize and text cursors over the grid | Ghostty, kitty, foot, xterm |2424 | Kitty graphics | chart images | kitty, Ghostty, WezTerm |25+| Sixel graphics (DA1 attribute 4, XTSMGRAPHICS) | chart images where kitty graphics aren't, outside tmux: [Charts](../sheets/charts.md) | foot, xterm, mlterm, Windows Terminal |2526 | Hyperlinks (OSC 8) | links in cells | most modern terminals |2627 | Clipboard (OSC 52) | copying ranges | most terminals; tmux needs `set-clipboard on` |2728 | Progress (OSC 9;4) | JEV and import progress in the tab | Ghostty, Windows Terminal, iTerm2 |@@ -139,7 +139,7 @@ Every session is a separate 012 with nothing shared but the process:
139139 its own workbook, undo history, clipboard, theme, JEV cache and140140 terminal state. What it learns about the terminal comes from the141141 client: the window size and its changes, the colors (from the `TERM`142-the client sends), the light or dark background, kitty graphics, and142+the client sends), the light or dark background, kitty or sixel graphics, and143143 the clipboard, which Ctrl+C sets on the client's machine through OSC 52.144144 145145 [Macros](../sheets/macros.md) can be recorded, run, renamed and deleted in a@@ -0,0 +1,219 @@
1+package chart2+3+import (4+ "image"5+ "image/color"6+ "slices"7+ "strconv"8+ "strings"9+)10+11+// Sixel is the other way images reach a terminal, for terminals without12+// kitty graphics (foot, xterm, mlterm, Windows Terminal and others). A13+// sixel image is pixels drawn at the cursor, not text: the UI draws it14+// after the frame over blank cells it keeps for it (see the ui package's15+// sixel.go). The image is sent whole each time, in at most as many16+// colors as the terminal has color registers, run-length encoded.17+18+// Sixel returns the DCS sequence that draws img at the cursor: pixels19+// with any coverage composited over bg, fully transparent ones left20+// unset so the cell background shows, in at most registers colors21+// (at least 2, at most 256).22+func Sixel(img *image.RGBA, bg color.RGBA, registers int) string {23+ registers = min(max(registers, 2), 256)24+ b := img.Bounds()25+ w, h := b.Dx(), b.Dy()26+ if w <= 0 || h <= 0 {27+ return ""28+ }29+ idx, pal := quantize(img, bg, registers)30+ var out strings.Builder31+ // P2 = 1: pixels not drawn keep what's under them. The raster32+ // attributes set square pixels and the image's size.33+ out.WriteString("\x1bP0;1;0q\"1;1;")34+ out.WriteString(strconv.Itoa(w) + ";" + strconv.Itoa(h))35+ for i, c := range pal {36+ out.WriteString("#" + strconv.Itoa(i) + ";2;" + percent(c.R) + ";" + percent(c.G) + ";" + percent(c.B))37+ }38+ bands := sixelBands(idx, w, h, len(pal))39+ out.WriteString(bands)40+ out.WriteString("\x1b\\")41+ return out.String()42+}43+44+// percent is a color channel as sixel's 0 to 100.45+func percent(v uint8) string { return strconv.Itoa((int(v)*100 + 127) / 255) }46+47+// sixelBands encodes the image of register indexes (-1 for unset), six48+// rows at a time: for each color in the band, its row of sixels,49+// run-length encoded, then "$" to go back for the next color, and "-"50+// between bands.51+func sixelBands(idx []int16, w, h, colors int) string {52+ var out strings.Builder53+ b := bands{w: w, rows: make([][]byte, colors), band: make([]int, colors)}54+ for y0 := 0; y0 < h; y0 += 6 {55+ if y0 > 0 {56+ out.WriteByte('-')57+ }58+ b.gather(idx[y0*w:min(y0+6, h)*w], y0)59+ for i, c := range b.used {60+ if i > 0 {61+ out.WriteByte('$')62+ }63+ out.WriteString("#" + strconv.Itoa(int(c)))64+ writeRuns(&out, b.rows[c])65+ }66+ }67+ return out.String()68+}69+70+// bands holds the rows of sixels of one band of six pixel rows.71+type bands struct {72+ w int73+ rows [][]byte // each color's row of sixels, made when first used74+ band []int // the band each color was last used in, plus one75+ used []int16 // the colors of the band, in order76+}77+78+// gather makes the rows of the band at y0 from its pixels' registers.79+func (b *bands) gather(px []int16, y0 int) {80+ b.used = b.used[:0]81+ for i, c := range px {82+ if c < 0 {83+ continue84+ }85+ if b.band[c] != y0+1 {86+ b.band[c] = y0 + 187+ b.used = append(b.used, c)88+ if b.rows[c] == nil {89+ b.rows[c] = make([]byte, b.w)90+ }91+ for x := range b.rows[c] {92+ b.rows[c][x] = '?'93+ }94+ }95+ b.rows[c][i%b.w] += 1 << (i / b.w) // each pixel has one color, so each bit is added once96+ }97+ slices.Sort(b.used)98+}99+100+// writeRuns writes a row of sixels, runs of four or more as "!n", and101+// leaves out the empty sixels at its end.102+func writeRuns(out *strings.Builder, row []byte) {103+ end := len(row)104+ for end > 0 && row[end-1] == '?' {105+ end--106+ }107+ for i := 0; i < end; {108+ j := i + 1109+ for j < end && row[j] == row[i] {110+ j++111+ }112+ if n := j - i; n >= 4 {113+ out.WriteString("!" + strconv.Itoa(n))114+ out.WriteByte(row[i])115+ } else {116+ for range n {117+ out.WriteByte(row[i])118+ }119+ }120+ i = j121+ }122+}123+124+// quantize maps each pixel of img, composited over bg, to a register,125+// -1 for a fully transparent pixel, and returns the registers' colors.126+// Colors are kept exactly while there are at most registers of them;127+// past that, their low bits are dropped until they fit, and each128+// register is the average of the colors it stands for. Registers are129+// numbered in the order their colors first appear, so the same image130+// always encodes the same way.131+func quantize(img *image.RGBA, bg color.RGBA, registers int) ([]int16, []color.RGBA) {132+ b := img.Bounds()133+ w, h := b.Dx(), b.Dy()134+ px := make([]uint32, w*h) // 0 for transparent, else 1<<24 | RGB135+ for y := range h {136+ for x := range w {137+ if c := over(img, b.Min.X+x, b.Min.Y+y, bg); c.A != 0 {138+ px[y*w+x] = 1<<24 | uint32(c.R)<<16 | uint32(c.G)<<8 | uint32(c.B)139+ }140+ }141+ }142+ shift := uint(0)143+ for ; shift < 8; shift++ {144+ if distinct(px, shift, registers) <= registers {145+ break146+ }147+ }148+ reg := map[uint32]int16{}149+ var sums [][4]int // R, G, B sums and count per register150+ idx := make([]int16, w*h)151+ last, lastReg := uint32(0), int16(-1) // runs of one color are common: look each up once152+ for i, p := range px {153+ if p == 0 {154+ idx[i] = -1155+ continue156+ }157+ r := lastReg158+ if p != last {159+ k := bucket(p, shift)160+ var ok bool161+ if r, ok = reg[k]; !ok {162+ r = int16(len(sums))163+ reg[k] = r164+ sums = append(sums, [4]int{})165+ }166+ last, lastReg = p, r167+ }168+ idx[i] = r169+ s := &sums[r]170+ s[0] += int(p >> 16 & 0xff)171+ s[1] += int(p >> 8 & 0xff)172+ s[2] += int(p & 0xff)173+ s[3]++174+ }175+ pal := make([]color.RGBA, len(sums))176+ for i, s := range sums {177+ pal[i] = color.RGBA{uint8(s[0] / s[3]), uint8(s[1] / s[3]), uint8(s[2] / s[3]), 255}178+ }179+ return idx, pal180+}181+182+// bucket is the key of pixel p with shift low bits of each channel183+// dropped.184+func bucket(p uint32, shift uint) uint32 {185+ m := uint32(0xff>>shift<<shift) * 0x010101186+ return p & (1<<24 | m)187+}188+189+// distinct counts the buckets of px at shift, stopping past limit.190+func distinct(px []uint32, shift uint, limit int) int {191+ seen := map[uint32]struct{}{}192+ last := uint32(0)193+ for _, p := range px {194+ if p == 0 || p == last {195+ continue196+ }197+ last = p198+ seen[bucket(p, shift)] = struct{}{}199+ if len(seen) > limit {200+ break201+ }202+ }203+ return len(seen)204+}205+206+// over is the pixel at x, y of the premultiplied img composited over207+// bg, opaque, or transparent where img has no coverage.208+func over(img *image.RGBA, x, y int, bg color.RGBA) color.RGBA {209+ i := img.PixOffset(x, y)210+ p := img.Pix[i : i+4 : i+4]211+ a := int(p[3])212+ if a == 0 {213+ return color.RGBA{}214+ }215+ return color.RGBA{mix(p[0], bg.R, a), mix(p[1], bg.G, a), mix(p[2], bg.B, a), 255}216+}217+218+// mix is the premultiplied channel c of coverage a over under.219+func mix(c, under uint8, a int) uint8 { return uint8(min(int(c)+int(under)*(255-a)/255, 255)) }@@ -0,0 +1,81 @@
1+package chart2+3+import (4+ "image"5+ "image/color"6+ "strings"7+ "testing"8+9+ "github.com/FelineStateMachine/012/internal/sheet"10+)11+12+// The encoder's bytes for a small image: a register per color in the13+// order they appear, runs of four or more as "!n", trailing empty14+// sixels left out, "$" between the colors of a band and "-" between15+// bands, a pixel with half coverage composited over the background and16+// a transparent one left unset.17+func TestSixelGolden(t *testing.T) {18+ img := image.NewRGBA(image.Rect(0, 0, 4, 7))19+ red := color.RGBA{255, 0, 0, 255}20+ for x := range 4 {21+ img.SetRGBA(x, 0, red)22+ }23+ img.SetRGBA(1, 1, color.RGBA{0, 128, 0, 128}) // premultiplied: half green24+ img.SetRGBA(0, 2, red)25+ img.SetRGBA(1, 2, red)26+ img.SetRGBA(2, 2, red)27+ img.SetRGBA(0, 6, color.RGBA{0, 0, 255, 255})28+ got := Sixel(img, color.RGBA{0, 0, 0, 255}, 256)29+ want := "\x1bP0;1;0q\"1;1;4;7" +30+ "#0;2;100;0;0#1;2;0;50;0#2;2;0;0;100" +31+ "#0DDD@$#1?A" + // red: rows 0 and 2 (1+4) three times, then row 0 alone32+ "-#2@" +33+ "\x1b\\"34+ if got != want {35+ t.Errorf("Sixel =\n%q\nwant\n%q", got, want)36+ }37+ img.SetRGBA(3, 2, red)38+ if got := Sixel(img, color.RGBA{0, 0, 0, 255}, 256); !strings.Contains(got, "#0!4D$") {39+ t.Errorf("a run of four isn't run-length encoded: %q", got)40+ }41+}42+43+// Past the terminal's registers, colors are merged until they fit, and44+// every pixel still names a register that exists.45+func TestSixelQuantize(t *testing.T) {46+ img := image.NewRGBA(image.Rect(0, 0, 40, 30))47+ for y := range 30 {48+ for x := range 40 {49+ img.SetRGBA(x, y, color.RGBA{uint8(x * 6), uint8(y * 8), uint8((x + y) * 3), 255})50+ }51+ }52+ for _, regs := range []int{16, 256} {53+ idx, pal := quantize(img, color.RGBA{}, regs)54+ if len(pal) > regs {55+ t.Errorf("%d registers: %d colors", regs, len(pal))56+ }57+ for _, i := range idx {58+ if i < 0 || int(i) >= len(pal) {59+ t.Fatalf("%d registers: pixel names register %d of %d", regs, i, len(pal))60+ }61+ }62+ }63+ if s := Sixel(img, color.RGBA{}, 16); strings.Count(s, ";2;") > 16 {64+ t.Errorf("more than 16 colors defined")65+ }66+}67+68+// A chart's image encodes, and an empty image draws nothing.69+func TestSixelChart(t *testing.T) {70+ img := Image(sheet.ChartColumn, single, 40, 12, Options{CellW: 8, CellH: 16}, testPalette)71+ if img == nil {72+ t.Fatal("no image")73+ }74+ s := Sixel(img, color.RGBA{0, 0, 0, 255}, 256)75+ if !strings.HasPrefix(s, "\x1bP0;1;0q\"1;1;") || !strings.HasSuffix(s, "\x1b\\") {76+ t.Errorf("not a sixel sequence: %.40q", s)77+ }78+ if Sixel(image.NewRGBA(image.Rect(0, 0, 0, 0)), color.RGBA{}, 256) != "" {79+ t.Error("an empty image drew something")80+ }81+}@@ -76,3 +76,22 @@ func BenchmarkImage(b *testing.B) {
7676 }7777 }7878 }79+80+// BenchmarkSixel is a chart's image encoded as sixel, in 256 colors, as81+// it is whenever its data, size or colors change.82+func BenchmarkSixel(b *testing.B) {83+ pal := Palette{Series: [Colors]color.RGBA{{205, 49, 49, 255}, {13, 188, 121, 255}, {36, 114, 200, 255}}, Grid: color.RGBA{128, 128, 128, 80}}84+ d := stressData(12)85+ for _, ct := range stressTypes {86+ for _, sz := range stressSizes {87+ img := Image(ct.t, d, sz.w, sz.h, Options{CellW: 10, CellH: 20}, pal)88+ b.Run(fmt.Sprintf("%s/%dx%d", ct.name, sz.w, sz.h), func(b *testing.B) {89+ n := 090+ for b.Loop() {91+ n = len(Sixel(img, color.RGBA{0, 0, 0, 255}, 256))92+ }93+ b.ReportMetric(float64(n), "bytes")94+ })95+ }96+ }97+}@@ -93,8 +93,8 @@ var Options = []Option{
9393 "`light:NAME,dark:NAME` picks one by the terminal's background and follows it when it changes.",9494 Check: checkTheme},9595 {Name: "chart-images", Kind: Bool, Group: GroupAppearance, Default: "true", Env: []string{"O12_CHART_IMAGES"}, Live: true,96- Desc: "Draw charts as images on terminals with kitty graphics (kitty, Ghostty, WezTerm). " +97- "When false, charts are always text."},96+ Desc: "Draw charts as images on terminals with kitty graphics (kitty, Ghostty, WezTerm) or, " +97+ "outside tmux, sixel graphics (foot, xterm, mlterm, Windows Terminal). When false, charts are always text."},9898 {Name: "notifications", Kind: Bool, Group: GroupAppearance, Default: "true", Env: []string{"O12_NOTIFICATIONS"}, Live: true,9999 Desc: "Send a desktop notification (OSC 9) when JEV answers or an import finishes while the " +100100 "terminal window is in the background."},@@ -226,6 +226,9 @@ func (m *Model) drawChart(i int, c sheet.Chart, selected bool) []string {
226226 w, h := chartInner(c)227227 o := m.term.chartOptions()228228 o.Chart = c.ChartOptions229+ if m.term.sixelOn() {230+ o.Image = m.term.six.placed(i) // see sixel.go231+ }229232 if i >= maxImages {230233 o.Image = false231234 }@@ -244,7 +247,7 @@ func (m *Model) drawChart(i int, c sheet.Chart, selected bool) []string {
244247 lines = append(lines, border.Render("┌─")+m.th.Title.Render(title)+245248 border.Render(strings.Repeat("─", max(c.W-3-ansi.StringWidth(title), 0))+"┐"))246249 for y := range h {247- lines = append(lines, border.Render("│")+" "+chartRow(&m.th, g, y, firstImageID+i, o.Image)+" "+border.Render("│"))250+ lines = append(lines, border.Render("│")+" "+chartRow(&m.th, g, y, firstImageID+i, o.Image, m.term.kitty)+" "+border.Render("│"))248251 }249252 corner := "┘"250253 if selected {@@ -260,8 +263,9 @@ func (m *Model) drawChart(i int, c sheet.Chart, selected bool) []string {
260263 }261264 262265 // chartRow renders row y of a drawn chart with the theme; with images,263-// the plot area is the image's placeholders.264-func chartRow(th *theme.Theme, g *chart.Grid, y, imageID int, image bool) string {266+// the plot area is the image's kitty placeholders, or blank cells for a267+// sixel image to cover.268+func chartRow(th *theme.Theme, g *chart.Grid, y, imageID int, image, kitty bool) string {265269 var b, run strings.Builder266270 var style lipgloss.Style267271 styled := false@@ -281,8 +285,10 @@ func chartRow(th *theme.Theme, g *chart.Grid, y, imageID int, image bool) string
281285 c := g.At(x, y)282286 key, st, isStyled := 0, lipgloss.Style{}, true283287 text := c.Text284- switch {285- case image && y >= g.Plot.Min.Y && y < g.Plot.Max.Y && x >= g.Plot.Min.X && x < g.Plot.Max.X:288+ switch inPlot := y >= g.Plot.Min.Y && y < g.Plot.Max.Y && x >= g.Plot.Min.X && x < g.Plot.Max.X; {289+ case image && inPlot && !kitty:290+ key, isStyled, text = 2, false, " "291+ case image && inPlot:286292 key, st = 1, th.ImageID(imageID)287293 text = chart.Placeholder(y-g.Plot.Min.Y, x-g.Plot.Min.X)288294 case c.Fg == chart.None && c.Bg == chart.None:@@ -4,6 +4,7 @@ import (
44 "fmt"55 "image/color"66 "log/slog"7+ "slices"78 "strconv"89 "strings"910 @@ -21,24 +22,30 @@ import (
2122 // replies to queries sent at startup. It survives File New and Open.2223 //2324 // Charts become images when the terminal answers a kitty graphics query24-// (kitty, Ghostty, WezTerm and others); everywhere else they stay text.25-// Images use the terminal's own palette, asked for with OSC 4 once26-// graphics are known to work, so bars match the text legend.25+// (kitty, Ghostty, WezTerm and others), or else when its primary device26+// attributes (DA1) list sixel graphics (foot, xterm, mlterm, Windows27+// Terminal; see sixel.go); everywhere else they stay text. Images use28+// the terminal's own palette, asked for with OSC 4 at startup, so bars29+// match the text legend.2730 type terminal struct {2831 noImages bool // chart-images = false in the config2932 noNotify bool // notifications = false in the config3033 kitty bool // answers kitty graphics queries34+ sixel bool // lists sixel in DA1, without kitty graphics, outside tmux35+ registers int // sixel color registers, as XTSMGRAPHICS reports them3136 tmux bool // inside tmux: sequences for the outer terminal need passthrough3237 cellW, cellH int // cell size in pixels, 0 until reported3338 palette map[int]color.RGBA39+ bg color.Color // the terminal's background, nil until reported3440 sent map[int]string // image id -> what was sent, to send only changes41+ six sixelState // the sixel images drawn and to draw3542 blurred bool // the terminal window doesn't have focus3643 }3744 3845 // newTerminal starts with what getenv, the terminal's environment, says:3946 // the process's own locally, the client's when served over SSH.4047 func newTerminal(getenv func(string) string) terminal {41- return terminal{tmux: getenv("TMUX") != "", palette: map[int]color.RGBA{}, sent: map[int]string{}}48+ return terminal{tmux: getenv("TMUX") != "", palette: map[int]color.RGBA{}, sent: map[int]string{}, registers: 256}4249 }4350 4451 // wrap prepares a sequence for the outer terminal.@@ -50,10 +57,13 @@ func (t *terminal) wrap(seq string) string {
5057 }5158 5259 // probes are the startup queries: kitty graphics support, live light53-// and dark changes (mode 2031), and the 16 palette colors (OSC 4), which54-// chart images and color scales draw with.60+// and dark changes (mode 2031), the 16 palette colors (OSC 4), which61+// chart images and color scales draw with, the sixel color registers62+// (XTSMGRAPHICS) and, last, the primary device attributes. Terminals63+// answer in order, so the kitty and XTSMGRAPHICS replies, if any, come64+// before DA1's, which then decides between kitty images, sixel and text.5565 func (t *terminal) probes() tea.Cmd {56- return tea.Raw(t.wrap(chart.Query()) + ansi.SetModeLightDark + paletteQuery())66+ return tea.Raw(t.wrap(chart.Query()) + ansi.SetModeLightDark + paletteQuery() + sixelRegistersQuery + ansi.RequestPrimaryDeviceAttributes)5767 }5868 5969 // paletteQuery asks for the 16 ANSI colors; terminals that don't answer@@ -84,6 +94,15 @@ func (t *terminal) handle(msg tea.Msg) tea.Cmd {
8494 // was asked for at startup.8595 return tea.Raw(ansi.WindowOp(16)) // 16: report the cell size in pixels8696 }97+ case uv.PrimaryDeviceAttributesEvent:98+ if slices.Contains(msg, 4) && !t.kitty && !t.tmux && !t.sixel {99+ t.sixel = true100+ return tea.Raw(ansi.WindowOp(16)) // sixel images are drawn at the cell size101+ }102+ case uv.UnknownCsiEvent:103+ if n, ok := parseRegisters(string(msg)); ok {104+ t.registers = n105+ }87106 case uv.CellSizeEvent:88107 if msg.Width > 0 && msg.Height > 0 {89108 t.cellW, t.cellH = msg.Width, msg.Height@@ -156,10 +175,12 @@ func (t *terminal) chartPalette(th *theme.Theme) chart.Palette {
156175 return p157176 }158177 159-// images reports whether charts are drawn as images.178+// images reports whether charts are drawn as kitty images.160179 func (t *terminal) images() bool { return t.kitty && !t.noImages }161180 162-// chartOptions are the drawing options for charts on this terminal.181+// chartOptions are the drawing options for charts on this terminal. For182+// sixel, whether a chart is an image depends on where it is: see183+// Model.drawChart.163184 func (t *terminal) chartOptions() chart.Options {164185 return chart.Options{Image: t.images(), CellW: t.cellW, CellH: t.cellH}165186 }@@ -224,7 +224,7 @@ func (m *Model) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
224224 m.clampView()225225 // Any edit may have queued JEV questions.226226 // Chart images follow any change, see graphics.go.227- return m, tea.Batch(cmd, m.jev.send(m.spans.Parent()), m.term.syncImages(m.sheet, m.displayCharts, &m.th, m.spans))227+ return m, tea.Batch(cmd, m.jev.send(m.spans.Parent()), m.term.syncImages(m.sheet, m.displayCharts, &m.th, m.spans), m.syncSixel(msg))228228 }229229 230230 // beginUpdate prepares for an input event and returns the sheet's state@@ -219,6 +219,7 @@ func (m *Model) handlePrefs(msg tea.Msg) bool {
219219 switch msg := msg.(type) {220220 case tea.BackgroundColorMsg:221221 m.prefs.light = !msg.IsDark()222+ m.term.bg = msg.Color222223 if p := m.applyTheme(); p != "" && m.note == "" {223224 m.note = m.th.Warning.Render(p)224225 }@@ -0,0 +1,293 @@
1+package ui2+3+import (4+ "fmt"5+ "hash/fnv"6+ "image"7+ "image/color"8+ "log/slog"9+ "slices"10+ "strconv"11+ "strings"12+ "time"13+14+ tea "charm.land/bubbletea/v2"15+ "github.com/charmbracelet/x/ansi"16+17+ "github.com/FelineStateMachine/012/internal/chart"18+)19+20+// Sixel chart images, for terminals without kitty graphics. A sixel21+// image is pixels the terminal draws at the cursor, which Bubble Tea's22+// renderer knows nothing of: it keeps drawing cells, and a cell it23+// writes over the image punches a hole in it. So:24+//25+// - A chart is an image only where the whole of its plot area shows: in26+// the grid, under no chart drawn later and no floating box (a menu,27+// a dialog, formula suggestions, a note). Elsewhere it is text. The28+// frame keeps blank cells where an image goes, so the renderer has29+// nothing to write there.30+// - Images are drawn after the frame: a sixelDrawMsg comes once the31+// frame has been still for sixelSettle, longer than a frame, and32+// draws each image with the cursor saved and restored around it.33+// - When an image moves, changes or goes (scrolling, resizing, an edit34+// of its data, an overlay opening over it), the screen is cleared and35+// redrawn whole, which erases the images, and they are drawn again.36+// When a row an image crosses changes, the images are drawn again.37+// - Never inside tmux: tmux answers DA1 for itself, and an image passed38+// through to the outer terminal lands where tmux doesn't know, so its39+// redraws would erase it.40+41+// sixelRegistersQuery asks for the number of sixel color registers42+// (XTSMGRAPHICS, read).43+const sixelRegistersQuery = "\x1b[?1;1;0S"44+45+// parseRegisters parses the XTSMGRAPHICS reply "CSI ? 1 ; 0 ; n S".46+func parseRegisters(s string) (int, bool) {47+ s, ok := strings.CutPrefix(s, "\x1b[?1;0;")48+ if !ok {49+ return 0, false50+ }51+ s, ok = strings.CutSuffix(s, "S")52+ n, err := strconv.Atoi(s)53+ if !ok || err != nil || n < 2 {54+ return 0, false55+ }56+ return min(n, 256), true57+}58+59+// sixelSettle is how long the frame stays still before images are drawn60+// over it: more than a frame at 60 fps, so the frame is out first.61+const sixelSettle = 25 * time.Millisecond62+63+// sixelState is what the sixel images are and where.64+type sixelState struct {65+ cache map[int]sixelImage // by chart66+ plan []sixelPlace // the images the frame keeps room for67+ drawn []sixelPlace // the images on screen68+ frame uint64 // the rows of the latest frame the plan crosses, hashed69+ drawnFrame uint64 // frame when the images were drawn70+ changed time.Time // when frame last changed71+ pending bool // a sixelDrawMsg is on its way72+ now func() time.Time // the clock; time.Now when nil73+ tick func() tea.Cmd // what sends sixelDrawMsg; sixelTick when nil74+}75+76+// sixelImage is a chart's image encoded, and where its plot area is in77+// the chart's contents.78+type sixelImage struct {79+ key string80+ plot image.Rectangle81+ seq string // "" when there is nothing to plot82+}83+84+// sixelPlace is chart's image at screen cells at.85+type sixelPlace struct {86+ chart int87+ at image.Rectangle88+ key string89+}90+91+// sixelDrawMsg draws the images once the frame has settled.92+type sixelDrawMsg struct{}93+94+func sixelTick() tea.Cmd {95+ return tea.Tick(sixelSettle, func(time.Time) tea.Msg { return sixelDrawMsg{} })96+}97+98+// later asks for a sixelDrawMsg.99+func (s *sixelState) later() tea.Cmd {100+ s.pending = true101+ if s.tick != nil {102+ return s.tick()103+ }104+ return sixelTick()105+}106+107+func (s *sixelState) clock() time.Time {108+ if s.now != nil {109+ return s.now()110+ }111+ return time.Now()112+}113+114+// sixelOn reports whether charts are drawn as sixel images: the115+// terminal has sixel and has said its cell size, and chart-images isn't116+// off.117+func (t *terminal) sixelOn() bool { return t.sixel && !t.noImages && t.cellW > 0 && t.cellH > 0 }118+119+// placed reports whether chart i is an image in the current frame.120+func (s *sixelState) placed(i int) bool {121+ return slices.ContainsFunc(s.plan, func(p sixelPlace) bool { return p.chart == i })122+}123+124+// syncSixel follows msg, after Update has handled it: it encodes the125+// charts that changed, works out where images go, clears the screen when126+// one drawn moved, changed or went, and asks for the images to be drawn127+// once the frame settles.128+func (m *Model) syncSixel(msg tea.Msg) tea.Cmd {129+ t := &m.term130+ if _, ok := msg.(sixelDrawMsg); ok {131+ return m.drawSixels()132+ }133+ var cmds []tea.Cmd134+ if _, ok := msg.(tea.WindowSizeMsg); ok && t.sixel {135+ cmds = append(cmds, tea.Raw(ansi.WindowOp(16))) // the font may have changed size too136+ }137+ if !t.sixelOn() {138+ t.six.plan, t.six.cache = nil, nil139+ if len(t.six.drawn) > 0 {140+ t.six.drawn = nil141+ cmds = append(cmds, tea.ClearScreen)142+ }143+ return tea.Batch(cmds...)144+ }145+ m.encodeSixels()146+ t.six.plan = m.sixelPlan()147+ if t.six.gone() {148+ t.six.drawn = nil149+ cmds = append(cmds, tea.ClearScreen)150+ }151+ if len(t.six.plan) > 0 && !t.six.pending {152+ cmds = append(cmds, t.six.later())153+ }154+ return tea.Batch(cmds...)155+}156+157+// gone reports whether an image drawn isn't in the plan as it is.158+func (s *sixelState) gone() bool {159+ for _, d := range s.drawn {160+ if !slices.Contains(s.plan, d) {161+ return true162+ }163+ }164+ return false165+}166+167+// drawSixels draws the planned images over the settled frame, unless168+// they are drawn already over the same rows.169+func (m *Model) drawSixels() tea.Cmd {170+ s := &m.term.six171+ s.pending = false172+ if !m.term.sixelOn() || len(s.plan) == 0 || s.frame == s.drawnFrame && slices.Equal(s.drawn, s.plan) {173+ return nil174+ }175+ if s.clock().Sub(s.changed) < sixelSettle {176+ return s.later()177+ }178+ var b strings.Builder179+ for _, p := range s.plan {180+ b.WriteString(ansi.SaveCursor + ansi.CursorPosition(p.at.Min.X+1, p.at.Min.Y+1))181+ b.WriteString(s.cache[p.chart].seq)182+ b.WriteString(ansi.RestoreCursor)183+ }184+ s.drawn, s.drawnFrame = slices.Clone(s.plan), s.frame185+ return tea.Raw(b.String())186+}187+188+// noteFrame records the rows of the frame content that the planned189+// images cross, so images are drawn again when those rows change.190+func (s *sixelState) noteFrame(content string) {191+ if len(s.plan) == 0 {192+ s.frame = 0193+ return194+ }195+ h := fnv.New64a()196+ lines := strings.Split(content, "\n")197+ for _, p := range s.plan {198+ fmt.Fprintf(h, "%d %v %s\n", p.chart, p.at, p.key)199+ for y := p.at.Min.Y; y < p.at.Max.Y && y < len(lines); y++ {200+ h.Write([]byte(lines[y]))201+ }202+ }203+ if f := h.Sum64(); f != s.frame {204+ s.frame, s.changed = f, s.clock()205+ }206+}207+208+// encodeSixels encodes the image of each chart whose data, size, colors209+// or cell size changed.210+func (m *Model) encodeSixels() {211+ t := &m.term212+ if t.six.cache == nil {213+ t.six.cache = map[int]sixelImage{}214+ }215+ o := t.chartOptions()216+ o.Image = true217+ pal, bg := t.chartPalette(&m.th), m.sixelBackground()218+ charts := m.displayCharts()219+ for i, c := range charts {220+ if i >= maxImages {221+ break222+ }223+ w, h := chartInner(c)224+ d := m.sheet.ChartData(c)225+ o.Chart = c.ChartOptions226+ key := fmt.Sprintf("%v %v %d %d %v %v %v %d", c.Type, d, w, h, o, pal, bg, t.registers)227+ if t.six.cache[i].key == key {228+ continue229+ }230+ span := m.spans.Start("chart.sixel", slog.String("type", c.Type.String()), slog.Int("w", w), slog.Int("h", h))231+ g := chart.Draw(c.Type, d, w, h, o)232+ si := sixelImage{key: key, plot: g.Plot}233+ if img := chart.Image(c.Type, d, w, h, o, pal); img != nil && !g.Plot.Empty() {234+ si.seq = chart.Sixel(img, bg, t.registers)235+ }236+ span.End(slog.Int("bytes", len(si.seq)))237+ t.six.cache[i] = si238+ }239+ for i := range t.six.cache {240+ if i >= len(charts) || i >= maxImages {241+ delete(t.six.cache, i)242+ }243+ }244+}245+246+// sixelBackground is the color images are composited over: the theme's247+// background, or the terminal's, or black.248+func (m *Model) sixelBackground() color.RGBA {249+ c := m.term.bg250+ if m.th.Palette != nil && m.th.Palette.Background != nil {251+ c = m.th.Palette.Background252+ }253+ if c == nil {254+ return color.RGBA{0, 0, 0, 255}255+ }256+ r, g, b, _ := c.RGBA()257+ return color.RGBA{uint8(r >> 8), uint8(g >> 8), uint8(b >> 8), 255}258+}259+260+// sixelPlan is where images go: the charts whose plot area shows whole261+// in the grid, under no chart drawn later and no floating box.262+func (m *Model) sixelPlan() []sixelPlace {263+ var over []image.Rectangle264+ for _, b := range m.floating() {265+ if len(b.Lines) > 0 {266+ over = append(over, image.Rect(b.X, b.Y, b.X+b.Width(), b.Y+b.Height()))267+ }268+ }269+ charts := m.displayCharts()270+ boxes := make([]image.Rectangle, len(charts))271+ for i, c := range charts {272+ x, y := m.chartScreen(c)273+ boxes[i] = image.Rect(x, y, x+c.W, y+c.H)274+ }275+ grid := image.Rect(m.hdrW(), gridTop, m.width, gridTop+m.visibleRows())276+ var plan []sixelPlace277+ for i := range charts {278+ img, ok := m.term.six.cache[i]279+ if !ok || img.seq == "" {280+ continue281+ }282+ at := img.plot.Add(boxes[i].Min.Add(image.Pt(2, 1))) // the border and a column of padding283+ if !at.In(grid) || overlapsAny(at, over) || overlapsAny(at, boxes[i+1:]) {284+ continue285+ }286+ plan = append(plan, sixelPlace{i, at, img.key})287+ }288+ return plan289+}290+291+func overlapsAny(r image.Rectangle, rs []image.Rectangle) bool {292+ return slices.ContainsFunc(rs, r.Overlaps)293+}@@ -0,0 +1,159 @@
1+package ui2+3+import (4+ "fmt"5+ "strings"6+ "testing"7+ "time"8+9+ tea "charm.land/bubbletea/v2"10+ uv "github.com/charmbracelet/ultraviolet"11+ "github.com/charmbracelet/x/ansi/kitty"12+)13+14+// ticked stands for a sixelDrawMsg asked for: tests deliver15+// sixelDrawMsg themselves, with the clock where they want it.16+type ticked struct{}17+18+// sixelModel is a wide model with a chart, on a terminal that has said19+// what a sixel terminal says: sixel in DA1, 16 color registers and its20+// cell size. Its clock stands still unless the test moves it.21+func sixelModel(t *testing.T) (*Model, *time.Time) {22+ t.Helper()23+ m := wideModel()24+ m.term.tmux = false // wherever the tests run25+ clock := time.Unix(1000, 0)26+ m.term.six.now = func() time.Time { return clock }27+ m.term.six.tick = func() tea.Cmd { return func() tea.Msg { return ticked{} } }28+ chartTable(t, m)29+ m.runCommand("insert.chart")30+ press(t, m, "<enter>", "<esc>")31+ if !strings.Contains(screen(m), "█") {32+ t.Fatalf("no bars in the text chart:\n%s", screen(m))33+ }34+ send(m, uv.UnknownCsiEvent("\x1b[?1;0;16S"))35+ if got := sixelOut(m, uv.PrimaryDeviceAttributesEvent{62, 4, 22}); !m.term.sixel || got != "\x1b[16t" {36+ t.Fatalf("sixel not detected, or no cell size asked for: %q", got)37+ }38+ if m.term.registers != 16 {39+ t.Errorf("%d registers, want 16", m.term.registers)40+ }41+ if got := sixelOut(m, uv.CellSizeEvent{Width: 8, Height: 16}); got != "<tick>" {42+ t.Fatalf("after the cell size: %q, want images asked for", got)43+ }44+ return m, &clock45+}46+47+// sixelOut updates m with msg and returns what the commands write to the48+// terminal, with "<clear>" for a screen cleared and "<tick>" for images49+// asked for.50+func sixelOut(m *Model, msg tea.Msg) string {51+ _, cmd := m.Update(msg)52+ m.View()53+ var b strings.Builder54+ for _, msg := range flatten(cmd) {55+ switch msg := msg.(type) {56+ case tea.RawMsg:57+ b.WriteString(msg.Msg.(string))58+ case ticked:59+ b.WriteString("<tick>")60+ default:61+ if fmt.Sprintf("%T", msg) == "tea.clearScreenMsg" {62+ b.WriteString("<clear>")63+ }64+ }65+ }66+ return b.String()67+}68+69+func TestSixelImages(t *testing.T) {70+ m, clock := sixelModel(t)71+ if len(m.term.six.plan) != 1 || strings.Contains(screen(m), "█") {72+ t.Fatalf("plan %v; the plot area isn't blank:\n%s", m.term.six.plan, screen(m))73+ }74+ if !strings.Contains(screen(m), "1,500") {75+ t.Error("the axes aren't text around the image")76+ }77+ // Not drawn until the frame has been still for sixelSettle.78+ if got := sixelOut(m, sixelDrawMsg{}); got != "<tick>" {79+ t.Errorf("drawn over a frame still changing: %q", got)80+ }81+ *clock = clock.Add(sixelSettle)82+ p := m.term.six.plan[0]83+ got := sixelOut(m, sixelDrawMsg{})84+ at := fmt.Sprintf("\x1b7\x1b[%d;%dH\x1bP0;1;0q", p.at.Min.Y+1, p.at.Min.X+1)85+ if !strings.HasPrefix(got, at) || !strings.HasSuffix(got, "\x1b\\\x1b8") {86+ t.Fatalf("image drawn as %.60q, want it at %q with the cursor saved", got, at)87+ }88+ if n := strings.Count(got, ";2;"); n == 0 || n > 16 {89+ t.Errorf("%d colors defined for 16 registers", n)90+ }91+ // Drawn once: a frame that doesn't change draws nothing.92+ if got := sixelOut(m, sixelDrawMsg{}); got != "" {93+ t.Errorf("drawn again over the same frame: %.40q", got)94+ }95+ // A row the image crosses changes: drawn again.96+ press(t, m, "<f5>", fmt.Sprintf("A%d", p.at.Min.Y-gridTop+1), "<enter>")97+ m.View() // as Bubble Tea does after every update98+ *clock = clock.Add(time.Second)99+ if got := sixelOut(m, sixelDrawMsg{}); !strings.HasPrefix(got, at) {100+ t.Errorf("not drawn again after its rows changed: %.40q", got)101+ }102+}103+104+// Scrolling moves the image: the screen is cleared, erasing it, and it is105+// drawn where it went. The command palette over it makes it text.106+func TestSixelMovesAndHides(t *testing.T) {107+ m, clock := sixelModel(t)108+ *clock = clock.Add(time.Second)109+ sixelOut(m, sixelDrawMsg{})110+ was := m.term.six.plan[0].at111+ if got := sixelOut(m, tea.MouseWheelMsg{X: 50, Y: gridTop + 2, Button: tea.MouseWheelDown}); !strings.Contains(got, "<clear>") {112+ t.Fatalf("scrolled without clearing: %q", got)113+ }114+ if len(m.term.six.plan) == 1 && m.term.six.plan[0].at == was {115+ t.Fatalf("the image didn't move: %v", m.term.six.plan)116+ }117+ *clock = clock.Add(time.Second)118+ if got := sixelOut(m, sixelDrawMsg{}); len(m.term.six.plan) == 1 && !strings.Contains(got, "\x1bP") {119+ t.Error("not drawn where it went")120+ }121+ sixelOut(m, tea.MouseWheelMsg{X: 50, Y: gridTop + 2, Button: tea.MouseWheelUp})122+ *clock = clock.Add(time.Second)123+ sixelOut(m, sixelDrawMsg{})124+ if got := sixelOut(m, tea.KeyPressMsg{Code: 'k', Mod: tea.ModCtrl}); len(m.term.six.plan) != 0 || !strings.Contains(got, "<clear>") {125+ t.Errorf("palette open: plan %v, %q", m.term.six.plan, got)126+ }127+}128+129+// Sixel is used only when kitty graphics aren't, never inside tmux, and130+// not with chart-images off.131+func TestSixelDetection(t *testing.T) {132+ m := wideModel()133+ m.term.tmux = false134+ m.Update(uv.PrimaryDeviceAttributesEvent{62, 22})135+ if m.term.sixel {136+ t.Error("sixel without attribute 4")137+ }138+ m.Update(uv.KittyGraphicsEvent{Options: kitty.Options{ID: 31}, Payload: []byte("OK")})139+ m.Update(uv.PrimaryDeviceAttributesEvent{62, 4})140+ if m.term.sixel {141+ t.Error("sixel on a terminal with kitty graphics")142+ }143+ m = wideModel()144+ m.term.tmux = true145+ m.Update(uv.PrimaryDeviceAttributesEvent{62, 4})146+ if m.term.sixel {147+ t.Error("sixel inside tmux")148+ }149+ m, _ = sixelModel(t)150+ m.term.noImages = true151+ if got := sixelOut(m, nil); m.term.sixelOn() || len(m.term.six.plan) != 0 || got != "" {152+ t.Errorf("sixel images with chart-images = false: %q", got)153+ }154+ for in, want := range map[string]int{"\x1b[?1;0;256S": 256, "\x1b[?1;0;1024S": 256, "\x1b[?1;3;0S": 0, "\x1b[?2;0;4S": 0} {155+ if n, _ := parseRegisters(in); n != want {156+ t.Errorf("%q: %d registers, want %d", in, n, want)157+ }158+ }159+}