Charts recorded by VHS showed axes around an empty plot. VHS runs ttyd with enableSixel=true, so xterm.js's image addon answers DA1 with 4 (?62;4;9;22c; there is no reply to the kitty graphics query) and 012 drew the plot as a sixel image over blank cells. xterm.js does show the image, on its xterm-image-layer canvas, but VHS records only the xterm-text-layer and xterm-cursor-layer canvases, so the recording had nothing in the plot. The same blank plot would show in any terminal that lists sixel in DA1 without drawing it. The cells under a sixel image now hold the text chart, and the image sets every pixel (the background composited in) so the text never shows through where it is drawn. Terminals that show the image look as before; those that don't, and recordings of the text layer, get the text chart. The demos no longer turn chart images off; charts.gif and nu-recipe.gif are rendered again.
- commit
- e893e7b620f74cb5bf975f87b7b37760ca5d1b3b
- parent
- 415bbd4a3b293cb95e93c5ee0a11df4117c86306
- author
- Dami <dami.etoile@gmail.com>
- kind
- fix risk score 1.9 of 3, Jev's estimate
| M | Makefile | 2 | +- |
| M | demos/ | 4 | +- |
| M | docs/ | 6 | +++- |
| M | docs/ | bin | |
| M | docs/ | bin | |
| M | docs/ | 21 | +++++++++----- |
| M | internal/ | 44 | ++++++++++++------------------ |
| M | internal/ | 15 | +++++----- |
| M | internal/ | 14 | +++------ |
| M | internal/ | 3 | +- |
| M | internal/ | 16 | +++++----- |
| M | internal/ | 6 | +++- |
@@ -116,7 +116,7 @@ demos: build
116116 mkdir -p demos/out/stills117117 bin/fakejev -addr $(FAKEJEV_ADDR) & pid=$$!; trap "kill $$pid" EXIT; \118118 export DEMOS_TTYD="$$(command -v ttyd)" PATH="$(CURDIR)/demos/lib:$$PATH"; \119- export XDG_CONFIG_HOME="$$(mktemp -d)" O12_JEV_CREDENTIAL_STORE=false O12_CHART_IMAGES=false O12_THEME= O12_LOCALE=en-US; \119+ export XDG_CONFIG_HOME="$$(mktemp -d)" O12_JEV_CREDENTIAL_STORE=false O12_THEME= O12_LOCALE=en-US; \120120 cd demos && for d in $(DEMOS); do echo "vhs $$d.tape"; vhs -q $$d.tape || exit 1; done121121 mkdir -p demos/out/media122122 for d in $(DEMOS); do ffmpeg -v error -y -i demos/out/$$d.gif -filter_complex \@@ -1,7 +1,7 @@
11 # Charts: Insert > Chart (here from the palette) charts the table22 # around the active cell; the editor bar switches the type with3-# Left/Right, and the chart redraws as its data changes. Terminals with the kitty graphics protocol get a real4-# image; this recording shows the text rendering every terminal gets.3+# Left/Right, and the chart redraws as its data changes. Terminals with kitty graphics or sixel get a real4+# image; VHS records only the text layer, so this shows the text chart under it.55 Output out/charts.gif66 Source lib/setup.tape77 @@ -101,8 +101,10 @@ so Alt+letter reaches the app. VHS has no mouse commands, so the tapes
101101 use the keyboard, and they reach commands through the palette (Ctrl+K)102102 rather than counting menu items, so a new menu item doesn't break them.103103 Tapes that change settings set them for the run (`O12_KEYMAP=vim`)104-rather than writing the config file the other tapes share, and charts105-are drawn as text (`O12_CHART_IMAGES=false`), since VHS records text.104+rather than writing the config file the other tapes share. VHS's ttyd105+has sixel on, so 012 draws charts as sixel images, but VHS records only106+xterm.js's text layer: the recordings show the text chart each image107+covers ([Charts](../sheets/charts.md)).106108 The nushell tapes need `nu`: `demos/lib/nu.tape` starts a session107109 without your config files, with a fixed prompt and 012 on the PATH, and108110 their data is in `demos/data`, so each recording comes out the same.Binary file.
Binary file.
@@ -47,11 +47,16 @@ kitty graphics protocol. Where it does (kitty, Ghostty, WezTerm) the plot is
4747 a real image, drawn in the terminal's own palette and placed with Unicode4848 placeholders, so redraws never erase it. Otherwise, where the terminal4949 lists sixel graphics among its device attributes (foot, xterm, mlterm,50-Windows Terminal), the plot is a sixel image in the terminal's palette,51-in as many colors as it has color registers. A sixel image is drawn over52-the finished screen, so it shows only where its whole plot area does: a53-chart scrolled partly out of the grid, or under another chart, a menu or54-a dialog, is text until it shows whole again, and the image is drawn55-again after scrolling, resizing or an edit to its data. Elsewhere, and56-inside tmux, the plot is drawn with block and braille characters. Axis57-labels, the legend and the frame are text either way.50+Windows Terminal, and terminals built on xterm.js with its image addon51+on, such as VS Code with `terminal.integrated.enableImages`), the plot is52+a sixel image in the terminal's palette, in as many colors as it has53+color registers. A sixel image is drawn over the finished screen, on top54+of the text chart, which it covers whole: a terminal that lists sixel but55+doesn't show the image, or a recording of the text alone (VHS records56+xterm.js's text layer, not its image layer), shows the text chart. The57+image shows only where its whole plot area does: a chart scrolled partly58+out of the grid, or under another chart, a menu or a dialog, is text59+until it shows whole again, and the image is drawn again after60+scrolling, resizing or an edit to its data. Elsewhere, and inside tmux,61+the plot is drawn with block and braille characters. Axis labels, the62+legend and the frame are text either way.@@ -11,14 +11,14 @@ import (
1111 // Sixel is the other way images reach a terminal, for terminals without1212 // kitty graphics (foot, xterm, mlterm, Windows Terminal and others). A1313 // sixel image is pixels drawn at the cursor, not text: the UI draws it14-// after the frame over blank cells it keeps for it (see the ui package's15-// sixel.go). The image is sent whole each time, in at most as many16-// colors as the terminal has color registers, run-length encoded.14+// after the frame, over the text chart the cells hold (see the ui15+// package's sixel.go), so every pixel is set and the text never shows16+// through. The image is sent whole each time, in at most as many colors17+// as the terminal has color registers, run-length encoded.1718 18-// Sixel returns the DCS sequence that draws img at the cursor: pixels19-// with any coverage composited over bg, fully transparent ones left20-// unset so the cell background shows, in at most registers colors21-// (at least 2, at most 256).19+// Sixel returns the DCS sequence that draws img at the cursor, every20+// pixel composited over bg, in at most registers colors (at least 2, at21+// most 256).2222 func Sixel(img *image.RGBA, bg color.RGBA, registers int) string {2323 registers = min(max(registers, 2), 256)2424 b := img.Bounds()@@ -28,8 +28,9 @@ func Sixel(img *image.RGBA, bg color.RGBA, registers int) string {
2828 }2929 idx, pal := quantize(img, bg, registers)3030 var out strings.Builder31- // P2 = 1: pixels not drawn keep what's under them. The raster32- // attributes set square pixels and the image's size.31+ // P2 = 1: pixels not drawn keep what's under them, which only the32+ // empty sixels past the image's last row are. The raster attributes33+ // set square pixels and the image's size.3334 out.WriteString("\x1bP0;1;0q\"1;1;")3435 out.WriteString(strconv.Itoa(w) + ";" + strconv.Itoa(h))3536 for i, c := range pal {@@ -44,7 +45,7 @@ func Sixel(img *image.RGBA, bg color.RGBA, registers int) string {
4445 // percent is a color channel as sixel's 0 to 100.4546 func percent(v uint8) string { return strconv.Itoa((int(v)*100 + 127) / 255) }4647 47-// sixelBands encodes the image of register indexes (-1 for unset), six48+// sixelBands encodes the image of register indexes, six4849 // rows at a time: for each color in the band, its row of sixels,4950 // run-length encoded, then "$" to go back for the next color, and "-"5051 // between bands.@@ -79,9 +80,6 @@ type bands struct {
7980 func (b *bands) gather(px []int16, y0 int) {8081 b.used = b.used[:0]8182 for i, c := range px {82- if c < 0 {83- continue84- }8583 if b.band[c] != y0+1 {8684 b.band[c] = y0 + 18785 b.used = append(b.used, c)@@ -122,7 +120,7 @@ func writeRuns(out *strings.Builder, row []byte) {
122120 }123121 124122 // quantize maps each pixel of img, composited over bg, to a register,125-// -1 for a fully transparent pixel, and returns the registers' colors.123+// and returns the registers' colors.126124 // Colors are kept exactly while there are at most registers of them;127125 // past that, their low bits are dropped until they fit, and each128126 // register is the average of the colors it stands for. Registers are@@ -131,12 +129,11 @@ func writeRuns(out *strings.Builder, row []byte) {
131129 func quantize(img *image.RGBA, bg color.RGBA, registers int) ([]int16, []color.RGBA) {132130 b := img.Bounds()133131 w, h := b.Dx(), b.Dy()134- px := make([]uint32, w*h) // 0 for transparent, else 1<<24 | RGB132+ px := make([]uint32, w*h) // 1<<24 | RGB135133 for y := range h {136134 for x := range w {137- if c := over(img, b.Min.X+x, b.Min.Y+y, bg); c.A != 0 {138- px[y*w+x] = 1<<24 | uint32(c.R)<<16 | uint32(c.G)<<8 | uint32(c.B)139- }135+ c := over(img, b.Min.X+x, b.Min.Y+y, bg)136+ px[y*w+x] = 1<<24 | uint32(c.R)<<16 | uint32(c.G)<<8 | uint32(c.B)140137 }141138 }142139 shift := uint(0)@@ -150,10 +147,6 @@ func quantize(img *image.RGBA, bg color.RGBA, registers int) ([]int16, []color.R
150147 idx := make([]int16, w*h)151148 last, lastReg := uint32(0), int16(-1) // runs of one color are common: look each up once152149 for i, p := range px {153- if p == 0 {154- idx[i] = -1155- continue156- }157150 r := lastReg158151 if p != last {159152 k := bucket(p, shift)@@ -191,7 +184,7 @@ func distinct(px []uint32, shift uint, limit int) int {
191184 seen := map[uint32]struct{}{}192185 last := uint32(0)193186 for _, p := range px {194- if p == 0 || p == last {187+ if p == last {195188 continue196189 }197190 last = p@@ -204,14 +197,11 @@ func distinct(px []uint32, shift uint, limit int) int {
204197 }205198 206199 // over is the pixel at x, y of the premultiplied img composited over207-// bg, opaque, or transparent where img has no coverage.200+// bg, opaque.208201 func over(img *image.RGBA, x, y int, bg color.RGBA) color.RGBA {209202 i := img.PixOffset(x, y)210203 p := img.Pix[i : i+4 : i+4]211204 a := int(p[3])212- if a == 0 {213- return color.RGBA{}214- }215205 return color.RGBA{mix(p[0], bg.R, a), mix(p[1], bg.G, a), mix(p[2], bg.B, a), 255}216206 }217207 @@ -13,7 +13,8 @@ import (
1313 // order they appear, runs of four or more as "!n", trailing empty1414 // sixels left out, "$" between the colors of a band and "-" between1515 // bands, a pixel with half coverage composited over the background and16-// a transparent one left unset.16+// a transparent one drawn in the background, so text under the image17+// never shows through.1718 func TestSixelGolden(t *testing.T) {1819 img := image.NewRGBA(image.Rect(0, 0, 4, 7))1920 red := color.RGBA{255, 0, 0, 255}@@ -27,9 +28,9 @@ func TestSixelGolden(t *testing.T) {
2728 img.SetRGBA(0, 6, color.RGBA{0, 0, 255, 255})2829 got := Sixel(img, color.RGBA{0, 0, 0, 255}, 256)2930 want := "\x1bP0;1;0q\"1;1;4;7" +30- "#0;2;100;0;0#1;2;0;50;0#2;2;0;0;100" +31- "#0DDD@$#1?A" + // red: rows 0 and 2 (1+4) three times, then row 0 alone32- "-#2@" +31+ "#0;2;100;0;0#1;2;0;0;0#2;2;0;50;0#3;2;0;0;100" +32+ "#0DDD@$#1ywy}$#2?A" + // red: rows 0 and 2 (1+4) three times, then row 0 alone; black around33+ "-#1?@@@$#3@" +3334 "\x1b\\"3435 if got != want {3536 t.Errorf("Sixel =\n%q\nwant\n%q", got, want)@@ -81,8 +82,8 @@ func TestSixelChart(t *testing.T) {
8182 }8283 8384 // A chart's sixel image decodes to the image composited over the84-// background, each channel within sixel's percent steps, and pixels85-// without coverage unset.85+// background, each channel within sixel's percent steps, pixels86+// without coverage included.8687 func TestSixelDecodes(t *testing.T) {8788 bg := color.RGBA{250, 250, 245, 255}8889 for _, ct := range []sheet.ChartType{sheet.ChartLine, sheet.ChartPie, sheet.ChartArea} {@@ -93,7 +94,7 @@ func TestSixelDecodes(t *testing.T) {
9394 for x := range b.Dx() {9495 want := over(img, x, y, bg)9596 c, set := got[image.Pt(x, y)]96- if set != (want.A != 0) || set && (diff(c.R, want.R) > 3 || diff(c.G, want.G) > 3 || diff(c.B, want.B) > 3) {97+ if !set || (diff(c.R, want.R) > 3 || diff(c.G, want.G) > 3 || diff(c.B, want.B) > 3) {9798 t.Fatalf("%v at %d,%d: %v %v, want %v", ct, x, y, c, set, want)9899 }99100 }@@ -228,11 +228,10 @@ func (m *Model) chartBoxes() []overlay.Box {
228228 // border with a resize handle in the corner.229229 func (m *Model) drawChart(i int, c sheet.Chart, selected bool) []string {230230 w, h := chartInner(c)231+ // A sixel image covers the text chart drawn under it (see sixel.go),232+ // so the plot stays text for kitty placeholders only.231233 o := m.term.chartOptions()232234 o.Chart = c.ChartOptions233- if m.term.sixelOn() {234- o.Image = m.term.six.placed(i) // see sixel.go235- }236235 if i >= maxImages {237236 o.Image = false238237 }@@ -251,7 +250,7 @@ func (m *Model) drawChart(i int, c sheet.Chart, selected bool) []string {
251250 lines = append(lines, border.Render("┌─")+m.th.Title.Render(title)+252251 border.Render(strings.Repeat("─", max(c.W-3-ansi.StringWidth(title), 0))+"┐"))253252 for y := range h {254- lines = append(lines, border.Render("│")+" "+chartRow(&m.th, g, y, firstImageID+i, o.Image, m.term.kitty)+" "+border.Render("│"))253+ lines = append(lines, border.Render("│")+" "+chartRow(&m.th, g, y, firstImageID+i, o.Image)+" "+border.Render("│"))255254 }256255 corner := "┘"257256 if selected {@@ -267,9 +266,8 @@ func (m *Model) drawChart(i int, c sheet.Chart, selected bool) []string {
267266 }268267 269268 // chartRow renders row y of a drawn chart with the theme; with images,270-// the plot area is the image's kitty placeholders, or blank cells for a271-// sixel image to cover.272-func chartRow(th *theme.Theme, g *chart.Grid, y, imageID int, image, kitty bool) string {269+// the plot area is the image's kitty placeholders.270+func chartRow(th *theme.Theme, g *chart.Grid, y, imageID int, image bool) string {273271 var b, run strings.Builder274272 var style lipgloss.Style275273 styled := false@@ -290,8 +288,6 @@ func chartRow(th *theme.Theme, g *chart.Grid, y, imageID int, image, kitty bool)
290288 key, st, isStyled := 0, lipgloss.Style{}, true291289 text := c.Text292290 switch inPlot := y >= g.Plot.Min.Y && y < g.Plot.Max.Y && x >= g.Plot.Min.X && x < g.Plot.Max.X; {293- case image && inPlot && !kitty:294- key, isStyled, text = 2, false, " "295291 case image && inPlot:296292 key, st = 1, th.ImageID(imageID)297293 text = chart.Placeholder(y-g.Plot.Min.Y, x-g.Plot.Min.X)@@ -24,7 +24,8 @@ import (
2424 // Charts become images when the terminal answers a kitty graphics query2525 // (kitty, Ghostty, WezTerm and others), or else when its primary device2626 // attributes (DA1) list sixel graphics (foot, xterm, mlterm, Windows27-// Terminal; see sixel.go); everywhere else they stay text. Images use27+// Terminal, xterm.js with its image addon; see sixel.go); everywhere28+// else they stay text. Images use2829 // the terminal's own palette, asked for with OSC 4 at startup, so bars2930 // match the text legend.3031 type terminal struct {@@ -24,9 +24,14 @@ import (
2424 //2525 // - A chart is an image only where the whole of its plot area shows: in2626 // the grid, under no chart drawn later and no floating box (a menu,27-// a dialog, formula suggestions, a note). Elsewhere it is text. The28-// frame keeps blank cells where an image goes, so the renderer has29-// nothing to write there.27+// a dialog, formula suggestions, a note). Elsewhere it is text.28+// - Under an image the frame keeps the text chart, which the image29+// covers whole (every pixel is set). The renderer writes those cells30+// only when the chart changes, and then the image is drawn again.31+// DA1 only says a terminal parses sixel: one that lists it without32+// showing images, or a recording of the text alone (VHS records33+// xterm.js's text layer, not its image layer), shows the text chart34+// rather than an empty plot.3035 // - Images are drawn after the frame: a sixelDrawMsg comes once the3136 // frame has been still for sixelSettle, longer than a frame, and3237 // draws each image with the cursor saved and restored around it.@@ -116,11 +121,6 @@ func (s *sixelState) clock() time.Time {
116121 // off.117122 func (t *terminal) sixelOn() bool { return t.sixel && !t.noImages && t.cellW > 0 && t.cellH > 0 }118123 119-// placed reports whether chart i is an image in the current frame.120-func (s *sixelState) placed(i int) bool {121- return slices.ContainsFunc(s.plan, func(p sixelPlace) bool { return p.chart == i })122-}123-124124 // syncSixel follows msg, after Update has handled it: it encodes the125125 // charts that changed, works out where images go, clears the screen when126126 // one drawn moved, changed or went, and asks for the images to be drawn@@ -68,8 +68,10 @@ func sixelOut(m *Model, msg tea.Msg) string {
6868 6969 func TestSixelImages(t *testing.T) {7070 m, clock := sixelModel(t)71- if len(m.term.six.plan) != 1 || strings.Contains(screen(m), "█") {72- t.Fatalf("plan %v; the plot area isn't blank:\n%s", m.term.six.plan, screen(m))71+ // The text chart stays under the image, for a terminal that says it72+ // has sixel but doesn't show it, or a recording of the text alone.73+ if len(m.term.six.plan) != 1 || !strings.Contains(screen(m), "█") {74+ t.Fatalf("plan %v; no text chart under the image:\n%s", m.term.six.plan, screen(m))7375 }7476 if !strings.Contains(screen(m), "1,500") {7577 t.Error("the axes aren't text around the image")