Automation that feels
like programming
Meet Go command buffers. Composable, testable, type-safe automation that works on local systems, inside containers, and over remote connections.
// Back up a fleet of databases.
var (
ctx, m = context.Background(), sys.Machine()
fsys = command.FS(m)
sem = semaphore.NewWeighted(3) // Limit to 3 concurrent backups.
wg sync.WaitGroup
)
for i := range 10 {
wg.Add(1)
go func() {
defer wg.Done()
sem.Acquire(ctx, 1)
defer sem.Release(1)
db := ssh.Machine(m, fmt.Sprintf("postgres@db%02d.prod", i+1))
_, err := io.Copy(
fs.CreateBuffer(ctx, fsys, fmt.Sprintf("db%02d.sql", i+1)),
command.NewReader(ctx, db, "pg_dumpall"),
)
if err != nil {
log.Printf("❌ db #%02d: %v", i+1, err)
} else {
log.Printf("✅ db #%02d: backup successful", i+1)
}
}()
}
wg.Wait()
// Generate a Jekyll site without installing Ruby locally.
ctx, m := context.Background(), sys.Machine()
jekyll := ctr.Machine(m, "jekyll/jekyll:latest")
defer command.Shutdown(ctx, jekyll)
sh, rb := command.Shell(m), command.Shell(jekyll, "jekyll")
// Copy source files in.
_, err := io.Copy(
rb.CreateBuffer(ctx, "/srv/jekyll/"),
sh.OpenBuffer(ctx, "."),
)
if err != nil {
log.Fatal(err)
}
if err := rb.Exec(ctx, "jekyll", "build"); err != nil {
log.Fatal(err)
}
// Copy generated files out.
_, err = io.Copy(
sh.CreateBuffer(ctx, "public/"),
rb.OpenBuffer(ctx, "/srv/jekyll/_site/"),
)
if err != nil {
log.Fatal(err)
}
log.Println("✅ Site built and extracted")
// Build a Go application with version stamps.
ctx, sh := context.Background(), command.Shell(sys.Machine(), "go")
if err := sh.Exec(ctx, "go", "mod", "tidy"); err != nil {
log.Fatalf("go mod tidy failed: %v", err)
}
if err := sh.Exec(ctx, "go", "test", "./..."); err != nil {
log.Fatalf("tests failed: %v", err)
}
ver, err := sh.ReadFile(ctx, "VERSION")
if err != nil {
ver = []byte("dev")
}
if err := sh.MkdirAll(ctx, "bin"); err != nil {
log.Fatalf("failed to create bin directory: %v", err)
}
err = sh.Exec(
command.WithEnv(ctx, map[string]string{"CGO_ENABLED": "0"}),
"go", "build",
"-ldflags", fmt.Sprintf(
"-X main.version=%s", strings.TrimSpace(string(ver)),
),
"-o", "bin/app", ".",
)
if err != nil {
log.Fatalf("build failed: %v", err)
}
info, err := sh.Stat(ctx, "bin/app")
if err != nil {
log.Fatalf("binary not found: %v", err)
}
fmt.Printf("Built %s (%d bytes)\n", info.Name(), info.Size())
// Release multi-platform binaries to GitHub.
ctx, sh := context.Background(), command.Shell(sys.Machine(), "git", "gh", "go")
tag, err := sh.Read(ctx, "git", "describe", "--tags", "--abbrev=0")
if err != nil {
log.Fatal(err)
}
prev, err := strconv.Atoi(strings.TrimPrefix(tag, "v"))
if err != nil {
log.Fatal(err)
}
version := fmt.Sprintf("v%d", prev+1)
if err := sh.Exec(ctx, "git", "tag", version); err != nil {
log.Fatal(err)
}
if err := sh.Exec(ctx, "git", "push", "origin", version); err != nil {
log.Fatal(err)
}
platforms := []struct{ os, arch string }{
{"linux", "amd64"},
{"linux", "arm64"},
{"darwin", "amd64"},
{"darwin", "arm64"},
{"windows", "amd64"},
}
release := []string{"gh", "release", "create", version, "--title", version}
for _, p := range platforms {
bin := fmt.Sprintf("bin/app-%s-%s", p.os, p.arch)
err := sh.Exec(
command.WithEnv(ctx, map[string]string{
"GOOS": p.os, "GOARCH": p.arch, "CGO_ENABLED": "0",
}),
"go", "build", "-o", bin, ".",
)
if err != nil {
log.Fatalf("%s/%s build failed: %v", p.os, p.arch, err)
}
release = append(release, bin)
}
if err := sh.Exec(ctx, release...); err != nil {
log.Fatal(err)
}
log.Printf("✅ Released %s", version)
Builds Deserve Better
Shell Scripts
Portable in theory. Full of sharp edges in practice. Code organization, modularity, and dependency management are challenging at best.
YAML Config
Branching, conditionals, and loops in configuration languages are awkward and frustrating. And how do you test YAML?
Polyglot Solutions
Writing automation in every language sounds good on paper, but makes common libraries and deep expertise impossible.
If your code needs maintenance and deployment, your team will need to learn a language to write that in. Even if that language is configuration - with enough time, all configuration systems become bad programming languages.
So instead, start with a programming language that was designed to be one. Start with Go.
With Go, you get:
- Backwards compatibility. Forever.
The
go1compatpromise means your automation will be runnable for years to come. - Dependency heaven. Go's module system is easy to use and produces a flat dependency structure.
- Batteries included. A formatter, linter, and test framework make it easy to work with code.
- Lightweight types. Enough to prevent trivial errors, yet not so much as to allow for complex hierarchies.
- Simplicity. Built from the ground up to be productive, anyone can learn Go in an afternoon.
- First-class concurrency. Goroutines permit natural concurrency without async/await or specialized functions.
Command buffers make running commands in Go as natural and as composable as shell scripting.
Commands as io.Reader
Reading a command buffer executes it. Commands start on first Read and complete at EOF. They can be read from, written to, or piped between.
Standard io
Buffers expose standard output as an io.Reader
and standard input as an io.WriteCloser.
Error values capture logs from standard error.
io.Copy pipes commands (and more!)
from one buffer to another.
const deployment = `apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
spec:
replicas: 3
template:
spec:
containers:
- name: app
image: myapp:latest`
sh := command.Shell(sys.Machine(), "kubectl")
_, err := io.Copy(
sh.NewWriter(ctx, "kubectl", "apply", "-f", "-"),
strings.NewReader(deployment),
)
One Machine Interface, Multiple Backends
A command.Machine is anything that can execute commands. Same
code, different execution context.
Local System
m := sys.Machine()
Container
m := ctr.Machine(sys.Machine(), "ubuntu:latest")
Remote SSH
m := ssh.Machine(sys.Machine(), "user@host.com")
BusyBox
m := sub.Machine(sys.Machine(), "busybox")
Switch contexts without changing code
m := sys.Machine()
if *useContainer {
m = ctr.Machine(m, "golang:latest")
defer command.Shutdown(ctx, m)
}
local := command.Shell(sys.Machine(), "git")
build := command.Shell(m, "go")
tmp, err := fs.Temp(ctx, build, "workdir/")
if err != nil {
log.Fatal(err)
}
defer build.RemoveAll(ctx, tmp.Path())
// Test using only the committed code.
_, err = io.Copy(tmp, local.NewReader(ctx, "git", "archive", "HEAD"))
if err != nil {
log.Fatal(err)
}
tmp.Close()
ctx = command.WithDir(ctx, tmp.Path())
if err := build.Exec(ctx, "go", "test", "./..."); err != nil {
log.Fatal(err)
}
Declarative Shell, Imperative Filling
command.Shell wraps a Machine with a declarative layer: you
declare which commands you need upfront. The rest is pure imperative control.
sh := command.Shell(sys.Machine(), "go", "git")
if err := sh.Exec(ctx, "go", "mod", "tidy"); err != nil {
log.Fatal(err)
}
if err := sh.Exec(ctx, "go", "test", "./..."); err != nil {
log.Fatal(err)
}
// "tee" would fail here. It's not a registered command.
// Use ergonomic shell methods to manage files instead.
if err := sh.WriteFile(ctx, ".gitignore", []byte("*.test\n")); err != nil {
log.Fatal(err)
}
Self-documenting
The list of required commands is explicit and visible.
Catch errors early
Know immediately if a command isn't available rather than discovering it halfway through.
Portable by default
Encourages using portable abstractions (like fs.ReadFile) instead of shell
commands.
Unified Filesystem Access
Work with files using the same patterns, whether local or remote.
const config = `port: 8080
database: postgres://prod-db:5432/app`
local := command.Shell(sys.Machine())
remote := command.Shell(
ssh.Machine(sys.Machine(), "deploy@prod.example.com"),
)
if err := remote.MkdirAll(ctx, "/opt/myapp"); err != nil {
log.Fatal(err)
}
_, err = io.Copy(
remote.CreateBuffer(ctx, "/opt/myapp/server"),
local.OpenBuffer(ctx, "./bin/server"),
)
if err != nil {
log.Fatal(err)
}
err = remote.WriteFile(ctx, "/opt/myapp/config.yaml", []byte(config))
if err != nil {
log.Fatal(err)
}
The lesiw.io/fs package extends
Go's fs.FS with context-aware operations. Files open lazily on first Read or
Write, making them composable with command pipelines.
Built for Testing
Mock machines make automation code testable. Program responses, verify calls, run tests without touching the real system.
func TestDeploy(t *testing.T) {
ctx, m := t.Context(), new(mock.Machine)
m.Return(strings.NewReader("main"), "git", "branch", "--show-current")
sh := command.Shell(m, "git")
if err := deploy(ctx, sh); err != nil {
t.Fatal(err)
}
calls := mock.Calls(sh, "git")
want := []mock.Call{
{Args: []string{"git", "branch", "--show-current"}},
{Args: []string{"git", "push", "origin", "main"}},
}
if !cmp.Equal(want, calls) {
t.Errorf("mismatch (-want +got):\n%s", cmp.Diff(want, calls))
}
}
Getting Started
go get lesiw.io/command
Quick Example
package main
import (
"context"
"fmt"
"log"
"lesiw.io/command"
"lesiw.io/command/sys"
)
func main() {
ctx, sh := context.Background(), command.Shell(sys.Machine(), "go")
if err := sh.Exec(ctx, "go", "version"); err != nil {
log.Fatal(err)
}
version, err := sh.Read(ctx, "go", "version")
if err != nil {
log.Fatal(err)
}
fmt.Println("Go version:", version)
}