Commands as byte streams.

The model

• A Buffer is a command's execution as an io.Reader: the command starts on the first Read and is finished at io.EOF.
• A Machine is anything that can run a command — one interface, one method: Command(ctx, args...) Buffer.
• Machines wrap machines: ssh, ctr, and sub each take a Machine and return a new one, so a container on a remote host is just two calls.
• Files are buffers too: command.FS gives any Machine a filesystem, and copying between machines is io.Copy. Directories stream as tar archives.
• A Shell makes automation portable: declare the external commands you truly need, and everything else — files, directories, OS detection — goes through operations that work on any machine.

Every example on this page compiles and runs against the current releases of lesiw.io/command and lesiw.io/fs.

Reading runs the command

There is none of os/exec's ceremony — no Start, no Wait, no pipe plumbing. A buffer executes because something consumes it, the same way a shell pipeline only moves when something reads the pipe.

package main

import (
    "context"
    "fmt"
    "io"
    "log"

    "lesiw.io/command"
    "lesiw.io/command/mem"
)

func main() {
    ctx, m := context.Background(), mem.Machine()

    out, err := io.ReadAll(command.NewReader(ctx, m, "echo", "Hello, world!"))
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("%s", out)
}

Run this program in the Go Playground — mem.Machine is an in-memory machine, so the example works anywhere.

Because a pipe is just bytes moving from a reader to a writer, piping one command into another is io.Copy:

// echo "hello, pipes" | tee hello.txt
_, err := io.Copy(
    command.NewWriter(ctx, m, "tee", "hello.txt"),
    command.NewReader(ctx, m, "echo", "hello, pipes"),
)

Run this program in the Go Playground. For pipelines of three or more commands, use command.Copy with command.NewFilter for the middle stages.

Three helpers cover the everyday cases, so most code never touches a buffer directly:

// version=$(go version)     — capture output, like command substitution.
version, err := command.Read(ctx, m, "go", "version")

// go test ./... >/dev/null  — run for the side effect, discard output.
err = command.Do(ctx, m, "go", "test", "./...")

// go vet ./...              — attach to the terminal, stream output live.
err = command.Exec(ctx, m, "go", "vet", "./...")

Environment variables travel on the context, so they compose the way the rest of Go does:

ctx = command.WithEnv(ctx, map[string]string{"CGO_ENABLED": "0"})
err := command.Exec(ctx, m, "go", "build", ".")

Anything that runs a command is a machine

The interface has one method. Everything else in the package is built on it.

type Machine interface {
    Command(ctx context.Context, arg ...string) Buffer
}

m := sys.Machine()

m := ssh.Machine(sys.Machine(),
    "ssh", "user@host")

m := ctr.Machine(sys.Machine(),
    "alpine:latest")

m := sub.Machine(sys.Machine(),
    "busybox")

m := mem.Machine()

m := new(mock.Machine)

Machines take machines. ssh.Machine needs somewhere to run ssh; ctr.Machine needs somewhere to run docker. That somewhere is another Machine — so environments nest by construction:

// A container on a remote build host.
host := ssh.Machine(sys.Machine(), "ssh", "admin@build.example.com")
m := ctr.Machine(host, "golang:latest")
defer command.Shutdown(ctx, m)

sh := command.Shell(m, "go")
if err := sh.Exec(ctx, "go", "test", "./..."); err != nil {
    log.Fatal(err)
}

A prefix glued onto a command — ssh host …, docker exec …, env FOO=bar … — is a machine that hasn't been named yet. Once it is one, the code that runs commands no longer depends on where they run: the same program works on a laptop, in a container, and on a production host.

A machine doesn't have to be a computer. It can be a single command, enriched — here, a shim that injects an environment variable into every go invocation:

m := command.HandleFunc(sys.Machine(), "go",
    func(ctx context.Context, args ...string) command.Buffer {
        ctx = command.WithEnv(ctx, map[string]string{
            "GOFLAGS": "-trimpath",
        })
        return sys.Machine().Command(ctx, args...)
    })

flags, err := command.Read(ctx, m, "go", "env", "GOFLAGS")
// flags == "-trimpath"

Files are buffers too

command.FS turns any machine into a filesystem, accessed through lesiw.io/fs — a context-aware cousin of io/fs with write support. On machines without native filesystem access, file operations are implemented with the commands the target system has: tee on Unix, Remove-Item on Windows. Your code doesn't see the difference.

fsys := command.FS(m)

err := fs.WriteFile(ctx, fsys, "greeting.txt", []byte("Hello from fs!\n"))

data, err := fs.ReadFile(ctx, fsys, "greeting.txt")

Run this program in the Go Playground.

Copying a file between machines is the same io.Copy as piping commands. The operations scp, docker cp, and cp share one implementation:

local := command.Shell(sys.Machine())
remote := command.Shell(ssh.Machine(sys.Machine(), "ssh", "deploy@prod.example.com"))

// Stream a local build to the remote host.
_, err := io.Copy(
    remote.CreateBuffer(ctx, "/opt/app/server"),
    local.OpenBuffer(ctx, "bin/server"),
)

A directory is a buffer too: a tar archive, indicated by a trailing slash. Copying a directory into a container looks exactly like copying a file:

box := ctr.Machine(sys.Machine(), "alpine:latest")
defer command.Shutdown(ctx, box)

local, remote := command.Shell(sys.Machine()), command.Shell(box)

dst, err := remote.Create(ctx, "/app/")   // trailing slash: a directory
if err != nil {
    log.Fatal(err)
}
defer dst.Close()
src, err := local.Open(ctx, "src/")
if err != nil {
    log.Fatal(err)
}
defer src.Close()
if _, err := io.Copy(dst, src); err != nil {
    log.Fatal(err)
}

Because commands and files are the same shape, they mix in one pipeline. This example dumps a database, compresses it in flight, and lands it on another machine, with no intermediate files:

backup := ssh.Machine(m, "ssh", "backup@vault.example.com")
_, err := command.Copy(
    command.NewWriter(ctx, backup, "tee", "db.sql.gz"),
    command.NewReader(ctx, m, "pg_dumpall"),
    command.NewFilter(ctx, m, "gzip"),
)

Shells make automation portable

command.Shell exists to keep automation portable. It wraps a machine with two things: operations that work on any machine for everything a filesystem can do, and an explicit list of the external commands your automation is allowed to run. Anything you didn't declare fails with command not found — the dependency list lives at the top of the file, and it stays short because most of what shell scripts shell out for doesn't need a command at all.

sh := command.Shell(sys.Machine(), "go")

if err := sh.Exec(ctx, "go", "vet", "./..."); err != nil {
    log.Fatal(err)
}

ver, err := sh.ReadFile(ctx, "VERSION")
if err != nil {
    ver = []byte("dev")
}

if err := sh.MkdirAll(ctx, "bin"); err != nil {
    log.Fatal(err)
}
err = sh.Exec(
    command.WithEnv(ctx, map[string]string{"CGO_ENABLED": "0"}),
    "go", "build", "-ldflags", "-X main.version="+string(ver),
    "-o", "bin/app", ".",
)

The build declares only go. Reading VERSION and creating the output directory go through the Shell's portable methods, which work on any machine — including Windows, where cat and mkdir -p do not exist.

Automation you can test

Automation written against Machine is code, so it tests like code. mock.Machine records every call and returns whatever you program:

func Deploy(ctx context.Context, sh *command.Sh) error {
    branch, err := sh.Read(ctx, "git", "branch", "--show-current")
    if err != nil {
        return fmt.Errorf("read branch: %w", err)
    }
    return sh.Exec(ctx, "git", "push", "origin", branch)
}

func TestDeploy(t *testing.T) {
    m := new(mock.Machine)
    m.Return(strings.NewReader("main\n"), "git", "branch", "--show-current")

    sh := command.Shell(m, "git")
    if err := Deploy(t.Context(), sh); err != nil {
        t.Fatal(err)
    }

    got := mock.Calls(sh, "git")
    want := []mock.Call{
        {Args: []string{"git", "branch", "--show-current"}},
        {Args: []string{"git", "push", "origin", "main"}},
    }
    if !cmp.Equal(want, got) {
        t.Errorf("git calls mismatch (-want +got):\n%s", cmp.Diff(want, got))
    }
}

No git repository, no network, no side effects — and the test asserts the exact commands your automation would have run in production.

Why Go for automation

Every configuration language grows conditionals, then loops, then modules, until it has become a programming language — one without a debugger, a formatter, or a test framework. Starting with a real language skips that trajectory, and Go is a good fit for this job:

• It keeps working. The Go 1 compatibility promise means automation written today still builds years from now.
• Dependencies are solved. Modules give automation code the same sharing and versioning as any other code.
• Concurrency is ordinary. Backing up one host and backing up a hundred are the same code plus a goroutine per host.

Command buffers are not a replacement for a five-line shell script. They're for the automation that outgrew one — the build that needs a real conditional, the deploy that spans three machines, the script someone finally asked you to write a test for.

Get started

go get lesiw.io/command

A complete program:

package main

import (
    "context"
    "fmt"
    "log"

    "lesiw.io/command"
    "lesiw.io/command/sys"
)

func main() {
    ctx, m := context.Background(), sys.Machine()

    version, err := command.Read(ctx, m, "go", "version")
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(version)
}

Keep reading

• lesiw.io/command — full API documentation, including a cookbook of shell idioms translated to Go.
• lesiw.io/fs — the filesystem abstraction: local, remote, and in-memory filesystems behind one interface.
• The cmdbuf guide — a condensed, example-first reference for writing idiomatic command buffer code. Written with coding agents in mind; humans welcome too.
• Source on GitHub.