Commands as byte streams.
A command buffer is a command as an io.Reader:
reading it runs it. Piping is io.Copy.
And anywhere a command can run — your own system, a
container, a host across the network — is the same
one-method Machine interface.
// Stream a database dump from a remote host into a local file.
db := ssh.Machine(sys.Machine(), "ssh", "postgres@db1.example.com")
_, err := io.Copy(
fs.CreateBuffer(ctx, command.FS(sys.Machine()), "backup.sql"),
command.NewReader(ctx, db, "pg_dumpall"),
)
No temp files, no scp, no shell quoting. Bytes stream from
pg_dumpall on the database host straight into
a local file. Swapping the source or destination machine is a
one-line change.
The model
- A Buffer is a command's execution as an
io.Reader: the command starts on the firstReadand is finished atio.EOF. - A Machine is anything that can run a
command — one interface, one method:
Command(ctx, args...) Buffer. - Machines wrap machines:
ssh,ctr, andsubeach take a Machine and return a new one, so a container on a remote host is just two calls. - Files are buffers too:
command.FSgives any Machine a filesystem, and copying between machines isio.Copy. Directories stream as tar archives. - A Shell makes automation portable: declare the external commands you truly need, and everything else — files, directories, OS detection — goes through operations that work on any machine.
Every example on this page compiles and runs against the current releases of lesiw.io/command and lesiw.io/fs.
Reading runs the command
There is none of os/exec's ceremony — no
Start, no Wait, no pipe plumbing.
A buffer executes because something consumes it, the same
way a shell pipeline only moves when something reads the pipe.
package main
import (
"context"
"fmt"
"io"
"log"
"lesiw.io/command"
"lesiw.io/command/mem"
)
func main() {
ctx, m := context.Background(), mem.Machine()
out, err := io.ReadAll(command.NewReader(ctx, m, "echo", "Hello, world!"))
if err != nil {
log.Fatal(err)
}
fmt.Printf("%s", out)
}
Run this program in the Go Playground
— mem.Machine is an in-memory machine, so the
example works anywhere.
Because a pipe is just bytes moving from a reader to a
writer, piping one command into another is
io.Copy:
// echo "hello, pipes" | tee hello.txt
_, err := io.Copy(
command.NewWriter(ctx, m, "tee", "hello.txt"),
command.NewReader(ctx, m, "echo", "hello, pipes"),
)
Run this program in the Go Playground.
For pipelines of three or more commands, use
command.Copy
with command.NewFilter for the middle stages.
Three helpers cover the everyday cases, so most code never touches a buffer directly:
// version=$(go version) — capture output, like command substitution.
version, err := command.Read(ctx, m, "go", "version")
// go test ./... >/dev/null — run for the side effect, discard output.
err = command.Do(ctx, m, "go", "test", "./...")
// go vet ./... — attach to the terminal, stream output live.
err = command.Exec(ctx, m, "go", "vet", "./...")
Environment variables travel on the context, so they compose the way the rest of Go does:
ctx = command.WithEnv(ctx, map[string]string{"CGO_ENABLED": "0"})
err := command.Exec(ctx, m, "go", "build", ".")
Anything that runs a command is a machine
The interface has one method. Everything else in the package is built on it.
type Machine interface {
Command(ctx context.Context, arg ...string) Buffer
}
Local system
m := sys.Machine()
Runs commands with os/exec.
Remote host
m := ssh.Machine(sys.Machine(),
"ssh", "user@host")
Any ssh command line works: ports, keys, sshpass, autossh, jump hosts.
Container
m := ctr.Machine(sys.Machine(),
"alpine:latest")
Docker, Podman, or nerdctl — detected automatically. Starts on first use.
Command prefix
m := sub.Machine(sys.Machine(),
"busybox")
Prefixes every command: an applet runner, a
kubectl exec, a wrapper CLI.
In-memory
m := mem.Machine()
Real implementations of echo, cat, tee, and tr over an in-memory filesystem. Runs in the Go Playground.
Mock
m := new(mock.Machine)
Programmable responses and call recording for tests.
Machines take machines. ssh.Machine needs
somewhere to run ssh; ctr.Machine
needs somewhere to run docker. That somewhere
is another Machine — so environments nest by construction:
// A container on a remote build host.
host := ssh.Machine(sys.Machine(), "ssh", "admin@build.example.com")
m := ctr.Machine(host, "golang:latest")
defer command.Shutdown(ctx, m)
sh := command.Shell(m, "go")
if err := sh.Exec(ctx, "go", "test", "./..."); err != nil {
log.Fatal(err)
}
Machine.
A prefix glued onto a command —
ssh host …,
docker exec …,
env FOO=bar … — is a machine that
hasn't been named yet. Once it is one, the code that runs
commands no longer depends on where they run: the same
program works on a laptop, in a container, and on a
production host.
A machine doesn't have to be a computer. It can be a single
command, enriched — here, a shim that injects an environment
variable into every go invocation:
m := command.HandleFunc(sys.Machine(), "go",
func(ctx context.Context, args ...string) command.Buffer {
ctx = command.WithEnv(ctx, map[string]string{
"GOFLAGS": "-trimpath",
})
return sys.Machine().Command(ctx, args...)
})
flags, err := command.Read(ctx, m, "go", "env", "GOFLAGS")
// flags == "-trimpath"
Files are buffers too
command.FS turns any machine into a filesystem,
accessed through
lesiw.io/fs —
a context-aware cousin of io/fs with write
support. On machines without native filesystem access, file
operations are implemented with the commands the target
system has: tee on Unix,
Remove-Item on Windows. Your code doesn't see
the difference.
fsys := command.FS(m)
err := fs.WriteFile(ctx, fsys, "greeting.txt", []byte("Hello from fs!\n"))
data, err := fs.ReadFile(ctx, fsys, "greeting.txt")
Run this program in the Go Playground.
Copying a file between machines is the same
io.Copy as piping commands. The operations
scp, docker cp, and cp share one implementation:
local := command.Shell(sys.Machine())
remote := command.Shell(ssh.Machine(sys.Machine(), "ssh", "deploy@prod.example.com"))
// Stream a local build to the remote host.
_, err := io.Copy(
remote.CreateBuffer(ctx, "/opt/app/server"),
local.OpenBuffer(ctx, "bin/server"),
)
A directory is a buffer too: a tar archive, indicated by a trailing slash. Copying a directory into a container looks exactly like copying a file:
box := ctr.Machine(sys.Machine(), "alpine:latest")
defer command.Shutdown(ctx, box)
local, remote := command.Shell(sys.Machine()), command.Shell(box)
dst, err := remote.Create(ctx, "/app/") // trailing slash: a directory
if err != nil {
log.Fatal(err)
}
defer dst.Close()
src, err := local.Open(ctx, "src/")
if err != nil {
log.Fatal(err)
}
defer src.Close()
if _, err := io.Copy(dst, src); err != nil {
log.Fatal(err)
}
Because commands and files are the same shape, they mix in one pipeline. This example dumps a database, compresses it in flight, and lands it on another machine, with no intermediate files:
backup := ssh.Machine(m, "ssh", "backup@vault.example.com")
_, err := command.Copy(
command.NewWriter(ctx, backup, "tee", "db.sql.gz"),
command.NewReader(ctx, m, "pg_dumpall"),
command.NewFilter(ctx, m, "gzip"),
)
Shells make automation portable
command.Shell exists to keep automation
portable. It wraps a machine with two things: operations
that work on any machine for everything a filesystem can
do, and an explicit list of the external commands your
automation is allowed to run. Anything you didn't declare
fails with command not found — the dependency
list lives at the top of the file, and it stays short
because most of what shell scripts shell out for doesn't
need a command at all.
sh := command.Shell(sys.Machine(), "go")
if err := sh.Exec(ctx, "go", "vet", "./..."); err != nil {
log.Fatal(err)
}
ver, err := sh.ReadFile(ctx, "VERSION")
if err != nil {
ver = []byte("dev")
}
if err := sh.MkdirAll(ctx, "bin"); err != nil {
log.Fatal(err)
}
err = sh.Exec(
command.WithEnv(ctx, map[string]string{"CGO_ENABLED": "0"}),
"go", "build", "-ldflags", "-X main.version="+string(ver),
"-o", "bin/app", ".",
)
The build declares only go. Reading VERSION and
creating the output directory go through the Shell's
portable methods, which work on any machine — including
Windows, where cat and mkdir -p
do not exist.
| Shell idiom | Command buffer equivalent |
|---|---|
cat file | sh.ReadFile(ctx, "file") |
echo x > file | sh.WriteFile(ctx, "file", data) |
mkdir -p dir | sh.MkdirAll(ctx, "dir") |
rm -rf dir | sh.RemoveAll(ctx, "dir") |
mktemp | sh.Temp(ctx, "prefix") |
uname -sm | sh.OS(ctx), sh.Arch(ctx) |
which cmd | command.NotFound(command.Do(ctx, m, "cmd", "--version")) |
tar -cf- dir | sh.Open(ctx, "dir/") |
a | b | io.Copy(command.NewWriter(…), command.NewReader(…)) |
set -x | command.Trace = os.Stderr |
Automation you can test
Automation written against Machine is code, so
it tests like code. mock.Machine records every
call and returns whatever you program:
func Deploy(ctx context.Context, sh *command.Sh) error {
branch, err := sh.Read(ctx, "git", "branch", "--show-current")
if err != nil {
return fmt.Errorf("read branch: %w", err)
}
return sh.Exec(ctx, "git", "push", "origin", branch)
}
func TestDeploy(t *testing.T) {
m := new(mock.Machine)
m.Return(strings.NewReader("main\n"), "git", "branch", "--show-current")
sh := command.Shell(m, "git")
if err := Deploy(t.Context(), sh); err != nil {
t.Fatal(err)
}
got := mock.Calls(sh, "git")
want := []mock.Call{
{Args: []string{"git", "branch", "--show-current"}},
{Args: []string{"git", "push", "origin", "main"}},
}
if !cmp.Equal(want, got) {
t.Errorf("git calls mismatch (-want +got):\n%s", cmp.Diff(want, got))
}
}
No git repository, no network, no side effects — and the test asserts the exact commands your automation would have run in production.
Why Go for automation
Every configuration language grows conditionals, then loops, then modules, until it has become a programming language — one without a debugger, a formatter, or a test framework. Starting with a real language skips that trajectory, and Go is a good fit for this job:
- It keeps working. The Go 1 compatibility promise means automation written today still builds years from now.
- Dependencies are solved. Modules give automation code the same sharing and versioning as any other code.
- Concurrency is ordinary. Backing up one host and backing up a hundred are the same code plus a goroutine per host.
Command buffers are not a replacement for a five-line shell script. They're for the automation that outgrew one — the build that needs a real conditional, the deploy that spans three machines, the script someone finally asked you to write a test for.
Get started
go get lesiw.io/command
A complete program:
package main
import (
"context"
"fmt"
"log"
"lesiw.io/command"
"lesiw.io/command/sys"
)
func main() {
ctx, m := context.Background(), sys.Machine()
version, err := command.Read(ctx, m, "go", "version")
if err != nil {
log.Fatal(err)
}
fmt.Println(version)
}
Keep reading
- lesiw.io/command — full API documentation, including a cookbook of shell idioms translated to Go.
- lesiw.io/fs — the filesystem abstraction: local, remote, and in-memory filesystems behind one interface.
- The cmdbuf guide — a condensed, example-first reference for writing idiomatic command buffer code. Written with coding agents in mind; humans welcome too.
- Source on GitHub.