CLI commands with google/subcommands

How to create CLI programs with multiple commands, flags and subcommands, and how to do it fast and simply?

google/subcommands makes it a breeze.

Let us see how.

The problem scope

First, let us agree on the topics covered here.

Any Go program is — at least in concept — run from a command line, be it an actual shell command line or some automation tool, using the execve(2) system call.

To define the specific words used in this tutorial, let us consider a typical *NIX command:

docker -D container inspect -s some_container

• docker is the program, or application
• -D is a global flag
• container is a root or top-level command
• inspect is a subcommand of the container command; more generally, it is also a command too, but a level 2 one.
• -s is a local flag for the inspect subcommand
• some_container is an argument to the inspect subcommand

In order to interpret that kind of structure, Go code needs to work from a single global variable, os.Args, the first element of which contains the absolute path to the program being run, while the other elements are copies of the arguments passed when starting the program.

To avoid having to redo that kind of very formalized work in every other program, various libraries/modules exist, with the most popular being:

The first three libraries are by far the best known and the richest in terms of functionality, but that comes with multiple dependencies, sometimes quite a lot (3136 for Cobra 1.3). While this is easily justified in humongous projects like Docker or Kubernetes, such a complexity is not as acceptable for typical projects like microservices or small, sharp tools for the CLI.

This is where google/subcommands shines, providing support for most commonly needed command creation tools without requiring any dependency, in a single module around 500 lines only, very easy to use in any application and with a negligible size cost even for the tiniest projects. This tutorial is about how to use it from the simplest to the most advanced scenario.

All along the successive refinement levels in this demo, we are going to be adding features to the same example code, which you can find on https://github.com/fgm/subcommands_demo, with every branch matching the eponymous section in this tutorial, containing 100% of the source code for the level being discussed, as well as a Makefile configured to demonstrate just the specifics of that level.

This tutorial covers 100% of the functionality in google/subcommands, hence its overall length ; you will however be able to create a complete application as early as level 1.2 : later levels describe features meant for projects ever more complex and demanding.

Level 1 : simple commands

We shall start by creating a couple of top-level commands, laying out our code in two directories: the project root and cmd/, with the following main files :

A word of advice: in each branch in the demo repository, the make command runs a different default sequence of tasks, meant to illustrate the changes brought by the current branch: you might wish to use that command on your own machine any time you change branch, to see the impact the code changes actually have.

1.1 Enabling builtin commands

The code for that level is available on level1.1-builtin_commands.

At this step, the code is just made up of two files: the application entry point in main.go:

1
2
3
4
5

func main() {
	ctx := context.Background()
	sts := cmd.Execute(ctx) // Runs the designated command on the CLI
	os.Exit(int(sts))       // Returns the command result to the program caller
}

…and the code triggering execution of the commands, in cmd/root.go :

10
11
12
13
14
15
16
17
18
19
20
21
22

// Execute sets up the command chain and runs it.
func Execute(ctx context.Context) subcommands.ExitStatus {
	for _, command := range [...]subcommands.Command{
		subcommands.CommandsCommand(), // Implement "commands"
		subcommands.FlagsCommand(),    // Implement "flags"
		subcommands.HelpCommand(),     // Implement "help"
	} {
		subcommands.Register(command, "")
	}

	flag.Parse()
	return subcommands.Execute(ctx)
}

The above code registers the three optional commands provided by subcommands proper, which enables listing existing commands and flags, as well as providing CLI help, then it parses global flags and ends up passing control to the actual command selected by the CLI arguments, and returning its result.

Our application scaffold is now ready and supports commands commands, flags, et help.

$ go run . help
Usage: subcommands_demo <flags> <subcommand> <subcommand args>

      Subcommands:
      commands         list all command names
      flags            describe all known top-level flags
      help             describe subcommands and their syntax
$

We can now create our first custom commands.

1.2 Writing custom commands

The code for that level is available on level1.2-custom_commands.

The best practices with custom commands is to have one file per command. This eases code navigation and comparison of commands between each other to ensure consistency of practices. We shall therefore add two files, one for each of the two top-level commands we are adding:

For google/subcommands, a command is provided by a variable, the type of which implements the subcommands.Command interface. Our two files will therefore be very similar to each other, based on the structure in file cmd/top2.go :

15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

type top2 struct{}

func (cmd *top2) Name() string {
	return "top2"
}

func (cmd *top2) Synopsis() string {
	return "top2 is an example top-level custom command with arguments"
}

func (cmd *top2) Usage() string {
	return fmt.Sprintf("%s arg1 arg2 ...", cmd.Name())
}

func (cmd *top2) SetFlags(fs *flag.FlagSet) {}

func (cmd *top2) Execute(_ context.Context, fs *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
	// Notice how the command line arguments are taken on the flag set, not on the variadic.
	fmt.Printf("In %s %v\n", cmd.Name(), fs.Args())
	return subcommands.ExitSuccess
}

For now, top2 just implements subcommands.Command and does nothing more:

• top2.Name() returns the name of the command
• top2.Synopsis returns a single-line summary of the command description
• top2.Usage() returns an example of how to use the command, and may spread over multiple text lines. The returned value is used by the help builtin command.
• top2.SetFlags() defines the local flags on the command: none for now.
• top2.Execute() is the actual command implementation.
  • the context.Context arguments supports passing context values, and using timeout and cancellation capabilities, which will be available to subcommands, just like in a http.Handler.
  • the flag.FlagSet holds the local flag definitions, actual values, and the command arguments.
    • when running that method, the name of the program, the global flags, as well as the command local flags, have all been parsed, and fs.Args() holds the command arguments with no extras.
    • if we run the program as demo top2 hello world, fs.Args() will contain []string{"hello", "world"}.
  • the third parameter is not used in this scenario.

Now that we have created our two commands, we can register all three of them with google/subcommands: top1, top2, and a 1 alias for top1. We can perform this by adding instances of the commands to the subcommands.Register call besides commands, flags and help, in root.go :

11
12
13
14
15
16
17
18
19
20
21

func Execute(ctx context.Context) subcommands.ExitStatus {
	for _, command := range [...]subcommands.Command{
		subcommands.CommandsCommand(),   // Implement "commands"
		subcommands.FlagsCommand(),      // Implement "flags"
		subcommands.HelpCommand(),       // Implement "help"
		&top1{},                         // Our first top-level command, without args
		&top2{},                         // Our second top-level command, with args
		subcommands.Alias("1", &top1{}), // An alias for our top1 command
	} {
		subcommands.Register(command, "")
	}

Once this is done, our program supports these two commands, as well as a "1" alias for command top1.

Many applications will not anything more when it comes to commands handling.

$ go run . commands
commands
flags
help
top1
top2
1
$ go run . top2 hello world
In top2 [hello world]
$

1.3 Passing non-CLI arguments to commands

The code for that level is available on level1.3-non_cli_arguments.

In the previous example, the top2 command takes arguments passed from the CLI.

Command code may however need extra arguments passed to all commands regardless of the CLI arguments, like a logger, an authentication service, etc.

That is where the variadic argument to subcommands.Execute enters into play, taking any number of values of any type, to be received by the Command.Execute() implementations on their variadic parameter. We shall be using it to pass two values :

11
12
13
14
15

func Execute(ctx context.Context) subcommands.ExitStatus {
	for _, command := range [...]subcommands.Command{
		/* ...snip... */
	flag.Parse()
	return subcommands.Execute(ctx, "meaning", 42)

Here they are handled in top1 :

31
32
33
34
35
36

func (cmd *top1) Execute(_ context.Context, _ *flag.FlagSet, args ...interface{}) subcommands.ExitStatus {
	// The variadic arguments are the ones passed to subcommands.Execute().
	// Unlike the CLI args, they are always a []interface{}.
	fmt.Printf("In %s.\nNon-CLI args: %#v\n", cmd.Name(), args)
	return subcommands.ExitSuccess
}

$ go run . top1
In top1.
Non-CLI args: []interface {}{"meaning", 42}
$

The salient issue here is how, as a command author, you need to coordinate the types and order of values passed in that variadic arguments, since they are passed as interface{} (aka any starting with Go 1.18), which will usually mean a type assertion in the Execute method to get a ready-to-use value.

1.4 Grouping commands

The code for that level is available on level1.4-command_groups.

When a project starts to grow and get ever more commands, grouping them by purpose may become useful. This is what the second, string, parameter of subcommands.Register is for. We use it in root.go, to associate each registered command to a given subcommands.Group.

In this example, we shall separate commands provided by google/subcommands, in the help group, from our custom commands, in the top group.

11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

func Execute(ctx context.Context) subcommands.ExitStatus {
	for _, command := range [...]struct {
		group string
		subcommands.Command
	}{
		{"help", subcommands.CommandsCommand()},  // Implement "commands"
		{"help", subcommands.FlagsCommand()},     // Implement "flags"
		{"help", subcommands.HelpCommand()},      // Implement "help"
		{"top", &top1{}},                         // Our first top-level command, without args
		{"top", &top2{}},                         // Our second top-level command, with args
		{"top", subcommands.Alias("1", &top1{})}, // An alias for our top1 command

	} {
		subcommands.Register(command.Command, command.group)
	}

The help command now displays our commands in these two different groups :

$ go run . help
Usage: subcommands_demo <flags> <subcommand> <subcommand args>

Subcommands for help:
        commands         list all command names
        flags            describe all known top-level flags
        help             describe subcommands and their syntax

Subcommands for top:
        top1, 1          top1 is an example top-level custom command without arguments
        top2             top2 is an example top-level custom command with arguments
$

1.5 Adding global or local flags

The code for that level is available on level1.5-flags.

Many programs use a few global flags, like a boolean -v used to increase the verbosity of all commands ; and some also use local flags allowing individual commands to adjust their own behaviour.

Let us start with the typical global boolean -v flag, and implementing it in our two existing commands. We can do this with the stdlib package flag, with which google/subcommands has a hidden integration.

To make sure that flag will be available to all commands and subcommands, we add its value to the context. Following best Go practices, we define an unexported specific context key type, and a variable of that type, in our root.go file :

10
11
12

type verboseKey struct{}

var VerboseKey = verboseKey{}

Now, in our Execute function, we define that the flag, parse the program CLI arguments, and add the parsed flag value to the context using that key :

21
22
23
24
25
26
27
28

func Execute(ctx context.Context) subcommands.ExitStatus {
	for _, command := range [...]struct {
		/* ...snip... */
  }

	verbose := flag.Bool("v", false, "Be more verbose")
	flag.Parse()
	ctx = context.WithValue(ctx, VerboseKey, *verbose)

Commands can now fetch the flag value from the context, as in this example taken from top1.go :

36
37
38
39

func (cmd *top1) Execute(ctx context.Context, _ *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
	if ctx.Value(VerboseKey).(bool) {
		fmt.Printf("In %s.\n", cmd.Name())
	}

Since we defined the flag type as boolean, and the key is of a unique unexported type, there can be no collision on the context value even in a large program, which means the type assertion will never fail.

For that global flag, we did not refer to google/subcommands at all, and it is correctly recognized by builtin command flags, but does not appear in the builtin help provided by the help command :

$ go run . help
Usage: subcommands_demo <flags> <subcommand> <subcommand args>

Subcommands for help:
        commands         list all command names
        flags            describe all known top-level flags
        help             describe subcommands and their syntax

Subcommands for top:
        top1, 1          top1 is an example top-level custom command without arguments
        top2             top2 is an example top-level custom command with arguments


Use "subcommands_demo flags" for a list of top-level flags
$ go run . flags
  -v    Be more verbose
$

Now, let us define a local flag on command top1. Since that flag will be specific to the command, it will be listed after the command name in the command line arguments, preventing a simple use of the flag package procedural API like we just did for global flags.

Local flags are defined on each command by implementing their SetFlags method to store the parsed flag value on a command instance field.

To that effect, we shall add a field to the top1 type implementing our command, in the top1.go file, to store the parsed flag value, which will be a string field in this example.

16
17
18

type top1 struct {
	prefix string
}

We then implement the top1.SetFlags method to define that flag and store its parsed value on the command instance :

32
33
34
35
36
37
38
39
40
41
42

func (cmd *top1) SetFlags(fs *flag.FlagSet) {
	fs.StringVar(&cmd.prefix, "prefix", "", "Add a prefix to the result")
}

func (cmd *top1) Execute(ctx context.Context, _ *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
	if ctx.Value(VerboseKey).(bool) {
		fmt.Printf("In %s.\n", cmd.Name())
	}
	fmt.Println(strings.Join(append([]string{cmd.prefix}, "hello"), ": "))
	return subcommands.ExitSuccess
}

The help command is now aware of our local flag :

$ go run . help top1
top1  -prefix string
        Add a prefix to the result
$

When subcommands.Execute prepares to invoke our top1 command instance, it starts by creating a new empty flag.FlagSet from the remaining arguments on the current one, which it passed to the SetFlags method on the command instance. That defines flags pointing to fields on said command instance, so that parsing will field values on it. These fields are thereafter available on the instance when eventually running the top1.Execute method, allowing the method code to access their values as plain typed fields on the method receiver, as demonstrated on line 40 for cmd.prefix.

$ go run . top1 -prefix today
today: hello
$

Another possible mechanism enabled by this process would be for the method implementation to just drop the flag, not pointing it anywhere, then extract the value from the fs *FlagSet argument in top1.Execute, as done below, but that leads to code which is much less readable than the recommended method above.

func (cmd *top1) SetFlags(fs *flag.FlagSet) {
  _ = fs.String("prefix", "", "Add a prefix to the result")
}

func (cmd *top1) Execute(ctx context.Context, fs *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
	prefix := fs.Lookup("prefix").Value.(fmt.Stringer).String()
	fmt.Println(strings.Join(append([]string{cmd.prefix}, "hello"), ": "))
	return subcommands.ExitSuccess
}

1.6 Labeling flags as important

The code for that level is available on level1.6-important_flags.

Some flags may be frequently used, making them worthy of being reported in the help command output.

Let us describe our -v global flag as an important one, and our code also include an unimportant boolean -debug flag. We can inform google/subcommands of that difference when declaring the flags in our root.go file :

21
22
23
24
25
26
27
28
29

func Execute(ctx context.Context) subcommands.ExitStatus {
	for _, command := range [...]struct {
		/* ...snip... */
	}

	debug := flag.Bool("debug", false, "Show debug information")
	verbose := flag.Bool("v", false, "Be more verbose")
	subcommands.ImportantFlag("v")
	flag.Parse()

Without the subcommands.ImportantFlag("v") statement, the help command does not document any top-level flag, as we saw in the level1.5 example above, where the help command mentions that top-level flags exist, but does not list any.

When we add this statement, however, the now important -v flag is included in the help output, while the mundane -debug flag remains unlisted.

$ go run . help
Usage: subcommands_demo <flags> <subcommand> <subcommand args>

Subcommands for help:
        commands         list all command names
        flags            describe all known top-level flags
        help             describe subcommands and their syntax

Subcommands for top:
        top1, 1          top1 is an example top-level custom command without arguments
        top2             top2 is an example top-level custom command with arguments


Top-level flags (use "subcommands_demo flags" for a full list):
  -v=false: Be more verbose
$

Both flags are listed indiscriminately by command flags, since its express purpose is to document them :

$ go run . flags
  -debug
        Show debug information
  -v    Be more verbose
$

With all the features we have now examined, google/subcommands covers the needs of most small applications. Later levels will mostly be relevant for more complexe projects, especially those which require unit testing coverage for commands themselves, instead of just requiring it for the service-level code the commands call upon, which is a more frequent requirement.

Level 2 : Reusing command code

The code for that level is available on level2.1-reuse.

Until this point we have been using one defined type per command, not counting aliases, as suggested by the minimal documentation provided by the google/subcommands repository. Doing so allows storing command-specific properties of commands, like local flags, on the instance of that type which is registered with google/subcommands.

In practice, however, many commands either do not use any flag, or share whichever flags they use ; and repeating identical code for each of these types is not entirely satisfying. But there is no actual need to use different types for different commands, especially if they share the same structure.

The main reason for different types is the Command.Execute method, which has to differ for each command since commands implement different features. Sharing types means using different methods for each command instance.

Various techniques are available for that purpose. For example, using a single top type, its Execute method could look up a property on the command instance to select and invoke a concrete runner function, as in this fragment :

11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

// In top.go
type top struct {
	// ...shared fields
	name string // command name
}

// Add the Name, Synopsis, and SetFlags methods to implement subcommands.Command.

// NewTop replaces both NewTop1 and NewTop2.
func NewTop(name string) {
	return &top{name: name}
}

func (t *top) Execute(ctx context.Context, fs *flag.FlagSet, args ...any) {
	switch t.name {
	case "top1":
		Top1Execute(ctx, fs, args) // A function in top1.go
  case "top2":
	  Top2Execute(ctx, fs, args) // A function in top2.go
	default:
		panic("Unexpected command %s, should never happen", t.name)
	}
}

20
21
22
23
24
25
26
27
28
29
30

// In root.go,
func Execute(ctx context.Context) subcommands.ExitStatus {
	for _, command := range [...]struct {
		group string
		subcommands.Command
	}{
		{"help", subcommands.CommandsCommand()},    // Implement "commands"
		{"help", subcommands.FlagsCommand()},       // Implement "flags"
		{"help", subcommands.HelpCommand()},        // Implement "help"
		{"top", NewTop("top1")},                    // Our first top-level command, without args
		{"top", NewTop("top2")},                    // Our second top-level command, with args

Onw downside with this approach is how it introduces coupling between

• the code in the shared Execute method and the various commands.
• the code in the shared NewTop factory function, which could requires modifications to accommodate the specifics of various commands

A better model is more direct and prevents such coupling. It keeps per-command factory functions, and uses a function field to store the runner function on the instance. That way the dedicated factory functions are used in root.go :

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

func Execute(ctx context.Context) subcommands.ExitStatus {
	for _, command := range [...]struct {
		group string
		subcommands.Command
	}{
		{"help", subcommands.CommandsCommand()},    // Implement "commands"
		{"help", subcommands.FlagsCommand()},       // Implement "flags"
		{"help", subcommands.HelpCommand()},        // Implement "help"
		{"top", NewTop1()},                         // Our first top-level command, without args
		{"top", NewTop2()},                         // Our second top-level command, with args

The top.go contains the reusable top type , with various fields to store :

• the constant values for which the Name, Synopsis and Usage will just be getters;
• the local flags shared by both our commands, in this example prefix, as declared by SetFlags
• a function field storing the runner function for the Execute method.

12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

type top struct {
	name, synopsis, usage string // Reuse support
	prefix                string // Actual features
	run                   func(context.Context, *top, *flag.FlagSet, ...interface{}) subcommands.ExitStatus
}

func (cmd top) Name() string {
	return cmd.name
}

func (cmd top) Synopsis() string {
	return cmd.synopsis
}

func (cmd top) Usage() string {
	return cmd.usage
}

func (cmd *top) SetFlags(fs *flag.FlagSet) {
	fs.StringVar(&cmd.prefix, "prefix", "", "Add a prefix to the result")
}

func (cmd *top) Execute(ctx context.Context, fs *flag.FlagSet, args ...interface{}) subcommands.ExitStatus {
	if cmd.run == nil {
		log.Printf("command %s is not runnable", cmd.name)
		return subcommands.ExitFailure
	}
	return cmd.run(ctx, cmd, fs, args)
}

A limitation of function fields vs methods lies in the fact that they do not receive the instance as a hidden first parameter like methods do, meaning that with the same signature, they cannot access the instance, which Execute needs to access the local flag values. The command instance must therefore be passed as an explicit extra argument, hence the *top parameter in the function signature, on line 15.

To remain as close as possible to a standard method expression, that parameter should be the first, but it would conflict with the Go usage of having context always be the first parameter of any function, hence the placement as the second parameter.

Commands using the shared type, like top2, define a runner function with that signature, which provides the runner with access to the instance as if they were methods; and their factory function assigns that runner to the instance during the instance initialization process, like in this code in top2.go :

12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

func top2Execute(ctx context.Context, cmd *top, fs *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
	if ctx.Value(VerboseKey).(bool) {
		fmt.Printf("In %s.\n", cmd.Name())
	}
	fmt.Println(strings.Join(append([]string{cmd.prefix}, fs.Args()...), ": "))
	return subcommands.ExitSuccess
}

func NewTop2() *top {
	const name = "top2"
	return &top{
		name:     name,
		synopsis: fmt.Sprintf("%s is an exemple top-level custom command with arguments", name),
		usage:    fmt.Sprintf("%s arg1 arg2 ...", name),
		prefix:   "",
		run:      top2Execute,
	}
}

This process removes coupling between commands and the Execute function. An unlimited number of commands can now reuse the shared command type, without having to modify the implementation in top.Execute.

$ go run . top1 -prefix today
today: hello
$ go run . top2 -prefix "the answer is" 42
the answer is: 42
$

Level 3 : Commanders

3.1 Procedural API vs object API

The code for that level is available on level3.1-object_api.

Up until now, we have been using the public functions exported by google/subcommands :

• subcommands.Alias and subcommands.Register to register command instances
• subcommands.ImportantFlag to label flags as global
• subcommands.CommandsCommand, subcommands.FlagsCommand and subcommands.HelpCommand for the builtin help and documentation features

That procedure-based API is actually only a facade simplifying the underlying object API : except for Alias, which does not depend on a Commander, each of these functions is actually a method call to the method with the same name on a default instance of the subcommands.Commander type, held in the exported mutable global variable subcommands.DefaultCommander, which is initialized when importing github.com/google/subcommands by the init pseudo-function, using the subcommands.go fragment below :

458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473

// DefaultCommander is the default commander using flag.CommandLine for flags
// and os.Args[0] for the command name.
var DefaultCommander *Commander

func init() {
	DefaultCommander = NewCommander(flag.CommandLine, path.Base(os.Args[0]))
}

// Register adds a subcommand to the supported subcommands in the
// specified group. (Help output is sorted and arranged by group
// name.)  The empty string is an acceptable group name; such
// subcommands are explained first before named groups. It is a
// wrapper around DefaultCommander.Register.
func Register(cmd Command, group string) {
	DefaultCommander.Register(cmd, group)
}

This is but one case of a model frequently used throughout the Go standard library, for instance by packages flag (flag.CommandLine) or http (http.DefaultClient).

We can switch to that object model by only changing our root.go file, in which we replace every procedural call (except Alias) by a method call on that default Commander instance :

21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

func Execute(ctx context.Context) subcommands.ExitStatus {
	commander := subcommands.DefaultCommander

	for _, command := range [...]struct {
		group string
		subcommands.Command
	}{
		{"help", commander.CommandsCommand()},      // Implement "commands"
		{"help", commander.FlagsCommand()},         // Implement "flags"
		{"help", commander.HelpCommand()},          // Implement "help"
		{"top", NewTop1()},                         // Our first top-level command, without args
		{"top", NewTop2()},                         // Our second top-level command, with args
		{"top", subcommands.Alias("1", NewTop1())}, // An alias for our top1 command

	} {
		commander.Register(command.Command, command.group)
	}

	debug := flag.Bool("debug", false, "Show debug information")
	verbose := flag.Bool("v", false, "Be more verbose")
	commander.ImportantFlag("v")
	flag.Parse()

	ctx = context.WithValue(ctx, DebugKey, *debug)
	ctx = context.WithValue(ctx, VerboseKey, *verbose)

	return commander.Execute(ctx, "meaning")

The program keeps operating in the exact same way as the previous version.

3.2 Using custom commanders

The code for that level is available on level3.2-custom_commander.

Our next step will be to create our own Commander instance, rather that using the shared mutable instance exported by google/subcommands, so we can keep its initialization and mutations under control. We will therefore create it in the Execute function in our root.go file :

22
23
24

func Execute(ctx context.Context) subcommands.ExitStatus {
	// See subcommands.DefaultCommander
	commander := subcommands.NewCommander(flag.CommandLine, os.Args[0])

Nothing else deviates from the previous version, but we are now one step closer to a testable code structure, by removing our dependency on the global subcommands.DefaultCommander.

Time to finish the work and make Execute completely testable by removing all remaining dependencies to other globals.

3.3 Creating a testable command structure

The code for that level is available on level3.3.

Our code is still encumbered by a number of obstacles before it can be unit tested easily :

Once we apply these changes, our latest version of main.go should become the only place where globals are still accessed :

11
12
13
14

func main() {
	sts := cmd.Execute(context.Background(), os.Stdout, os.Stderr, os.Args, log.LstdFlags, cmd.Describe)
	os.Exit(int(sts))
}

We also need to create a Describe factory instantiating our commands in root.go, so that Execute now receives them instead of creating them itself :

22
23
24
25
26
27
28
29
30
31
32
33

type description struct {
  group   string
  command subcommands.Command
}

func Describe(outW io.Writer, logger *log.Logger) []description {
	return []description{
		{"top", NewTop1(outW, logger)},                         // Our first top-level command, without args
		{"top", NewTop2(outW, logger)},                         // Our second top-level command, with args
		{"top", subcommands.Alias("1", NewTop1(outW, logger))}, // An alias for our top1 command
	}
}

…and our new version of Execute now receives all that data it needs by injection instead of referencing imported variables or creating instances itself.

39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87

func Execute(ctx context.Context,
	outW io.Writer, // Standard output for command results
	errW io.Writer, // Error output for logs
	args []string,  // CLI args to avoid depending on the flag global
	logFlags int,   // Log flags to make error message testing easier
	describe func(outW io.Writer, logger *log.Logger) []description, // Command registration descriptions
) subcommands.ExitStatus {
	// Do not depend on log.Default().
	logger := log.New(errW, "", logFlags)

	// Create a flag.FlagSet from args to avoid depending on global os.Args.
	// Continue on error to support testing instead of the ExitOnError on flag.CommandLine
	if len(args) < 1 {
		logger.Printf("Expected at least one argument for the program name, got none")
		return subcommands.ExitFailure
	}
	fs := flag.NewFlagSet(args[0], flag.ContinueOnError)

	// Create a custom commander to avoid depending on global flag.CommandLine and os.Args
	commander := subcommands.NewCommander(fs, args[0])

	descriptions := []description{
		{"help", commander.CommandsCommand()}, // Implement "commands"
		{"help", commander.FlagsCommand()},    // Implement "flags"
		{"help", commander.HelpCommand()},     // Implement "help"
	}
	if describe != nil {
		descriptions = append(descriptions, describe(outW, logger)...)
	}
	for _, command := range descriptions {
		commander.Register(command.command, command.group)
	}

	debug := fs.Bool("debug", false, "Show debug information")
	verbose := fs.Bool("v", false, "Be more verbose")
	commander.ImportantFlag("v")

	// Parse must not receive the program name, hence the slice.
	if err := fs.Parse(args[1:]); err != nil {
		// Our logger has been configured above.
		logger.Printf("Error parsing CLI flags: %v", err)
		return subcommands.ExitUsageError
	}

	ctx = context.WithValue(ctx, DebugKey, *debug)
	ctx = context.WithValue(ctx, VerboseKey, *verbose)

	return commander.Execute(ctx, "meaning", 42)
}

• 40-41: outW and errW are the two io.Writer passed by main().
• 41,47: errW is used to create a log.Logger
• 44: describe is the command description factory, passed by main()
• 55: a new FlagSet is created from received parameters, to break the dependency on os.Args
• 58: a Commander is created from that new FlagSet and received arguments, to break the dependency on flag.CommandLine, and indirect dependency to os.Args
• 66: outW and our new logger are passed to describe so it can pass them into the command factory functions, allowing command instances to work on injected writers instead of global variables.
• 72, 73: global flags are defined on the new FlagSet instead of global flag.CommandLine
• 77: flags parsing is run on the FlagSet instead of global flag.CommandLine

Execute is no longer using any global variable, and passes injected dependencies to the top1 et top2 command instances. We have to modify these too, so that they use the injected writer and logger instead of shared globals. This is how it applies in top1.go :

15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

func top1Execute(ctx context.Context, cmd top, fs *flag.FlagSet, _ ...any) subcommands.ExitStatus {
	if ctx.Value(VerboseKey).(bool) {
		cmd.logger.Printf("In %s.\n", cmd.Name())
	}
	if l := fs.NArg(); l != 0 {
		cmd.logger.Printf("%s expects no arguments, called with %d: %v", cmd.Name(), l, fs.Args())
		return subcommands.ExitFailure
	}
	message := "hello"
	if cmd.prefix != "" {
		message = strings.Join(append([]string{cmd.prefix}, message), ": ")
	}
	fmt.Fprintln(cmd.outW, message)
	return subcommands.ExitSuccess
}

func NewTop1(outW io.Writer, logger *log.Logger) *top {
	const name = "top1"
	return &top{
		logger:   logger,
		name:     name,
		outW:     outW,
		prefix:   "",
		run:      top1Execute,
		synopsis: fmt.Sprintf("%s is an exemple top-level custom command without arguments", name),
		usage:    name,
	}
}

The code in top2 is very similar, and the top type in top.go has been extended as show at lines 34-40 above to store the injected dependencies.

This enables us to establish a 100% test coverage rate very simply, without needing any mocking tool.

$ go test -race -count=1 -cover ./cmd
ok  	github.com/fgm/subcommands_demo/cmd	0.035s	coverage: 100.0% of statements
$

Level 4 : adding nested subcommands

The code for that level is available on level4.1-nesting.

With more complex applications, some commands will need a hierarchy of subcommands, but google/subcommands does not describe how to implement them, leading to a false impression that it does not support them.

That support is actually a consequence of our use of a custom commander created from received parameters passed to subcommands.NewCommander : since flags are now defined on a local FlagSet and arguments received, all it takes to implement a command accepting subcommands is to create a Commander instance from the arguments not consumed after resolving the command itself.

This is how our new top3 works, with its sub31 and sub32 nested commands, to be found in top3.go. Feature-wise, it is similar to top1 and top2, but its runner function top31Execute now includes the subcommands handling logic in addition to the base top3 logic.

23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58

func top3Execute(ctx context.Context, cmd top, topFS *flag.FlagSet, args ...any) subcommands.ExitStatus {
	name := cmd.Name()
	if ctx.Value(VerboseKey).(bool) {
		cmd.logger.Printf("In %s.\n", cmd.Name())
	}
	// Handle command called without subcommands.
	if topFS.NArg() == 0 {
		return top3Internal(ctx, cmd, topFS, args)
	}

	// Create a flag.FlagSet from args to use only remaining args
	// Continue on error to support testing.
	fs := flag.NewFlagSet(cmd.Name(), flag.ContinueOnError)

	// Create a custom commander to restart evaluation below this command.
	commander := subcommands.NewCommander(fs, name)

	descriptions := []description{
		{name, commander.CommandsCommand()}, // Implement "commands"
		{name, commander.FlagsCommand()},    // Implement "flags"
		{name, commander.HelpCommand()},     // Implement "help"
		{name, NewSub31(cmd.outW, cmd.logger)},
		{name, NewSub32(cmd.outW, cmd.logger)},
	}
	for _, command := range descriptions {
		commander.Register(command.command, command.group)
	}

	// Parse must not receive the command name.
	if err := fs.Parse(topFS.Args()); err != nil {
		cmd.logger.Printf("Error parsing %s flags: %v", name, err)
		return subcommands.ExitUsageError
	}

	return commander.Execute(ctx, fs)
}

• near the beginning (lines 29-31), the command checks whether it receives arguments, which could be subcommand names. If there are none, it then transfers control to its own internal logic, which performs the work desired for top3 without a subcommand.
• otherwise, it implements a logic similar to the one in our root Execute function :
  • create a local FlagSet,
  • create a local Commander from that FlagSet and available arguments,
  • register subcommands on the Commander, including builtin commands,
  • parse available arguments for local flags and arguments on the nested commands
  • run the subcommand designated by arguments, if any.

$ go run . top3
hello top3
$ go run . top3 commands
commands
flags
help
sub31
sub32
$ go run . top3 help sub31
sub31  -prefix string
    	Add a prefix to the result
$ go run . top3 sub31 -prefix today
today: hello sub31
$

Level 5 : beyond NewCommander

5.1 Controlling outputs beyond our code

The code for that level is available on level5.1-newcommander.

Although we have injected everything for our own logic to support unit testing, some error outputs remain, from the subcommands and flag packages themselves, in error situations, and can be seen when running tests. We could have foreseen it, because NewCommander does not take any argument for the standard or output error to use by the instance it returns..

$ go test -race -count=1 -v  ./cmd 2>&1 | grep -vE '(CONT|RUN|PASS|PAUSE)'
flag provided but not defined: -bad
Usage: Test_Execute <flags> <subcommand> <subcommand args>

Subcommands for help:

  ...snip (33 lines total)...

flag provided but not defined: -bad
top1  -prefix string
    	Add a prefix to the result
flag provided but not defined: -bad
c  -prefix string
    	Add a prefix to the result
ok  	github.com/fgm/subcommands_demo/cmd	0.036s
$

However, while that redirection is not available at instance creation, it is still available after creation, as we do here in root.go for top-level commands and global flags :

40
41
42
43
44
45
46
47
48
49
50
51
52
53
54

func Execute(ctx context.Context,
	outW io.Writer, // Standard output for command results
	errW io.Writer, // Error output for logs
	args []string, // CLI args to avoid depending on the flag global
	logFlags int, // Log flags to make error message testing easier
	describe func(outW io.Writer, logger *log.Logger) []description, // Command registration descriptions
) subcommands.ExitStatus {
  /* ...snip... */
	fs := flag.NewFlagSet(args[0], flag.ContinueOnError)
	fs.SetOutput(errW)

	// Create a custom commander to avoid depending on global flag.CommandLine and os.Args
	commander := subcommands.NewCommander(fs, args[0])
	commander.Output = outW
	commander.Error = errW

Redirecting the additional properties removes most of the parasitic output we were seeing during tests :

$ go test -race -count=1 -v  ./cmd 2>&1 | grep -vE '(CONT|RUN|PASS|PAUSE)'
flag provided but not defined: -bad
flag provided but not defined: -bad
ok  	github.com/fgm/subcommands_demo/cmd	0.045s
$

The two remaining uncaptured messages appear to be a bug in version 1.2.0 of google/subcommands, for which a pull request exists.

We have now covered all the useful features of the google/subcommands package. All that remains is two related sets of features which are unlikely to be used in most practical cases.

5.2 Commanders introspection

The code for that level is available on level5.2-visit.

The subcommands.Commander type includes exported methods allowing code to examine its internal flags and commands data.

Let us create a top-level visit command, similar to those we have been writing, and which demonstrates all these methods. It is located in the visit.go file.

• The VisitAll methods implements the Visitor pattern, allowing introspection of the flags on the Commander instances:

21
22
23
24
25
26
27
28
29
30

func visitAll(commander *subcommands.Commander, w io.Writer) {
	fmt.Fprintln(w, "VisitAll show all the commander flags:")
	tw := tabwriter.NewWriter(w, 0, 0, 1, ' ', tabwriter.Debug)
	fmt.Fprintln(tw, "\tName\tDefault\tValue\tUsage\t")
	commander.VisitAll(func(f *flag.Flag) {
		fmt.Fprintf(tw, "\t%s\t%s\t%s\t%s\t\n", f.Name, f.DefValue, f.Value, f.Usage)
	})
	tw.Flush()
	fmt.Fprintln(w)
}

Running "demo visit":
VisitAll show all the commander flags:
 |Name  |Default |Value |Usage                  |
 |debug |false   |false |Show debug information |
 |v     |false   |true  |Be more verbose        |

• The VisitAllImportant methods is almost identical, but only visits flags labeled as important:

VisitAllImportant only shows the "important" flags:
 |Name |Default |Value |Usage           |
 |v    |false   |true  |Be more verbose |

• The VisitGroups method implements the Visitor pattern, allowing introspection of the CommandGroup instances created from the group names used when registering commands. It is quite limited, as the public methods on the CommandGroup type do not provide access to the grouped commands:

43
44
45
46
47
48
49
50
51
52

func visitGroups(commander *subcommands.Commander, w io.Writer) {
	fmt.Fprintln(w, "VisitGroups only visits the command groups, not the commands:")
	tw := tabwriter.NewWriter(w, 0, 0, 1, ' ', tabwriter.Debug)
	fmt.Fprintln(tw, "\tName\tLen\t")
	commander.VisitGroups(func(group *subcommands.CommandGroup) {
		fmt.Fprintf(tw, "\t%s\t%d\t\n", group.Name(), group.Len())
	})
	tw.Flush()
	fmt.Fprintln(w)
}

VisitGroups only visits the command groups, not the commands:
 |Name |Len |
 |help |3   |
 |top  |5   |

• The VisitCommands method implement the Visitor pattern in a much more useful way, providing access to the Command instances registered on the Commander, and from there to their CommandGroup and local flags:

43
44
45
46
47
48
49
50
51
52
53
54
55
56

func visitCommands(commander *subcommands.Commander, w io.Writer) {
	fmt.Fprintln(w, "VisitCommands visits the commands themselves:")
	tw := tabwriter.NewWriter(w, 0, 0, 1, ' ', tabwriter.Debug)
	fmt.Fprintln(tw, "\tGroup\tName\tSynopsis\tFlags\t")
	commander.VisitCommands(func(group *subcommands.CommandGroup, c subcommands.Command) {
		fs := flag.NewFlagSet("visit", flag.ContinueOnError)
		c.SetFlags(fs)
		var flags []string
		fs.VisitAll(func(f *flag.Flag) { flags = append(flags, f.Name) })
		fmt.Fprintf(tw, "\t%s\t%s\t%s\t%s\t\n", group.Name(), c.Name(), c.Synopsis(), strings.Join(flags, ", "))
	})
	tw.Flush()
	fmt.Fprintln(w, "(command usage omitted for readability)")
}

VisitCommands visits the commands themselves:
 |Group |Name     |Synopsis                                                            |Flags  |
 |help  |commands |list all command names                                              |       |
 |help  |flags    |describe all known top-level flags                                  |       |
 |help  |help     |describe subcommands and their syntax                               |       |
 |top   |top1     |top1 is an exemple top-level custom command without arguments       |prefix |
 |top   |top2     |top2 is an exemple top-level custom command with arguments          |prefix |
 |top   |1        |top1 is an exemple top-level custom command without arguments       |prefix |
 |top   |top3     |top3 is an exemple top-level custom command with nested subcommands |prefix |
 |top   |visit    |demoes commander Visit* functions                                   |       |
(command usage omitted for readability)

Because these visitor callbacks do not receive the Commander instance, the visit command will need to carry an injected commander instance, as commands are not aware of the Commander invoking them. We perform this with a new feature in root.go :

27
28
29

type CommanderAware interface {
	SetCommander(commander *subcommands.Commander)
}

45
46
47
48
49
50
51
52

func Execute(ctx context.Context,
  /* ...snip... */
	for _, command := range descriptions {
		if vc, ok := command.command.(CommanderAware); ok {
			vc.SetCommander(commander)
		}
		commander.Register(command.command, command.group)
	}

Our visit command is an instance of the visitCmd type, which includes a subcommands.Commander field, and an associated visitCmd.SetCommander setter, making it a CommanderAware implementation, allowing the Execute function to inject it with the active Commander as show on lines 48-49.

5.3 Overriding builtin output

The code for that level is available on level5.3-explain.

The last and most esoteric feature in google/subcommands is the ability to swap the implementation of the builtin CommandsCommand, FlagsCommand and HelpCommand, using matching function field on the subcommands.Commander type :

• Explain is a func(io.Writer) field, initialized in NewCommander with the unexported method expression cdr.writer
• ExplainGroup is a func(io.Writer, *CommandGroup) field, initialized in NewCommander with unexported function explainGroup
• ExplainCommand is a func(w io.Writer, c subcommands.Command) field, initialized in NewCommander with unexported function explain

Since these function fields are missing receiver access, but they still need it, our new explain command, defined in cmd/explain.go, has to be CommanderAware, so it can replace the fields with functions having access to the Commander instance, using the one instance it was injected with :

Examples below first show the default builtin output, then the output of a customized version

• ExplainCommand : builtin output, then YAML API version:

Running "demo explain":

Demoes overriding ExplainCommand to describe top3.
- Builtin version using unexported explain:
top3  -prefix string
    	Add a prefix to the result
- Custom version in YAML format:
top3:
    flags:
        - name: prefix
          default: ""
          usage: Add a prefix to the result
    synopsis: top3 is an exemple top-level custom command with nested subcommands
    usage: top3

• ExplainGroup : builtin output, then YAML API version:

Demoes overriding ExplainGroup.
- Builtin version using private explainGroup:
Subcommands for help:
	commands         list all command names
	flags            describe all known top-level flags
	help             describe subcommands and their syntax	
...snip...
- Custom version in YAML format, without access to group contents:
help: 3
top: 6

• Explain : builtin output, then neutral version suggesting how to write a custom description

Demoes overriding Explain.
- Builtin version using private commander.explain:
Usage: demo <flags> <subcommand> <subcommand args>


Top-level flags (use "demo flags" for a full list):
  -v=false: Be more verbose

- Custom version, build from commander methods:
Use any commander.(Explain|Visit)* methods

Conclusions

• Pros
  • google/subcommands is an excellent fit for small, simple projects
  • it is uncommonly lightweight, and very stable due to its having no dependencies at all
  • starting with it does not prevent evolving from a simple version to a multi-level command hierarchy with a mix of local and global flags
  • it does not make it difficult to provide unit test coverage for the commands themselves instead of limiting one to the service code the commands call upon.
• Cons
  • it does not natively merge CLI flags and environment variables
  • it does not natively include merging configuration files with flags
  • creating complex commands with validation hooks has no direct support: all steps will have to be implemented together in the Execute method
  • it does not natively include the persistent flags concept, that is flags defined at one level and also applying to subcommands. These can however be passed manually, for example in the context like we did for global flags.
  • creating multi-level command hierarchies requires a bit more work than with some other tools.

To summarize, that package is likely to be your tool of choice for your next microservice or CLI tool, but not if you are building your next giant monolith.