The heartbeat from part 1 runs, ticks along, and shuts down politely when you ask it to. It also talks to absolutely no one. A service people can actually call needs an API, and for a typed, fast, streaming-capable one, gRPC is the obvious first move.
The catch is that a production-grade gRPC server is rather more than
grpc.NewServer(). You want health checks an orchestrator understands,
reflection so you can poke at it without the .proto file in hand, a graceful
shutdown that doesn’t guillotine calls that are still in flight, and TLS, which
is where most people’s first attempt quietly goes wrong. The good news: part 1
already gave us the thing that carries all of it. A gRPC server is just another
service to register on the controller.
Why gRPC, and not just REST
Worth a moment on why we’re reaching for gRPC at all, because for plenty of services a plain JSON-over-HTTP API is the right call and less faff. gRPC earns its place when a few of these matter:
- A contract that’s enforced, not hoped for. The
.protois the single source of truth, and both ends are generated from it. You don’t hand-write JSON marshalling, and you don’t find out at runtime that the client and server disagree about a field’s type. Evolve the schema carefully (add fields by number) and old clients keep working. - Clients in any language, for free. The same
.protogenerates a Go server and a Python, TypeScript, Rust or Java client with nobody writing an SDK by hand. For an internal service that several teams call, that one point can decide it. - It’s built for service-to-service traffic. Binary protobuf is smaller and quicker to encode than JSON, calls multiplex down a single HTTP/2 connection, and streaming (from the client, the server, or both at once) is a first-class thing rather than something you bolt onto REST with websockets.
- Deadlines, cancellation and a health protocol come built in, rather than conventions you reinvent for every service.
The trade-offs are real. A browser doesn’t speak gRPC natively, and a binary
protocol is fiddlier to poke at than a JSON endpoint you can curl (which is
exactly why reflection and grpcurl exist). That’s not a reason to avoid it;
it’s the reason this series doesn’t stop at gRPC. In part 4 we put a REST/JSON
face on this very service, so the things that call it get the typed, fast core
and the things that can’t speak gRPC still get a friendly HTTP surface. You don’t
have to pick a side.
Define the contract
gRPC starts with a schema. Here’s a small macguffin service, macguffin.proto:
// proto/macguffin/v1/macguffin.proto
syntax = "proto3";
package macguffin.v1;
option go_package = "gitlab.com/myorg/macguffinsvc/internal/gen/macguffin/v1;macguffinv1";
service MacguffinService {
rpc GetMacguffin(GetMacguffinRequest) returns (Macguffin);
rpc ListMacguffins(ListMacguffinsRequest) returns (ListMacguffinsResponse);
rpc CreateMacguffin(CreateMacguffinRequest) returns (Macguffin);
}
message Macguffin {
string id = 1;
string name = 2;
int32 quantity = 3;
}
message GetMacguffinRequest { string id = 1; }
message ListMacguffinsRequest { int32 page_size = 1; }
message ListMacguffinsResponse { repeated Macguffin macguffins = 1; }
message CreateMacguffinRequest { string name = 1; int32 quantity = 2; }
From proto to Go
If gRPC in Go is new to you, this is the part that catches people out: you don’t
write the Go for those messages and that service interface, you generate it
from the .proto. The proto is the source of truth; a compiler turns it into Go
you import and build against. Same goes for a client in any other language, all
from the same file.
That compiler is protoc, and on its own it’s a faff. You install it, then a
separate plugin for each output you want (protoc-gen-go for the message types,
protoc-gen-go-grpc for the client and server stubs), keep their versions in
step, and drive the lot with a command line of -I include paths and --*_out
flags that’s easy to get subtly wrong.
buf is the friendlier way to run exactly that. It wraps protoc and its plugins behind a couple of small config files, handles the plugin versions, and turns that gnarly invocation into a single word. It’s become the usual way to work with protobuf in Go, and it’s what we’ll use here.
At a minimum you need three binaries on your PATH: buf itself, and the two
plugins it drives. go install is the quickest way to get them:
go install github.com/bufbuild/buf/cmd/buf@latest
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
Those land in $(go env GOPATH)/bin, so make sure that’s on your $PATH. Then
describe what you want generated in a buf.gen.yaml:
# buf.gen.yaml
version: v2
plugins:
- local: protoc-gen-go
out: internal/gen
opt: paths=source_relative
- local: protoc-gen-go-grpc
out: internal/gen
opt: paths=source_relative
Each part of that earns its place. version: v2 is buf’s config format. The
plugins list names the generators to run, and we run two, because gRPC in Go
arrives in two halves:
protoc-gen-goturns the messages into Go structs, theMacguffintype and the request and response types, in amacguffin.pb.gofile.protoc-gen-go-grpcturns theserviceinto the client and server scaffolding, in amacguffin_grpc.pb.gofile, including theMacguffinServiceServerinterface you’re about to implement.
out: internal/gen is where the files land, and paths=source_relative lays
them out mirroring the proto’s own folders (so proto/macguffin/v1/... becomes
internal/gen/macguffin/v1/...) rather than deriving the path from the
go_package line. Then run it:
buf generate
Both files appear under internal/gen/macguffin/v1, and we’re ready to write the
implementation.
Running that by hand once is fine; remembering to run it every time the .proto
changes is where it goes wrong, and the generated code quietly drifts out of
step, usually right before a demo. Wire it into go generate instead. Drop a
one-line directive in a file at your module root, say gen.go:
// gen.go (at your module root)
package macguffinsvc
//go:generate buf generate
Now go generate ./... regenerates everything from the proto, and it’s the same
one command for any other generator you add later. Run it whenever the schema
changes, and in CI if you want to catch a stale checkout.
If OpenAPI is your map of the territory
If your mental model of an API contract is an OpenAPI (Swagger) document, a
.proto is the same idea wearing fewer clothes: a typed, language-neutral
description of a service that both ends generate from. The thing you notice
first is how much less of it there is. Here’s that Macguffin message again, and
the same shape written as an OpenAPI schema:
message Macguffin {
string id = 1;
string name = 2;
int32 quantity = 3;
}
Macguffin:
type: object
properties:
id:
type: string
name:
type: string
quantity:
type: integer
format: int32
And that pattern holds across the whole service. The proto above, three calls and five messages, is about twenty lines. Describe the same surface in OpenAPI and you’re closer to a hundred, because OpenAPI also pins down the HTTP verbs, paths, status codes and content types: the transport details a proto leaves out on purpose. That isn’t OpenAPI being bloated; it’s describing more. But when the contract is the thing you care about, the proto says it with less ceremony, and it doesn’t wed your API to HTTP, which is exactly what lets us serve this same service over gRPC now and REST later. (We’ll generate a real OpenAPI document from this proto in part 5, for the readers who still want one.)
Implement it
Generating the code gave you the message types and, more to the point, an
interface to satisfy. Open macguffin_grpc.pb.go and you’ll find
MacguffinServiceServer, one method per RPC in the proto:
// internal/gen/macguffin/v1/macguffin_grpc.pb.go (generated)
type MacguffinServiceServer interface {
GetMacguffin(context.Context, *GetMacguffinRequest) (*Macguffin, error)
ListMacguffins(context.Context, *ListMacguffinsRequest) (*ListMacguffinsResponse, error)
CreateMacguffin(context.Context, *CreateMacguffinRequest) (*Macguffin, error)
mustEmbedUnimplementedMacguffinServiceServer()
}
That interface is the server-side contract. Each method takes the request message you defined and hands back the response message, plus an error. Writing the type that honours it, the actual logic behind each call, is the part that’s yours: the proto pins down the shape of the conversation, and this is what the service actually does when one happens.
The one curious line is mustEmbedUnimplementedMacguffinServiceServer().
Alongside the interface, buf generated an UnimplementedMacguffinServiceServer
struct with a do-nothing stub for every method, and you embed it in your own
type. It earns its keep twice over. It satisfies that unexported method, so your
type counts as a real implementation. And it future-proofs you: add a new RPC to
the proto later and your existing server still compiles, falling back to the
stub (which returns a clean “unimplemented” error) until you write the real
method.
Before we satisfy that interface, one separation worth making up front. The gRPC server is a delivery mechanism, not the place the data lives. If we stuff the map of macguffins straight inside it and then build an HTTP server next part, we’d have two servers each hoarding their own copy of the same data. So keep the domain, the macguffins and what you can do with them, in its own type, and let each transport be a thin layer over it.
Here’s that domain: an in-memory store standing in for the repository a real service would have. Nothing in it knows about gRPC, HTTP, or any wire format.
// internal/macguffin/store.go
package macguffin
import "sync"
// Macguffin is the domain type. The JSON tags will let a hand-written HTTP
// handler serve it directly in part 3.
type Macguffin struct {
ID string `json:"id"`
Name string `json:"name"`
Quantity int32 `json:"quantity"`
}
type Store struct {
mu sync.Mutex
items map[string]Macguffin
seq int
}
func (s *Store) Get(id string) (Macguffin, bool) {
s.mu.Lock()
defer s.mu.Unlock()
m, ok := s.items[id]
return m, ok
}
List and Create are the same shape, and NewStore seeds it with a single
maltese-falcon. Now the gRPC server is thin: it embeds the stub, holds a store,
and each method calls the store and translates the result into the generated
protobuf type.
// internal/grpcsvc/server.go
package grpcsvc
import (
"context"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
macguffinv1 "gitlab.com/myorg/macguffinsvc/internal/gen/macguffin/v1"
"gitlab.com/myorg/macguffinsvc/internal/macguffin"
)
type Server struct {
macguffinv1.UnimplementedMacguffinServiceServer
store *macguffin.Store
}
func New(store *macguffin.Store) *Server {
return &Server{store: store}
}
func (s *Server) GetMacguffin(_ context.Context, req *macguffinv1.GetMacguffinRequest) (*macguffinv1.Macguffin, error) {
m, ok := s.store.Get(req.GetId())
if !ok {
return nil, status.Errorf(codes.NotFound, "macguffin %q not found", req.GetId())
}
return toProto(m), nil
}
// toProto maps the domain type to the generated protobuf DTO.
func toProto(m macguffin.Macguffin) *macguffinv1.Macguffin {
return &macguffinv1.Macguffin{Id: m.ID, Name: m.Name, Quantity: m.Quantity}
}
ListMacguffins and CreateMacguffin are the same: call the store, map the
result. The one habit worth keeping is to return real gRPC status codes
(codes.NotFound here) rather than bare errors, so callers get something they
can branch on.
That toProto step is worth a second look, because it comes back round later.
The domain has one shape, the proto has its own generated Macguffin, so the
adapter maps between them. It’s a small price for a single transport. In part 3
we add a second transport with its own generated type and pay that price again,
and part 4 is where we stop paying it twice.
Wire it onto the controller
This is the part that earns its keep. First, generate a serve command, the
same way the CLI
series
generated its commands:
gtb generate command \
--name serve \
--short "Run the macguffin service"
That scaffolds two files: pkg/cmd/serve/cmd.go (generated, and wired into your
command tree for you) and pkg/cmd/serve/main.go, which holds a RunServe
function for your logic. Fill it in:
// pkg/cmd/serve/main.go
package serve
import (
"context"
"gitlab.com/phpboyscout/go-tool-base/pkg/controls"
gtbgrpc "gitlab.com/phpboyscout/go-tool-base/pkg/grpc"
"gitlab.com/phpboyscout/go-tool-base/pkg/props"
macguffinv1 "gitlab.com/myorg/macguffinsvc/internal/gen/macguffin/v1"
"gitlab.com/myorg/macguffinsvc/internal/grpcsvc"
"gitlab.com/myorg/macguffinsvc/internal/macguffin"
)
func RunServe(ctx context.Context, p *props.Props, _ *ServeOptions, _ []string) error {
controller := controls.NewController(ctx, controls.WithLogger(p.Logger))
store := macguffin.NewStore()
grpcSrv, err := gtbgrpc.Register(ctx, "grpc", controller, p.Config, p.Logger)
if err != nil {
return err
}
macguffinv1.RegisterMacguffinServiceServer(grpcSrv, grpcsvc.New(store))
controller.Start()
controller.Wait()
return nil
}
That’s the whole server. gtbgrpc.Register does four things in one call: it
builds a *grpc.Server, wires the standard gRPC health service to the
controller’s health reports (the ones we met in part 1), registers Start,
Stop and Status against the controller so the lifecycle is handled, and
hands you back the server to register your own service on, which is the
RegisterMacguffinServiceServer line. After that it’s the same
Start() / Wait() we used for the heartbeat.
It reads its port from config (server.grpc.port, falling back to
server.port), so a minimal config is:
server:
grpc:
port: 50051
reflection: true
Poke it
Build, run mytool serve, and reach for grpcurl. Reflection is on, so you
don’t need the .proto to hand:
$ grpcurl -plaintext localhost:50051 list
grpc.health.v1.Health
grpc.reflection.v1.ServerReflection
grpc.reflection.v1alpha.ServerReflection
macguffin.v1.MacguffinService
$ grpcurl -plaintext localhost:50051 macguffin.v1.MacguffinService/ListMacguffins
{
"macguffins": [
{ "id": "m-1", "name": "maltese-falcon", "quantity": 1 }
]
}
And the health service is already answering, wired straight to the controller, without you registering a thing:
$ grpcurl -plaintext localhost:50051 grpc.health.v1.Health/Check
{ "status": "SERVING" }
That’s the lifecycle work from part 1 paying out: the controller’s health is the gRPC health, and a SIGTERM still drains and stops the server cleanly.
Now turn on TLS
Here’s the bit people brace for. Plaintext gRPC is fine on a laptop and unacceptable the moment it leaves one. With go-tool-base it’s a config change, not a code change.
The fiddly part of local TLS is usually the certificate. A hand-rolled
self-signed one means passing a -cacert to every client and clicking past
browser warnings. mkcert makes that go
away: it creates a local certificate authority and installs it into your
system’s (and your browser’s) trust stores, so anything it signs is simply
trusted. Set the CA up once:
mkcert -install
Then mint a certificate for the names the service answers on:
mkcert localhost 127.0.0.1 ::1
That writes localhost+2.pem (the certificate) and localhost+2-key.pem (the
key), signed by your now-trusted local CA. Doing this properly now pays off
later: in part 3 the HTTP server, and in part 5 the API docs in a browser, both
lean on that certificate being trusted with no warning.
Point the tool’s config at the pair, under the shared server.tls block:
server:
grpc:
port: 50051
reflection: true
tls:
enabled: true
cert: ./localhost+2.pem
key: ./localhost+2-key.pem
No code changes. Run mytool serve again and it comes up over TLS:
INFO starting gRPC server tls=true addr=:50051
Because the certificate is signed by a CA your machine already trusts, the client needs no extra flags:
$ grpcurl localhost:50051 macguffin.v1.MacguffinService/GetMacguffin -d '{"id":"m-1"}'
{ "id": "m-1", "name": "maltese-falcon", "quantity": 1 }
A plaintext client is now refused, as it should be. (In production you’d point
those same two config keys at whatever your real CA issues; the wiring doesn’t
change.) Two details are worth knowing about what just happened, because both
are easy to get wrong by hand.
The server uses a hardened TLS config (1.2 minimum, AEAD cipher suites, X25519),
so you’re not accidentally shipping a weak handshake. And the listener
advertises HTTP/2 over ALPN, the h2 protocol gRPC rides on, which sounds like
a footnote until you discover that recent gRPC clients refuse a TLS connection
that doesn’t offer it. The framework sets it for you; it’s the single most
common reason a hand-rolled gRPC-over-TLS server works with old tooling and
mysteriously rejects a current client. All of that lives in the shared
pkg/tls package.
I put the certificate under server.tls rather than server.grpc.tls
deliberately. That shared block is the cert every transport falls back to, so
the HTTP server in the next part and the transports after it can use the same
one, with a per-transport override only where you actually need it.
The short version
A few files in, you have a real gRPC API: a typed contract, an implementation, health an orchestrator understands, reflection for poking, a clean shutdown, and TLS, and the only part that was actually yours to write was the service logic. The reference for the server helpers is the gRPC component doc, and the add-a-gRPC-service how-to has the manual-wiring path if you ever want it.
Next part puts an HTTP face on the very same controller, REST handlers and the same health endpoints an orchestrator probes, sharing that one certificate.