First Light on PHP Boy Scout

Introducing afmpeg and ffmpeg-wasi: FFmpeg with no install, no CGO, no disk

Sun, 28 Jun 2026 00:00:00 +0000

keryx renders short promo reels, and the way it does that, today, is the way nearly everything does: it shells out to the ffmpeg binary. Which is fine, until you ask it to render a project that doesn’t exist on disk. keryx can work on an in-memory project, a repo cloned straight into RAM with no local checkout, and the moment it tries to hand that to ffmpeg, the whole thing falls over. The binary wants real files in a real directory. There aren’t any.

I went looking for a way out and didn’t find one I could live with. The bindings that use purego/dlopen are immature and still need the host’s libav libraries installed. The CGO bindings to libav are mature and can absolutely work in memory, but they’re CGO, and CGO takes away the thing I most want from a Go program: a clean static cross-compile to a single binary that runs anywhere. The famous ffmpeg.wasm is an emscripten build aimed at the browser, which is the opposite target from a server-side Go tool. And the one existing WASI-capable build pins FFmpeg 5.1, which is end-of-life, and I’m not shipping an out-of-support media decoder whose entire job is parsing untrusted files. Every road had a tollbooth I wasn’t willing to pay.

So I built two new things instead, and because they’re so closely tied I’m introducing them together.

ffmpeg-wasi: current FFmpeg, sandboxed, CGO-free

ffmpeg-wasi (repo) is the foundation, and it’s the harder of the two to build. It takes FFmpeg’s media libraries, the libav* family, and builds them to wasm32-wasi, then drives them with a small purpose-built engine, producing a single .wasm artifact that runs anywhere a WASI runtime does. No native FFmpeg install, no C toolchain at deploy time, no shelling out to a binary. It’s built to run under wazero, the zero-dependency pure-Go WebAssembly runtime, so a Go program can transcode, filter and mux media embedded, sandboxed, and CGO-free, still cross-compiling to one static binary.

The interesting bit, the reason this didn’t already exist, is a wall that FFmpeg 7.0 put up. The 7.x series rewrote the command-line tool to be mandatorily multithreaded, and a pure-Go WASI runtime can’t run that, because the threading model it needs (wasi-threads, spawning real threads) isn’t something wazero does. Every project trying to get current FFmpeg into WASI hits that wall, which is exactly why the existing build froze at the last single-threaded CLI, 5.1, and went EOL there. ffmpeg-wasi goes under the wall instead of over it: it doesn’t compile the CLI at all. It links the libav* libraries directly, which build single-threaded without complaint, and drives them with its own engine. That’s the move nobody else has made, and it’s the whole reason this can track current, maintained FFmpeg rather than a frozen one.

afmpeg: the pure-Go binding that lives in memory

afmpeg (repo) was the catalyst for all this, and it’s the part a Go developer actually touches. It’s a small, idiomatic Go API (New, Run, Probe, Close) sitting on top of the ffmpeg-wasi artifact, with one important twist: its I/O is bridged to an afero.Fs. afero isn’t in the standard library, but it’s the filesystem abstraction a great deal of Go already reaches for when it wants to swap a real disk for something else, and that “something else” is exactly the point here. The inputs and outputs of a media job can live entirely in memory, or in any afero backend you like, and ffmpeg is none the wiser. keryx gets to render its in-memory project without ever touching the disk, which was the whole reason I started.

This isn’t a roadmap post. Both projects are released and run today: afmpeg is at v0.4.0, ffmpeg-wasi at n8.1.2-1 (current FFmpeg, not the EOL 5.1), and between them they do real in-memory transcodes, verified end to end, WAV to AAC and H.264 rescaled and re-encoded with x264. Stripped of keryx’s reel-specific filtergraph, the shape a caller actually deals with is about this small:

// Compile the ffmpeg-wasi module once, then reuse the runtime.
rt, err := afmpeg.New(ctx, afmpeg.WithModuleFile("ffmpeg-wasi-lgpl.wasm"))
if err != nil {
 return err
}
defer rt.Close(ctx)

// The whole job lives in memory: no temp dir, nothing on disk.
fs := afero.NewMemMapFs()
afero.WriteFile(fs, "in.wav", input, 0o644)

// Drive it with the ffmpeg arguments you already know...
res, err := rt.Run(ctx, fs, "-i", "in.wav", "-c:a", "aac", "out.m4a")
if err != nil || res.ExitCode != 0 {
 return fmt.Errorf("render failed: %w (%s)", err, res.Stderr)
}

// ...then read the finished file straight back out of memory.
out, _ := afero.ReadFile(fs, "out.m4a")

The .wasm itself is a published release artifact you pin by SHA-256 (or let afmpeg fetch and verify for you with WithModuleURL), so the licence boundary stays explicit and nothing surprising ends up in your binary.

It doesn’t do everything yet, and I’d rather say so than let you find out the hard way. Single input to single output and Probe work now; the full multi-pad filter_complex and multi-output muxing are the next thing on the bench. But the part that nobody had cracked, getting current FFmpeg to run sandboxed, pure-Go, over a virtual filesystem with nothing on disk, is done, and you can pull it today.

Why it’s two repos

There’s a reason these are separate projects rather than one, and it’s about licences, not tidiness. The moment you compile FFmpeg you’re handling LGPL and GPL code, and I wanted that boundary to be obvious rather than smeared across one repo where nobody’s quite sure what’s covered by what. So the build tooling and the (L)GPL .wasm artifacts live in ffmpeg-wasi, with no grey areas, and afmpeg stays a clean permissive layer on top of the published artifact. I’m also shipping both LGPL and GPL builds of the artifact, so anyone who just wants the output and doesn’t fancy doing their own FFmpeg build can pick the licence that suits them and get on with it.

Both repos are public, so you can rebuild or relink either one yourself, and the doc sites are now linked in the nav up top.

There’s a stack of stories behind these two already, the threading wall and the spike that found the way under it, the afero-to-WASI filesystem bridge, the day I built a reel-shaped API and then reverted it before merge for being too narrow, the licence decision in full. I’ll be writing those up as the work lands. For now this is the headline: I needed an FFmpeg I could embed in a Go program with no install, no CGO, and no disk, couldn’t buy one off the shelf, so I built it in the open, and as of today it’s there to pull.

I built my wife a judge

Mon, 22 Jun 2026 00:00:00 +0000

The morning after Hailey shot a wedding, we were both wrecked, lying in bed and not really moving, and she said she might just pay for Aftershoot to get through the cull. And my ears pricked up.

She’d been on her feet for the best part of fourteen hours the day before. She had four thousand photos waiting for her. And I, lying there next to her, realised that for the first time the thing I do in my spare time, the obsession that mostly just pays my half of the bills, might actually be of some use to her.

Four thousand photos and the weeks that follow

A bit of background. My wife is a photographer. Not full-time yet, but that’s the plan, and she’s been chipping away at it around life and the bills ever since she set up Echos Photography a couple of years ago. Her real love is landscape and animals, the patient stuff, but every so often she’ll take on a wedding or some portraits, and last weekend she was the photographer for a family friend’s big day. A lovely affair. She shot it; I flew the drone for a bit of aerial photography and video, which mostly means I got to stand in a field feeling useful while she did the actual work.

Then it ends, and the real job starts. Four thousand frames, and before a single one gets edited somebody has to sit and look at every one and decide which are even worth keeping. The blinks, the soft ones, the same moment fired off eight times in a burst where only one frame is sharp. Done by hand, in Lightroom, one at a time, that can take her weeks. Months, sometimes, depending on what else life is throwing at her. So when she said she’d pay a subscription to make the first pass go away, I understood completely. It’s a genuinely grim job.

I just thought I could do better… for her, specifically.

Why my ears pricked up

I’ve been building developer tools for a while now. go-tool-base has reached the point where it’s actually pleasant to build things on, and the work I’d been doing on keryx, a little studio you drive from a browser rather than a command line, had shown me the shape of something a non-developer could use without ever touching a terminal. The pieces were sitting there… they just needed pointing at a different problem.

So I built her a judge. That’s what the name means, as it happens. krites is Ancient Greek for “the judge”, and that’s the whole product in one word: the thing that looks at every frame and rules on it, keep, maybe, reject, with its reasons, so the human doesn’t have to do the first exhausting pass by hand. It judges; she still decides. It proposes, she disposes. The tool never gets the last word, and it was never meant to.

A tool that’s only trying to please one person

Here’s the thing an off-the-shelf product can never quite be: krites is opinionated entirely in Hailey’s favour. It’s built around her workflow, her kit, her idea of what makes a frame worth keeping, and it’s designed so that she can use it with no setup and no technical knowledge whatsoever. The day it asks her to edit a config file is the day I’ve failed.

I’ve also kept it deliberately dim. There’s very little machine learning in it, and that’s a choice rather than a limitation. Most of the first pass, deciding whether a photo is in focus or hopelessly blown out, whether it’s the eighth near-identical frame in a burst, turns out to be arithmetic, and arithmetic is simple, fast and predictable in a way a model never quite is. I’ll reach for ML where there’s genuinely no other way (telling whether someone’s mid-blink is the obvious one), and when I do it’ll lean on the Apple hardware she already owns, with a deliberate gap left so it can grow onto other machines later. But the rule is: no cleverness she doesn’t need.

And it all runs on her own machine, on purpose. No subscription, which matters when the photography isn’t paying for itself yet. No reliance on a connection, which matters because we’re hoping to spend a good chunk of the near future on the road (there’s a whole vanlife plan brewing, with a work-in-progress demo site slowly coming together), and a campsite in the middle of nowhere is exactly where you want to be culling a backlog and exactly where you won’t have signal. And nothing leaves the laptop, which matters most of all, because these are other people’s weddings. Their photos shouldn’t have to go and live on somebody’s cloud to be useful to the woman they hired.

Why not just pay for the thing that exists

Aftershoot is good. Genuinely good. The easy version of this post is the one where I sneer at the incumbent, and I won’t: they’ve built something clever, and plenty of photographers are well served by it.

But it’s a product, and a product is a compromise by definition. It’s built for the average of everyone, and it can only ever evolve in the directions a company decides are profitable. Hailey isn’t the average of everyone. Something I build is hers, and it can grow in any direction she can imagine, including the unprofitable, niche, only-makes-sense-for-one-photographer directions that no company would ever green-light. That’s not really a technical argument. It’s closer to pride. I’d rather build my wife the tool than rent her someone else’s.

Early days, and the people who made it possible

A word on where this actually is, though. What’s shipped today is the cull, the first pass I’ve described. The straightening, the cropping, the colour work, the little retouches that eat her evenings, the part where it learns her taste over time, that’s all roadmap, and I’ll write about each piece as it lands. krites is at the very start of its life.

None of it would exist at all without three things, and only one of them is mine. go-tool-base finally being solid enough to stand on. Being able to build, with a lot of AI assistance, far faster than I ever could have managed alone, fast enough that “I could probably build that” became “I have built that” inside a weekend. And Hailey, whose infinite patience and gracious kindness in indulging my passions is the only reason any of my obsessions ever get the room to become something.

Which is the part I keep coming back to. I’ve written before about the cost of the compulsion to build, the evenings it takes and the people it takes them from. krites is the same compulsion, for once, pointed the other way. The thing that usually pulls me away from her, aimed squarely at giving her her evenings back. I don’t think I’ve enjoyed building anything more.

She still gets the final say on every photo. Obviously. She’s the judge that matters. I just built her a faster one to argue with.

rust-tool-base: the same idea, in a language that argues back

Wed, 22 Apr 2026 00:00:00 +0000

I built go-tool-base because I was sick of rebuilding the same CLI scaffolding every time I started a new Go tool. You’d think that would have taught me a lesson about doing things more than once. Apparently not, because I’ve now started building rust-tool-base: the same idea, the same itch, for Rust.

In my defence, there’s method in it.

The same itch, a different language

go-tool-base exists because I kept writing the same couple of hundred lines of wiring every time I started a new Go CLI. Config loading, logging setup, an update check, an error path, a help system. None of it was the tool. All of it had to be there before the tool could be.

Lately I’ve been learning Rust, and two things collided. The first is how I tend to learn a language. I’ve always picked them up reasonably quickly, and the way I do it isn’t with a tutorial that builds a toy, it’s by rebuilding something whose shape I already know cold, so that every decision is about the language rather than the problem. The second is that every time I started a Rust CLI of any size, I hit the very same gap I’d already filled once in Go.

So rather than learn Rust on a throwaway, I decided to learn it by building rust-tool-base: the same idea, the same niche, for Rust.

One housekeeping note before the series gets going. You don’t need to write Rust to follow it. The posts lean on a handful of language ideas, and rather than stop and re-explain each one mid-flow, I’ve gathered them into a short primer. If a bit of syntax trips you up along the way, that’s where to look. If you already write Rust, ignore me and read on.

The gap in Rust

The Rust ecosystem has a well-earned reputation for sharp, focused crates and a deliberate shortage of big opinionated frameworks. clap for argument parsing, figment for layered config, tracing for logging, miette for errors, ratatui for terminal UI, reqwest and tokio underneath. Each of them is genuinely best-in-class.

What nobody hands you is the assembly. Wiring those into one coherent product, and then adding self-update, AI integration, an MCP server, embedded documentation, credential handling, telemetry and a scaffolder, is real work, and it’s the same work on every project.

The closest existing neighbours stop short of it. cli-batteries is a thin preamble: argument parsing plus a logging subscriber plus panic and signal handling. starbase has a proper session and lifecycle model but is CLI-agnostic and shaped around the moonrepo tooling it came from. cargo-dist and cargo-release are about release packaging, not the runtime. Good tools, all of them, but none is the opinionated, full-lifecycle, scaffolded base that go-tool-base is in the Go world. That space is empty, and rust-tool-base is built to fill it.

Why it is not a port

The obvious way to build this would be to open go-tool-base and translate it file by file. I’m not doing that, and the reason matters enough that it’s the rule the whole project is built around.

go-tool-base is full of Go. It leans on a Props struct that carries the framework’s services in loosely-typed fields. It configures things with functional options. It registers commands using package-level init(). It threads a context.Context through every call. Those are all good, idiomatic Go. Transliterated into Rust they’d become code that argues with the compiler on every single line, because Rust has its own answers to every one of those problems and they are emphatically not the Go answers.

So rust-tool-base reaches the same outcomes by Rust’s means. Commands still self-register, but through link-time machinery instead of init(). There’s still one context object per command, but it’s strongly typed rather than a loosely-keyed bag. Configuration is still layered, but it lands in your own typed struct instead of a string-keyed lookup. Same philosophy, same shape of product, an entirely different ecosystem underneath. The README says it plainly: it’s a sibling, not a port.

Why do it twice at all

Three reasons, and they reinforce each other.

The first is plain usefulness. The next time I want a Rust CLI tool, I want the same head start go-tool-base already gives me in Go.

The second is the learning. Rebuilding a system I understand forces me to meet Rust’s idioms where they actually bite, not where a tutorial gently stages them. You learn ownership properly when a real design is pushing back at you.

The third is the one I didn’t expect, and it’s the subject of the next post. Building the same framework twice, in two languages, turns out to be the cleanest way to find out which of your original decisions were genuine design and which were merely idiom. The design survives the move. The idiom does not. Sorting one from the other has been the most interesting part so far.

Boiling it down

rust-tool-base is the Rust sibling of go-tool-base: the same batteries-included, scaffolded, opinionated CLI framework, aimed at the same gap, which in Rust is the gap between a pile of excellent crates and a coherent product.

It’s not a port. Transliterating Go idioms into Rust produces code that fights the language, so RTB reaches the same outcomes through Rust’s own mechanisms instead. The posts after this one walk through the specific cases: how commands register, how the builder works, how errors are reported, and a few things RTB can do that the Go version structurally can’t. First, though, the thing the exercise taught me about my own design.

go-tool-base: I got tired of reinventing the wheel

Wed, 18 Mar 2026 00:00:00 +0000

If you’ve written more than two or three command-line tools in Go, you’ll recognise the shape of the first afternoon. I certainly do! You reach for Cobra for the command tree, Viper for config, and then you start the part nobody ever puts in the README… the plumbing.

Where does config live? A file, an env var, an embedded default? In what order do they override each other? How does the tool tell the user there’s a newer version, and how does it actually update itself? What does logging look like, and is it the same logging the next tool will use? And how do you wire all of that into each command without every command reaching into a pile of globals?

None of it is hard. That’s the problem! It’s not hard, it’s just there, every single time, and every single time I’d find myself reinventing it slightly differently to the last time. Different override precedence here. A subtly different update flow there. Logging that didn’t quite match the tool I’d written three months earlier. Each new tool was a fresh re-litigation of decisions I’d already made and then promptly forgotten.

Now, I’ve banged on about the Boy Scout rule for years (leave the codebase better than you found it), but it has an uncomfortable corollary. If you keep turning up to the same campsite and finding it in the same mess, at some point the honest thing to do is to stop tidying it and go and build a better campsite.

First, just packages

So I started pulling the recurring pieces out into their own packages. Nothing grand. A config package that did the hierarchical merge the way I always ended up doing it anyway. A version package that knew how to compare semver and spot a development build. A setup package that handled first-run bootstrap and self-updating from a release. They lived as separate repos, and if you go digging through my GitHub history you can still find the scruffy ancestors of them scattered about.

Separate packages was the right first move. It forced each piece to stand on its own and earn its keep on a real project before I trusted it on the next one. A package that’s only ever been used in the repo it was born in hasn’t really been tested… it’s just been agreed with.

But separate packages come with a tax. Each one has its own release cadence, its own changelog, its own CI. Worse, they have to agree with each other at the seams, and when they’re versioned independently those seams drift. I’d bump the config package, and the setup package that depended on it would quietly need a matching bump, and the tool that used both would need telling about both. I’d traded “reinvent the wheel” for “keep a dozen wheels in sync”, and I’m really not convinced that’s a better deal.

Then, one library

Once the packages had been used enough (used in anger, on real tools, by people who weren’t me) the shape of them stopped moving. The interfaces settled. The arguments about precedence and defaults were over, because the answers had survived contact with reality.

That’s the point where separate packages stop being a virtue and start being friction. So I forged them into one and called it go-tool-base. One module, one version number, one changelog, and one set of seams that are now internal and can’t drift, because they ship together.

The heart of it is a dependency-injection container, a Props struct, that holds the things every command needs: the logger, the config, the embedded assets, the filesystem handle, the error handler, the tool’s own metadata. Commands are handed Props explicitly rather than reaching for globals, which means a command is just a function of its inputs and is therefore trivially testable. That one decision has quietly paid for itself on every tool I’ve built since.

Around that container sits all the stuff I was so tired of rewriting: hierarchical config, structured logging, version checking, self-update from GitHub or GitLab releases, an interactive TUI documentation browser, AI integration, service lifecycle management. A new tool inherits the lot and gets to spend its first afternoon on the thing that’s actually novel… its own logic.

Finally, a generator

A library still leaves you staring at a blank main.go. You still have to know the conventions, wire the container, lay out the directories, register the commands. All knowable, but all boilerplate. And boilerplate is exactly the enemy I set out to kill in the first place.

So go-tool-base ships a generator. gtb generate project scaffolds a complete, working, idiomatic project: directory layout, the wired Props container, the command tree, CI, the whole lot. gtb generate command adds a new command and registers it for you. The generator also handles upkeep: when the framework’s conventions move, it can regenerate the scaffolding of an existing project without trampling all over the code you’ve written on top. (That last bit turned out to be a properly interesting problem in its own right, and a future post.)

The goal is blunt. Creating a CLI tool should be about the tool, not the scaffolding. The first afternoon should be spent on the part that’s actually worth writing.

One thing I was careful about

There’s a nasty failure mode with “batteries-included” frameworks: the day you outgrow them, they hold you hostage. You either stay inside the framework’s worldview forever, or you face a rewrite. I’ve been burned by that before and I had no intention of inflicting it on anyone else.

So go-tool-base generates idiomatic, standard-library-compliant Go. There’s no magic runtime you can’t see, no clever code you couldn’t have written by hand. If you ever outgrow the framework the generated code stands on its own and you walk away with a perfectly normal Go project. A framework should be a starting point you’re glad you took, not a room you can’t get out of.

Where this leaves me

go-tool-base exists because I was spending the first afternoon of every Go CLI tool rebuilding the same plumbing, and rebuilding it slightly wrong relative to last time. It started life as separate packages so each piece could earn its place on real projects; once they’d stopped moving I forged them into a single library so the seams couldn’t drift; and then I wrapped a generator around it so a new tool starts as a working project rather than a blank file.

It’s a framework for the boring 80% (config, versioning, updates, logging, lifecycle) so you can spend your time on the 20% that’s actually yours.

Over the coming posts I’ll dig into the individual pieces… the generator that won’t clobber your edits, the credential handling, the self-update integrity checks, and a few Go techniques I’m rather pleased with along the way. Stay tuned!

Introducing ZuQ - A Simple ZeroMQ Queuing Daemon

Tue, 19 Mar 2013 00:00:00 +0000

We recently had the need to create a queuing system to replace an implementation of RabbitMQ that was being used on a previous project. The reasoning behind this is that the requirements of the project required a very custom implementation of a queuing system that would drastically alter in architecture as the project grew and RabbitMQ just wasn’t going to fit the bill. However to start with we required something super simple and efficient that could be expanded and developed as required. After a little investigation and a lot of recommendation from others we decided to use ZeroMQ as our transport layer for that very reason, as we could build something which could span across multiple servers and was fast.

The diagram above helps describe our basic queuing system. We have a queue daemon that is continuously listening for connections and two clients, one that populates the queue and the other that retrieves from the queue.

The Clients

Each client is written in PHP and uses a 0mq socket to communicate with a service, in this case our queue service. We used a SOCKET_REQ type of socket in order to have a request/response communication with our queue service.

function client_socket(\ZMQContext $context)
{
 // SOCKET_REQ used to create a client that sends requests to and receive from a service
 $client = new \ZMQSocket($context,\ZMQ::SOCKET_REQ);
 $client->connect("tcp://localhost:5555");
 // SOCKOPT_LINGER = 0 Configure socket to not wait at close time
 $client->setSockOpt(\ZMQ::SOCKOPT_LINGER, 0);
 return $client;
}

public function injectIntoQueue()
{
 $context = new \ZMQContext();
  $client = $this->client_socket($context);
  $msg = "This is a message";
  $retries_left = 3;
  $read = $write = array();
  while ($retries_left) {
    // We send a request, then we wait to get a reply
        $client->send($msg);
        $expect_reply = true;
        while ($expect_reply) {
        // Poll socket for a reply, with timeout
            $poll = new \ZMQPoll();
            $poll->add($client, \ZMQ::POLL_IN);
            $events = $poll->poll($read, $write, 2500);
            // If we got a reply, process it
            if ($events > 0) {
            // We got a reply from the server, must match sequence
                $reply = $client->recv();
                if (intval($reply) == $msg) {
                $retries_left = 0;
                    $expect_reply = false;
                }
             } elseif (--$retries_left == 0) {
            break;
             } else {
               // Old socket will be confused; close it and open a new one
                $client = $this->client_socket($context);
                // Send request again, on new socket
                $client->send($msg);
 }
 }
 }
}

You can see from the code above, have a 3 strike rule. The reasoning behind this is that if the client fails to connect to the queue service more than 3 times, we can stop trying to inject into the queue and move on to the next item. As we ultimately intend to adapt the lazy pirate pattern we have made it so that if the socket times out, we can then create a new socket and retry. Without this, as the architecture becomes more complicated we may then end up in a situation where we might have errors, thus the recommend solution is to create a new socket. Once the client has sent its message to the queue, we poll for a response (i.e. which is the message we sent returned back). Once we have a response that is valid, meaning that the queue has been populated, we can stop polling until the next message.

The Frontend Client

protected function getFromQueue()
{
 $context = new \ZMQContext();
 $worker = new \ZMQSocket($context, \ZMQ::SOCKET_REQ);
 $read = $write = array();
 // Set random identity to make tracing easier
 $worker->connect("tcp://localhost:5556");
 // Tell queue we're ready for work
 $worker->send("ready");
 $reply = $worker->recv();
 return $reply;
}

Our Frontend Client is much simpler as it is part of a process that is being continually updated, therefore it doesn’t need the same connection retries are the Backend Client. We simply send “ready” to the queue system and if the queue is populated it will return us the first item.

The Queuing Service

This is a continuously running executable created using c++.

zmq::context_t context(1);
zmq::socket_t frontend (context, ZMQ_ROUTER);
zmq::socket_t backend (context, ZMQ_ROUTER);
backend.bind("tcp://*:5555");
frontend.bind("tcp://*:5556");

We create two sockets of type ZMQ_ROUTER which is an advanced pattern used for extending request/reply sockets. This means when we improve our queuing system we will be able to route packets to specific recipients using an address in the message.

After creating our sockets, we initialise them

// Initialize poll set
zmq::pollitem_t items [] = {
 { frontend, 0, ZMQ_POLLIN, 0 },
 { backend, 0, ZMQ_POLLIN, 0 }
};

//poll the sockets - this seems to poll both sockets at the same time
zmq::poll (items, 2, -1);

Backend Handler

If we get a message from the backend, we check the contents to see if it contains purge at which point we empty the queue, otherwise we push the msg contents onto the queue. Finally we send the message back to the backend to show that the message has been received.

//receive msg from client
if (items [1].revents & ZMQ_POLLIN) {
 //get message from client
 zmq::message_t message(0);
 string client_addr = s_recv (backend);
 string empty = s_recv (backend);
 assert (empty.size() == 0);
 string msg = s_recv (backend);

 //allow the backend to purge the queue
 if(msg == "purge") {
 while (!queue.empty()) queue.pop();
 } else {
 queue.push(msg);
 }

 //send response back to the backend
 s_sendmore (backend, client_addr);
 s_sendmore (backend, "");
 s_send (backend, msg);
}

Frontend Handler

If we get the “ready” message from the frontend client, we pop a message off the queue and return it to the frontend client. If the queue is empty we send an “empty” message back instead.

// Handle activity on frontend
if (items [0].revents & ZMQ_POLLIN) {
 //get message from worker
 zmq::message_t message(0);
 string worker_addr = s_recv (frontend);
 string empty = s_recv (frontend);
 assert (empty.size() == 0);}
 string msg = s_recv (frontend); string queueMsg;
 if(msg == "ready") {
 if(queue.size() > 0) {
 queueMsg = queue.front();
 queue.pop();
 } else {
 queueMsg = "empty";
 }
 }
 //send reply to worker with contents of queue
 s_sendmore (frontend, worker_addr);
 s_sendmore (frontend, "");
 s_send (frontend, queueMsg);
}

That is our complete queuing system using ZeroMQ with PHP and C++.

Summary

Using the above has allowed us to create a very simple in memory queue daemon that we can use to quickly pass data from one system to a another. On the whole it works well and we are looking to expand on it in the near future to increase both its functionality and scalability.

You can find the queueing daemon (christened as “ZuQ”) on github @ https://github.com/zucchi/ZuQ