TL;DR: An LLM returns prose, and prose is miserable to program against. You end up regex-ing a paragraph for the one number you wanted. Structured output fixes this: you define a struct, the framework derives a JSON Schema from it, the model is made to answer in that shape, and you get a typed value back. go-tool-base’s Ask and rust-tool-base’s chat_structured both do this, and because the schema is derived from your type, the two cannot drift apart.

The problem with a paragraph

You ask an LLM to analyse a log file and tell you the severity of what it found and a suggested fix. It comes back with three well-written paragraphs. Somewhere in there is the word “critical,” and somewhere is the fix.

Your program now has to extract those two facts from prose, and prose has no contract. The next run, the model phrases it differently. It leads with a caveat. It says “severe” where it said “critical” before. It puts the fix first. Anything that worked by finding “critical” in the text is now wrong, and you didn’t change a line. Parsing free text for structured facts is a game you lose slowly.

What you actually wanted was not a paragraph. It was a value: a thing with a severity field and a fix field, that you can branch on and store and pass around.

Ask for the struct, not the prose

go-tool-base’s chat package draws the line with two methods. Chat gives you text. Ask gives you a struct.

You define the Go type you want back:

type Analysis struct {
    Severity string `json:"severity"`
    Fix      string `json:"fix"`
}

var result Analysis
err := client.Ask(ctx, "Analyse this log file: "+logText, &result)

The framework generates a JSON Schema from that struct, sends it to the model as the required response format, and unmarshals the reply straight into result. You never handle the prose. You get result.Severity and result.Fix, typed, ready to use. If you want the model’s answer to drive a switch statement, this is the method that lets it.

The struct is the schema is the contract

The detail that makes this hold up over time: you do not write the schema. The struct is the schema.

The framework derives the JSON Schema from your type. In go-tool-base that is GenerateSchema[T](); in rust-tool-base the schema comes from your Rust type through schemars. Either way there is one definition, your type, and the schema is a projection of it.

That matters because otherwise two things have to agree. There is the schema you tell the model to obey, and there is the type you unmarshal the answer into. Hand-write the schema and those two can drift: add a field to the struct, forget to add it to the schema, and the model is never told to produce it, so it silently never appears. Deriving the schema from the type collapses the two into one. They cannot disagree, because there is only one of them.

Both frameworks, with one extra step in Rust

go-tool-base does this with Ask and a ResponseSchema set on the client config. rust-tool-base does it with chat_structured::<T>, where T is any type that is both deserialisable and JsonSchema.

rust-tool-base adds one step worth calling out. Before it deserialises the model’s reply into your T, it validates the raw response against the schema with a JSON Schema validator. That splits the failure into two distinct, named cases: the response did not match the schema, or it matched the schema but still would not deserialise. A model that returns subtly wrong JSON fails loudly and specifically, with an error that says which of those happened, instead of quietly handing you a zero-valued struct that you debug an hour later.

When you’d reach for it

The line is simple, and it is about who reads the answer.

If a human reads the answer, prose is right. Chat, free text, let the model write well. A summary, an explanation, an interactive reply: leave those as prose.

If a program consumes the answer, you want a value. Classification, extraction, a code review scored out of a hundred with a list of issues, a yes-or-no with reasons: anything where the next thing that happens is your code branching on the result. There, Ask and chat_structured turn the LLM from something you have to interpret into something that returns a value, and a typed value is a thing you can actually build on.

Summary

An LLM returns prose by default, and prose has no contract, so a program that picks structured facts out of it breaks the moment the model rephrases.

Structured output asks for the value instead. You define a struct, the framework derives a JSON Schema from it, the model is constrained to that shape, and you get a typed result. go-tool-base’s Ask and rust-tool-base’s chat_structured both work this way, with the schema derived from your type so the schema and the type cannot drift; rust-tool-base additionally validates the response against the schema before deserialising. Use it whenever the answer feeds code rather than a human. It is one of the four methods that make up go-tool-base’s small chat interface, and it is the one that makes an LLM safe to program against.