You want an AI agent to run your media work end to end. A brief comes in, and the pipeline reframes it for every channel, color-corrects to the brand, adds captions, checks the result, and exports, without a person clicking through each step or correcting the agent after every move. The agent is ready for that. The question is whether the pipeline underneath it is.
That part trips up most teams. What decides whether an agent can actually run your work is a handful of plain design choices in the pipeline beneath it. There are five of them, and once you can name them, "agent-drivable" stops being a buzzword and becomes a checklist. Full disclosure: Layermetry sells one of the off-the-shelf versions of this, so we will lay out both paths and let you decide. This piece is the practical walkthrough of how the pieces fit. For the bigger case on why an agent-drivable surface matters at all, we make that argument here.
What "agent-drivable" actually has to mean
Start with the distinction that keeps this honest. One clean tool call working is not the same as an agent running a whole job on its own. A job needs the agent to act, check its own work, recover from failures, coordinate with other agents, and do it all without a human watching the screen. That bar is higher than "the agent can call one function."
Five pieces, and how they line up
The pieces are few and they fit in a clear order. Each one answers a question the agent will hit while it works.
- Typed operations answer "what can I do, and exactly how do I ask for it?"
- Machine-readable results answer "did that work, and what do I do next?"
- Safe retries answer "the call failed halfway, can I just run it again?"
- Shared state answers "where is the work, so the next agent can pick it up?"
- Observability answers "what actually happened in there when something went wrong?"
The first two make a single agent reliable. The next two make a crew of agents work together. The last one makes the whole thing debuggable when it does not. Here is the shape before we walk each piece.
Typed operations: give the agent a menu, not a blank canvas
The first piece is the one everything else sits on. An agent should pick from a fixed set of named operations, each with a clear set of inputs, rather than improvise free-form instructions a system has to interpret.
Why a fixed menu beats free text
It is the difference between a form with labeled fields and a blank page. The form removes guesswork. A typed operation is the form: crop, with a target size and a focus point; caption, with a language and a position. The agent fills in known fields, and the pipeline never has to guess what "make it a bit tighter" meant.
This is the shape the industry standardized on. The open protocol that defines how agents call tools requires each tool to declare its inputs as a schema, and through 2026 it became the common standard across the major model providers and editors, with tens of thousands of public tool servers in use (protocol governance moved to a neutral foundation in late 2025). You are not inventing a private convention. You are following the one the ecosystem already agreed on.
Keep each one small and self-describing
Two details matter more than people expect. Keep operations small and specific, one action each, so the agent has fewer wrong buttons to press. Guidance from a leading AI lab is blunt here: more tools do not automatically mean better results. And give each operation a plain description of what it does, so the agent picks the right one without trial and error. Teams underinvest here, and it quietly sets how reliable the whole pipeline turns out to be.
Machine-readable results: let the agent read, not just look
A pipeline can have perfect typed operations and still leave the agent blind. The fix is the second piece: after every operation, hand back a result the agent can read, not just a finished picture.
A picture is not an answer
If crop returns only the cropped image, the agent has no idea whether it worked, what changed, or what comes next. It stops and waits for a human to look, which breaks the premise of an agent running the job on its own. Instead, the operation should return a small structured payload: what changed, the new dimensions, whether it succeeded, and any warning. The agent reads that, judges its own output, and moves on.
This is now the norm, not an aspiration. By 2026 every major model provider ships a way to force results into a declared, machine-readable shape, and the 2025 revision of the tool-calling standard added a structured-output schema: if an operation declares an output schema, the server must return results that conform to it (the standard now defines a structured output schema for tool results). A result the agent can parse is what turns a one-shot call into a step in a self-checking loop. That single property is the line between "the agent can call this" and "the agent can run with this."
Safe retries: the piece you cannot bolt on later
The third piece is the one to design in from the start, because it is the one you cannot patch in afterward. Agents retry constantly. A request times out, a connection drops, the agent re-plans mid-task and runs the same step again. That is normal behavior, not an edge case, so every operation has to be safe to run twice.
Why "run it again" is dangerous by default
Most pipelines fail this quietly. If a step changes the file in place and an agent re-runs it, a blur gets applied twice or a counter double-increments, and nobody is watching the canvas to catch it. The work is just wrong. The standard fix is well understood in distributed systems: make each operation take the current state and return a new state, so running it again produces the same result as running it once. Engineers call this idempotent: safe to run twice. The common implementation is an idempotency key the caller sends with each request, so the pipeline recognizes a repeat and returns the original answer instead of doing the work again (the pattern is documented as core retry-safety practice).
One caution worth stating plainly. Safe retries give you predictable behavior under repeats. They do not guarantee an operation runs exactly once. It is worth keeping that distinction clear from the start, so design for "a retry is harmless," not "a retry can never happen." This is a foundational choice, which is why the surface you adopt or build matters so much.
Shared state and observability: what a crew of agents needs
The first three pieces make a single agent reliable. The last two are what you add the moment more than one agent touches the work, or the moment something goes wrong and you have to find out why.
Shared state: where the work lives between agents
When a brief fans out to a crew, one agent reframing, one color-correcting, one captioning, one checking the result, they need a single place the work lives so each picks up where the last left off. The common pattern is a shared, addressable workspace the agents read from and write to, rather than passing the whole file hand to hand. Each specialist acts when its part is ready, and the shared state keeps everyone on the same version (shared-workspace coordination is a documented 2026 multi-agent pattern). Get it wrong and agents overwrite each other or lose the thread. Get it right and a crew behaves like a team.
Observability: how you debug a system that decides its own steps
This is the piece that pays off most once a system is live. When an agent chooses its own steps, you cannot debug it by reading your code, because the path changes every run. You need a record of what actually happened: which operation ran, what it returned, where the agent changed course. The industry has converged on a vendor-neutral way to capture this, where every model call and tool call becomes a traceable span in a timeline of the run (open observability conventions now cover agent and tool steps directly).
The catch is that this is not a one-time setup. Change one agent or one model and the whole flow can behave differently, so the observability to catch it is a standing cost, not a box you tick once.
Build it yourself, or adopt the pattern
So you have the five pieces. The honest question is whether you assemble them yourself or take them off the shelf, and there is a real case for each.
What each path actually costs
Building it yourself gives you total control and no dependency. If a custom media pipeline is your actual product, that may be exactly right. You also own all five pieces forever, which is months of work plus the standing maintenance the observability piece alone guarantees.
Adopting the pattern flips that. You ship in a sprint, the hard pieces come built and maintained, and you spend your effort on what makes you different, your workflow and your models, instead of the plumbing. The trade is a dependency on the layer, though one built on the open standards above rather than a private format. And it is not all-or-nothing. A good layer ships native operations and lets you add your own alongside them, so adopting does not cost you the customization building would have given.
Either way you are not betting on a fringe idea. The open standards under all five pieces, the tool-calling protocol, structured outputs, idempotency keys, and the tracing conventions, are the same ones the broader agent ecosystem settled on through 2025 and 2026. The only real question is how much of the plumbing you want to own.
For the deeper argument on why the surface matters more than the agent, see the full case here.
FAQ
Do I need all five pieces, or can I start with a couple?
You can start with typed operations and machine-readable results, and a single agent will work. The moment more than one agent runs a job, or the agent retries on its own, the other three stop being optional. Safe retries keep a re-run from corrupting a file, shared state lets a crew hand off without losing the thread, and observability is how you find out why a run went wrong. Most teams add them in that order as the work grows.
Can I add this to a pipeline I already have?
Some of it, yes. You can wrap existing calls in a typed layer and add structured results without a rebuild. The hard retrofit is safe retries. If a step changes a file in place, a re-run can double or corrupt the work, and fixing that means changing how the step handles state. Plan for that one piece to be real work, and design the rest around it.
Is observability really worth the effort up front?
Yes, because it is the only way to debug a system that decides its own steps. When a run goes wrong you need to see which operation ran, what it returned, and where the agent changed course. Industry observability conventions now treat every tool call and model step as a traceable span, so the tooling exists. The cost is that it is a standing one, not a one-time setup, since changing one agent or one model can shift the whole flow.
The pieces are the pipeline
Here is the whole thing in one line. Whether an agent can run your media work is decided less by the agent you pick and more by five plain pieces underneath it: typed operations it can call, results it can read, retries that are safe, state it can share, and a record of what happened. Get those right and a crew of agents does the work while you set the direction and judge the result. Skip them and even the best agent stalls and waits for a human.
That is a design decision you own. Assemble the five pieces yourself if a custom pipeline is your product, or take them off the shelf and spend your effort on what makes you different. To see what the adopted version looks like in practice, the operations are in our docs.