A Structured Output is All You Need

Learn how to use structured outputs to extract structured data from text.

Nadeesha Cabral on 09-03-2025

There's a tendency to jump straight to complex autonomous agents for problems that could be solved with simpler approaches. Let's examine why structured outputs are sufficient for most AI use cases, and when agents actually provide value.

The Power of Structured Outputs

Structured outputs are the foundation of nearly all productive LLM applications. They provide:

  1. Predictable formats - Data in consistent, machine-readable structures
  2. Clear constraints - Well-defined schemas limit the output space
  3. Reliable extraction - Transform unstructured content into structured data consistently

Looking at Inferable's implementation, we see how straightforward this is:

const { events } = await ctx.llm.structured({
  input: `Federal reserve was founded in 1913 after the banking panic of 1907.`,
  schema: z.object({
    events: z.array(
      z.object({
        event: z.string(),
        year: z.number(),
      }),
    ),
  }),
});

The LLM handles the natural language processing, but the output conforms to a pre-defined structure that's immediately usable.

When Agents Are Overkill

Consider a customer support workflow that categorizes tickets and generates responses. This could be implemented with a React Agent:

const agentResult = await ctx.agents.react({
  name: "ticketResponder",
  instructions: "Categorize and respond to customer support tickets",
  input: JSON.stringify({ ticketContent }),
  tools: ["searchKnowledgeBase", "getCustomerInfo"],
  resultSchema: z.object({
    category: z.string(),
    response: z.string(),
    escalate: z.boolean()
  })
});

But this introduces unnecessary complexity. The same functionality can be achieved more efficiently with structured outputs:

// Step 1: Categorize the ticket
const { category, priority } = await ctx.llm.structured({
  input: ticketContent,
  schema: z.object({
    category: z.enum(["billing", "technical", "account", "other"]),
    priority: z.enum(["low", "medium", "high"]),
  })
});

// Step 2: Get relevant information if needed
let knowledgeItems = [];
if (category === "technical") {
  knowledgeItems = await searchKnowledgeBase(category);
}

// Step 3: Generate response with context
const { response, escalate } = await ctx.llm.structured({
  input: JSON.stringify({ 
    ticket: ticketContent, 
    category, 
    knowledgeItems 
  }),
  schema: z.object({
    response: z.string(),
    escalate: z.boolean()
  })
});

if (escalate) {
  // Send to a human
} else {
  // Send response
}

This workflow:

  • Has clear, discrete steps
  • Maintains control flow in your code, not the agent
  • Is easier to debug, test, and monitor
  • Allows for more specific prompting at each stage

When Agents Actually Make Sense

Agents become valuable when dealing with:

  1. Dynamic, multi-step reasoning that requires adjusting strategies based on intermediate results
  2. Unbounded exploration problems where the path to the answer isn't predetermined
  3. Tool-use with complex decision trees requiring multiple iterations

Here's a concrete example where an agent provides genuine value - a research assistant that needs to recursively explore and refine information:

const researchResult = await ctx.agents.react({
  name: "marketResearcher",
  instructions: `Find comprehensive information about the renewable energy market in Europe.
    Start with broad searches, then drill down into specific trends.
    Compare data across countries and identify growth opportunities.`,
  input: JSON.stringify({ topic: "European renewable energy market" }),
  tools: ["search", "fetchArticle", "queryDataset"],
  resultSchema: z.object({
    keyFindings: z.array(z.string()),
    marketSize: z.object({
      value: z.number(),
      unit: z.string(),
      year: z.number()
    }),
    growthRate: z.number(),
    keyPlayers: z.array(z.string()),
    countryComparison: z.record(z.string())
  })
});

This works well as an agent because:

  • The search space is large and unbounded
  • Multiple tools must be used iteratively
  • Results from one search inform the next search
  • The agent needs to track what it's learned and what gaps remain

Even in this case, if the problem can be broken down into well-defined steps with clear inputs and outputs, structured outputs may still be simpler and more reliable. But that workflow might not be maintainable.

Therefore, at a certain point of complexity, it makes sense to use an agent.

A note on chat applications: Chat applications are by definition unbounded problems. You can't predict the next question or the next action, and the conversation can go in any direction. Depending on the constraints of the problem, it might make sense to use an agent-based approach here.

Practical Example: Refactoring an Agent to Structured Outputs

Consider this agent for processing support tickets:

const result = await ctx.agents.react({
  name: "ticketProcessor",
  instructions: "Process customer support ticket and update database",
  input: JSON.stringify({ ticket }),
  tools: ["searchCustomer", "updateTicket", "sendResponse"],
  resultSchema: z.object({
    resolution: z.string(),
    nextSteps: z.array(z.string())
  })
});

This can be refactored to a more controlled workflow:

// Extract customer info
const { customerId, issue } = await ctx.llm.structured({
  input: ticket.body,
  schema: z.object({
    customerId: z.string().optional(),
    issue: z.string(),
    category: z.enum(["billing", "technical", "account"])
  })
});

// Get customer details if available
let customer = null;
if (customerId) {
  customer = await ctx.memo("getCustomer", async () => {
    return searchCustomer(customerId);
  });
}

// Generate response with appropriate context
const { resolution, nextSteps } = await ctx.llm.structured({
  input: JSON.stringify({ 
    issue, 
    customerInfo: customer,
    knowledgeBase: await getRelevantArticles(issue)
  }),
  schema: z.object({
    resolution: z.string(),
    nextSteps: z.array(z.string())
  })
});

// Perform necessary actions
await updateTicket(ticket.id, { resolution, status: "resolved" });
await sendResponse(ticket.id, resolution);

This structured approach provides:

  • Clear visibility into each processing step
  • Explicit control over external system interactions
  • Better testability of individual components
  • More predictable error handling

Why People Think They Need Agents (But Often Don't)

Despite the advantages of structured outputs, there's a persistent belief that autonomous agents are necessary for many AI applications. Here's why, and where these concerns fall short:

Misconception 1: "Structured Outputs Are Unreliable"

Many developers have struggled with LLMs returning malformed JSON or outputs that don't match the requested schema. This experience leads to the belief that structured outputs aren't reliable enough for production use.

However, by providing feedback to the LLM about schema violations and implementing automatic retries, structured outputs become significantly more reliable. Also, providing a minimal schema (rather than a verbose JSON schema) makes it easier for the LLM to conform to the schema as well.

Here's how we implement the minimal schema.

Misconception 2: "Structured Outputs Can't Be Model-Agnostic"

There's a belief that different models require different prompting strategies to reliably produce structured outputs, making it hard to switch models or maintain compatibility.

Again, as long as the LLM can generate text, it can generate structured outputs using the aforementioned techniques. More capable models will need fewer retries, but depending on the complexity of the schema, it's possible to make a small model work.

Misconception 3: "Control Flow Becomes Awkward"

The most legitimate concern is that structuring an application around direct LLM calls creates a rigid, procedural code flow that can be harder to maintain than agent-based approaches.

The Real Tradeoff: This is a classic tradeoff between development complexity and runtime reliability:

Development Complexity (Agents) ↔ Runtime Reliability (Structured Outputs)

With agents, your code is simpler but runtime behavior is less predictable. With structured outputs, your code is more verbose but behavior is more consistent.

The Better Approach: The solution isn't to abandon structured control flow, but to improve it through:

  1. Higher-level abstractions - Creating reusable patterns for common flows
  2. Workflow orchestration - Using Inferable's workflow capabilities to manage complex sequences
  3. State management - Leveraging contextual state to simplify multi-step processes

For example, you can create higher-level abstractions for common patterns:

// A higher-level abstraction for classification + action
async function classifyAndAct(input, classificationSchema, actionFn) {
  const classification = await ctx.llm.structured({
    input,
    schema: classificationSchema
  });
  
  return await actionFn(classification);
}

// Usage becomes much cleaner
const result = await classifyAndAct(
  email.body,
  z.object({ intent: z.enum(["inquiry", "complaint", "feedback"]), urgency: z.number() }),
  async (classification) => {
    if (classification.urgency > 8) {
      return await escalateToManager(email, classification);
    } else {
      return await routeToAppropriateTeam(email, classification);
    }
  }
);

Agent failure modes

Autonomous agents have a few failure modes that most developers don't account for, unless they've worked with them extensively.

1. Control flow issues

Infinite loops are a common failure mode for agents. It happens when the agent's control flow is unbounded, and it can keep picking the same action over and over again. The agent's run loop is a recursive function that needs to have a deterministic exit condition.

This is why sometimes even the coding agents will ask you to "continue" manually.

2. Errors of commission

This is the most common failure mode for agents. They produce the wrong output, or the wrong action. With no control flow mechanism to guard against this, it's easy to end up with a broken application.

3. Errors of omission

They are also prone to errors of omission, where they simply don't take an action at all. This is much harder to catch, and is a common source of bugs.

Some of these issues can be mitigated with "Plan and Execute" agents, which plan out the steps of the agent, and then execute the steps in a controlled manner. However, your mileage may vary, depending on how good the planning step is.

Current Market

As I'm writing this, most of the "agents" that are advertised are actually LLM-based workflows that use a pre-determined control flow. They are not true "agents" in the sense of having a mind of their own. People much smarter than me have written about this, so I won't go into it too much. See Building Effective Agents and Simon Willison's post on the same topic for more.

Written by Inferable Team

Ready to build your own AI solutions? Inferable makes it easy to create powerful AI workflows, agents, and automations without wrestling with complex LLM APIs.

Try Inferable Free