Hot on the heels of Griptape 1.1, we are excited to bring you version 1.2 of the Griptape Framework. The 1.2 release adds support for Structured Output to all Prompt Drivers, the ability to query vector stores with vectors, as well as enhancements to thread handling with Assistants. As usual, we have also squashed some bugs. Let’s jump straight in with a deep dive into the new Structured Output feature.
Structured Output
Structured Output is a critical feature for developers building applications on LLMs. How can you reliably work with the output from LLMs or build logic based on that output if the output is structured in an unpredictable way? The answer is, of course, that you can’t.
With the new Structured Output feature in version 1.2 of the Griptape Framework, developers can provide a schema object in the output_schema attribute of a PromptTask when creating an instance of any structure that uses a PromptTask. The schema class is provided by the Schema python library. This is defined as tool dependency within the Griptape Framework and automatically installed along with the Framework.
Let’s try an example. As the author of this post is English, this one is going to be weather themed. Let’s say we are interested in the typical temperature at certain times of the year in specific cities. We want to work with this information programmatically so we want our output to be in the following format.
{"city": string, "results": [{"month": integer, "daytime_temperature": integer, "nighttime_temperature": integer, "units": string}]}
We can define a schema for this using the Schema Library like this. Note that we have a nested Schema in this example. Your schema can be as simple or a complex as you need to match your requirements:
schema.Schema(
{
"city": str,
"results": [
schema.Schema(
{
"month": int,
"daytime_temperature": int,
"nighttime_temperature": int,
"units": str,
}
)
],
}
)
The nested schema allows us to make a request to the model asking for the typical day and night time temperature in multiple months and get the results back as dict containing a nested list. Let’s try that in a complete sample application and take a look at the results. Here is our code:
import schema
from rich.pretty import pprint
from griptape.structures import Agent
from griptape.drivers import (
OpenAiChatPromptDriver,
OllamaPromptDriver,
AnthropicPromptDriver,
)
agent = Agent(
input="what are the typical daytime and nighttime temperatures in {{ args[0] }} in January and July?",
prompt_driver=OpenAiChatPromptDriver(
model="gpt-4o",
structured_output_strategy="native", # optional we'll discuss this later in the post
),
output_schema=schema.Schema(
{
"city": str,
"results": [
schema.Schema(
{
"month": int,
"daytime_temperature": int,
"nighttime_temperature": int,
"units": str,
}
)
],
}
),
)
result = agent.run("New York").output.value
print(type(result)) # verify output.value type
pprint(result) # pretty print the result
The results should be something like this:
<class 'dict'>
{'city': 'New York', 'results': [{'month': 1, 'daytime_temperature': 39, 'nighttime_temperature': 26, 'units': 'Fahrenheit'}, {'month': 7, 'daytime_temperature': 85, 'nighttime_temperature': 70, 'units': 'Fahrenheit'}]}
The Structured Output feature in Griptape ensures that the results are a dict object in the format specified in the schema definition that I provided. The results also confirm what I already know about the temperature in New York. It’s warm in the summer and pretty cold at this time of year, in January.
Models can be very effective in determining how to match the schema to the desired output format. You can test this by adding another entry to the nested list within the schema definition: "short_units": str. If you run the application again after making this addition to the schema definition, you will see that the model figures out that this should be a short representation of either Fahrenheit or Celsius, and populates the response accordingly. With this change the entry for January in the nested list changes to this:
[{'month': 1, 'daytime_temperature': 39, 'nighttime_temperature': 26, 'units': 'Fahrenheit', 'short_units': '°F'}, ... ]
You can experiment with changing the agent configuration from agent.run(“New York”) to agent.run(“Helsinki”) or (to “Sydney” if you want somewhere a little warmer) to check whether this works for cities where the metric system is used for temperature.
Selecting a Structured Output Strategy
You might have noticed that we set the structured_output_strategy attribute to "native" in the application example. This is the default value for OpenAI’s gpt-4o model, so this was optional (as we mentioned in the comment). Setting the structured_output_strategy attribute allows you to control how Griptape makes the model generate structured output. As usual, the Griptape Framework sets an optimal default for each different model. For 99% of cases we recommend that you do not set a different structured output strategy, so test with the default option before making changes.
The three different options, in order of their relative effectiveness for 99% of use-cases, are as follows:
-
native- The Driver will use the LLM's structured output functionality provided by the model provider’s API, if available. -
tool- The Driver will add theStructuredOutputToolTool and do its best to force the LLM to use the Tool to validate its output -
rule- The Driver will add aJsonSchemaRuleto the Task's system prompt. This does not guarantee that the LLM will output JSON and should only be used as a last resort.
To see how these different strategies work we can turn on DEBUG logging and inspect the requests that are made to the LLM providers API endpoint with each of the different structured_output_strategy options.
Using OpenAI’s gpt-4o model with structured_output_strategy="native', the request that is made to the LLM contains the response_format parameter with the following payload:
"response_format": {
"type": "json_schema",
"json_schema": {
"name": "Output",
"schema": {
"type": "object",
"properties": {
"months": {
"type": "array",
"items": {
"type": "string"
}
}
},
"required": [
"months"
],
"additionalProperties": false,
"$id": "Output",
"$schema": "http://json-schema.org/draft-07/schema#"
},
"strict": true
}
}
Using the tool strategy the response_format parameter is not used and instead we include a tools parameter in the request that instructs the model to use StructuredOutputTool from the Griptape Framework.
"tools": [
{
"function": {
"name": "StructuredOutputTool_provide_output",
"description": "Used to provide the final response which ends this conversation.",
"parameters": {
"type": "object",
"properties": {
"values": {
"type": "object",
"properties": {
"months": {
"type": "array",
"items": {
"type": "string"
}
}
},
"required": [
"months"
],
"additionalProperties": false
}
},
"required": [
"values"
],
"additionalProperties": false,
"$id": "Parameters Schema",
"$schema": "http://json-schema.org/draft-07/schema#"
}
},
"type": "function"
}
]
Lastly, with structured_output_strategy="rule" the framework injects a system prompt into the messages parameter in the request that looks like this.
"messages": [
{
"role": "system",
"content": "Output valid JSON that matches this schema: {\"type\": \"object\", \"properties\": {\"months\": {\"type\": \"array\", \"items\": {\"type\": \"string\"}}}, \"required\": [\"months\"], \"additionalProperties\": false, \"$id\": \"Output Schema\", \"$schema\": \"http://json-schema.org/draft-07/schema#\"}\nNo markdown, code snippets, code blocks, or backticks."
},
{
"role": "user",
"content": "Give me a list of the months in the year in order"
}
]
Not all models support native and/or tools so we select the default structured_output_strategy on the basis of the best option supported by each specific model.
We’re excited to hear how you put the new Structured Output features to work in your applications!
Other changes & fixes in Griptape 1.2
For a full list of the all the fixes included in the Griptape 1.2 release, please head over to the CHANGELOG for the Griptape Framework at https://github.com/griptape-ai/griptape/blob/main/CHANGELOG.md
How to get started
Griptape Framework 1.2 is available on PyPi now and you can download it with pip, poetry, or another Python package manager of your choice. We would love to hear your feedback on the changes and new features. If you want to ask any questions about the other features in this release or learn more about structured output, please head over to the Griptape Discord.