How to Use JSON Schema Output in Dify?

JSON Schema is a specification for describing JSON data structures. Developers can define JSON Schema structures to specify that LLM outputs strictly adhere to the defined data or content, such as generating clear document or code structures.

Models Supporting JSON Schema Functionality

• gpt-4o-mini-2024-07-18 and later versions

• gpt-4o-2024-08-06 and later versions

For more information on the structured output capabilities of OpenAI series models, please refer to Structured Outputs.

Usage of Structured Outputs

1. Connect the LLM to tools, functions, data, and other components within the system. Set strict: true in the function definition. When enabled, the Structured Outputs feature ensures that the parameters generated by the LLM for function calls precisely match the JSON schema you provided in the function definition.

2. When the LLM responds to users, it outputs content in a structured format according to the definitions in the JSON Schema.

Enabling JSON Schema in Dify

Switch the LLM in your application to one of the models supporting JSON Schema output mentioned above. Then, in the settings form, enable JSON Schema and fill in the JSON Schema template. Simultaneously, enable the response_format column and switch it to the json_schema format.

The content generated by the LLM supports output in the following format:

• Text: Output in text format

Defining JSON Schema Templates

You can refer to the following JSON Schema format to define your template content:

{
    "name": "template_schema",
    "description": "A generic template for JSON Schema",
    "strict": true,
    "schema": {
        "type": "object",
        "properties": {
            "field1": {
                "type": "string",
                "description": "Description of field1"
            },
            "field2": {
                "type": "number",
                "description": "Description of field2"
            },
            "field3": {
                "type": "array",
                "description": "Description of field3",
                "items": {
                    "type": "string"
                }
            },
            "field4": {
                "type": "object",
                "description": "Description of field4",
                "properties": {
                    "subfield1": {
                        "type": "string",
                        "description": "Description of subfield1"
                    }
                },
                "required": ["subfield1"],
                "additionalProperties": false
            }
        },
        "required": ["field1", "field2", "field3", "field4"],
        "additionalProperties": false
    }
}

Step-by-step guide:

1. Define basic information:

   • Set name: Choose a descriptive name for your schema.

   • Add description: Briefly explain the purpose of the schema.

   • Set strict: true to ensure strict mode.

2. Create the schema object:

   • Set type: "object" to specify the root level as an object type.

   • Add a properties object to define all fields.

3. Define fields:

   • Create an object for each field, including type and description.

   • Common types: string, number, boolean, array, object.

   • For arrays, use items to define element types.

   • For objects, recursively define properties.

4. Set constraints:

   • Add a required array at each level, listing all required fields.

   • Set additionalProperties: false at each object level.

5. Handle special fields:

   • Use enum to restrict optional values.

   • Use $ref to implement recursive structures.

Example

1. Chain of thought（routine）

JSON Schema Example

{
    "name": "math_reasoning",
    "description": "Records steps and final answer for mathematical reasoning",
    "strict": true,
    "schema": {
        "type": "object",
        "properties": {
            "steps": {
                "type": "array",
                "description": "Array of reasoning steps",
                "items": {
                    "type": "object",
                    "properties": {
                        "explanation": {
                            "type": "string",
                            "description": "Explanation of the reasoning step"
                        },
                        "output": {
                            "type": "string",
                            "description": "Output of the reasoning step"
                        }
                    },
                    "required": ["explanation", "output"],
                    "additionalProperties": false
                }
            },
            "final_answer": {
                "type": "string",
                "description": "The final answer to the mathematical problem"
            }
        },
        "additionalProperties": false,
        "required": ["steps", "final_answer"]
    }
}

Prompts

You are a helpful math tutor. You will be provided with a math problem,
and your goal will be to output a step by step solution, along with a final answer.
For each step, just provide the output as an equation use the explanation field to detail the reasoning.

UI generation（root recursion mode）

JSON Schema Example

{
        "name": "ui",
        "description": "Dynamically generated UI",
        "strict": true,
        "schema": {
            "type": "object",
            "properties": {
                "type": {
                    "type": "string",
                    "description": "The type of the UI component",
                    "enum": ["div", "button", "header", "section", "field", "form"]
                },
                "label": {
                    "type": "string",
                    "description": "The label of the UI component, used for buttons or form fields"
                },
                "children": {
                    "type": "array",
                    "description": "Nested UI components",
                    "items": {
                        "$ref": "#"
                    }
                },
                "attributes": {
                    "type": "array",
                    "description": "Arbitrary attributes for the UI component, suitable for any element",
                    "items": {
                        "type": "object",
                        "properties": {
                            "name": {
                                "type": "string",
                                "description": "The name of the attribute, for example onClick or className"
                            },
                            "value": {
                                "type": "string",
                                "description": "The value of the attribute"
                            }
                        },
                      "additionalProperties": false,
                      "required": ["name", "value"]
                    }
                }
            },
            "required": ["type", "label", "children", "attributes"],
            "additionalProperties": false
        }
    }

Prompts

You are a UI generator AI. Convert the user input into a UI.

Example Output:

Tips

• Ensure that the application prompt includes instructions on how to handle cases where user input cannot produce a valid response.

• The model will always attempt to follow the provided schema. If the input content is completely unrelated to the specified schema, it may cause the LLM to generate hallucinations.

• If the LLM detects that the input is incompatible with the task, you can include language in the prompt to specify returning empty parameters or specific sentences.

• All fields must be required, For details, please refer to Supported Schemas.
• additionalProperties: false must always be set in objects.
• The root level object of the schema must be an object.

Appendix

• Introduction to Structured Outputs
• Structured Output