Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Schema template --save --schema support #787

Merged
merged 5 commits into from
Feb 27, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 34 additions & 4 deletions docs/templates.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,13 @@ You can also save default parameters:
llm --system 'Summarize this text in the voice of $voice' \
--model gpt-4 -p voice GlaDOS --save summarize
```

Add `--schema` to bake a {ref}`schema <usage-schemas>` into your template:

```bash
llm --schema dog.schema.json 'invent a dog' --save dog
```

If you add `--extract` the setting to {ref}`extract the first fenced code block <usage-extract-fenced-code>` will be persisted in the template.
```bash
llm --system 'write a Python function' --extract --save python-function
Expand Down Expand Up @@ -68,15 +75,18 @@ This will open the system default editor.

:::{tip}
You can control which editor will be used here using the `EDITOR` environment variable - for example, to use VS Code:

export EDITOR="code -w"

```bash
export EDITOR="code -w"
```
Add that to your `~/.zshrc` or `~/.bashrc` file depending on which shell you use (`zsh` is the default on macOS since macOS Catalina in 2019).
:::

You can also create a file called `summary.yaml` in the folder shown by running `llm templates path`, for example:
```bash
$ llm templates path
llm templates path
```
Example output:
```
/Users/simon/Library/Application Support/io.datasette.llm/templates
```

Expand Down Expand Up @@ -120,6 +130,26 @@ You can combine system and regular prompts like so:
system: You speak like an excitable Victorian adventurer
prompt: 'Summarize this: $input'
```
### Schemas

Use the `schema:` key to embed a JSON schema (as YAML) in your template. The easiest way to create these is with the `llm --schema ... --save name-of-template` command - the result should look something like this:

```yaml
name: dogs
schema_object:
properties:
dogs:
items:
properties:
bio:
type: string
name:
type: string
type: object
type: array
type: object
```


### Additional template variables

Expand Down
15 changes: 14 additions & 1 deletion docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,20 @@ llm --schema '{
}
}' -m gpt-4o-mini 'invent two dogs'
```
The JSON returned from the model should match that schema.
LLM will pass this to the model, whish should result in JSON returned from the model matching that schema.

You can also save the JSON schema to a file and reference the filename using `--schema`:

```bash
llm --schema dogs.schema.json 'invent two dogs'
```
Or save your schema {ref}`to a template <prompt-templates>` like this:

```bash
llm --schema dogs.schema.json --save dogs
# Then to use it:
llm -t dogs 'invent two dogs'
```

Be warned that different models may support different dialects of the JSON schema specification.

Expand Down
7 changes: 6 additions & 1 deletion llm/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -299,7 +299,7 @@ def prompt(
model_aliases = get_model_aliases()

def read_prompt():
nonlocal prompt
nonlocal prompt, schema

# Is there extra prompt available on stdin?
stdin_prompt = None
Expand All @@ -318,6 +318,7 @@ def read_prompt():
and sys.stdin.isatty()
and not attachments
and not attachment_types
and not schema
):
# Hang waiting for input to stdin (unless --save)
prompt = sys.stdin.read()
Expand Down Expand Up @@ -356,6 +357,8 @@ def read_prompt():
to_save["extract"] = True
if extract_last:
to_save["extract_last"] = True
if schema:
to_save["schema_object"] = schema
path.write_text(
yaml.dump(
to_save,
Expand All @@ -374,6 +377,8 @@ def read_prompt():
template_obj = load_template(template)
extract = template_obj.extract
extract_last = template_obj.extract_last
if template_obj.schema_object:
schema = template_obj.schema_object
prompt = read_prompt()
try:
prompt, system = template_obj.evaluate(prompt, params)
Expand Down
1 change: 1 addition & 0 deletions llm/templates.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ class Template(BaseModel):
# Should a fenced code block be extracted?
extract: Optional[bool] = None
extract_last: Optional[bool] = None
schema_object: Optional[dict] = None

model_config = ConfigDict(extra="forbid")

Expand Down
6 changes: 6 additions & 0 deletions tests/test_templates.py
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,12 @@ def test_templates_list(templates_path, args):
{"system": "write python", "extract": True},
None,
),
# So should schemas
(
["--schema", '{"properties": {"name": {"type": "string"}}}'],
{"schema_object": {"properties": {"name": {"type": "string"}}}},
None,
),
),
)
def test_templates_prompt_save(templates_path, args, expected_prompt, expected_error):
Expand Down
Loading