Comparing prompts
In AEL, the Agent Engineering Language, a system prompt will be business logic: changing it will change what your agent does. ael prompt diff (command names may change before launch) will show how the prompts of your working version differ from a released one, so a prompt change can be reviewed like a change to code.
Status
Planned for AEL Beta 0.0.1. AEL is not available yet.
Usage
ael prompt diff --baseline <release>
ael prompt diff --baseline <release> --json
| Option | Meaning |
|---|---|
--baseline <release> | The released version to compare with. |
--json | The differences in machine-readable form. |
What it shows
For each agent and node whose prompts changed, it will show:
- the prompt text before and after, with the changed passages marked;
- where each prompt comes from: written inline, read from a file, from the environment or from configuration.
A prompt written inline or kept in a file will be compared as the exact text the agent will use, so a change of one word shows. A prompt taken from the environment, or from configuration resolved when a run starts, will be shown by where it comes from rather than by its text.
A prompt change will be a new version
An agent's version will be identified by the exact text of every prompt it resolves, together with its input builders, its input and output types, its tools and permissions, its model bindings and its configuration. A prompt-only change will give the agent a new version, even when no code changes.
A change that goes beyond prompt text, such as a changed type, a new tool or a wider permission, will need more than a prompt release: the deployment plan will show it and require it to be authorized. A prompt update alone will never be able to widen a tool's permissions.
What it never does
ael prompt diff will call no model and run no agent. It will change nothing, and never show secret values: a prompt that refers to a secret will show the reference, not its value.
In a release
Run an evaluation, compare the prompts, then plan the deployment:
ael eval --suite support-regression
ael prompt diff --baseline <release>
ael deploy plan
Runs already under way will stay pinned to the version they were admitted with, to the end. Deployment plans describes promotion and rollback.
Not settled yet
How released versions are named, and so what you give as <release>, will be described when the release format is final.
Related
- System prompts: prompt sources and fallback.
- Evaluations and prompt releases: versions, suites, promotion and rollback.