Reporting, logs and replay

Time and token reports

Time and token usage will be reported by default on every channel: the command line, HTTP, gRPC, WebSocket and sockets. Each operation will report:

• its elapsed time;
• input tokens, output tokens and total tokens.

A server package will add its channel's report only when you add that server: a console program will carry no server code for reporting.

Honest numbers

• Every model call will count once: retries, fallbacks, parallel branches, repairs, verifiers and child agents will each be counted, and never twice.
• A count will be marked as reported by the provider, estimated or unknown. An unknown count will never be shown as zero, and an estimate will never be shown as exact.
• A count will be reported as a known zero only when AEL can tell that no model work happened. When it cannot, for example because an outside program you started may have called a model, the count will be unknown.
• A report will show tokens used, not tokens reserved for work that has not happened.
• Times will be measured with a clock that only moves forward. Durations of work that ran in parallel will never be added up to claim a longer total.

Turning reports off

You will be able to turn off each field without disabling the limits behind it:

• all reporting at once;
• time alone, or tokens alone;
• input, output or total tokens, one at a time;
• one channel.

Settings will come from, in order: an authorized override for one operation, a named deployment environment setting, the deployment's configuration, your project, then the defaults. The settings an operation starts with will stay fixed until it ends. You will be able to show times in automatic units, or in nanoseconds, microseconds, milliseconds, seconds, minutes or hours.

• Turning reporting off will never turn off budgets, limits or required audit records.
• A total can reveal a hidden count: to keep output tokens private, hide the total as well.
• The names of the settings will be confirmed when the syntax is final.

Structured logs

Structured logs will go to the console, a file, or custom or remote sinks, with redaction, rotation and retention.

• Events. Every log event will carry the tenant, the run, the agent and node, the prompt and configuration versions, a severity and a timestamp. Events will cover prompt fallbacks, model attempts, retrieval, tool and delegation calls, verification decisions, retry waits, budget exhaustion and each run's outcome.
• Sinks. You will send events to the console, to files as JSON lines, to a remote collector or to a sink you write in AEL, and route them by severity or by agent.
• Rotation and retention. Files will roll over by size or time, and old files will be removed by your retention rules.
• Redaction. By default, logs will record metadata and redacted errors. Recording prompts, inputs or outputs will be a separate choice, with its own access and retention rules. Credentials will never be logged.
• Bounded. A slow or failing sink will never hold up your agents indefinitely: its buffer will be bounded, and a failure will be reported through a fallback route.
• Audit apart. The program's own audit records will be kept apart from your application's logs, and your code will not be able to erase them.

Console and logging describes events, sinks and redaction in detail, and Logging and monitoring where logs go on each kind of target.

Record and replay

You will record control and orchestration runs and replay them to find the first point where behaviour diverged.

• A recording will hold what came from outside the run: inputs, timing, scheduling choices, random seeds and the responses of models, tools and services, with the identity of the build, prompts and configuration.
• Replay will feed the recorded responses back in, and never make a live call or a real side effect.
• A divergence report will show the first event that differs, the source and prompt involved, and the expected and observed typed values.
• An exact replay will need the same build and target. Replaying across a changed build, prompt or machine will be a separate, best-effort mode, and will say so.
• A recording with gaps will be marked incomplete rather than treated as whole. Recordings will have size limits, redaction and retention rules like logs.
• Replay will never need a model's hidden reasoning, and it will not make a live model's output reproducible.

The command that records and replays runs has not been named yet; the Command-line reference lists it among the capabilities whose commands will be named before launch.