Model client

What the client will cover

• Requests and conversations: the OpenAI-compatible request styles for a single response and for a chat of messages.
• Realtime sessions: a session of events that stays open for continuous exchanges, run as a session of its own and not as a one-off request.
• Streaming: output read as it is produced, with backpressure, and input sent as a stream where the endpoint accepts it.
• Tool calls: the model asks for a tool, your agent decides whether to run it, and the result goes back, each call matched to its request.
• Typed output: structured output decoded into your own types and checked before your code sees it, or the raw output if you choose it.
• More than text: text, structured data, images, audio and files, where the endpoint and model accept them.

Model bindings

A node will name the model it uses through a binding:

# nodes/summarize.node.ael
node summarize(input: Request) -> Summary {
    config {
        prompt: file("prompts/summarize.md");
        model: binding("primary_model");
        max_attempts: 2;
    }
    return write_summary(input);
}

The binding primary_model will be set by the deployment or by configuration, not by the node. It will record:

• the endpoint's address and which request style it speaks;
• the model;
• a reference to the credential, never the credential itself;
• a timeout;
• default parameters, and which parameters a single call may override.

Moving to another endpoint will change the binding, not your code. A binding without its model or credential will be reported as a configuration error before any request is sent.

Parameters

A binding will use one of two modes:

• A parameter the endpoint or model does not accept will fail with an explicit error. It will never be dropped silently.
• Leaving parameters to their defaults will never remove AEL's own limits: the output ceiling and the run's budget will still apply.
• Where the endpoint does not say which value it used, the record will say unknown; AEL will not invent one.

Compatibility

An address that calls itself OpenAI-compatible will not be taken to promise every feature. AEL will check what each binding's endpoint and model can do. A feature that is not there will fail with an explicit error, or use a fallback that is documented for it; it will never be guessed.

Where models run

You will choose local or cloud models: local model servers, optional packages for major cloud providers, and clients for inference endpoints you run yourself.

• An endpoint that speaks the OpenAI-compatible interface will be called directly, with no package.
• A cloud provider with its own sign-in or request format will be reached through an optional official package; see the Package catalog.
• No model and no inference server will be bundled into your program or installed for you.
• Prompt-driven agents will run on small devices through a gateway, and each device will keep its safe local behaviour when offline. See Microcontroller boards.

Usage and budgets

• Every call will record its input, output and total tokens. Each count will say where it came from: reported by the provider, estimated, known to be zero, or unknown. Unknown usage will never be shown as zero.
• Retries, output repair, timeouts and cancellation will draw on one shared budget, so nested retries cannot multiply cost. Routing, fallback, fan-out and verifier calls will draw on it too.
• A run's time and token usage will be reported by default when it ends; see Console and logging.

Failures

A call will return a typed result that keeps these failures apart:

• a parameter, input kind or capability the endpoint does not accept;
• an input over the endpoint's limit;
• a missing binding or credential;
• a request the endpoint refused as unauthorized;
• a timeout or a cancellation;
• an output that ended early or could not be decoded into your type.

An output that ended early will never count as a success, even if what arrived looks complete. Rate limits will be respected, with waits that never pass the deadline.

Security

• Remote endpoints will be reached over HTTPS, authenticated with TLS. A plain connection will be only for a local endpoint that you explicitly allow.
• Credentials will stay references: never in your source, your logs or a command-line argument.
• A model's output will be data. It will not be able to grant a permission, call a tool your agent did not allow, or raise a limit.

Model policies

Model policies will cover exactly one call, a bounded loop, routing, ordered fallback, parallel fan-out and an independent verifier model. They will belong to agents and nodes: see Models and providers.

Testing

Tests will run agents against mock and recorded model responses, with no network and no credentials. See Testing.

Not yet described

The format of a binding in deployment settings, and the names of the client's types and operations, will be published when the library is final.