Apply explicit prompt, output, stop-token, memory, and concurrency ceilings.
Load-time ceilings
Use LocalGgufExecutionLimits to bound prompt characters, generated tokens, and stop-token count before sessions execute.
Request-time ceiling
Set MaximumTokens on every generation request inside the configured upper bound.
Host queue policy
Limit concurrent generations according to measured CPU and memory behavior. A responsive UI should queue, cancel, or reject work rather than saturate the system silently.
Stop reporting
Record whether generation ended through a stop token, maximum token count, cancellation, failure, or host policy.
Available now
- The public UAIX.LmRuntime package family and canonical package documentation.
- The host-facing workflow and evidence distinctions described on this page.
Host responsibilities
- Select and trust inputs, define policy and limits, integrate the UI or service, retain application evidence, and test the exact deployment.
Not claimed
- Bounds are not performance guarantees.
- A larger limit is not evidence that the host can safely sustain it.
Operational boundary
What this page covers: Apply explicit prompt, output, stop-token, memory, and concurrency ceilings.
Available now: The documented package surfaces and this host-facing guidance are public.
Host responsibility: Own acquisition, trust, configuration, presentation, persistence, transport, policy, tests, and deployment.
Not claimed: Universal compatibility, production certification, safety certification, hidden fallbacks, or behavior beyond retained evidence.
Canonical sources: Canonical package documentation · Current release metadata
Last reviewed UTC:
Source and review record
Prerequisites
- Read Get Started
- Review host/runtime boundaries