Pass host cancellation into bounded generation and observe committed tokens synchronously.
Goal
Pass host cancellation into bounded generation and observe committed tokens synchronously.
Applicability
Use this example as a host-architecture reference for the named framework. Resolve and pin dependencies through the validation solution or the consuming repository policy.
Required packages
- UAIX.LmRuntime.LocalEndpoint
Host responsibilities
- Acquire and approve the model outside request or generation paths.
- Prepare the model-specific prompt and enforce limits.
- Own cancellation, UI dispatch, error presentation, persistence, and evidence retention.
Complete code
using System;
using System.Threading;
using UAIX.LmRuntime.LocalEndpoint;
namespace GgufRuntime.Examples.Cancellation;
/// <summary>Runs bounded generation with host cancellation and synchronous token observation.</summary>
public static class GenerationRunner
{
/// <summary>Generates one response while observing committed tokens.</summary>
/// <param name="session">The isolated local GGUF session.</param>
/// <param name="preparedPrompt">The host-prepared prompt.</param>
/// <param name="cancellationToken">The host cancellation token.</param>
/// <returns>The complete generation result.</returns>
public static LocalGgufGenerationResult Generate(
LocalGgufSession session,
String preparedPrompt,
CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(session);
ArgumentException.ThrowIfNullOrWhiteSpace(preparedPrompt);
return session.GenerateGreedy(
new LocalGgufGenerationRequest
{
Prompt = preparedPrompt,
MaximumTokens = 64,
ResetSession = true,
AddSpecialTokens = true,
ParseSpecialTokens = false,
RemoveSpecialTokens = true
},
token => Console.WriteLine(
$"{session.SessionId}:{token.Sequence}:{token.TokenId}:{token.SelectedLogit}"),
cancellationToken);
}
}Expected behavior
The host either produces the narrow result described by the example or surfaces a specific identity, validation, load, generation, cancellation, or disposal failure. No hidden fallback is introduced.
Failure cases
- Invalid or untrusted input identity.
- GGUF structure, architecture, tokenizer, storage, kernel, or memory incompatibility.
- Cancellation or host shutdown.
- Framework dispatch or lifecycle misuse.
Security and trust notes
Do not accept remote model uploads, arbitrary paths, arbitrary URLs, provider keys, commands, or private-network targets. The loopback example binds to 127.0.0.1 and still requires a new threat model before broader exposure.
Disposal requirements
The host must finish or cancel generation, dispose the session, then dispose the model. Service and UI examples must connect this order to application shutdown.
Evidence produced
- Resolved package identity and version from the build.
- Model hash and byte count from intake.
- Session identifier, bounds, cancellation state, selected path, result, timing method, and limitations when the host records them.
Build-validation status
State: Experimental. Last compilation attempt UTC: 2026-06-25T00:00:00Z. The source was API-reviewed against current canonical public documentation; local compilation is blocked until a .NET SDK and package restore are available. CI is configured to produce the retained build result.
Open canonical package documentation.
Operational boundary
What this page covers: Pass host cancellation into bounded generation and observe committed tokens synchronously.
Available now: The complete source and validation-project mapping are included in this release.
Host responsibility: Provide real inputs, run restore/build/tests, test the host lifecycle, and retain exact evidence.
Not claimed: A successful local build in this environment, finished downloadable application, universal compatibility, or production/safety certification.
Canonical sources: Canonical package guide · NuGet
Last reviewed UTC: