Load one verified GGUF model, create an isolated session, run bounded generation, and dispose in order.
Goal
Load one verified GGUF model, create an isolated session, run bounded generation, and dispose in order.
Applicability
Use this example as a host-architecture reference for the named framework. Resolve and pin dependencies through the validation solution or the consuming repository policy.
Required packages
- UAIX.LmRuntime.LocalEndpoint
Host responsibilities
- Acquire and approve the model outside request or generation paths.
- Prepare the model-specific prompt and enforce limits.
- Own cancellation, UI dispatch, error presentation, persistence, and evidence retention.
Complete code
using System;
using System.Globalization;
using System.IO;
using System.Threading;
using UAIX.LmRuntime.LocalEndpoint;
namespace GgufRuntime.Examples.ConsoleQuickStart;
/// <summary>
/// Loads one verified local GGUF artifact and performs bounded deterministic generation.
/// </summary>
public static class Program
{
/// <summary>
/// Runs the verified local-model example.
/// </summary>
/// <param name="arguments">Model path, SHA-256, byte count, and host-prepared prompt.</param>
/// <returns>Zero after generation; two when required arguments are missing.</returns>
public static Int32 Main(String[] arguments)
{
if (arguments.Length != 4)
{
Console.Error.WriteLine(
"Usage: ConsoleQuickStart <model.gguf> <sha256> <byte-count> <prepared-prompt>");
return 2;
}
String modelPath = Path.GetFullPath(arguments[0]);
String expectedSha256 = arguments[1];
Int64 expectedByteCount = Int64.Parse(arguments[2], CultureInfo.InvariantCulture);
String preparedPrompt = arguments[3];
String trustedModelRoot = Path.GetDirectoryName(modelPath)
?? throw new InvalidOperationException("The model path has no parent directory.");
LocalGgufRuntime runtime = new LocalGgufRuntime();
LocalUaixRuntimeContext uaixContext = new LocalUaixRuntimeContext
{
LoadedUaixProfilePresent = true,
LoadedUaixProfileId = "profile1",
LoadedUaixProfileDisplayName = "Local model profile",
LoadedUaixLoadSessionId = "load1",
LoadedUaixUaiRelativePath = "Memories/Profiles/profile1/.uai",
LoadedUaixSessionRelativePath = "Memories/Sessions/load1.json",
LongTermMemoryRootId = "wiki1",
LongTermMemoryRootRelativePath = "Profiles/profile1",
LongTermMemoryMode = LocalUaixLongTermMemoryMode.Isolated,
RuntimeExecutionAllowed = false,
MemoryCanOverridePolicy = false,
CommandExecutionAllowed = false,
NetworkAccessAllowed = false,
ProviderApisAllowed = false,
WebsitePromptIntakeAllowed = false,
TelemetryEnabled = false,
AutoExportAllowed = false
};
_ = LocalGgufRuntime.VerifyUaixRuntimeContext(uaixContext);
using LocalGgufModel model = runtime.LoadVerifiedModel(
modelPath,
new LocalGgufFileExpectation
{
ModelSha256 = expectedSha256,
ModelByteCount = expectedByteCount
},
new LocalGgufModelLoadOptions
{
AllowedRootDirectory = trustedModelRoot,
RejectReparsePoints = true,
MaximumModelBytes = expectedByteCount,
ExecutionLimits = new LocalGgufExecutionLimits
{
MaximumPromptCharacters = 32_768,
MaximumGeneratedTokens = 256,
MaximumStopTokenCount = 32
}
});
using LocalGgufSession session = model.CreateSession(
new LocalGgufSessionContext
{
SessionId = "session1",
UaixRuntimeContext = uaixContext
});
LocalGgufGenerationResult result = session.GenerateGreedy(
new LocalGgufGenerationRequest
{
Prompt = preparedPrompt,
MaximumTokens = 128,
ResetSession = true,
AddSpecialTokens = false,
ParseSpecialTokens = false,
EmitTokenizerTrace = false,
RemoveSpecialTokens = false,
UnparseSpecialTokens = true,
CleanSpaces = false
},
CancellationToken.None);
Console.WriteLine(result.GeneratedText);
return 0;
}
}Expected behavior
The host either produces the narrow result described by the example or surfaces a specific identity, validation, load, generation, cancellation, or disposal failure. No hidden fallback is introduced.
Failure cases
- Invalid or untrusted input identity.
- GGUF structure, architecture, tokenizer, storage, kernel, or memory incompatibility.
- Cancellation or host shutdown.
- Framework dispatch or lifecycle misuse.
Security and trust notes
Do not accept remote model uploads, arbitrary paths, arbitrary URLs, provider keys, commands, or private-network targets. The loopback example binds to 127.0.0.1 and still requires a new threat model before broader exposure.
Disposal requirements
The host must finish or cancel generation, dispose the session, then dispose the model. Service and UI examples must connect this order to application shutdown.
Evidence produced
- Resolved package identity and version from the build.
- Model hash and byte count from intake.
- Session identifier, bounds, cancellation state, selected path, result, timing method, and limitations when the host records them.
Build-validation status
State: Experimental. Last compilation attempt UTC: 2026-06-25T00:00:00Z. The source was API-reviewed against current canonical public documentation; local compilation is blocked until a .NET SDK and package restore are available. CI is configured to produce the retained build result.
Open canonical package documentation.
Operational boundary
What this page covers: Load one verified GGUF model, create an isolated session, run bounded generation, and dispose in order.
Available now: The complete source and validation-project mapping are included in this release.
Host responsibility: Provide real inputs, run restore/build/tests, test the host lifecycle, and retain exact evidence.
Not claimed: A successful local build in this environment, finished downloadable application, universal compatibility, or production/safety certification.
Canonical sources: Canonical package guide · NuGet
Last reviewed UTC: