InferenceClient

Namespace: Daisi.SDK.Clients.V1.Host

Factory

var factory = new InferenceClientFactory();
var infClient = factory.Create();

Properties

• InferenceSessionManager SessionManager get
  Handles setting up and maintaining the Session.
• string? InferenceId get
  Unique Inference Session Id from Orc.

Methods

• Task<CloseInferenceResponse> CloseAsync(bool closeOrcSession = true)
  Closes the inference session with the Orc and Host. Closes the whole Orc Session by default as well.
• AsyncServerStreamingCall<SendInferenceResponse> Send(string)
  Sends an inference request using plain text. Requires connected session, but that will be handled by the client.
• AsyncServerStreamingCall<SendInferenceResponse> Send(SendInferenceRequest)
  Sends an inference request with optional parameters. Requires connected session, but that will be handled by the client.
• InferenceStatsResponse Stats(InferenceStatsRequest)
  Retrieves the statistics from the last Send request as well as the total Inference Session.

Walkthrough: Send an inference request to a host

1. Create an InferenceClientFactory.
2. Create an InferenceClient.
3. Create a SendInferenceRequest, if you want specific criteria set on the request. Otherwise, you can just send a string with your user input.
4. Call Send and handle stream.

// Create the factory.
var infClientFactory = new InferenceClientFactory();

// Create a new Inference Session
var infClient = infClientFactory.Create();

// Use CreateDefault so that you don't have to setup all of the settings manually.
var sendInferenceRequest = SendInferenceRequest.CreateDefault();
sendInferenceRequest.Text = "Tell me a joke about an orc";

// Send the request to the Host. 
// The client determines FOC or DC.
var call = infClient.Send(sendInferenceRequest);

// This also works if you don't want to create the full SendInferenceRequest first.
// var call = infClient.Send("Tell me a joke about an orc");

await foreach(var resp in call.ResponseStream.ReadAllAsync())
{
    Console.WriteLine(resp.Content);
}