uncloseai.com/languages/rust/README.md

UncloseAI Rust Client

A Rust client library for interacting with vLLM, Ollama, and OpenAI-compatible APIs.

Features

• 🔍 Automatic Model Discovery - Discovers available models from configured endpoints
• 💬 Chat Completions - Both streaming and non-streaming modes
• 🎙️ Text-to-Speech - Generate audio from text with multiple voice options
• 🔄 Multiple Endpoints - Support for multiple model and TTS endpoints
• 🛡️ Error Handling - Comprehensive error handling with custom error types
• 🦀 Type Safe - Full type safety with Rust's type system
• ⚡ Async/Await - Built on Tokio for high-performance async I/O

Installation

Add to your Cargo.toml:

[dependencies]
uncloseai = "1.0"
tokio = { version = "1.42", features = ["full"] }
futures-util = "0.3"

Or use as a local dependency:

[dependencies]
uncloseai = { path = "../path/to/uncloseai" }

Quick Start

use uncloseai::{UncloseAI, ChatMessage};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Initialize client (auto-discovers from environment variables)
    let client = UncloseAI::new(None).await?;

    // Non-streaming chat
    let response = client.chat(
        "auto",
        vec![ChatMessage::user("Hello!")],
        None
    ).await?;
    println!("{}", response.choices[0].message.content);

    // Streaming chat
    let mut stream = client.chat_stream(
        "auto",
        vec![ChatMessage::user("Write a story")],
        None
    ).await?;

    while let Some(chunk) = stream.next().await {
        if let Ok(chunk) = chunk {
            if let Some(content) = &chunk.choices[0].delta.content {
                print!("{}", content);
            }
        }
    }

    Ok(())
}

Configuration

Environment Variables

# Model endpoints (numbered 1-9999)
export MODEL_ENDPOINT_1="https://hermes.ai.unturf.com/v1"
export MODEL_ENDPOINT_2="https://qwen.ai.unturf.com/v1"

# TTS endpoints (numbered 1-9999)
export TTS_ENDPOINT_1="https://speech.ai.unturf.com/v1"

Programmatic Configuration

use uncloseai::{UncloseAI, UncloseAIConfig};

let config = UncloseAIConfig {
    endpoints: Some(vec!["https://api.example.com/v1".to_string()]),
    tts_endpoints: Some(vec!["https://tts.example.com/v1".to_string()]),
    api_key: Some("your-api-key".to_string()),
    timeout: 30,
    debug: true,
};

let client = UncloseAI::new(Some(config)).await?;

API Reference

UncloseAI

Main client struct for interacting with AI APIs.

async fn new(config: Option<UncloseAIConfig>) -> Result<UncloseAI, UncloseAIError>

Initialize the client.

Parameters:

• config - Optional configuration. If None, uses defaults and auto-discovers from environment

Returns:

• Result<UncloseAI, UncloseAIError> - Initialized client or error

Example:

// Auto-discover from environment
let client = UncloseAI::new(None).await?;

// Explicit configuration
let config = UncloseAIConfig {
    endpoints: Some(vec!["https://api.example.com/v1".to_string()]),
    ..Default::default()
};
let client = UncloseAI::new(Some(config)).await?;

fn list_models(&self) -> &[ModelInfo]

List all discovered models with their metadata.

Returns:

• Slice of ModelInfo structs with id, endpoint, and max_tokens

Example:

let models = client.list_models();
for model in models {
    println!("{} - {} tokens", model.id, model.max_tokens);
}

async fn chat(&self, model: &str, messages: Vec<ChatMessage>, options: Option<ChatOptions>) -> Result<ChatResponse, UncloseAIError>

Send a non-streaming chat completion request.

Parameters:

• model - Model ID or "auto" for first available
• messages - Vector of ChatMessage with role and content
• options - Optional ChatOptions for max_tokens, temperature, etc.

Returns:

• Result<ChatResponse, UncloseAIError> - Chat completion response or error

Example:

let response = client.chat(
    "auto",
    vec![
        ChatMessage::system("You are a helpful assistant."),
        ChatMessage::user("What is AI?")
    ],
    Some(ChatOptions {
        max_tokens: Some(100),
        temperature: Some(0.7),
        ..Default::default()
    })
).await?;

println!("{}", response.choices[0].message.content);

async fn chat_stream(&self, model: &str, messages: Vec<ChatMessage>, options: Option<ChatOptions>) -> Result<impl Stream<Item = Result<ChatChunk, UncloseAIError>>, UncloseAIError>

Send a streaming chat completion request.

Parameters:

• Same as chat()

Returns:

• Result<Stream<...>, UncloseAIError> - Stream of chat chunks or error

Example:

use futures_util::StreamExt;

let mut stream = client.chat_stream(
    "auto",
    vec![ChatMessage::user("Write a haiku")],
    None
).await?;

while let Some(chunk) = stream.next().await {
    if let Ok(chunk) = chunk {
        if let Some(content) = &chunk.choices[0].delta.content {
            print!("{}", content);
        }
    }
}

async fn tts(&self, text: &str, voice: &str, model: &str) -> Result<Vec<u8>, UncloseAIError>

Generate speech from text.

Parameters:

• text - Text to convert to speech
• voice - Voice to use (alloy, echo, fable, onyx, nova, shimmer)
• model - TTS model (tts-1 or tts-1-hd)

Returns:

• Result<Vec<u8>, UncloseAIError> - Audio data (MP3 format) or error

Example:

use std::fs::File;
use std::io::Write;

let audio = client.tts("Hello!", "alloy", "tts-1").await?;
let mut file = File::create("speech.mp3")?;
file.write_all(&audio)?;

Types

ChatMessage

Message in a chat conversation.

Constructors:

• ChatMessage::system(content) - Create a system message
• ChatMessage::user(content) - Create a user message
• ChatMessage::assistant(content) - Create an assistant message

Fields:

• role: String - Message role (system, user, assistant)
• content: String - Message content

ChatOptions

Options for chat completions.

Fields:

• max_tokens: Option<u32> - Maximum tokens to generate
• temperature: Option<f32> - Sampling temperature (0.0 - 2.0)
• top_p: Option<f32> - Nucleus sampling parameter (0.0 - 1.0)

ModelInfo

Information about a discovered model.

Fields:

• id: String - Model ID
• endpoint: String - Endpoint URL
• max_tokens: u32 - Maximum context length

UncloseAIError

Error types for the library.

Variants:

• ConnectionError(String) - Network connection errors
• ModelNotFoundError(String) - Requested model not available
• StreamingError(String) - Errors during streaming
• ApiError(String) - General API errors

Usage Examples

Basic Chat

use uncloseai::{UncloseAI, ChatMessage, ChatOptions};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = UncloseAI::new(None).await?;

    let response = client.chat(
        "auto",
        vec![
            ChatMessage::system("You are a helpful assistant."),
            ChatMessage::user("What is Rust?")
        ],
        Some(ChatOptions {
            max_tokens: Some(100),
            ..Default::default()
        })
    ).await?;

    println!("{}", response.choices[0].message.content);
    Ok(())
}

Streaming Chat

use uncloseai::{UncloseAI, ChatMessage};
use futures_util::StreamExt;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = UncloseAI::new(None).await?;

    let mut stream = client.chat_stream(
        "auto",
        vec![ChatMessage::user("Write a haiku about code")],
        None
    ).await?;

    while let Some(chunk) = stream.next().await {
        if let Ok(chunk) = chunk {
            if let Some(content) = &chunk.choices[0].delta.content {
                print!("{}", content);
                std::io::stdout().flush()?;
            }
        }
    }

    println!(); // newline
    Ok(())
}

Multi-Turn Conversation

let mut messages = vec![
    ChatMessage::system("You are a helpful assistant."),
    ChatMessage::user("What is AI?"),
];

// First response
let response1 = client.chat("auto", messages.clone(), None).await?;
let assistant_msg = response1.choices[0].message.content.clone();
messages.push(ChatMessage::assistant(assistant_msg));

// Follow-up question
messages.push(ChatMessage::user("Can you explain more?"));
let response2 = client.chat("auto", messages, None).await?;

Text-to-Speech

use std::fs::File;
use std::io::Write;

let audio = client.tts("Hello from UncloseAI!", "alloy", "tts-1").await?;
let mut file = File::create("output.mp3")?;
file.write_all(&audio)?;

Using Specific Models

// List available models
let models = client.list_models();
for model in models {
    println!("{} - {} tokens", model.id, model.max_tokens);
}

// Use specific model
let response = client.chat(
    &models[0].id,
    vec![ChatMessage::user("Hello")],
    None
).await?;

Error Handling

use uncloseai::{UncloseAI, UncloseAIError, ChatMessage};

match client.chat("non-existent-model", vec![ChatMessage::user("Hello")], None).await {
    Ok(response) => println!("{}", response.choices[0].message.content),
    Err(UncloseAIError::ModelNotFoundError(msg)) => println!("Model error: {}", msg),
    Err(UncloseAIError::ConnectionError(msg)) => println!("Connection error: {}", msg),
    Err(e) => println!("Other error: {}", e),
}

Running Examples

# Set environment variables
export MODEL_ENDPOINT_1="https://hermes.ai.unturf.com/v1"
export MODEL_ENDPOINT_2="https://qwen.ai.unturf.com/v1"
export TTS_ENDPOINT_1="https://speech.ai.unturf.com/v1"

# Run example
cargo run --example basic

# Or run the binary
cargo run

Docker Usage

# Build
docker build -t uncloseai-rust .

# Run examples
docker run -e MODEL_ENDPOINT_1="https://..." uncloseai-rust

Compatibility

Tested with:

• ✅ vLLM (v0.5.0+)
• ✅ Ollama (v0.1.0+)
• ✅ OpenAI API (compatible endpoints)

Dependencies

• reqwest - HTTP client with streaming support
• serde / serde_json - Serialization/deserialization
• tokio - Async runtime
• futures-util / futures-core - Stream utilities

License

MIT License - See LICENSE file for details

Contributing

Contributions welcome! Please submit pull requests or open issues.

Support

For issues, questions, or contributions, please visit: https://github.com/yourusername/uncloseai

Changelog

v1.0.0 (2025-10-13)

• Initial release
• Streaming and non-streaming chat support
• Text-to-speech generation
• Automatic model discovery
• Type-safe API with comprehensive error handling
• Full async/await support with Tokio