diff --git a/index.html b/index.html index b575a64..59f2a7d 100644 --- a/index.html +++ b/index.html @@ -3,13 +3,14 @@ - Using Hermes AI - Free AI Service + Using Free Hermes AI Service | ai.unturf.com + +

Welcome to ai.unturf.com

-

At ai.unturf.com, we offer a free AI service powered by the model NousResearch/Hermes-3-Llama-3.1-8B. Our mission is to provide accessible AI tools for everyone, embodying the principles of free beer and free freedom. You can interact with our model without any cost, and you are encouraged to contribute and build upon our open-source framework.

+

At ai.unturf.com, we offer a free AI service powered by the model NousResearch/Hermes-3-Llama-3.1-8B. Our mission is to provide accessible AI tools for everyone, embodying the principles of both free as in beer & free as in freedom. You can interact with our model without any cost, and you are encouraged to contribute and build upon the open-source code & models that we use.

Using the Hermes AI Model

-

This guide explains how to use the official OpenAI client to interact with the Hermes AI model hosted at hermes.ai.unturf.com/v1. You can use this endpoint without an API key.

+

This guide explains how to use the official OpenAI client to interact with the Hermes AI model hosted at hermes.ai.unturf.com/v1. You can use this endpoint without an API key.

Installing the OpenAI Client

Python

@@ -104,6 +107,7 @@ for chunk in response: # The 'ChatCompletionChunk' object exposes the 'choices' attribute directly if hasattr(chunk.choices[0].delta, "content"): print(chunk.choices[0].delta.content, end="") +

Node.js Example

@@ -135,6 +139,7 @@ async function getResponse() { } getResponse(); +

Streaming

@@ -171,8 +176,68 @@ async function streamResponse() { } streamResponse(); +

Feel free to modify the messages array to ask different questions or adjust the temperature and max_tokens parameters according to your needs. Happy querying!

+ +

How we run inference if you wanted to try to contribute

+ +

We use vLLM to run models, currently full f16 safetensors. We make sure to use a virtualenv to hold the dependencies.

+

We are considering supporting ollama for better quant support.

+

+cd ~
+python3 -m venv env
+source env/bin/activate
+pip install vllm
+python -m vllm.entrypoints.openai.api_server --model NousResearch/Hermes-3-Llama-3.1-8B --host 0.0.0.0 --port 18888 --max-model-len 16000
+
+    
+ + +

Questions, Comments, Discussions

+ +
+ +
+ + + +