modified: index.html
This commit is contained in:
root
parent
1823dbadf1
commit
62674dfc5f
1 changed files with 78 additions and 16 deletions
94
index.html
94
index.html
|
|
@ -6,7 +6,7 @@
|
|||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<meta name="theme-color" content="#43a047">
|
||||
<meta name="color-scheme" content="light dark">
|
||||
<title>Using Free Hermes AI Service | ai.unturf.com</title>
|
||||
<title>Using Free LLM & Text To Speech Artificial Intelligence Service | ai.unturf.com</title>
|
||||
|
||||
<!-- PicoCSS -->
|
||||
<link rel="stylesheet" href="https://unpkg.com/@picocss/pico@latest/css/pico.classless.min.css">
|
||||
|
|
@ -44,7 +44,7 @@
|
|||
<header>
|
||||
<hgroup>
|
||||
<h1 class="unturf" style="font-family: 'ChunkFiveRegular';">unturf.</h1>
|
||||
<p>Welcome to ai.unturf.com - Free AI Service Powered by Hermes AI</p>
|
||||
<p>Welcome to ai.unturf.com - Free LLM & Text To Speech Artificial Intelligence Service</title>
|
||||
</hgroup>
|
||||
<nav>
|
||||
<ul>
|
||||
|
|
@ -56,9 +56,10 @@
|
|||
</header>
|
||||
|
||||
<main>
|
||||
<h2>Using the Hermes AI Model</h2>
|
||||
<p>At <strong>ai.unturf.com</strong>, we offer a free AI service powered by the model <a href="https://nousresearch.com/hermes3/" target="_blank">NousResearch/Hermes-3-Llama-3.1-8B</a>.</p>
|
||||
<p>Our mission is to provide accessible AI tools for everyone, embodying the principles of both free as in beer & free as in freedom. You can interact with our model without any cost, and you are encouraged to contribute and build upon the open-source code & models that we use.</p>
|
||||
<h2>Introducing Hermes AI and TTS Speech Endpoints</h2>
|
||||
<p>At <strong>ai.unturf.com</strong>, we offer free AI services powered by the <a href="https://nousresearch.com/hermes3/" target="_blank">NousResearch/Hermes-3-Llama-3.1-8B</a> model and a TTS (Text-to-Speech) endpoint. Our mission is to provide accessible AI tools for everyone, embodying the principles of both free as in beer & free as in freedom. You can interact with our models without any cost, and you are encouraged to contribute and build upon the open-source code & models that we use.</p>
|
||||
|
||||
<p>We intend to be a drop in replacement, you can use the existing open source OpenAI client to communicate with us.</p>
|
||||
|
||||
<h3>Installing the OpenAI Client</h3>
|
||||
<h4>Python</h4>
|
||||
|
|
@ -76,8 +77,9 @@
|
|||
<p>Run the following command to install it:</p>
|
||||
<pre><code>npm install</code></pre>
|
||||
|
||||
<h2>Python Example</h2>
|
||||
<h3>Non-Streaming</h3>
|
||||
<h2>Using the Hermes AI Model</h2>
|
||||
<h3>Python Example</h3>
|
||||
<h4>Non-Streaming</h4>
|
||||
<pre><code class="python"># Python Fizzbuzz Example
|
||||
from openai import OpenAI
|
||||
|
||||
|
|
@ -97,7 +99,7 @@ response = client.chat.completions.create(
|
|||
print(response.choices[0].message.content)
|
||||
</code></pre>
|
||||
|
||||
<h3>Streaming</h3>
|
||||
<h4>Streaming</h4>
|
||||
<pre><code class="python"># Streaming response in Python
|
||||
from openai import OpenAI
|
||||
|
||||
|
|
@ -189,7 +191,52 @@ async function streamResponse() {
|
|||
streamResponse();
|
||||
</code></pre>
|
||||
|
||||
<h2>How we run inference if you wanted to try to contribute</h2>
|
||||
<h2 id="tts">Using the Text To Speech Endpoint</h2>
|
||||
<h3>Python Example</h3>
|
||||
<pre><code class="python"># TTS Speech Example in Python
|
||||
import openai
|
||||
|
||||
client = openai.OpenAI(
|
||||
api_key = "YOLO",
|
||||
base_url = "https://speech.ai.unturf.com/v1",
|
||||
)
|
||||
|
||||
with client.audio.speech.with_streaming_response.create(
|
||||
model="tts-1",
|
||||
voice="alloy",
|
||||
input="I think so therefore, Today is a wonderful day to build something people love!"
|
||||
) as response:
|
||||
response.stream_to_file("speech.mp3")
|
||||
</code></pre>
|
||||
|
||||
<h3>Node.js Example</h3>
|
||||
<pre><code class="javascript">const OpenAI = require('openai');
|
||||
|
||||
const client = new OpenAI({
|
||||
baseURL: "https://speech.ai.unturf.com/v1",
|
||||
apiKey: "YOLO",
|
||||
});
|
||||
|
||||
async function getSpeech() {
|
||||
try {
|
||||
const response = await client.audio.speech.with_streaming_response.create({
|
||||
model: "tts-1",
|
||||
voice: "alloy",
|
||||
input: "I think so therefore, Today is a wonderful day to build something people love!"
|
||||
});
|
||||
|
||||
response.stream_to_file("speech.mp3");
|
||||
} catch (error) {
|
||||
console.error("Error:", error.response ? error.response.data : error.message);
|
||||
}
|
||||
}
|
||||
|
||||
getSpeech();
|
||||
</code></pre>
|
||||
|
||||
<h2>How we run inference</h2>
|
||||
|
||||
<p>This section is optional. This is only if you wanted to try to contribute idle GPU time to the project or if you wanted to reproduce everything in your own cluster.</p>
|
||||
|
||||
<p>We use vLLM to run models, currently full f16 safetensors. We make sure to use a virtualenv to hold the dependencies.</p>
|
||||
<p>We are considering supporting ollama for better quant support.</p>
|
||||
|
|
@ -202,7 +249,9 @@ pip install vllm
|
|||
python -m vllm.entrypoints.openai.api_server --model NousResearch/Hermes-3-Llama-3.1-8B --host 0.0.0.0 --port 18888 --max-model-len 16000
|
||||
</code></pre>
|
||||
|
||||
<p>If you want to see how we setup the proxy, check out <a href="https://git.unturf.com/-/snippets/3">/etc/caddy/Caddyfile</a></p>
|
||||
<p>The Speech endpoint or TTS uses <a href="https://github.com/matatonic/openedai-speech?tab=readme-ov-file#nvidia-gpu-cuda">openedai-speech</a> running via Docker.
|
||||
|
||||
<p>If you want to see how we setup the proxy, check out <a href="https://git.unturf.com/-/snippets/3">/etc/caddy/Caddyfile</a></p>
|
||||
|
||||
<pre><code>
|
||||
|
||||
|
|
@ -233,17 +282,30 @@ hermes.ai.unturf.com {
|
|||
}
|
||||
}
|
||||
|
||||
</code></pre>
|
||||
speech.ai.unturf.com {
|
||||
reverse_proxy <removed>:8000
|
||||
log {
|
||||
output file /var/log/caddy/speech.ai.unturf.com.log {
|
||||
roll_size 50mb
|
||||
roll_keep 5
|
||||
}
|
||||
}
|
||||
tls {
|
||||
on_demand
|
||||
}
|
||||
}
|
||||
|
||||
</code></pre>
|
||||
|
||||
<p>We will likely implement a rate limit based on client IP address.</p>
|
||||
|
||||
<h2 id="client-side">Web Client-Only Solution: Interact with AI Services Directly from Static Sites or CDNs</h2>
|
||||
|
||||
<p><b>Because we don't require a valid API key, we don't have any real need for a server.</b></p>
|
||||
<p><b>Because we don't require a valid API key, we don't have any real need for a server.</b></p>
|
||||
|
||||
<p>Add this LLM to any static site or CDN.</p>
|
||||
|
||||
<p>This web client-only solution uses <a href="https://uncloseai.com/uncloseai.js" target="_blank">uncloseai.js</a> to make the browser act as a client, directly interacting with the API without needing an intermediary server. By eliminating the need for a valid API key, the API handles requests on behalf of the browser client, making it efficient and accessible thin client, especially those on battery power like phones & laptops.</p>
|
||||
<p>This web client-only solution uses <a href="https://uncloseai.com/uncloseai.js" target="_blank">uncloseai.js</a> to make the browser act as a client, directly interacting with the API without needing an intermediary server. By eliminating the need for a valid API key, the API handles requests on behalf of the browser client, making it efficient and accessible thin client, especially those on battery power like phones & laptops.</p>
|
||||
|
||||
<div id="chat-container">
|
||||
<div id="chat-box"></div>
|
||||
|
|
@ -254,12 +316,12 @@ hermes.ai.unturf.com {
|
|||
<button onclick="handleUserInput()">Send</button>
|
||||
</div>
|
||||
|
||||
<br/>
|
||||
<p>This static site has a live LLM demostation. Feel free to message us in the box below.</p>
|
||||
<br/>
|
||||
<p>This static site has a live LLM demonstration. Feel free to message us in the box below.</p>
|
||||
|
||||
<h2>Questions & Comments & Discussions</h2>
|
||||
|
||||
Use the <a href="https://www.remarkbox.com" target="_blank">Remarkbox</a> below to tell us what you think!
|
||||
Use the <a href="https://www.remarkbox.com" target="_blank">Remarkbox</a> below to tell us what you think!
|
||||
|
||||
<div id="remarkbox-div">
|
||||
<noscript>
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue