Russell Ballestrini
cae51b3578
- Add smart-translate widget feature with AI language detection
- Implement triple-format language display (code • English • Native)
- Add native script names with proper Unicode (中文, Español, العربية, etc.)
- Export handleSmartTranslate to global scope for HTML onclick
- Update demo.html to showcase both translate and smart-translate options
- Add comprehensive widget documentation with code examples
🤖 Generated with Hermes Staff
Co-Authored-By: Claude <noreply@anthropic.com>
443 lines
17 KiB
HTML
443 lines
17 KiB
HTML
<!DOCTYPE html>
|
|
<html lang="en">
|
|
|
|
<head>
|
|
<meta charset="UTF-8">
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
<meta name="theme-color" content="#43a047">
|
|
<meta name="color-scheme" content="light dark">
|
|
<title>Using Free LLM & Text To Speech Artificial Intelligence Service | uncloseai.com</title>
|
|
|
|
<!-- PicoCSS -->
|
|
<link rel="stylesheet" href="/css/pico.classless.min.css">
|
|
<!-- ChunkFive Font -->
|
|
<link rel="stylesheet" href="/css/chunkfive/stylesheet.css" type="text/css" charset="utf-8" />
|
|
|
|
<style>
|
|
body {
|
|
max-width: 960px;
|
|
margin: 0 auto;
|
|
}
|
|
</style>
|
|
|
|
<!-- Highlight.js for syntax highlighting -->
|
|
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.10.0/styles/a11y-dark.min.css" />
|
|
<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.6.0/highlight.min.js"></script>
|
|
|
|
<!-- Theme Switcher Script -->
|
|
<script>
|
|
function switchTheme(theme) {
|
|
if (theme === "auto") {
|
|
document.documentElement.removeAttribute('data-theme');
|
|
} else {
|
|
document.documentElement.setAttribute('data-theme', theme);
|
|
}
|
|
}
|
|
|
|
// Disable custom styling for uncloseai.js - use PicoCSS instead
|
|
window.UNCLOSEAI_CUSTOM_STYLING = false;
|
|
</script>
|
|
|
|
<script src="https://uncloseai.com/uncloseai.js" type="module"></script>
|
|
<!--
|
|
<script defer data-domain="ai.unturf.com" src="https://analytics.unturf.com/js/plausible.js"></script>
|
|
-->
|
|
</head>
|
|
|
|
<body>
|
|
|
|
<header>
|
|
<hgroup>
|
|
<a href="https://uncloseai.com"><h1 class="unturf" style="font-family: 'ChunkFiveRegular';">russell@unturf. presents, uncloseai</h1></a>
|
|
<p>Welcome to uncloseai.com - Free LLM & Text To Speech Artificial Intelligence Service</p>
|
|
</hgroup>
|
|
<nav>
|
|
<ul>
|
|
<li><a href="#" onclick="switchTheme('auto')">Auto</a></li>
|
|
<li><a href="#" onclick="switchTheme('light')">Light</a></li>
|
|
<li><a href="#" onclick="switchTheme('dark')">Dark</a></li>
|
|
</ul>
|
|
</nav>
|
|
</header>
|
|
|
|
<main>
|
|
<h2>Introducing uncloseai & Hermes & TTS Speech Endpoints</h2>
|
|
<p>At <strong>uncloseai.</strong>, we offer free AI services powered by the <a href="https://nousresearch.com/hermes3/" target="_blank">NousResearch/Hermes-3-Llama-3.1-8B</a> model and a TTS (Text-to-Speech) endpoint. Our mission is to provide accessible AI tools for everyone, embodying the principles of both free as in beer & free as in freedom. You can interact with our models without any cost, and you are encouraged to contribute and build upon the open-source code & models that we use.</p>
|
|
|
|
<p>We intend to be a drop in replacement, you can use the existing open source OpenAI client to communicate with us.</p>
|
|
|
|
<h2 id="client-side">Web Client-Only Solution: Interact with AI Services Directly from Static Sites or CDNs</h2>
|
|
|
|
<p><b>Because we don't require a valid API key, we don't have any real need for a server.</b></p>
|
|
|
|
<p>Add this LLM to any static site or CDN.</p>
|
|
|
|
<p>This web client-only solution uses <a href="https://uncloseai.com/uncloseai.js" target="_blank">uncloseai.js</a> to make the browser act as a client, directly interacting with the API without needing an intermediary server. By eliminating the need for a valid API key, the API handles requests on behalf of the browser client, making it efficient and accessible thin client, especially those on battery power like phones & laptops.</p>
|
|
|
|
<h3>Install uncloseai.js on any website</h3>
|
|
|
|
<p>Add AI capabilities to any static website or web application with just one line of code. uncloseai.js automatically creates a floating "uncloseai." button that provides access to Hermes AI, text-to-speech, file upload, and more.</p>
|
|
|
|
<h4>Quick Installation</h4>
|
|
<p>Add this script tag to your HTML - that's it! The floating AI button appears automatically:</p>
|
|
<pre><code class="html"><script src="https://uncloseai.com/uncloseai.js" type="module"></script></code></pre>
|
|
|
|
<h4>Configuration Options</h4>
|
|
<p>To use your CSS framework's styling instead of uncloseai.js custom styles:</p>
|
|
<pre><code class="javascript"><script>
|
|
window.UNCLOSEAI_CUSTOM_STYLING = false;
|
|
</script></code></pre>
|
|
|
|
<p>To disable the floating button (if you only want the API functions):</p>
|
|
<pre><code class="javascript"><script>
|
|
window.UNCLOSEAI_FLOATING_BUTTON = false;
|
|
</script></code></pre>
|
|
|
|
<h4>System Message Customization</h4>
|
|
<p>Customize the AI's behavior by appending additional instructions to the system message:</p>
|
|
<pre><code class="javascript"><script type="module">
|
|
import { setSystemMessageAppend } from './src/config.js';
|
|
|
|
// Add custom instructions
|
|
setSystemMessageAppend(`
|
|
You are embedded on a documentation website about CSS frameworks.
|
|
- Help users understand CSS concepts and provide examples
|
|
- Reference the current page content when answering
|
|
- Suggest best practices for responsive design
|
|
`);
|
|
</script></code></pre>
|
|
|
|
<p>The custom text is appended to the base Hermes system message, allowing you to:</p>
|
|
<ul>
|
|
<li>Add domain-specific knowledge</li>
|
|
<li>Provide context about your website/application</li>
|
|
<li>Set specific behavioral guidelines</li>
|
|
<li>Include relevant instructions without losing core functionality</li>
|
|
</ul>
|
|
|
|
<h4>Class-Based Integration</h4>
|
|
<p>Embed AI features directly into your HTML using CSS classes and data attributes:</p>
|
|
|
|
<p><strong>Full Interface:</strong> Complete AI chat with all features</p>
|
|
<pre><code class="html"><div class="uncloseai" data-features="full"></div></code></pre>
|
|
|
|
<p><strong>Chat Only:</strong> Just the AI chat interface</p>
|
|
<pre><code class="html"><div class="uncloseai" data-features="chat"></div></code></pre>
|
|
|
|
<p><strong>Specific Features:</strong> Choose which buttons to include</p>
|
|
<pre><code class="html"><div class="uncloseai" data-features="tts,translate,read"></div>
|
|
<div class="uncloseai" data-features="translate"></div>
|
|
<div class="uncloseai" data-features="smart-translate"></div></code></pre>
|
|
|
|
<p><strong>Available Features:</strong> <code>chat</code>, <code>tts</code>, <code>translate</code>, <code>smart-translate</code>, <code>upload</code>, <code>read</code>, <code>full</code></p>
|
|
|
|
<h4>What You Get</h4>
|
|
<ul>
|
|
<li><strong>🤖 Floating AI Assistant</strong> - Always-accessible "uncloseai." button in bottom right</li>
|
|
<li><strong>📖 Page-Aware</strong> - Hermes AI understands your page content and generates contextual introductions</li>
|
|
<li><strong>🔊 Text-to-Speech</strong> - Convert any text to speech with multiple voice options</li>
|
|
<li><strong>💾 Conversation History</strong> - Persistent chat history with delete functionality in browser localstorage</li>
|
|
<li><strong>🌐 Translation</strong> - Pixel perfect HTML DOM transation into 18 languages!</li>
|
|
<li><strong>🎨 Framework Compatible</strong> - Works with PicoCSS, PureCSS, Bootstrap, and more</li>
|
|
</ul>
|
|
|
|
<p><strong>Live Demo:</strong> <a href="https://ai.unturf.com/demo.html">Try uncloseai.js in action here</a> - see a page with various isolated features working on a real page!</p>
|
|
|
|
<h3>Installing the OpenAI Client</h3>
|
|
<h4>Python</h4>
|
|
<p>To install the OpenAI package for Python, use <code>pip</code>:</p>
|
|
<pre><code>pip install openai</code></pre>
|
|
|
|
<h4>Node.js</h4>
|
|
<p>To install the OpenAI package for Node.js, you can use <code>npm</code> in your <code>package.json</code>:</p>
|
|
<pre><code>{
|
|
"dependencies": {
|
|
"openai": "^v4.67.3" // Use the latest version
|
|
}
|
|
}
|
|
</code></pre>
|
|
<p>Run the following command to install it:</p>
|
|
<pre><code>npm install</code></pre>
|
|
|
|
<h2>Using the Hermes AI Model</h2>
|
|
<h3>Python Example</h3>
|
|
<h4>Non-Streaming</h4>
|
|
<pre><code class="python"># Python Fizzbuzz Example
|
|
from openai import OpenAI
|
|
|
|
client = OpenAI(base_url="https://hermes.ai.unturf.com/v1", api_key="choose-any-value")
|
|
|
|
#MODEL = "NousResearch/Hermes-3-Llama-3.1-8B"
|
|
MODEL = "adamo1139/Hermes-3-Llama-3.1-8B-FP8-Dynamic"
|
|
|
|
messages = [{"role": "user", "content": "Give a Python Fizzbuzz solution in one line of code?"}]
|
|
|
|
response = client.chat.completions.create(
|
|
model=MODEL,
|
|
messages=messages,
|
|
temperature=0.5,
|
|
max_tokens=150
|
|
)
|
|
|
|
print(response.choices[0].message.content)
|
|
</code></pre>
|
|
|
|
<h4>Streaming</h4>
|
|
<pre><code class="python"># Streaming response in Python
|
|
from openai import OpenAI
|
|
|
|
client = OpenAI(base_url="https://hermes.ai.unturf.com/v1", api_key="choose-any-value")
|
|
|
|
MODEL = "adamo1139/Hermes-3-Llama-3.1-8B-FP8-Dynamic"
|
|
|
|
messages = [
|
|
{"role": "user", "content": "Give a Python Fizzbuzz solution in one line of code?"}
|
|
]
|
|
|
|
response = client.chat.completions.create(
|
|
model=MODEL,
|
|
messages=messages,
|
|
temperature=0.5,
|
|
max_tokens=150,
|
|
stream=True, # Enable streaming
|
|
)
|
|
|
|
for chunk in response:
|
|
if hasattr(chunk.choices[0].delta, "content"):
|
|
print(chunk.choices[0].delta.content, end="")
|
|
</code></pre>
|
|
|
|
<h2>Node.js Example</h2>
|
|
<h3>Non-Streaming</h3>
|
|
<pre><code class="javascript">const OpenAI = require('openai');
|
|
|
|
const client = new OpenAI({
|
|
baseURL: "https://hermes.ai.unturf.com/v1",
|
|
apiKey: "dummy-api-key",
|
|
});
|
|
|
|
const MODEL = "adamo1139/Hermes-3-Llama-3.1-8B-FP8-Dynamic";
|
|
|
|
const messages = [{"role": "user", "content": "Give a Python Fizzbuzz solution in one line of code?"}];
|
|
|
|
async function getResponse() {
|
|
try {
|
|
const response = await client.chat.completions.create({
|
|
model: MODEL,
|
|
messages: messages,
|
|
temperature: 0.5,
|
|
max_tokens: 150,
|
|
});
|
|
|
|
console.log(response.choices[0].message.content);
|
|
} catch (error) {
|
|
console.error("Error:", error.response ? error.response.data : error.message);
|
|
}
|
|
}
|
|
|
|
getResponse();
|
|
</code></pre>
|
|
|
|
<h3>Streaming</h3>
|
|
<pre><code class="javascript">
|
|
const OpenAI = require('openai');
|
|
|
|
const client = new OpenAI({
|
|
baseURL: "https://hermes.ai.unturf.com/v1",
|
|
apiKey: "dummy-api-key",
|
|
});
|
|
|
|
const MODEL = "adamo1139/Hermes-3-Llama-3.1-8B-FP8-Dynamic";
|
|
|
|
const messages = [{"role": "user", "content": "Give a Python Fizzbuzz solution in one line of code?"}];
|
|
|
|
async function streamResponse() {
|
|
try {
|
|
const stream = await client.chat.completions.create({
|
|
model: MODEL,
|
|
messages: messages,
|
|
temperature: 0.5,
|
|
max_tokens: 150,
|
|
stream: true, // Enable streaming
|
|
});
|
|
|
|
// Use async iterator to read each chunk
|
|
for await (const chunk of stream) {
|
|
const msg = chunk.choices[0].delta.content;
|
|
process.stdout.write(msg); // Print each chunk as it arrives
|
|
}
|
|
} catch (error) {
|
|
console.error("Error:", error.response ? error.response.data : error.message);
|
|
}
|
|
}
|
|
|
|
streamResponse();
|
|
</code></pre>
|
|
|
|
<h2 id="tts">Using the Text To Speech Endpoint</h2>
|
|
<h3>Python Example</h3>
|
|
<pre><code class="python"># TTS Speech Example in Python
|
|
import openai
|
|
|
|
client = openai.OpenAI(
|
|
api_key = "YOLO",
|
|
base_url = "https://speech.ai.unturf.com/v1",
|
|
)
|
|
|
|
with client.audio.speech.with_streaming_response.create(
|
|
model="tts-1",
|
|
voice="alloy",
|
|
speed=0.9,
|
|
input="I think so therefore, Today is a wonderful day to build something people love!"
|
|
) as response:
|
|
response.stream_to_file("speech.mp3")
|
|
</code></pre>
|
|
|
|
<h3>Node.js Example</h3>
|
|
<pre><code class="javascript">const OpenAI = require('openai');
|
|
|
|
const client = new OpenAI({
|
|
baseURL: "https://speech.ai.unturf.com/v1",
|
|
apiKey: "YOLO",
|
|
});
|
|
|
|
async function getSpeech() {
|
|
try {
|
|
const response = await client.audio.speech.with_streaming_response.create({
|
|
model: "tts-1",
|
|
voice: "alloy",
|
|
speed: 0.9,
|
|
input: "I think so therefore, Today is a wonderful day to build something people love!"
|
|
});
|
|
|
|
response.stream_to_file("speech.mp3");
|
|
} catch (error) {
|
|
console.error("Error:", error.response ? error.response.data : error.message);
|
|
}
|
|
}
|
|
|
|
getSpeech();
|
|
</code></pre>
|
|
|
|
<h2>How we run inference</h2>
|
|
|
|
<p>This section is optional. This is only if you wanted to try to contribute idle GPU time to the project or if you wanted to reproduce everything in your own cluster.</p>
|
|
|
|
<p>We use vLLM to run models, currently full f16 safetensors. We make sure to use a virtualenv to hold the dependencies.</p>
|
|
<p>We are considering supporting ollama for better quant support.</p>
|
|
<p>Stand up a replica cluster on a new domain.</p>
|
|
<pre><code>
|
|
sudo apt-get install gcc python3.12-dev
|
|
cd ~
|
|
python3 -m venv env
|
|
source env/bin/activate
|
|
pip install vllm
|
|
python -m vllm.entrypoints.openai.api_server --model adamo1139/Hermes-3-Llama-3.1-8B-FP8-Dynamic --host 0.0.0.0 --port 18888 --max-model-len 82000
|
|
</code></pre>
|
|
|
|
<p>The Speech endpoint or TTS uses <a href="https://github.com/matatonic/openedai-speech?tab=readme-ov-file#nvidia-gpu-cuda">openedai-speech</a> running via Docker.
|
|
|
|
<p>If you want to see how we setup the proxy, check out <a href="https://git.unturf.com/-/snippets/3">/etc/caddy/Caddyfile</a></p>
|
|
|
|
<pre><code>
|
|
|
|
ai.unturf.com {
|
|
root * /opt/www
|
|
file_server
|
|
log {
|
|
output file /var/log/caddy/ai.unturf.com.log {
|
|
roll_size 50mb
|
|
roll_keep 5
|
|
}
|
|
}
|
|
tls {
|
|
on_demand
|
|
}
|
|
}
|
|
|
|
hermes.ai.unturf.com {
|
|
reverse_proxy <removed>:18888
|
|
log {
|
|
output file /var/log/caddy/hermes.ai.unturf.com.log {
|
|
roll_size 50mb
|
|
roll_keep 5
|
|
}
|
|
}
|
|
tls {
|
|
on_demand
|
|
}
|
|
}
|
|
|
|
speech.ai.unturf.com {
|
|
reverse_proxy <removed>:8000
|
|
log {
|
|
output file /var/log/caddy/speech.ai.unturf.com.log {
|
|
roll_size 50mb
|
|
roll_keep 5
|
|
}
|
|
}
|
|
tls {
|
|
on_demand
|
|
}
|
|
}
|
|
|
|
</code></pre>
|
|
|
|
<p>We will likely implement a rate limit based on client IP address.</p>
|
|
|
|
|
|
<h2>Questions & Comments & Discussions</h2>
|
|
|
|
Use the <a href="https://www.remarkbox.com" target="_blank">Remarkbox</a> below to tell us what you think!
|
|
|
|
<div id="remarkbox-div">
|
|
<noscript>
|
|
<iframe id=remarkbox-iframe src="https://my.remarkbox.com/embed?nojs=true" style="height:600px;width:100%;border:none!important" tabindex=0></iframe>
|
|
</noscript>
|
|
</div>
|
|
<script src="https://my.remarkbox.com/static/js/iframe-resizer/iframeResizer.min.js"></script>
|
|
<script>
|
|
var rb_owner_key = "944c8dfa-8b2b-11ef-af0e-29ab4fb285a0";
|
|
var thread_uri = window.location.href;
|
|
var thread_title = window.document.title;
|
|
var thread_fragment = window.location.hash;
|
|
|
|
var rb_src = "https://my.remarkbox.com/embed" +
|
|
"?rb_owner_key=" + rb_owner_key +
|
|
"&thread_title=" + encodeURI(thread_title) +
|
|
"&thread_uri=" + encodeURIComponent(thread_uri) +
|
|
thread_fragment;
|
|
|
|
function create_remarkbox_iframe() {
|
|
var ifrm = document.createElement("iframe");
|
|
ifrm.setAttribute("id", "remarkbox-iframe");
|
|
ifrm.setAttribute("scrolling", "no");
|
|
ifrm.setAttribute("src", rb_src);
|
|
ifrm.setAttribute("frameborder", "0");
|
|
ifrm.setAttribute("tabindex", "0");
|
|
ifrm.setAttribute("title", "Remarkbox");
|
|
ifrm.style.width = "100%";
|
|
document.getElementById("remarkbox-div").appendChild(ifrm);
|
|
}
|
|
create_remarkbox_iframe();
|
|
iFrameResize(
|
|
{
|
|
checkOrigin: ["https://my.remarkbox.com"],
|
|
inPageLinks: true,
|
|
initCallback: function(e) { e.iFrameResizer.moveToAnchor(thread_fragment) }
|
|
},
|
|
document.getElementById("remarkbox-iframe")
|
|
);
|
|
</script>
|
|
|
|
<script>hljs.highlightAll();</script>
|
|
</main>
|
|
|
|
<footer>
|
|
<small>Stylesheets by <a href="https://picocss.com" target="_blank">PicoCSS</a></small>
|
|
<small>& <a href="https://highlightjs.org/" target="_blank">highlight.js</a></small>
|
|
</footer>
|
|
|
|
</body>
|
|
|
|
</html>
|