Russell Ballestrini
53df255b9b
- Replace old multi-line HTML setup with single script tag - Add type="module" attribute for proper ES module loading - Highlight new features: floating AI assistant, TTS, translations, etc. - Remove deprecated custom chat interface example from both blog posts 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
83 lines
4.2 KiB
ReStructuredText
83 lines
4.2 KiB
ReStructuredText
Free LLM Endpoints & Dynamic Distributed Model Inference Client with UncloseAI.js
|
||
=====================================================================================
|
||
|
||
:author: Russell Ballestrini
|
||
:slug: free-llm-endpoints-dynamic-distributed-model-inference-client-uncloseai
|
||
:date: 2025-02-19 21:28
|
||
:tags: Code, DevOps, AI
|
||
:status: published
|
||
|
||
.. note::
|
||
This post is a follow-up to our `original post published on 2024-10-21 </free-hermes-ai-unturf-com-uncloseai/>`_. Over the last few months, we've partnered with **TDC** - `the disruptive collective <https://matrix.to/#/#disruptive-collective:matrix.org>`_ to make cutting-edge AI models accessible from any device, via our always available model servers to our new bot's for discord, matrix, & web!
|
||
|
||
|
||
Overview
|
||
--------
|
||
|
||
We’ve been pushing the boundaries of accessible AI. We built `ai.unturf.com <https://ai.unturf.com>`_ and `uncloseai.js <hhttps://git.unturf.com/engineering/unturf/ai.unturf.com/-/blob/master/uncloseai.js>`_ to provide you with free, state-of-the-art AI tools that are as *free as in beer* and as *free as in freedom*. A huge shout out to **TDC** for powering all the development work on the various bots on the telegram, discord, & matrix platforms!
|
||
|
||
Dynamic Model & Endpoint Discovery
|
||
------------------------------------
|
||
Instead of maintaining a static list of models—which can quickly become outdated—our service is designed for dynamic discovery. Each of our model server endpoints offers an API route that returns the current list of supported models. This means you can always retrieve the latest model roster directly from the source.
|
||
|
||
We encourage you to review the dynamic model querying approaches used by both `uncloseai.js` and `opencompletion.com <https://opencompletion.com>`_ to see how they simplify integration & keep things current.
|
||
|
||
Here’s where you can find the model lists for each endpoint:
|
||
|
||
- **Hermes Endpoints:**
|
||
|
||
|
||
hermes1:
|
||
|
||
* `https://hermes.ai.unturf.com/v1 <https://hermes.ai.unturf.com/v1>`_
|
||
* `https://hermes.ai.unturf.com/v1/models <https://hermes.ai.unturf.com/v1/models>`_
|
||
|
||
hermes2:
|
||
|
||
* `https://hermes2.ai.unturf.com/v1 <https://hermes2.ai.unturf.com/v1>`_
|
||
* `https://hermes2.ai.unturf.com/v1/models <https://hermes2.ai.unturf.com/v1/models>`_
|
||
|
||
|
||
By dynamically querying each endpoint for its supported models, you ensure your integration remains up to date without needing to hardcode model names.
|
||
|
||
Example Usage
|
||
-------------
|
||
Below is a quick cURL example for interacting with our new `adamo1139/Hermes-3-Llama-3.1-8B-FP8-Dynamic` endpoint:
|
||
|
||
.. code-block:: bash
|
||
|
||
curl -X POST \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"prompt": "Give a Python Fizzbuzz solution in one line of code",
|
||
"model": "adamo1139/Hermes-3-Llama-3.1-8B-FP8-Dynamic",
|
||
"temperature": 0.5,
|
||
"max_tokens": 150
|
||
}' \
|
||
https://hermes.ai.unturf.com/v1/completions
|
||
|
||
Integration Made Easy
|
||
---------------------
|
||
Our service supports any programming language. With client libraries available for Python and Node.js—and a web-client-only solution via `uncloseai.js`—integrating AI into your application or website has never been easier. Try to use the default openai client oin your native language.
|
||
|
||
The browser-based integration is ideal for static sites or CDNs, eliminating the need for a dedicated backend server or API key.
|
||
|
||
Add the following HTML snippet to your site to get started:
|
||
|
||
.. code-block:: html
|
||
|
||
<script src="https://uncloseai.com/uncloseai.js" type="module"></script>
|
||
|
||
That's it! This single line provides a floating AI assistant button with page-aware contextual help, text-to-speech functionality, translation capabilities, conversation history storage, and compatibility with any CSS framework.
|
||
|
||
|
||
**Try It Out!**
|
||
|
||
|
||
There is a demo connected to this static page!
|
||
|
||
Dive in and explore our service—use it as-is or build your own custom client. Let's keep democratizing AI and empower everyone to harness its power.
|
||
|
||
We’ve also open-sourced `Matrix & Discord Bots <https://git.unturf.com/engineering/unturf/discord.agents.ai.unturf.com#unturfdiscordbot>`_ to showcase dynamic model loading in action.
|
||
|
||
Join our `Discord server <https://discord.gg/TZPmQdPj8a>`_ to collaborate with us on building agents, environments, and much more!
|