Update 2025-02-19-free-llm-endpoints-dynamic-distributed-model-inference-client-uncloseai.rst
This commit is contained in:
parent
37037f95dc
commit
c50a62b206
1 changed files with 26 additions and 46 deletions
|
|
@ -1,51 +1,36 @@
|
|||
Free LLM Endpoints & Dynamic Distributed Model Inference Client with UncloseAI.js
|
||||
###################################################################################
|
||||
=====================================================================================
|
||||
|
||||
:author: Russell Ballestrini
|
||||
:slug: free-llm-endpoints-dynamic-distributed-model-inference-client-uncloseai
|
||||
:date: 2025-02-19 21:28
|
||||
:tags: Code, DevOps, AI
|
||||
:status: published
|
||||
:author: Russell Ballestrini
|
||||
:slug: free-llm-endpoints-dynamic-distributed-model-inference-client-uncloseai
|
||||
:date: 2025-02-19 21:28
|
||||
:tags: Code, DevOps, AI
|
||||
:status: published
|
||||
|
||||
.. note::
|
||||
This post is a follow-up to our `original post published on 2024-10-21 </free-hermes-ai-unturf-com-uncloseai/>`_. Over the last few months, we've partnered with **TDC (`the disruptive collective <https://matrix.to/#/#disruptive-collective:matrix.org>`_)** & **https://naptha.ai** to make cutting-edge AI models accessible from any device!
|
||||
|
||||
.. note::
|
||||
This post is a follow-up to our original post published on 2024-10-21 — roughly 4 months ago. Over that time, we've teamed up with **TDC (the disruptive collective)** and **naptha.ai** to bring new models accessible from any device!
|
||||
|
||||
Overview
|
||||
--------
|
||||
Over the past 4 months, we've been hard at work pushing the boundaries of accessible AI. We built `ai.unturf.com <https://ai.unturf.com>`_ & `uncloseai.js <https://uncloseai.com/uncloseai.js>`_ to provide you with free, state-of-the-art AI tools that are as *free as in beer* and as *free as in freedom*. And a big shout out to **naptha.ai** (https://naptha.ai) for powering the model servers, internet connectivity, and the electric that keep these publicly accessible models running!
|
||||
|
||||
Model & Endpoint Distribution
|
||||
-----------------------------
|
||||
Our AI service is dynamically distributed across three endpoints:
|
||||
We’ve been pushing the boundaries of accessible AI. We built `ai.unturf.com <https://ai.unturf.com>`_ and `uncloseai.js <https://uncloseai.com/uncloseai.js>`_ to provide you with free, state-of-the-art AI tools that are as *free as in beer* and as *free as in freedom*. A huge shout out to **naptha.ai** (https://naptha.ai) for powering the model servers, connectivity, and the electricity that keep these publicly accessible models running!
|
||||
|
||||
**Hermes Endpoint** (https://hermes.ai.unturf.com/v1)
|
||||
---------------------------------------------------------
|
||||
- ``adamo1139/Hermes-3-Llama-3.1-8B-FP8-Dynamic``
|
||||
Dynamic Model & Endpoint Discovery
|
||||
------------------------------------
|
||||
Instead of maintaining a static list of models—which can quickly become outdated—our service is designed for dynamic discovery. Each of our model server endpoints offers an API route that returns the current list of supported models. This means you can always retrieve the latest model roster directly from the source.
|
||||
|
||||
**Node2 Endpoint** (https://node2.naptha.ai/inference)
|
||||
---------------------------------------------------------
|
||||
- ``internlm/internlm2_5-7b-chat``
|
||||
- ``jinaai/jina-embeddings-v2-base-en``
|
||||
- ``ibm-granite/granite-3.1-8b-instruct``
|
||||
- ``meetkai/functionary-small-v3.1``
|
||||
- ``Team-ACE/ToolACE-8B``
|
||||
- ``Qwen/Qwen2.5-7B-Instruct``
|
||||
- ``NousResearch/Hermes-3-Llama-3.1-8B``
|
||||
- ``meta-llama/Llama-3.1-8B-Instruct``
|
||||
- ``gpt-4o-mini``
|
||||
We encourage you to review the dynamic model querying approaches used by both `uncloseai.js` and [opencompletion.com](https://opencompletion.com) to see how they simplify integration and keep things current. Here’s where you can find the model lists for each endpoint:
|
||||
|
||||
**Node3 Endpoint** (https://node3.naptha.ai/inference)
|
||||
---------------------------------------------------------
|
||||
- ``Qwen/QwQ-32B-Preview``
|
||||
- ``deepseek-ai/DeepSeek-R1-Distill-Qwen-32B``
|
||||
- ``katanemo/Arch-Function-7B``
|
||||
- ``microsoft/phi-4``
|
||||
- ``mistralai/Mistral-Small-24B-Instruct-2501``
|
||||
- ``gpt-4o-mini``
|
||||
- **Hermes Endpoint:** Query models at `https://hermes.ai.unturf.com/v1/models`
|
||||
- **Node2 Endpoint:** Query models at `https://node2.naptha.ai/inference/models`
|
||||
- **Node3 Endpoint:** Query models at `https://node3.naptha.ai/inference/models`
|
||||
|
||||
By dynamically querying each endpoint for its supported models, you ensure your integration remains up to date without needing to hardcode model names.
|
||||
|
||||
Example Usage
|
||||
-------------
|
||||
Here's a quick cURL example to interact with our Hermes endpoint:
|
||||
Below is a quick cURL example for interacting with our Hermes endpoint:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
|
|
@ -61,7 +46,7 @@ Here's a quick cURL example to interact with our Hermes endpoint:
|
|||
|
||||
Integration Made Easy
|
||||
---------------------
|
||||
Our service supports multiple programming languages. With client libraries for Python and Node.js—and a web-client-only solution via `uncloseai.js`—integrating our AI into your app or website has never been easier. The browser-based integration is ideal for static sites or CDNs, eliminating the need for a backend server or API key.
|
||||
Our service supports multiple programming languages. With client libraries available for Python and Node.js—and a web-client-only solution via `uncloseai.js`—integrating AI into your application or website has never been easier. The browser-based integration is ideal for static sites or CDNs, eliminating the need for a dedicated backend server or API key.
|
||||
|
||||
HTML Snippet for Browser Integration
|
||||
--------------------------------------
|
||||
|
|
@ -87,17 +72,12 @@ Add the following snippet to your site to get started:
|
|||
|
||||
Try It Out!
|
||||
-----------
|
||||
Dive in and explore our service—use it as-is or build your own client. Let's keep democratizing AI and empower everyone to harness its power.
|
||||
Dive in and explore our service—use it as-is or build your own custom client. Let's keep democratizing AI and empower everyone to harness its power.
|
||||
|
||||
We open sourced a discord bot to showcase the dynamic model loading.
|
||||
We’ve also open-sourced a Discord bot that showcases dynamic model loading in action. Join our Discord server at [https://discord.gg/TZPmQdPj8a](https://discord.gg/TZPmQdPj8a) to collaborate with us on building agents, environments, and much more!
|
||||
|
||||
Join the Discord server here https://discord.gg/TZPmQdPj8a and help us build agents, environments, & more!
|
||||
|
||||
|
||||
Bot repository:
|
||||
|
||||
https://git.unturf.com/engineering/unturf/discord.agents.ai.unturf.com#unturfdiscordbot
|
||||
We also have a discord Bot & a matrix bot agents with access to all the models:
|
||||
[https://git.unturf.com/engineering/unturf/discord.agents.ai.unturf.com#unturfdiscordbot](https://git.unturf.com/engineering/unturf/discord.agents.ai.unturf.com#unturfdiscordbot)
|
||||
|
||||
.. note::
|
||||
Big shout out to **naptha.ai** (https://naptha.ai) for running the model servers, internet, and electric that make these publicly accessible models possible!
|
||||
|
||||
A big thank you to **naptha.ai** (https://naptha.ai) for powering the model servers, connectivity, and the electricity that make these publicly accessible models possible!
|
||||
Loading…
Add table
Add a link
Reference in a new issue