engineering/uncloseai.com
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

f

Browse source
This commit is contained in:
root 2024-10-19 22:57:45 +00:00
parent dda8ed43ec
commit 36ec72c857
1 changed files with 6 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

8
index.html
View file

@ -194,6 +194,7 @@ streamResponse();
<p>We use vLLM to run models, currently full f16 safetensors. We make sure to use a virtualenv to hold the dependencies.</p>
<p>We are considering supporting ollama for better quant support.</p>
<p>Stand up a replica cluster on a new domain.</p>
<pre><code>
cd ~
python3 -m venv env
@ -236,10 +237,13 @@ hermes.ai.unturf.com {
</code></pre>
<p>We will likely implement a rate limit based on client IP address.</p>
<h2 id="client-side">Client Side Only Example, chat with this page.</h2>
<h2 id="client-side">Client Side Only Example, chat with this page.</h2>
<p>Because we don't use API keys we don't have any real need for a server.</p>
<p>view page source, it's _all_ there!</p>
<p>view the page source, it's _all_ there!</p>
<p>TODO: a Javascript CDN bundle of the minimum viable web client for this demo will be hosted and provided free of charge.</p>
<div id="chat-container">
<div id="chat-box"></div>