From 36ec72c857bd3d32f5296bc22997f2c196f06b19 Mon Sep 17 00:00:00 2001 From: root Date: Sat, 19 Oct 2024 22:57:45 +0000 Subject: [PATCH] f --- index.html | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/index.html b/index.html index bcb2e13..622d42f 100644 --- a/index.html +++ b/index.html @@ -194,6 +194,7 @@ streamResponse();

We use vLLM to run models, currently full f16 safetensors. We make sure to use a virtualenv to hold the dependencies.

We are considering supporting ollama for better quant support.

+

Stand up a replica cluster on a new domain.


 cd ~
 python3 -m venv env
@@ -236,10 +237,13 @@ hermes.ai.unturf.com {
 	

We will likely implement a rate limit based on client IP address.

+

Client Side Only Example, chat with this page.

-

Client Side Only Example, chat with this page.

+

Because we don't use API keys we don't have any real need for a server.

-

view page source, it's _all_ there!

+

view the page source, it's _all_ there!

+ +

TODO: a Javascript CDN bundle of the minimum viable web client for this demo will be hosted and provided free of charge.