diff --git a/index.html b/index.html index bcb2e13..622d42f 100644 --- a/index.html +++ b/index.html @@ -194,6 +194,7 @@ streamResponse();
We use vLLM to run models, currently full f16 safetensors. We make sure to use a virtualenv to hold the dependencies.
We are considering supporting ollama for better quant support.
+Stand up a replica cluster on a new domain.
cd ~
python3 -m venv env
@@ -236,10 +237,13 @@ hermes.ai.unturf.com {
We will likely implement a rate limit based on client IP address.
+Because we don't use API keys we don't have any real need for a server.
-view page source, it's _all_ there!
+view the page source, it's _all_ there!
+ +TODO: a Javascript CDN bundle of the minimum viable web client for this demo will be hosted and provided free of charge.