Skip to content
This repository has been archived by the owner on Aug 28, 2024. It is now read-only.

Update README.md #262

Merged
merged 1 commit into from
Aug 28, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
124 changes: 2 additions & 122 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -1,125 +1,5 @@
# NO LONGER ACTIVE

<p align="center">
<a href="https://discord.gg/JeXhcmjZVm">
<img alt="discord-invite" src="https://dcbadge.vercel.app/api/server/JeXhcmjZVm?style=flat">
</a>
</p>
We are focusing all of our effort on KubeAI, a platform focused on making it simple to operationalize AI on Kubernetes.

NOTICE: Changes coming! runbooks.git (previously substratus.git) will be refactored to focus on Notebooks on K8s.

🎵 Fine-tune LLM models with no/low code
📔 Provide a Colab style seamless Notebook experience
☁️ Provide a unified ML platform across clouds
⬆️ Easy to install with minimal dependencies

Looking for serving?
🚀 [substratusai/lingo](https://github.com/substratusai/lingo): Serve popular OSS LLM models in minutes on CPUs or GPUs

Support the project by adding a star on GitHub! ❤️

## Quickstart

Create a local Kubernetes cluster using Kind.

[embedmd]:# (../install/kind/up.sh bash /kind.*/ $)
```bash
kind create cluster --name substratus --config - <<EOF
apiVersion: kind.x-k8s.io/v1alpha4
kind: Cluster
nodes:
- role: control-plane
extraPortMappings:
- containerPort: 30080
hostPort: 30080
EOF
```

Install Substratus.

```bash
kubectl apply -f https://raw.githubusercontent.com/substratusai/substratus/main/install/kind/manifests.yaml
```

Import a small Open Source LLM.

```bash
kubectl apply -f https://raw.githubusercontent.com/substratusai/substratus/main/examples/facebook-opt-125m/base-model.yaml
```

[embedmd]:# (../examples/facebook-opt-125m/base-model.yaml yaml)
```yaml
apiVersion: substratus.ai/v1
kind: Model
metadata:
namespace: default
name: facebook-opt-125m
spec:
image: substratusai/model-loader-huggingface
params:
name: facebook/opt-125m
```

Serve the LLM.

```bash
kubectl apply -f https://raw.githubusercontent.com/substratusai/substratus/main/examples/facebook-opt-125m/base-server.yaml
```

[embedmd]:# (../examples/facebook-opt-125m/base-server.yaml yaml)
```yaml
apiVersion: substratus.ai/v1
kind: Server
metadata:
name: facebook-opt-125m
spec:
image: substratusai/model-server-basaran
model:
name: facebook-opt-125m
```

Checkout the progress of the Model and the Server.

```bash
kubectl get ai
```

When they report a `Ready` status, start a port-forward.

```bash
kubectl port-forward service/facebook-opt-125m-server 8080:8080
```

Open your browser to [http://localhost:8080/](http://localhost:8080/) or curl the LLM's API.

*PS: Because of the small size of this particular LLM, expect comically bad answers to your prompts.*

```bash
curl http://localhost:8080/v1/completions \
-H "Content-Type: application/json" \
-d '{ \
"model": "facebook-opt-125m", \
"prompt": "Who was the first president of the United States? ", \
"max_tokens": 10\
}'
```

Delete the local cluster.

[embedmd]:# (../install/kind/down.sh bash /kind.*/ $)
```bash
kind delete cluster --name substratus
```

If you want to try out a more capable LLM, running on substantial hardware, try [Kind with
GPU support](https://www.substratus.ai/docs/quickstart/local-kind?kind-mode=gpu),
or try [deploying Substratus in GKE](https://www.substratus.ai/docs/quickstart/gcp).

## Docs
* [Overview](https://www.substratus.ai/docs/overview)
* [Architecture](https://www.substratus.ai/docs/architecture)

## Creators
Feel free to contact any of us:
* [Nick Stogner](https://www.linkedin.com/in/nstogner/)
* [Sam Stoelinga](https://www.linkedin.com/in/samstoelinga/)
Instead see: https://github.com/substratusai/kubeai