"Self-host a local AI stack and access it from anywhere" | Tailscale Blog
-
NOTE: I got to this article by clicking on a banner ad that Tailscale bought in the release notes of OpenWebUI's GitHub. However, I thought it was still neat and worth sharing. Please discuss and share any alternatives for the paid/proprietary/problematic pieces of this setup:
Proxmox for virtualization, NixOS for repeatability, Docker for packaging, and Tailscale for secure access from anywhere. We’ll wire up an NVIDIA A4000 GPU to a NixOS VM via PCIe passthrough, set up Ollama and Open WebUI, and get chatting with local large language models all entirely within your Tailnet.
-
NOTE: I got to this article by clicking on a banner ad that Tailscale bought in the release notes of OpenWebUI's GitHub. However, I thought it was still neat and worth sharing. Please discuss and share any alternatives for the paid/proprietary/problematic pieces of this setup:
Proxmox for virtualization, NixOS for repeatability, Docker for packaging, and Tailscale for secure access from anywhere. We’ll wire up an NVIDIA A4000 GPU to a NixOS VM via PCIe passthrough, set up Ollama and Open WebUI, and get chatting with local large language models all entirely within your Tailnet.
I was going to click, until I saw the computer system it is running on.
-
I was going to click, until I saw the computer system it is running on.
You mean an A4000? My 1070 can run llama 3.1 comfortably (the low B versions). My 7800XT answers instantly.
