Distribution

Web Containers

!!!

• https://webcontainers.io/ (opens in a new tab)
  • https://webassembly.org/ (opens in a new tab)
  • https://pglite.dev/ (opens in a new tab)
  • https://pglite.dev/docs/about (opens in a new tab)

Rust

• Rust multi platform distribution

Distribution options

• app packaged with the model
• app without the model (if user has ollama and runs it locally)
  • provide an option to use custom model (set path to model)

• docker

  • open-webui example (opens in a new tab)

• What about packages? (docker images?)

  • Use github packages (opens in a new tab) (?)
  • https://github.com/orgs/open-webui/packages?repo_name=open-webui (opens in a new tab)
  • https://github.com/open-webui/open-webui/pkgs/container/open-webui (opens in a new tab)

• What about keeping docker images updated?

  • https://containrrr.dev/watchtower/ (opens in a new tab)

How should the product be distributed?

Examples:

JanAI

JanAI (opens in a new tab)

Jan runs on any hardware. From PCs to multi-GPU clusters, Jan supports universal architectures:

• NVIDIA GPUs (fast)
• Apple M-series (fast)
• Apple Intel
• Linux Debian
• Windows x64

jan downloads (opens in a new tab) - .exe for Windows - dmg for MacOS Intel / M1,2,3,4 - Linux .deb / .AppImage

Llamafile

https://github.com/ggerganov/llama.cpp (opens in a new tab)

https://github.com/jart/cosmopolitan (opens in a new tab)

Cosmopolitan Libc makes C a build-once run-anywhere language, like Java, except it doesn't need an interpreter or virtual machine. Instead, it reconfigures stock GCC and Clang to output a POSIX-approved polyglot format that runs natively on Linux + Mac + Windows + FreeBSD + OpenBSD + NetBSD + BIOS with the best possible performance and the tiniest footprint imaginable.

open-webui

Consider communicating changelog in the product (not just in github) as in open-webui (opens in a new tab).