How to Run a Private Remote AI Assistant on DGX Spark

Hardware + price warning · checked 24 July 2026Do not buy DGX Spark on the assumption that Windows support will arrive later.

DGX Spark and the pre-release RTX Spark family publish strikingly similar headline numbers, up to one petaflop and up to 128 GB unified memory, but NVIDIA currently lists them as different product categories: DGX Spark is a Linux companion system, while RTX Spark is a Windows primary system. Public documentation does not establish that their drivers, firmware or boards are interchangeable, and NVIDIA has announced no Windows driver, Windows image or upgrade path for DGX Spark.

Buy DGX Spark only if DGX OS works for the machine's useful life. Future Windows support is possible in theory, but today it is speculation. There is also no public evidence for claims about NVIDIA's commercial motive; the support gap itself is the decision-relevant fact.

Indicative MSRP · NVIDIA Marketplace Germany€4,800

NVIDIA's own DGX Spark listing displayed this price and was out of stock when checked. Use it as a reference for the NVIDIA-branded system, not a ceiling for partner products. OEMs choose their own configurations and selling prices.

Check the current NVIDIA Marketplace price →

NVIDIA platform comparison →DGX Spark software requirements →Pre-release RTX Spark example →

NVIDIA AI Enterprise check · GB10 / DGX SparkDo not treat preinstallation, NIM access or a Spark badge as a production entitlement.

A 90-day NVIDIA AI Enterprise - DGX Spark evaluation exists only when it is purchased, requested or explicitly issued on the NVIDIA Entitlement Certificate (EC); not every OEM offer includes it. Record the EC/order line, GPU metric, start and end date, support route and renewal before deployment. It is an evaluation, not a public free-forever entitlement to the complete supported production suite.

Free components remain useful for development: Omniverse and NVIDIA AI Workbench are free, and the standard NVIDIA Developer Program NIM route is for development, research and test up to 16 GPUs with community support. Production self-hosting generally needs NVIDIA AI Enterprise.

DGX Spark 90-day evaluation and EC activation →NVIDIA AI Enterprise licensing guide →NIM developer-program route and production boundary →Omniverse license and support route →AI Workbench introduction →

docker ps -a --filter 'name=^/open-webui$'
docker inspect open-webui --format '{{.Config.Image}}'
docker inspect open-webui --format \
  '{{range .Mounts}}{{println .Name "->" .Destination}}{{end}}'
docker volume inspect open-webui open-webui-ollama

# Check the state before using docker exec.
running=$(docker inspect open-webui --format '{{.State.Running}}')
printf 'running=%s\n' "$running"
if [ "$running" = true ]; then
  docker exec open-webui ollama --version
else
  printf '%s\n' 'Ollama version check skipped: open-webui is stopped.'
fi

The volume inspection output includes Docker's current host mountpoint. Treat that path as implementation state, not a user-managed application folder.

docker exec requires a running container. The inspection block now reports the stopped state without failing. If you need the bundled Ollama version while docker ps -a shows Exited (0), start Open WebUI from NVIDIA Sync first (preferred), or use the one-shot check below. A stopped container does not mean either named volume is missing.

When the Sync launcher is stopped, an optional one-shot check restores the prior stopped state:

was_running=$(docker inspect open-webui --format '{{.State.Running}}')
if [ "$was_running" != true ]; then docker start open-webui >/dev/null; fi
docker exec open-webui ollama --version
if [ "$was_running" != true ]; then docker stop open-webui >/dev/null; fi

Pulling the moving :ollama tag changes the local image, but it does not replace an existing container. Before an update, review the Open WebUI release notes and back up at least the application volume; the Ollama model-volume backup is optional but can be very large. Replace YYYY-MM-DD with the actual backup date.

mkdir -p "$PWD/open-webui-backups"
docker run --rm \
  -v open-webui:/data:ro \
  -v "$PWD/open-webui-backups:/backup" \
  alpine tar czf /backup/open-webui-YYYY-MM-DD.tar.gz /data

# Optional: preserve downloaded Ollama models as well.
docker run --rm \
  -v open-webui-ollama:/data:ro \
  -v "$PWD/open-webui-backups:/backup" \
  alpine tar czf /backup/ollama-models-YYYY-MM-DD.tar.gz /data

Stop Open WebUI with the x beside its Sync launcher. In a Sync terminal, pull the image and remove only the stopped container. Do not remove either named volume.

docker pull ghcr.io/open-webui/open-webui:ollama
docker rm open-webui

Launch Open WebUI from Sync again. Its script now creates a new container from the pulled image and reattaches both volumes. Then inspect startup and the bundled runtime:

docker ps --filter 'name=^/open-webui$'
docker logs --tail 100 open-webui
docker exec open-webui ollama --version

The combined image carries its own Ollama build. It can lag the newest standalone release, and current Ollama model metadata can declare a minimum required runtime version. If a new model refuses to pull or load, compare docker exec open-webui ollama --version with the model requirement before diagnosing the GPU. Updating the combined image is the clean route; replacing Ollama inside a running container creates an unrepeatable installation.

When a model's minimum required version outpaces the bundled build, decoupling Ollama from Open WebUI is a practical lower-complexity step before a vLLM migration. That is an operational comparison, not a vendor support or cost guarantee. Decoupling is what upgrades Ollama, not the tag switch: a moving :main tag tracks Open WebUI on its own schedule, while the host Ollama service tracks the standalone release independently.

Install standalone Ollama with the official installer. On the Spark, NVIDIA's recipe uses that same installer, and it detects the GB10 Blackwell GPU.

curl -fsSL https://ollama.com/install.sh | sh

Stop the newly installed service before copying. Create the destination for the ollama service account, copy the model store, restore ownership, then verify one real GPU-backed inference before switching Open WebUI. The old named volume remains untouched as the immediate rollback copy.

sudo systemctl stop ollama
sudo install -d -o ollama -g ollama /usr/share/ollama/.ollama
docker run --rm \
  -v open-webui-ollama:/data:ro \
  -v /usr/share/ollama/.ollama:/store \
  alpine cp -a /data/. /store/
sudo chown -R ollama:ollama /usr/share/ollama/.ollama
sudo systemctl start ollama

ollama list
model_tag='replace-with-model-tag'
ollama run "$model_tag" 'Reply with exactly: migration verified'
ollama ps

Then stop the Sync launcher, rename the retained combined container so the open-webui name is free, and start the ordinary image against the host service. Replace YYYY-MM-DD and the secret placeholder before running the commands. Docker host networking keeps Open WebUI on host port 12000 and lets it reach loopback-bound Ollama without exposing port 11434 to the LAN.

docker stop open-webui
docker rename open-webui open-webui-bundled-rollback-YYYY-MM-DD

docker run -d \
  --network=host \
  -e PORT=12000 \
  -e OLLAMA_BASE_URL=http://127.0.0.1:11434 \
  -e WEBUI_SECRET_KEY='<replace-with-a-long-random-secret>' \
  -v open-webui:/app/backend/data \
  --name open-webui \
  --restart unless-stopped \
  ghcr.io/open-webui/open-webui:main

The current DGX Spark run pairs RadixArk/Qwen3.8-27B-NVFP4-BF16-LMHead with the incoai/Qwen3.8-27B-DFlash2 drafter in a digest-pinned SGLang container. Two top-level VS Code agent tasks produced 161 successful local model calls. SGLang returned 166 successful chat-completion responses when five separate readiness and acceptance calls are included, with no inference HTTP failure in the recorded run.

Across 1,242 server windows with one active request, generation throughput averaged 20.4 tok/s and had an 18.6 tok/s median. Another 102 windows with two active requests averaged 36.6 aggregate tok/s. The DFlash windows averaged a 3.39-token acceptance length and a 34.2% acceptance rate. VS Code round-trip latency had a 20.1-second median and a 278.8-second 95th percentile across the 161 calls; that client timing includes prompt handling, reasoning, generation, transport and harness overhead.

During the VS Code interval, SGLang's prefill records represented 14,316,770 prompt tokens: 290,146 newly processed tokens and 14,026,624 reused cache tokens, for a 98.0% cached share. Across 84 complete uncached 2,048-token chunks, input throughput had a 1,005 tok/s median and a 1,239.1 tok/s mean. High cache reuse is useful for an agent loop, but it also means this session is not a cold-prompt benchmark.

The recipe below pins the tested container digest and the locally observed target and draft revisions. It binds the endpoint to loopback, limits the container to 96 GB and leaves system memory outside the service. Re-test before changing any pin, memory fraction or speculative setting.

cache_root="${XDG_CACHE_HOME:-${HOME}/.cache}"
mkdir -p "$cache_root/huggingface" "$cache_root/sglang-qwen38"

docker run -d \
  --name sglang-qwen38 \
  --restart unless-stopped \
  --gpus all \
  --network host \
  --ipc host \
  --shm-size 16g \
  --memory 96g \
  --memory-swap 96g \
  --ulimit memlock=-1 \
  --ulimit stack=67108864 \
  -e HF_HOME=/root/.cache/huggingface \
  -e SGLANG_CACHE_DIR=/root/.cache/sglang \
  -v "$cache_root/huggingface:/root/.cache/huggingface" \
  -v "$cache_root/sglang-qwen38:/root/.cache/sglang" \
  --entrypoint "" \
  lmsysorg/sglang@sha256:616a3e97f45191af975896cfa644279096cb31bd408a071c2e99ca7209c3cafe \
  python3 -m sglang.launch_server \
    --trust-remote-code \
    --model-path RadixArk/Qwen3.8-27B-NVFP4-BF16-LMHead \
    --revision 009632fef96dd349150baa780c984e62e70e91fe \
    --served-model-name qwen38-dev \
    --context-length 262144 \
    --tp-size 1 \
    --host 127.0.0.1 \
    --port 30000 \
    --mem-fraction-static 0.60 \
    --kv-cache-dtype auto \
    --attention-backend flashinfer \
    --chunked-prefill-size 2048 \
    --max-running-requests 4 \
    --mamba-radix-cache-strategy extra_buffer_lazy \
    --mamba-ssm-dtype float32 \
    --max-mamba-cache-size 16 \
    --cuda-graph-max-bs-decode 4 \
    --disable-prefill-cuda-graph \
    --speculative-algorithm DFLASH \
    --speculative-draft-model-path incoai/Qwen3.8-27B-DFlash2 \
    --speculative-draft-model-revision dedf8df68adfb1afeaf7b7480c0a0243108177b4 \
    --speculative-num-draft-tokens 8 \
    --speculative-draft-model-quantization unquant \
    --reasoning-parser qwen3 \
    --tool-call-parser qwen3_coder \
    --enable-metrics \
    --enable-cache-report \
    --sleep-on-idle

The tested startup loaded 22.68 GB for the target and 3.64 GB for the draft. It allocated 19.48 GB of FP8 target KV cache and 12.18 GB of BF16 draft KV cache for 638,179 tokens, then reported 41.52 GB available after CUDA graph capture. End-to-end engine startup took 165.2 seconds. The 638,179-token pool is shared across active requests; four configured running requests do not mean four simultaneous 262,144-token contexts.

Check health and the served model before connecting an interface. Then run a bounded acceptance task that must read once, edit once and verify the diff. Stop if the same tool request repeats, if no edit appears, or if the harness changes the requested model name.

curl -fsS http://127.0.0.1:30000/health
curl -fsS http://127.0.0.1:30000/v1/models

Run these commands in a terminal inside the logged-in GNOME desktop. A plain SSH shell may not carry the graphical values.

echo "$XDG_SESSION_TYPE"
echo "$DISPLAY"
echo "$XAUTHORITY"
xrandr --listmonitors
systemctl --user show-environment |
  grep -E '^(DISPLAY|XAUTHORITY|XDG_SESSION_TYPE)='

Continue only with the display belonging to the logged-in X11 desktop. GDM can use :0 for its greeter and :1 for the user session. Never force either value in generic instructions.

Add this exact line to ~/.xprofile, then run it once from the current graphical terminal:

dbus-update-activation-environment --systemd DISPLAY XAUTHORITY

Remove every hardcoded Environment="DISPLAY=..." shown by systemctl --user cat sunshine.service. In the user-service drop-in, keep the session ordering and replace any old pre-start check with this one:

[Unit]
After=graphical-session.target
Wants=graphical-session.target
StartLimitBurst=5
StartLimitIntervalSec=60

[Service]
ExecStartPre=
ExecStartPre=/bin/sh -c 'i=0; while [ "$$i" -lt 60 ]; do env="$$(systemctl --user show-environment)"; printf "%%s\n" "$$env" | grep -q "^DISPLAY=" && printf "%%s\n" "$$env" | grep -q "^XAUTHORITY=" && exit 0; i=$$((i + 1)); sleep 1; done; echo "DISPLAY/XAUTHORITY not available in systemd user environment" >&2; exit 1'

The format string must be %%s. systemd treats a single % as a specifier, so %s can be expanded before the shell sees it.

Back up ~/.config/sunshine/sunshine.conf. Remove resolutions, nvenc_tuning_info, bitrate, channels, nvenc_multipass, nvenc_rc, fps and codec when the tested release reports them as unrecognized. Set requested resolution, frame rate, bitrate and codec in Moonlight. Preserve site-specific csrf_allowed_origins locally.

adapter_name = /dev/dri/renderD128
capture = x11
encoder = nvenc
min_threads = 2
nvenc_preset = 1
nvenc_spatial_aq = 1
nvenc_twopass = quarter_res
nvenc_vbv_increase = 0
output_name = <numeric ID reported as connected>
qp = 20

journalctl --user -u sunshine.service -n 200 --no-pager |
  grep "Detected display"

Choose the numeric id: <N> on the row that says connected: true. Do not copy an example number, assume HDMI is ID 0, or use a connector string such as USB-C-2; the reproduced Sunshine build converted that string into an invalid display number.

systemctl --user daemon-reload
systemctl --user enable sunshine.service
systemctl --user restart sunshine.service

systemctl --user is-enabled sunshine.service
systemctl --user is-active sunshine.service

journalctl --user -u sunshine.service --since "2 minutes ago" --no-pager |
  grep -E \
  "Detected display|Configuring selected display|Found .*encoder|System tray|Fatal|Unable to"

Success means the connected ID is selected and the log finds h264_nvenc, hevc_nvenc and av1_nvenc. If cuda::cuda_t doesn't support any format other than AV_PIX_FMT_NV12 appears during probing but all three encoders are found afterward, it is not the startup failure.