nicegui-llm-bridge

Drive any NiceGUI 3.x app from an LLM. No browser. No DOM. Pure wire.

[!CAUTION] v0.0.1-alpha — YOLO ship, single overnight build, only smoke-tested against the bundled examples/gallery.py and one probe against live nicegui.io. No unit tests beyond the round-trip E2E. Wire protocol is NiceGUI-3.x-specific and will break on major-version drift. Feedback from agent operators welcome — see issue #1 for the questions we'd most like answered.

This is the third member of the nicegui-tui / nicegui-wire / nicegui-llm-bridge trio. It treats NiceGUI's Client abstraction as exactly what it is — a consumer of an outbox — and adds an LLM-shaped rendering target:

• nicegui-tui → renders into a Textual TUI in-process
• nicegui-wire → renders into Textual / framebuffer over the network
• nicegui-llm-bridge → renders into an LLM's cognition, via MCP

What it does

Connects to a running NiceGUI page via the Socket.IO wire (reusing nicegui-wire's engine), keeps a live element-tree mirror, and exposes it to an LLM as indented bullets:

- div#3
  - div#4 text='nicegui-llm-bridge gallery'
  - q-tabs#6 model-value='basics' [interactive: update:modelValue]
    - q-tab#7 label='Basics', name='basics'
  - q-tab-panels#12 model-value='basics' [interactive: update:modelValue]
    - q-tab-panel#13 name='basics'
      - div#17 text='count: 0'
      - q-btn#18 label='Increment' [interactive: click]
      - q-btn#19 label='Warn' [interactive: click]

Then the LLM can act: click(18) → page rerenders → LLM sees count: 1.

MCP tools

Tool	Purpose
open(url)	Connect to a NiceGUI page and return the indented-bullet view
read()	Re-emit current mirror (no network roundtrip)
actions()	JSON list of every interactive element
click(element_id)	Fire a click; return post-action view
fill(element_id, value)	Set a form value; return post-action view
choose(element_id, option)	Pick a radio/select option by visible label

All element references are raw integer IDs (#18), surfaced inline so the LLM can never guess — it must read before it acts.

Why indented bullets

For tree-shaped data, indentation column = depth. No pointer chasing, no global indices to cross-reference, matches the training distribution (READMEs, file trees, doc outlines). Edge lists, JSON, and adjacency lists are all worse for LLM tree reading — see arXiv 2511.10234 (Lost in Serialization) for the empirics.

Snapshot-on-demand

The background wire connection ingests every server update into a local mirror. The LLM never sees the firehose — it only sees what read() / click() / fill() return. A ui.log tailing at 10 Hz is invisible until the LLM asks.

Try it

# 1. Set up venv + install (one-time)
python3 -m venv .venv
.venv/bin/pip install -e .

# 2. Launch the gallery (terminal A)
.venv/bin/python examples/gallery.py

# 3. Drive it (terminal B)
.venv/bin/python examples/smoke.py     # CLI demo, prints transcript
.venv/bin/python tests/test_mcp_e2e.py # full MCP round-trip

Use from Claude Code

Add to ~/.claude.json mcpServers (or use the bundled .mcp.json if you launch Claude Code from this repo):

{
  "mcpServers": {
    "nicegui-bridge": {
      "command": "/Users/evnchn/nicegui-llm-bridge/.venv/bin/python",
      "args": ["-m", "nicegui_llm_bridge.mcp_server"]
    }
  }
}

Then in Claude Code: open a NiceGUI app (examples/gallery.py in another terminal), and ask Claude to drive it. Claude will call open → actions → click/fill like a screen reader would.

Architecture

NiceGUI server ──Socket.IO──> nicegui-wire (tree decode, event encode)
                                        │
                                        ▼
                              ElementTree mirror
                                        │
                                        ▼
                           serialize.render() → bullets
                                        │
                                        ▼
                              Session (snapshot-on-demand)
                                        │
                                        ▼
                              FastMCP tools (stdio)
                                        │
                                        ▼
                                  LLM consumer

Status

v0.0.1-alpha, YOLO-shipped 2026-06-03 ~10:30 HKT. Works against NiceGUI 3.12.x. Probed against https://nicegui.io (380 elements in 1.22 s, 15 KB YAML view, 42 actionable elements surfaced).

Known limitations:

• ui.html and arbitrary Vue components → render as opaque entries (nicegui-html#39 with no inner content)
• ui.aggrid / ui.echart → schema only, not data
• Page navigation (multi-route apps) → not yet; open() reconnects from scratch
• Quasar client-side toggles (q-drawer / q-menu burgers) don't roundtrip the wire — they live in DOM, not the outbox
• q-btn content nested in nicegui-html looks empty to the LLM until we surface innerHTML (planned)
• Wire protocol is NiceGUI 3.x specific; major-version drift will break

Credits

Wire decoding lifted from nicegui-wire. Gallery example adapted from nicegui-tui.