# ChatForge: a Local NPU AI Assistant for the Copilot Key URL: https://www.laserlloyd.com/projects/chatforge-local-npu-ai-assistant/ Published: 2026-10-02 Updated: 2026-10-19 Description: ChatForge: a Windows 11 tray assistant running Qwen2.5-1.5B on the Intel NPU — hotkey popup, nine tools, files in, Office documents out. MIT, free. Reuse: free for personal use — full policy at https://www.laserlloyd.com/llms.txt Our laptop shipped with a dedicated AI key and a dedicated AI chip, and out of the box they have never met: the key opens Microsoft Copilot in the cloud, and the NPU sits there doing nothing. That bothered us more than it should have. So now the key opens ChatForge, our own assistant. It runs a 1.5B Qwen on the NPU at 42–51 tokens per second. No account, no subscription, no per-token bill, and the conversation never leaves the PC. When a question is too big for a 1.5B brain, our GPU server or a cloud model is one click away in the same little popup. Updated 2026-10-19 (ChatForge 0.2.0). ChatForge now installs with a double-click (no terminal, Python or git), sets up its local model from one button on first start, runs that model on the CPU when a PC has no NPU, and asks before starting with Windows. A StudioForge server on another PC now needs an API key. The install steps below are the 0.2.0 ones; what changed and how to upgrade is in What's new in 0.2. This article is the tour: what it does, what the NPU is actually like to live with (including the model that compiled perfectly and then spoke in tongues), and the tricks that make a model this small give real answers instead of the classic small-model shrug. tl;dr Get it: github.com/LaserLloyd/ChatForge — MIT, free, version 0.2.0. Download the zip there, double-click Install ChatForge, click Set up local AI. (The zip in the Downloads box below is the older 0.1.0, which needs a terminal to install.) What it is: a free, MIT-licensed notification-area assistant for Windows 11. Ctrl+Alt+C (or the Copilot key, remapped with PowerToys) opens a 420 × 620 popup at the lower right; Escape puts it away. The local model: Qwen2.5-1.5B-Instruct (INT4, a 0.94 GB download) served by OpenVINO Model Server on the Intel "AI Boost" NPU, or on the CPU when there is no NPU. On the NPU, first load compiles in about 45 s; after that it loads from cache in about 3 s and decodes at 42–51 tok/s. It unloads itself after 10 idle minutes. Bigger models, same menu: StudioForge (your own GPU server), MiniMax, OpenAI, DeepSeek, or any OpenAI-compatible URL. Keys live in the Windows Credential Manager, never in a config file. Nine tools: web and news search, page fetch, weather, Wikipedia, exchange rates, date and time, a calculator — and create_document, which saves a Word, Excel or PowerPoint file the model wrote. Attach up to 10 files per message and ask about them. What it is not: an agent. No tool runs a program or changes a setting; the one tool that writes can only add a new file to its own folder. ## Where to get it One MIT-licensed repo: github.com/LaserLloyd/ChatForge, including the runtime notes with every NPU measurement this article quotes. On the GitHub page, Code > Download ZIP gets you the current version with the installer. The Downloads box at the end of this page holds the older 0.1.0 snapshot. ## What you end up with A question away from any app: press the key, type, read, Escape. Every answer says which model wrote it and how fast, so you always know whether you got the free local one or something with a meter on it. When a question needs live data, a tool chip shows what was looked up; when you ask a bigger model for a report, the reply carries a document card with **Open** and **Show in folder**. A weather question, answered on the NPU: a tool chip above the reply, a table in it. A spreadsheet in, a Word report out — on a cloud model. The model menu: your last-used models first, then only the providers you set up. The conversations, figures, location and server name in the screenshots are example data; the UI is the real thing, rendered in Edge (the engine behind WebView2) against the project's development mock. ## How it fits together One Python process owns everything: the tray icon, the global hotkey, the popup and Settings windows (pywebview on WebView2), and an asyncio core that streams answers and runs tools. The model server is a child process it fully owns — started when the popup opens with the local model selected, killed with the app by a Windows job object, so a crash can't leave a model squatting in your RAM. Ctrl+Alt+C or Copilot key (PowerToys) Popup 420 × 620, lower right question + attached files Chat engine streams, fits context, recovers ≤ 6 rounds 9 tools search · weather · fetch calc · create_document Qwen2.5-1.5B on the NPU OpenVINO Model Server, local no account · 42–51 tok/s Your GPU server StudioForge or any OpenAI-compatible URL Cloud providers MiniMax · OpenAI · DeepSeek keys in Credential Manager local model fails? fallback provider answers Very little of it was written from scratch, and we mean that as a boast: the process supervisor, downloader, tray and logging came from StudioForge, the chat UI and Markdown pipeline from DisPatch, and the streaming client from our benchmarking tool. All of them had already survived months of daily use, and every adapted module says so at the top. That reuse is why the first commit and v0.1.0 are two days apart (the evening of 2026-09-30 to 2026-10-02), with an LLM agent doing the assembly against a written plan of about 1,000 lines. ## Install {#install} ChatForge 0.2.0 installs in three steps on Windows 11, with no terminal and no administrator rights: 1. **Get it.** On the GitHub page choose **Code > Download ZIP** and extract it. 2. **Double-click `Install ChatForge`.** It fetches uv (hash-checked) and Python 3.12 if they are missing, installs the app for your user in `%LOCALAPPDATA%\Programs\ChatForge`, adds Start menu and desktop shortcuts, asks once whether to start with Windows, and opens ChatForge. The installer is not signed, so Windows shows "Windows protected your PC" the first time: **More info > Run anyway**. 3. **Click "Set up local AI"** on the card in the popup. It installs the model runtime (OpenVINO Model Server, 139 MB) and downloads the starter model, `OpenVINO/Qwen2.5-1.5B-Instruct-int4-ov` (0.94 GB, Apache-2.0), with one progress bar and a Cancel button, checks every file's SHA-256 against Hugging Face's, and points the chat at it. About a minute on a fast connection. **Use StudioForge** and **Use an API key** on the same card skip the local model. To upgrade, run `Install ChatForge` again from a newer zip; settings and data stay in `%LOCALAPPDATA%\ChatForge`. `Uninstall ChatForge` removes the app and keeps your data unless you add `--purge`. The repository's deploy/windows/README.md covers the installer's options, and the README has a paste-into-PowerShell version of the same three steps plus the developer route (`git clone`, `uv sync --extra dev`, `uv run chatforge doctor`). The first time the model loads on an NPU, the NPU compiles it: about 45 s, with a countdown in the popup header so it doesn't feel broken. Every load after that comes from the compile cache in about 3 s. On a CPU it loads in a few seconds and the card and status line say "slower". We have not re-run the installer for this article; the steps above are the README's, and the README's screenshots of it were taken on a Windows 11 PC with no NPU. ## The Copilot key problem Here's the part nobody tells you when you buy the laptop. The Copilot key is not a key you can bind. It sends `Win+Shift+F23`, Windows registers that combination for itself, and the official Settings picker for it only offers packaged, signed apps. Your own program is not on the guest list for your own keyboard. The workable route is PowerToys Keyboard Manager: remap the shortcut `Win (Left) + Shift (Left) + F23` to `Ctrl+Alt+C` (ChatForge's hotkey) and turn on PowerToys' *Run at startup*. One remap, and the key Microsoft reserved for Copilot opens your assistant instead. If the key ever opens Windows Search or Copilot again, PowerToys isn't running — that's the entire failure mode, and it's first in the troubleshooting list because it happens. ## Making a 1.5B model worth asking A 1.5B model with a 4096-token window is a goldfish with a library card. The interesting engineering in ChatForge works with that fact rather than pretending it away: - **A short tool list.** Nine tools exist, but the local model is only offered five (search, page fetch, weather, date/time, calculator). Small models get confused by long tool menus; cutting the list is what keeps its tool calls clean. - **Look it up first.** A question with clear live-data intent ("weather", "latest", "price of") runs the matching tool before the model's first round, so the model answers from the result instead of guessing. Plain pattern matching, no extra model call. - **Refusal recovery.** Small models love to answer "I don't have access to real-time data" even with a search tool sitting right there. A reply that refuses in the first person, or promises a lookup it never makes ("let me search for that…" and then doesn't), is dropped; the engine runs the lookup itself and the model answers again. The popup shows "Looking it up…" while this happens, and you never see the first draft's shrug. - **No summarising.** When a chat outgrows the window, the oldest turns are simply left out of the prompt, and a divider in the popup shows exactly where the model's view starts. Nothing is paraphrased behind your back, fitting costs no extra model call, and the model is told that older messages were cut so it asks rather than invents. - **A loop guard and a budget.** A message gets at most 6 rounds of tool calls, and a repeated identical call is refused — a 1.5B model will happily call the weather tool forever if you let it. - **A safety net.** If the NPU model fails to load, crashes or times out, the same message can be re-sent to a provider you pick. None of this needs the NPU — the same engine drives the big models — but it's what makes the free local model answer "will it rain this weekend" correctly, at full NPU speed, instead of refusing politely. ## The model that compiled fine and answered in garbage The plan was to ship Qwen3-4B as the local model. Bigger model, newer family, obvious choice. The repo's docs/RUNTIME-NOTES.md records the gate that killed it: `Qwen3-4B-int4-ov` compiles for the NPU without a single complaint and then produces garbage output with OVMS 2026.4 — while the same model files answer correctly on CPU at 24 tok/s. The fault is NPU-specific, every NPU configuration tried failed, and the alternative quantisations that might have worked can't do tool calls in OVMS 2026.4. On an NPU, a clean compile proves nothing. Gate the output before you trust it. What shipped instead, measured on a Core Ultra 5 226V (Lunar Lake): | Qwen2.5-1.5B-Instruct INT4 on the NPU | Measured | |---|---| | First compile (one-off) | 44.5 s | | Load from compile cache | 2.7–3.2 s | | Decode | 42–51 tok/s | | Time to first token (short prompts) | 0.5–0.8 s | | Compile cache on disk | 306 MiB | | Prompt window (static) | 4096 tokens | One tuning note from the same file: OVMS accepts a `BEST_PERF` compile flag that buys about 10% more decode speed (48.3 tok/s) for a 135 s compile — three times the default. ChatForge leaves it off, and we think that's right: the one-off 45 s is already at the edge of what a first-run user will sit through before deciding the thing is broken. ## Files in, documents out Attach up to 10 files per message (paperclip, drop, or paste — 20 MB each): code and text, CSV, JSON, HTML, Word, Excel, PowerPoint, OpenDocument, RTF, emails, PDFs, and even old `.doc` files read directly. Old `.xls`/`.ppt` are converted through Microsoft Office when it's installed. Pictures get checked, turned upright, shrunk to 1568 px and stripped of metadata — camera, GPS and time are gone before anything is stored or sent; models that can't see images get whatever text Windows OCR finds in them instead. In the other direction, `create_document` lets a StudioForge or cloud model save what it wrote: `.docx`, `.xlsx` and `.pptx` are built from the model's Markdown by ChatForge itself with the Python standard library only, no Office needed. Files land in `Documents\ChatForge`, a taken name becomes `report (2).docx` rather than an overwrite, and the reply shows a card with **Open** and **Show in folder**. The small local model is not offered this tool; asking a goldfish to write your quarterly report is on you. ## What it deliberately is not ChatForge is not an agent, and the boundary is structural rather than a promise in the prompt. Eight of the nine tools are read-only lookups; the ninth can only add a new file to its own folder and never overwrites anything; no tool runs a program, touches a setting, or reads a file you didn't attach. `fetch_url` blocks private and local addresses (DNS rebinding included), the calculator is a whitelisted evaluator with no `eval`, API keys live in the Windows Credential Manager and are registered with the log redactor, and the model server dies with the app, by construction. It keeps one conversation; there is no archive of everything you've ever asked. The worst a confused model can do here is give a wrong answer and leave a new file in a folder — which is exactly as much blast radius as a tray assistant deserves. ## What's new in 0.2 {#whats-new} ChatForge 0.2.0 (18 October 2026) is the release where a Windows 11 user no longer needs a terminal, Python or git. From the download to a first chat is three actions: double-click `Install ChatForge`, click **Set up local AI**, type. - **A double-click installer and uninstaller**, per user, no administrator rights, with shortcuts and an entry in Installed apps (see Install (#install)). - **A one-button starter model** on the first-run card: the runtime and the 1.5B Qwen in one download, checked, with Cancel. - **CPU when there is no NPU.** On a PC without an Intel NPU the local model runs on the CPU, and the card and status line say so. The README's capture shows the CPU path at about 45 tokens per second on one such PC. `local.device` in `config.toml` stays `NPU`, so a later NPU is picked up with no config change. - **Autostart is asked, never silent.** The installer asks once; a source install gets the same question as a card in the popup. Change it later with `chatforge autostart enable` or `disable`. - **New installs default to the Glacier theme.** Existing installs keep theirs. - **An add-on interface.** Python packages can register under the `chatforge.extensions` entry point and get settings sections, turn and event hooks, and `chat.send` / `chat.cancel`. The API is version 1 and may still change while ChatForge is 0.x (docs/extensions.md (https://github.com/LaserLloyd/ChatForge/blob/main/docs/extensions.md)). First start: one card, one button. While the runtime and model download, the same card shows a progress bar and a Cancel button. More captures are in the README. **Upgrading from 0.1.0.** If you installed from a clone, `git pull` then `uv sync` (the lock file changed and Python is now capped below 3.14). Or switch to the installer. On first start your `config.toml` is migrated: your autostart setting is kept and you are not asked again. Your data folder does not change. Two things you may have to do: - **A StudioForge server on another PC needs an API key.** StudioForge 2.0 refuses requests without one from another machine. Create a key on the StudioForge machine (`studioforge keys create`) and paste it into Settings > Providers > StudioForge. A refused request now says exactly that. - **Clone users who ran Python 3.14** need 3.12 or 3.13. **Voice, optional and early.** The repository now carries a push-to-talk add-on, `packages/chatforge-voice` (release 0.1): hold a key, speak, and ChatForge answers out loud, with speech recognition (Parakeet) and the voice (Chatterbox-Turbo) on the CPU and nothing leaving the PC. It is installed separately into ChatForge's environment, its models (about 1.4 GB) are downloaded from a small voice panel it opens in your browser, and hands-free listening is not part of it. We have not tried it on our laptop yet, so treat it as a preview. **Still true:** the starter model has a 4096-token window and sometimes gets sums wrong in its head (say "use the calculator"); the NPU figures come from one Lunar Lake machine and no other chip is verified; Linux and macOS are not supported, and there is no Docker version because a tray app with a hotkey and an NPU has nothing to run in a container. ## Gotchas - **Bare `python` on stock Windows is the Microsoft Store stub.** Use `py -3.12 -m uv run …` throughout, or the launchers in `launchers\`. This bites before anything else gets a chance to. - **The Copilot key cannot be a hotkey.** It's `Win+Shift+F23` and Windows keeps it. PowerToys remap or nothing (above). - **Don't assume a compile means a working model.** Qwen3-4B compiles and emits garbage on the NPU; the catalog badges it *Avoid*, and anything it doesn't know gets *Untested*. - **A driver update or sleep/resume can sour the compile cache.** Settings > Models > **Clear cache**, then Load. A fallback provider keeps you answered meanwhile. - **The small model degrades in long chats** — stops calling tools, loops, answers badly. That's a 1.5B model losing a long context, not a bug to chase: press Clear chat, or switch to a bigger model for the hard question. - **Old `.xls`/`.ppt` attachments need Office installed** (the conversion is hidden, on a temp copy). Without Office, save as `.xlsx`/`.pptx` first. HEIC photos need the optional `pillow-heif`, or export as JPEG. - **Licence note if you redistribute:** the app is MIT, but `pystray` (the tray library) is LGPL-3.0, imported unmodified at runtime. Fine as installed; if you ship a frozen bundle that embeds it, the LGPL relinking obligation is yours. ## Where this leaves us Two days from first commit to a v0.1.0 we actually use says more about reusing proven parts than about speed. The supervisor, the chat UI and the streaming client all came from projects already running in this house. The NPU turned out to be the easy half; the real work was making a 1.5B model useful (tool subset, look-it-up-first, refusal recovery) and refusing scope: no agent powers and no chat archive, and nothing goes to the cloud unless you ask. It's Windows-only, the version number says 0.2.0 and means it, and the test suite is what gives us the nerve to keep moving: CI runs ruff and the Python unit tests on Windows and Ubuntu, and the JS tests on Node 24. ## Versions The 0.2.0 tree is commit `de30788` (2026-10-19); its 183 JS tests pass on Linux (Node v24.18.0). The original 0.1.0 check, from 2026-10-02: ```text $ git log -1 --format='%h %ci' 2edd2d4 2026-10-02 18:04:38 +0900 # ChatForge v0.1.0 $ uv run pytest -q 1520 passed, 6 skipped, 6 deselected in 50.65s # on Linux, Python 3.13.14 $ npm run -s test:js tests 114 / pass 114 / fail 0 # Node v24.18.0 ``` Runtime on the Windows side: OpenVINO Model Server 2026.4.0 (`python_on`), model `OpenVINO/Qwen2.5-1.5B-Instruct-int4-ov`, measured on an Intel Core Ultra 5 226V. The NPU figures above are from the repo's `docs/RUNTIME-NOTES.md`, which also records the failed Qwen3-4B configurations. Related: StudioForge: a GPU-only LLM server · DisPatch: self-hosted AI chat · DeepSeek Harness (dsh) first look · Running DeepSeek locally: a 4-step guide · What our AI agents actually cost