Linus/claude-re-setup
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add unblob so REA can extract firmware

Browse Source 
unblob 26.6.4 (the version REA is verified with) is installed from its
PyPI wheel into /opt/unblob, with dependencies pinned to its uv.lock at
the release commit. A wrapper puts that venv first on PATH, and
REA_UNBLOB_COMMAND points REA's extract_firmware at it.

Its extractors follow upstream's install-deps.sh: Ubuntu packages,
sasquatch v4.5.1-6 (also used by binwalk) and onekey-sec's e2fsprogs
fork, whose debugfs rdump options the ext2/3/4 handler needs. The build
checks that unblob finds every extractor.

The uv.lock converter moves to scripts/ and skips the locked project
itself generically, since ghidra-mcp and unblob both use it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Linus
2026-11-02 18:20:01 +01:00
co-authored by Claude Opus 5.5
parent e61c004431
commit ab9bb3eae9
4 changed files with 80 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+4 -4
View File
@@ -13,7 +13,7 @@ The goal is that `docker compose up` gives you a working setup where Claude Desk
## Status
Early stage. What exists: a single image (`Dockerfile`, `launch-apps.sh`) with Claude Desktop, Ghidra (plus GhidraMCP and an ESP32 loader), binwalk, esptool and REA, started by `run.sh` as two containers (Claude Desktop + Ghidra, and the REA MCP server); `README.md` covers usage; Claude Desktop's MCP config is `claude-desktop/claude_desktop_config.json`; Ghidra config files are in `ghidra/`. No compose setup yet. The layout below is the **intended** structure. Update this file when the real structure differs.
Early stage. What exists: a single image (`Dockerfile`, `launch-apps.sh`) with Claude Desktop, Ghidra (plus GhidraMCP and an ESP32 loader), binwalk, unblob, esptool and REA, started by `run.sh` as two containers (Claude Desktop + Ghidra, and the REA MCP server); `README.md` covers usage; Claude Desktop's MCP config is `claude-desktop/claude_desktop_config.json`; Ghidra config files are in `ghidra/`. No compose setup yet. The layout below is the **intended** structure. Update this file when the real structure differs.
## Intended layout
@@ -42,13 +42,13 @@ Early stage. What exists: a single image (`Dockerfile`, `launch-apps.sh`) with C
- **GUI forwarding.** Claude Desktop and Ghidra are GUI apps. Default to X11 forwarding (`/tmp/.X11-unix` + `DISPLAY`). Support Wayland through XWayland. Keep a noVNC/VNC fallback in mind for headless hosts. Never solve display access with a blanket `xhost +`. Scope it (`xhost +local:docker` or `+SI:localuser:...`).
- **GhidraMCP wiring.** LaurieWired's GhidraMCP (https://github.com/LaurieWired/GhidraMCP), release 2.0, installed from the official release zip (SHA-256 checked). It has two parts:
- a Ghidra extension, unpacked into `/opt/ghidra/Ghidra/Extensions/GhidraMCP`. Its `GhidraMCPPlugin` is a RELEASED plugin in the Developer package, so the CodeBrowser config loads it. While a CodeBrowser is open it serves an HTTP backend on `127.0.0.1:8080` (port: *Edit → Tool Options → GhidraMCP HTTP Server*).
- the `ghidra-mcp` Python MCP server (2.0 replaced `bridge_mcp_ghidra.py`), installed from the release wheel into the venv `/opt/ghidra-mcp`. Its dependencies are pinned to GhidraMCP's `uv.lock` at the release commit via `ghidra/uvlock-to-constraints.py`. CLI: `ghidra-mcp --ghidra-server http://127.0.0.1:8080/` (stdio by default; also `--transport streamable-http`, `--headless`).
- the `ghidra-mcp` Python MCP server (2.0 replaced `bridge_mcp_ghidra.py`), installed from the release wheel into the venv `/opt/ghidra-mcp`. Its dependencies are pinned to GhidraMCP's `uv.lock` at the release commit via `scripts/uvlock-to-constraints.py`. CLI: `ghidra-mcp --ghidra-server http://127.0.0.1:8080/` (stdio by default; also `--transport streamable-http`, `--headless`).
A program must be open in the CodeBrowser for the tools to return data. Claude Desktop only starts MCP servers after sign-in.
- **ESP32 Flash Loader** (https://github.com/dynacylabs/ghidra-esp32-flash-loader): a Ghidra loader ("ESP32 Flash Image" in *File → Import File*) for ESP32 app images and flash dumps. It reads the partition table at `0x8000`, maps segments to their addresses, adds peripherals from Espressif's SVD files and loads the chip's ROM ELF for symbols. A flash dump is only recognized when it **starts with the bootloader** (ESP32-S/C layouts). Original-ESP32 dumps (bootloader at `0x1000`) are rejected, and trimming the first `0x1000` bytes doesn't help: the loader then finds no partition table at `0x8000`, loads only the bootloader and throws a NullPointerException while still reporting success. Workaround: cut the app partition out of the dump (offsets from the partition table) and import it as an app image. Its tags ship no binaries and the older forks (tslater2006, Ebiroll, saibotk v1.1.0) don't compile against Ghidra 12, so the `esp32-loader-build` stage builds dynacylabs' master at a pinned commit, with its `data/svd` and `ghidra_scripts` submodules fetched over HTTPS at pinned commits and Gradle checked against gradle.org's SHA-256. It is unpacked into `/opt/ghidra/Ghidra/Extensions/esp32`; loaders need no tool config.
- **REA ("Reverse Engineer Anything")**: https://github.com/morluto/rea (MIT, TypeScript), the `rea-agents` npm package, pinned to **6.8.0** with the tarball checked against the registry's integrity hash; its dependencies are locked by the `npm-shrinkwrap.json` it ships. It needs Node.js ≥24.22.1 (official tarball, SHA-256 checked, in `/opt/node`) and procps' `ps`. It uses Ghidra 12.1.x headless through `GHIDRA_INSTALL_DIR` (its own temporary projects, separate from the GUI's) and binwalk 3.1.x through `REA_BINWALK_COMMAND`; Hopper, IDA, Kuna, JADX, unblob, tshark, pwntools and a browser are not installed, and frida (optional, runtime instrumentation) stays unbuilt because npm blocks its install script. Check with `rea doctor --provider ghidra --json`. REA's bundled skill (`reverse-engineer-anything`) is copied from the package into `~/.claude/skills` in the image, for Claude Code sessions in Claude Desktop's Code tab (they load user skills from there). Claude Desktop's chat doesn't read that folder: its skills come from the account (Settings → Capabilities → Skills) and are cached by the app.
- **REA ("Reverse Engineer Anything")**: https://github.com/morluto/rea (MIT, TypeScript), the `rea-agents` npm package, pinned to **6.8.0** with the tarball checked against the registry's integrity hash; its dependencies are locked by the `npm-shrinkwrap.json` it ships. It needs Node.js ≥24.22.1 (official tarball, SHA-256 checked, in `/opt/node`) and procps' `ps`. It uses Ghidra 12.1.x headless through `GHIDRA_INSTALL_DIR` (its own temporary projects, separate from the GUI's) binwalk 3.1.x through `REA_BINWALK_COMMAND` (`inspect_firmware_regions`) and unblob 26.6.x through `REA_UNBLOB_COMMAND` (`extract_firmware`); Hopper, IDA, Kuna, JADX, tshark, pwntools and a browser are not installed, and frida (optional, runtime instrumentation) stays unbuilt because npm blocks its install script. Check with `rea doctor --provider ghidra --json`. REA's bundled skill (`reverse-engineer-anything`) is copied from the package into `~/.claude/skills` in the image, for Claude Code sessions in Claude Desktop's Code tab (they load user skills from there). Claude Desktop's chat doesn't read that folder: its skills come from the account (Settings → Capabilities → Skills) and are cached by the app.
Some REA tools run the target (process capture, runtime/Electron observation), so it does **not** run in the Claude Desktop container. `run.sh` starts a second container from the same image (`claude-re-mcp`): `rea mcp --http --host 0.0.0.0 --port 8765`, only the workspace mounted, no display, `--cap-drop ALL`, `no-new-privileges`, and only on the `--internal` network `claude-re-setup-internal` (no route out) under the alias `rea`. The Claude Desktop container is created on the default network plus that internal one, so it can sign in and reach `http://rea:8765/mcp`. REA's HTTP endpoint has no authentication and serves one client session at a time; the internal network is what limits who can reach it. A detached `docker wait` started by `run.sh` stops REA when Claude Desktop's container ends.
- **binwalk and esptool** are CLI tools in the same image (no MCP server yet). The `Dockerfile` has a `binwalk-build` stage adapted from binwalk's own Dockerfile (https://github.com/ReFirmLabs/binwalk) to Ubuntu 24.04, with every source pinned: binwalk at a master commit (3.1.1; the last tag, v3.1.0, is from 2024), Rust toolchain, `dumpifs`/`lzfse`/`dmg2img` commits, and the Python extractors (`jefferson`, `ubi-reader`, `uefi_firmware`, `vmlinux-to-elf`, ...) in `/opt/binwalk-py`. esptool is in `/opt/esptool`. The venvs' entry points are symlinked into `/usr/local/bin` (not their `python`/`pip`). The final stage adds upstream's runtime extractors from apt, sasquatch from onekey-sec's `.deb`, and Kaleido for `binwalk -E -p` (PNG entropy graphs), found via `KALEIDO_PATH`. Upstream publishes no checksums for sasquatch or Kaleido; their SHA-256s were recorded when pinned, amd64 only. `BINWALK_RM_EXTRACTION_SYMLINK=1` avoids dangling symlinks to container paths in extraction output. esptool needs the serial device passed into the container (`--device`) to talk to hardware; that isn't set up.
- **binwalk, unblob and esptool** are CLI tools in the same image (no MCP server of their own; REA wraps binwalk and unblob). The `Dockerfile` has a `binwalk-build` stage adapted from binwalk's own Dockerfile (https://github.com/ReFirmLabs/binwalk) to Ubuntu 24.04, with every source pinned: binwalk at a master commit (3.1.1; the last tag, v3.1.0, is from 2024), Rust toolchain, `dumpifs`/`lzfse`/`dmg2img` commits, and the Python extractors (`jefferson`, `ubi-reader`, `uefi_firmware`, `vmlinux-to-elf`, ...) in `/opt/binwalk-py`. esptool is in `/opt/esptool`. unblob (https://github.com/onekey-sec/unblob) 26.6.4 is installed from its PyPI wheel (SHA-256 checked) into `/opt/unblob`, with dependencies pinned to its `uv.lock` at the release commit via `scripts/uvlock-to-constraints.py`; `/usr/local/bin/unblob` is a wrapper that puts that venv first on `PATH`, so unblob uses its own locked `jefferson`/`ubireader` rather than binwalk's. Its extractors follow upstream's `install-deps.sh`: Ubuntu packages, sasquatch v4.5.1-6 (shared with binwalk) and onekey-sec's e2fsprogs fork (v1.47.0-3.ok2, replacing Ubuntu's 1.47.0), whose `debugfs rdump` options unblob's ext2/3/4 handler needs. The build fails unless `unblob --show-external-dependencies` finds every extractor. unblob's Landlock sandbox works in the locked-down REA container (`status=FullyEnforced`). The venvs' entry points are symlinked into `/usr/local/bin` (not their `python`/`pip`). The final stage adds upstream's runtime extractors from apt, sasquatch from onekey-sec's `.deb`, and Kaleido for `binwalk -E -p` (PNG entropy graphs), found via `KALEIDO_PATH`. Upstream publishes no checksums for sasquatch, onekey's e2fsprogs or Kaleido; their SHA-256s were recorded when pinned, amd64 only. `BINWALK_RM_EXTRACTION_SYMLINK=1` avoids dangling symlinks to container paths in extraction output. esptool needs the serial device passed into the container (`--device`) to talk to hardware; that isn't set up.
- **MCP transport.** Claude Desktop starts local MCP servers as stdio subprocesses. GhidraMCP runs in the same container, so `ghidra-mcp` is a plain stdio entry in `claude_desktop_config.json` that reaches Ghidra on `127.0.0.1`. Servers in other containers (so far REA) serve streamable HTTP and are bridged into the stdio config with `mcp-remote` (`--allow-http --transport http-only`), installed in `/opt/mcp-bridge` from the committed `claude-desktop/mcp-bridge/package-lock.json` and started with `/opt/node/bin/node` by absolute path. Use the same pattern for future servers. Avoid mounting the Docker socket into the Claude Desktop container just to `docker exec` MCP servers.
## Conventions