mcpwith.codemode.cc github

EDU AI LAB: MCP codemode — new way how LLM can exec code inside MCP srv (Contrast & Compare 3 old ways)

We all see emerging Agentic Infra as replacement of IaC (Infrastruture as Code) — 4 principles here are universal, I select observablity as area where we work with very large, comples, interconected, timed data structures, ideal use to demonstrate pros and cons of diferent aproaches + plus every AI project need observability so may you will learn something usefull by the way too.

The four harness patterns — Agent Skills + CLI, MCP LLM, MCP codemode, ZTA

HEAD 2 HEAD — 4 principles

Blueprint detailing how an AI agent harness connects to the Arize Phoenix observability platform. It maps the end-to-end execution flow, telemetry tracking spans, and architectural characteristics across four AI Harness patterns:

01

Agent Skills + CLI

Invocation of pre-defined agent skills through a terminal interface and shell binaries.

Pros

  • Rapid Prototyping: Directly leverages standard OS binaries, custom CLI scripts, and existing terminal tools without custom protocol wrappers.
  • Standardized System Telemetry: Phoenix natively captures process exit codes, stdout/stderr streams, and low-level system logs.

Cons

  • Brittle Parsing: Reliant on parsing unstructured text output, which frequently breaks downstream reasoning loops.
  • Expanded Attack Surface: Unrestricted command-line access carries significant command injection and sandbox-escape risks.
02

MCP LLM

Standard Model Context Protocol exposing JSON-RPC tool definitions into the LLM context window.

Pros

  • Native Semantic Awareness: Tools are embedded directly into model context, allowing fluid multi-turn planning and dynamic schema discovery.
  • Fine-Grained Prompt Observability: Phoenix tracks complete prompt/response cycles, token usage, tool invocation latency, and protocol metadata.

Cons

  • Heavy Context Overhead: Loading extensive MCP tool schemas directly into prompts rapidly exhausts available context space.
  • Compounding Costs: High latency and token consumption across multi-step agent reasoning chains.
03

MCP codemode

Sandboxed Python execution on an MCP server returning processed prompt, code, and response data.

Pros

  • Token Efficiency: Processes and filters large datasets in-memory on the server, returning only curated summary data to the agent rather than raw context.
  • Granular Traceability: Phoenix records execution flow, variable states, code performance benchmarks, and runtime exceptions.

Cons

  • Runtime Sandboxing Complexity: Requires strict isolation boundaries (such as a restricted Python subset) to prevent unauthorized execution.
  • Dynamic Debugging Friction: Code execution failures require robust runtime recovery paths to prevent the harness from stalling.
04

ZTA – Zero Token Architecture

Deterministic orchestration using a Python SDK or direct API calls, reporting final status back to the agent.

Pros

  • Maximum Cost & Speed Efficiency: Payload movement occurs entirely outside the LLM context, virtually eliminating token overhead and latency bottlenecks.
  • Deterministic Policy Governance: Strict API schemas provide tight compliance, predictable auditing, and deterministic security controls.

Cons

  • Zero Adaptive Reasoning: Inflexible handling of novel edge cases; changes require programmatic code updates rather than dynamic agent adaptation.
  • Higher Engineering Burden: Demands custom SDK and API pipeline maintenance rather than generic declarative agent prompts.

Part of EDU AI LAB — “Local AI is not CHEAP!”

EDU AI LAB — localai.isnot.cheap github

Hey!!! Local AI is not CHEAP at ALL! — every AI token carries a cost. Metering and billing internal teams by the token is essential: it creates accountability against wasteful compute loops and directly amortizes the infrastructure, pipeline, and staffing investments. This lab contrasts & compares agentic-infra approaches on that shared stack.

Related project

Inspirations

MCP Explained Simply — What It Is and Why It Exists
Making Agents Useful: MCP, Tool Calling, and Sandboxed Execution
Code Mode: Why AI Agents Are Ditching Tool Calls for Code
Monty: the ultrafast Python interpreter by Agents for Agents — Samuel Colvin, Pydantic
Code Mode: Let the Code do the Talking — Sunil Pai, Cloudflare
Stop Burning LLM Tokens on Routine Tasks! Introducing Zero Token Architecture (ZTA)
ZTA: Zero Token Architecture — Kelsey Hightower | PlatformCon 2026

Code Mode design ‘deep dive’

Code Mode: Why AI Agents Are Ditching Tool Calls for Code

Code Mode in Production Harnesses — Pi 1.0, OpenCode 2.0 & Across the Ecosystem

Both Pi 1.0 and OpenCode 2.0 now ship native Code Mode out of the box, alongside OpenAI Codex CLI, Goose, and Claude. Instead of dumping every MCP schema into the prompt and streaming raw intermediate tool payloads through the LLM context window, the agent writes a small program that orchestrates tool calls in parallel, filters data in-memory, and returns only the distilled result.

Harness Native Code Mode? Sandbox Runtime & Language Tool Discovery & Catalog
Pi 1.0 Yes (codemode tool) QuickJS (JS, 256 MB cap) exposure: "codemode", searchTools() (BM25), describeTool(), ALL_TOOLS
OpenCode 2.0 Yes (execute tool) Confined JS Runtime + fetch codemode: true (default for MCP), tools.<ns>.<tool>, search()
OpenAI Codex CLI Yes (functions.exec) codex-code-mode-host (JS, local or remote WS) code_mode_host = true (default), Promise.allSettled() batching
Goose (AAIF) Yes (code_execution) pctx / Deno (TypeScript) 3 meta-tools: list_functions, get_function_details, execute_typescript
Claude Code CLI / API Hybrid (MCPSearch + PTC) Tool Search in CLI; Python container in API PTC defer_loading: true (ENABLE_TOOL_SEARCH=auto:5) + code_execution_20260120
Tau (huggingface/tau) Via Extension Python 3.12+ (tau.register_tool) Minimal 4-tool core (read/write/edit/bash); embed Monty via extension
Pi 1.0

codemode tool (QuickJS Sandbox)

Harness-side QuickJS VM (256 MB cap, top-level await, no filesystem/network/timers) that exposes MCP & built-in tools as JS functions.

How it is enabled & configured

  • Auto-enabled for MCP: Connects automatically when any MCP server in ~/.pi/agent/mcp.json or .pi/mcp.json uses the default "exposure": "codemode", or always-on via "defaultTools": ["+codemode"] in settings.json.
  • Exposure control: Per-server exposure and per-tool toolExposure let you route tools as codemode (script-only, ~40% fewer prompt tokens), deferred (loaded via search), direct, or hidden.

How it is used

  • Tool orchestration: Scripts call await tools.mcp__<server>__<tool>(args) or built-ins like tools.read() / tools.bash(), discovering unlisted schemas on demand with searchTools() (BM25), describeTool(), and ALL_TOOLS.
  • State & auxiliary models: Persist small JSON state across turns via store(k, v) / load(k), emit curated output via return, text(), or image(), and invoke non-LLM models with models.classify() and models.generateImages().
// @options: {"max_output_tokens": 2000, "timeout_ms": 60000}
const hits = await searchTools("phoenix spans");
const [proj, spans] = await Promise.all([
  tools.mcp__phoenix__list_projects({}),
  tools.mcp__phoenix__get_spans({ project: "default", limit: 200 }),
]);
const slow = spans.filter(s => s.latency_ms > 1500);
store("slow_span_ids", slow.map(s => s.id));
return { projects: proj.length, slowCount: slow.length };
OpenCode 2.0

execute tool (Confined Code Mode Runtime)

Confined JS runtime for scripting catalog tools and fetch requests while enforcing per-tool permissions on every nested call.

How it is enabled & configured

  • Default for MCP servers: Every server under mcp.servers in opencode.jsonc defaults to "codemode": true (set "codemode": false to keep a server on the provider's native tool list).
  • Granular permissions: Permission execute (*) gates Code Mode itself, while every nested call inside the script still enforces its own permission rule (e.g. <server>_<tool>).

How it is used

  • Namespaced catalog & search: The agent calls execute({ code }) to run JS against tools in the catalog (or discovered via search()) using tools.<server>.<tool>(input) (or tools.<ns>["tool-name"](input)) plus native fetch.
  • Built-in namespaces & concurrency: Beyond MCP servers, scripts orchestrate tools.opencode.* (sessions, worktrees, model lookup, MCP resources) and tools.browser.* (desktop browser automation), running independent calls concurrently with Promise.allSettled().
const [resA, resB] = await Promise.allSettled([
  tools.phoenix.list_projects({}),
  tools.opencode.list_mcp_resources({ server: "phoenix" }),
]);
const projects = resA.status === "fulfilled" ? resA.value : [];
return {
  activeProjects: projects.map(p => p.name),
  resources: resB.status === "fulfilled" ? resB.value.resources.length : 0,
};
OpenAI Codex CLI

functions.exec (codex-code-mode-host)

Dedicated local or remote WebSocket execution host enabled by default in Codex CLI (v0.146+).

  • Configured via ~/.codex/config.toml: Enabled by default under [features] with code_mode = true, code_mode_host = true, and optional code_mode_only = true (or remotely via --code-mode-host wss://...).
  • How it is used: The agent writes JS in functions.exec to batch nested built-in tools (exec_command, apply_patch, wait) and MCP tools with Promise.allSettled() while preserving host approval policies.
Goose (AAIF)

code_execution extension (pctx / Deno)

Built-in MCP facade powered by Port of Context (pctx), a sandboxed Deno TypeScript runtime.

  • Configured via goose configure: Toggle the built-in code_execution extension (or pass --with-builtin code_execution,developer).
  • How it is used: Hides raw MCP schemas behind 3 meta-tools — list_functions, get_function_details, and execute_typescript — so the agent discovers schemas on demand and chains calls in TypeScript.
Claude Code CLI & API

MCPSearch + Programmatic Tool Calling (PTC)

Two-tier context reduction across the Claude Code CLI and the Claude Developer Platform.

  • Claude Code CLI (MCPSearch): Activates automatically when MCP schemas exceed 5% of context (ENABLE_TOOL_SEARCH=auto:5), deferring schema loading (defer_loading: true) for ~85% fewer prompt tokens.
  • Claude API / SDK (PTC): Tools marked with allowed_callers: ["code_execution_20260120"] become callable Python functions inside a sandboxed container, letting Claude loop and filter 20+ calls in Python before returning final stdout.
Tau (huggingface/tau)

Minimalist Python Harness + Extension API

Hugging Face’s readable Python port of Pi’s architecture (tau_ai → tau_agent → tau_coding).

  • Intentionally minimal core: Ships only 4 built-in tools (read, write, edit, bash — Pattern 01) without a built-in MCP client or JS VM.
  • Python-native Code Mode via Extensions: Because Tau runs on Python 3.12+, you can register a custom AgentTool via tau.register_tool(...) in ~/.tau/extensions/ that embeds Pydantic Monty (or calls a FastMCP Code Mode server) directly in-process.
Pi Agent Just Added CODEMODE… and It’s INSANE!