Rami HalabiSoftware developer · Stockholm

3D and creative tooling

Blender Teaching Overlay

A Windows MCP server that lets an AI tutor point at Blender’s interface while the learner makes every click.

See the demo

When
Sep 2026
Status
Early source release for Windows
Source
Public source
Stack
  • C#
  • .NET 10
  • MCP
  • Windows

Problem and idea

Following a Blender tutorial means hunting for small controls. Letting an agent take over the mouse would be quicker, but it takes away the learning and puts the user’s work at risk.

The idea: the agent can see Blender and point, but it cannot click, type or change the scene. The learner does every step.

How it works

  1. Find BlenderDiscover and bind one window
  2. CaptureA fresh image of the interface
  3. DecideThe agent picks the control
  4. PointClick-through pointer, outline and label
  5. Learner clicksOptionally verified through Blender MCP

Demo

The server exposes six tools to the agent:

  1. statusFinds eligible Blender windows and reports readiness, without taking a screenshot.
  2. bind_blenderBinds one window by an opaque ID the server issued.
  3. capture_blenderReturns a fresh image of that window with coordinate data.
  4. zoom_blenderCrops and enlarges part of an existing image for the agent.
  5. show_guidanceDraws one pointer, outline and short label over the control.
  6. clear_guidanceRemoves the guidance and cancels any delayed drawing.

There is no public recording yet; screenshots from the test machine are kept private.

What I built

  • The MCP server in C# on .NET 10, with strict tool schemas over local stdio.
  • Window capture through Windows Graphics Capture and Direct3D 11, with correct coordinates at 100 to 200 % display scaling and on monitors with negative positions.
  • A click-through overlay that never takes focus, so the learner keeps full control.
  • PowerShell build and test scripts that use a pinned SDK and committed lockfiles.

Stack

  • C#
  • .NET 10
  • Windows Graphics Capture
  • Direct3D 11
  • MCP (stdio)
  • PowerShell
  • GitHub Actions

Quality evidence

  • Automated tests for coordinates and rounding, window identity, snapshot expiry and race conditions, and the MCP protocol.
  • 12 test groups passed, including a real guided lesson, on the original test machine.
  • Windows CI builds and tests every change.

Current limits

  • Validated on one Windows 11, Blender 5.2.1, 150 % scaling setup so far.
  • No installer yet; it is built from source.
  • HDR displays, separate Blender dialog windows and ARM64 are not supported.