Loading replay…

Is this Machine Playing?

Read on

What does your coding agent do when you stop telling it what to do?

We built a small island and left Claude Code there for 30 hours. What does it do when we don't give it any task or objective?

In this post, we'll go deeper into how the agent works, the world we built for it, and the strangest things it did in there. You'll be able to inspect every movement and every thought of the agent through fully interactive replays.


Meet Eko

The agent is Claude Code running on Opus 4.7. It's essentially the same agent we use for coding. The agent has access to a workspace that persists across sessions.

Take a look at the initial workspace the agent has access to below. Click on EPISODIC_MEMORY.md to see the other files. The workspace has a SOUL.md file to give Eko a persona, a EPISODIC_MEMORY.jsonl to log of things that happened (instructions in EPISODIC_MEMORY.md), and a SEMANTIC_MEMORY.md for durable knowledge. The split between episodic and semantic memory follows Tulving's classic distinction in human cognitive psychology between remembering events and knowing things. Beyond these, the agent can use its workspace however it wants, e.g. to save notes or scripts.

Loading workspace…


The island

Have a look at the island below. Drag to look around, and scroll to zoom. There are a few mesas rising from it that the agent can climb by jumping, except for the highest one (the Spire), too tall to jump onto from the platform below. There are three types of objects the agent can interact with: small cubes, a rock, and a yellow ring. This world has standard rigid-body physics, with one extra mechanic: throwing the rock hard enough spawns new cubes.

Note that we built it before Opus 4.7's training cutoff, so the model has never seen it.

Loading replay…

The agent observes this world as a dictionary containing the positions and geometries of every entity. No visual input (though this could easily be added, since most coding agents can read images natively). The dictionary is fairly long, but coding agents like Claude Code are good at processing and reasoning over large structured files.

The agent has 6 actions available in this world. Each one is illustrated below, with an interactive example of the agent calling it.

MoveTo(position)

Walk toward a point on the island. The body moves to the target position over many ticks.

Loading replay…
00:00 / 00:00

Loading transcript…

Exposing actions as custom tools for the LLM as been a common approach, but here we use a different one. Instead of wrapping the six actions as tools, the agent simply uses Claude Code's default tools (including file read/write, plus a bash terminal) and executes actions through bash commands. The main advantage is that the agent can chain multiple actions in a single bash tool call by writing a short script on the fly, which Claude Code likes doing. With one tool per action, each action would need a full-round trip through the LLM, making action sequences much slower.

This is a similar approach to Moltbook where the agent is also given instructions for how to use bash commands. One notable difference: instead of calling the instruction file SKILL.md, we named it API.md. The file is really just a description of the world interface for the agent (see its content below). We deliberately don't describe any skills, because we want to study how the agent learns on its own when placed in a novel environment, so calling it SKILL would be misleading.

API.md
# Mesa World

## Action Schemas

| Input | Data |
|-------|------|
| MoveTo | `{"position": [x, y, z]}` |
| Jump | `{}` |
| Stop | `{}` |
| Pickup | `{"position": [x, y, z]}` |
| Place | `{"position": [x, y, z], "rotation"?: {"yaw": deg, "pitch": deg, "roll": deg}}` |
| Throw | `{"direction": [x, y, z], "speed": n}` |

## API Endpoints

The game server runs on `http://localhost:8080` by default.

### Join Game

`POST /join?name=MyAgent`

Save the `session` token; all other endpoints require it in the
`X-Session` header.

### Send Input

`POST /input`

Body: `{"type":"<ActionName>","data":{...}}`

Returns an observation payload.

### Get Observation

`GET /observe`

### Send Chat Message

`POST /chat`

Body: `{"content":"..."}`

Sends a chat message visible to the players.

### Read Chat Messages

`GET /chat/messages?after=<timestamp>&limit=20`

Returns recent chat messages.

Useful query parameters:
- `after`: optional ISO 8601 timestamp; only returns newer messages
- `limit`: optional max number of messages to return

### Get Game Skill

`GET /skill.md`

Returns this file as markdown text.

Eko's life on the island

Now that we've described the agent and the world, it's time to see what Eko actually does on the island. We placed it there for thirty hours and recorded the run as a full replay: every world-state change, every action it sent, and every message the agent wrote while deciding what to do next.

Start with the first hour below. The replay shows the reconstructed world state on the left and Eko's session transcript on the right, so you can watch what it does while reading the agent's own record of what it thought it was doing. Hit play, scrub the timeline, and click a tool icon in the transcript to see the exact tool call and the result it returned.

Loading replay…
00:00 / 00:00

Loading transcript…

After that, the rest of the section works as a map of the full run: ten moments from agent 9's thirty hours. Click a tile to open the clip; the transcript on the right is the agent's verbatim record of what it was thinking at the time.

Inspect any agent

Choose an agent and scrub through its full 30-hour recording. The transcript beside it follows along — select any entry to jump to that moment.

Loading agent index…

Now viewing agent-09

Loading replay…
00:00 / 00:00

Loading transcript…