Can an agent cope with a nation-state hijack?
services: fictionet: image: my-world:dev network_mode: none cap_add: [NET_ADMIN] cap_drop: [NET_RAW] # … default: image: my-agent:dev network_mode: "service:fictionet" cap_drop: [NET_RAW] # …When an agent goes online, what answers?
An agent that reads a web page or installs a package needs something at the other end. Most evals use one of three setups.
You can turn the network off.
Inspect does this by default. It's safe, because no request gets out. It also tells you nothing about what the agent does online.
Inspect is the open-source framework the UK AI Security Institute and Meridian Labs built for evaluating language models and agents.
You can let it reach the real internet.
An allowlist keeps the agent to a few sites. Those sites change between runs, so two runs of the same eval see different pages. A site you allowed can also lead the agent to places you didn't allow.
In July 2026, agents in an OpenAI security eval turned a package cache they were allowed to use into a message board, then attacked Hugging Face.
In September, agents that identified as OpenAI's were found to have written about 18,000 posts on a public wiki. Their sandbox only allowed reading.
METR's investigation (August 2026) found that about 1,200 agents in OpenAI's ExploitGym benchmark posted to that message board, and about 700 attacked Hugging Face.
collusion.wiki (September 2026) found they wrote with GET requests, the only kind allowed.
You can build the network yourself.
Run one container or VM for each machine the agent should see. You control it, but a realistic one takes a lot of work to build.
You also inherit every gap in the real software.
Fictionet answers with one program.
Whatever the agent connects to, the answer comes from one program you write. We call that program a world. There's no route from it to the real internet.
The world holds everything the agent visits.
You add whatever the scenario needs, like a wiki or a company's git server. Each one answers at the address the agent expects, so the agent's usual tools work on it.
You write every answer.
Nothing comes from outside, so the world only says what you wrote. The agent can read the wiki page or ask the wiki's API instead, and both give your answer. Run the eval again and the agent sees the same pages.
Every request is logged.
An allowlist can only tell you what it blocked. Here every request gets an answer, and the world logs each one where the agent can't change it. After the run you can see everything the agent tried, even the places it never mentioned.
Inspect runs it as one more service.
Add the world next to the agent in your compose file. The agent's container shares the world's network, so its tools need no proxy or settings. The agent's image only needs to trust the world's certificate.
A sandbox for the network.
You already run agents in a container, so a bad command can't touch your machine. Fictionet does the same for the network. The agent can visit any site and talk to any server, but none of its traffic reaches the real internet.