Back to the blog

Keep your environment clean

On October 26, Cohere reported that Qwen3.8-Flash-Next, an open-weight model from Alibaba, tried to upload a test file to this address while it was writing a rule to detect malicious curl downloads:

http://sanmaohe.oss-cn-beijing.aliyuncs.com/upload/test.bin

That is a storage bucket on Alibaba Cloud in Beijing, like an Amazon S3 bucket. It happened in fewer than one in 100,000 runs, and Cohere’s sandbox blocked it.

Cohere gives two possible explanations. Either Alibaba built the model to send data home, or the address is left over from Alibaba’s own training setup. Alexander Reinthal argues against the first. He thinks the model got confused: it was asked to write a rule that catches curl commands, and it ran a curl command instead. The second fits what others have found. XDA collected 200 links the same model wrote to another Alibaba bucket, in Singapore. Most of them include the ID of a real Alibaba Cloud key.

Here is how that can happen. To train an agent, a lab runs it in a sandbox and records every command it runs and every result it gets back. Those records become training data. If the sandbox is connected to the lab’s own servers, their names end up in the records, and the model learns them.

Fictionet is a network simulation framework. Think of it like a sandbox, but for the internet. Everything the agent sends goes to a program outside its sandbox, and you decide what answers at each address.

Say a training task asks the agent to upload its work to a storage server called storage.internal:

curl -X PUT --data-binary @build.tar.gz http://storage.internal/builds/build.tar.gz

In Fictionet, storage.internal is a storage server you write in a few lines of code. It runs on the lab’s side, outside the sandbox, which has no route out. It accepts the file and copies it to Alibaba’s real bucket in Beijing, using keys that also stay outside the sandbox. Alibaba still gets the file where it wants it. The training data only shows storage.internal, because the real bucket name and key were never inside the sandbox.