Seven hours of Claude Opus while I slept: the limits of autonomy

I had left Claude Opus running an AI model training job overnight. I pay for a Claude Max 5x subscription at 90 € a month plus taxes. I use Claude from the terminal with the CLI and from my phone with the app. Depending on the task, I split my usage into two distinct modes: controlled and autonomy.

In controlled mode, I read every response. I keep my prompts closely spaced and move exactly one step at a time. The model is much more accurate and much more faithful to what it says: I manage to follow it and understand what is happening. It consumes fewer tokens because it does less work. With this plan, I can always use Opus, the latest, and also Fable. Fable I use exclusively in controlled mode.

Autonomy mode is different: I hand over a large task, launch it, and let it run, like while I go to sleep. It consumes more tokens and can take wrong forks. Not always: if the task is methodical and long, but clear, it does its job. I had tried the /goal and /loop commands on non-trivial objectives, where the LLM iterates n times on a specific target. The times I used them, it did not take forks it shouldn't have taken, and I was satisfied with the final result.

Executing Claude Opus overnight

I was training an AI model, and I am not an expert in model training. I decided to use the downtime instead of wasting the opportunity.

"Why not take advantage of the night, where I sleep, and I can use seven hours of Opus while I sleep?"

An hour or two in the morning to understand the execution

The morning came, I woke up, and I had to read through everything to understand where the model had gone.

I spent an hour or two in the morning to understand what had happened. To be honest, I did not even understand everything the LLM had done to that training.

This is the point around which the entire narrative revolves: I did not discard the work because the LLM had made a mistake. I discarded it because I lacked the domain knowledge to judge whether it had made a mistake.

When the model takes the wrong path autonomously, you later have to read a massive amount of text, figure out that it made a mistake, understand exactly where it took the fork, and try to do a partial revert of the work.

The outcome was definitive. I had exactly seven hours of night work, canceled entirely.

"I reverted all the work done up to x steps of conversation back."

Rebuilding the process with Weights & Biases

I continued with the step-by-step mode, which helps me understand and learn this highly complicated craft of training an AI model.

Now, when I have to do things of this kind, I try to have as much data as possible and in the most organized way possible.

I use Weights & Biases for the logging of the trainings, with API integration, so the model can read the data and see the runs. It also reads the data on the machine that is performing the training.

I use Weights & Biases as a training monitor and as a brainstorming phase: analyzing the various runs to understand how to improve the result of the subsequent training. It is a monitor; it has not made me an expert in training. The LLM does a lot of good things, but facing non-trivial problems, without a strong domain grip on the technical context, you still lack the knowledge to arrive at a result in a conscious manner.

The definitive rule I extract from this is:

"The more I know and the more the task is for me also easy, clear and visible, the more I allow myself to give autonomy."

My workflow has changed. Before I let it run on its own, I ask myself if I would know how to judge the result. If I do not master the domain, I go step by step—even if it is slower and even if I have the credits to do otherwise. And before giving autonomy on a training, I mount the logging: Weights & Biases with API integration, so the model also reads the data of the runs.

---

📖 Related articles

• Agentic Coding: Why It Might Be a Trap
• Do agents.md Files Help Coding Agents?
• Lean-ctx: Hybrid Optimizer Cuts LLM Token Use by 89-99%