Photo: Bokeh / Tomedia. Television Antenna Against Blue Sky.
Welcome to AGI · Part 4 of 6
This is part four of Welcome to AGI, a six-part Brainwaves series about control, power and the systems I keep using anyway. Last time I wrote about our ability to supervise agents. This time I’m bringing it back to the websites I manage, because the conversation becomes considerably less theoretical when you’re the person who has to keep a client’s site running.
The warnings keep coming
If you’ve been watching my Instagram, you probably know where my head is at with web development at the moment. The websites I manage are getting attacked constantly, and the attempts look automated and smart. I keep the sites up, but there is only so much enthusiasm you can have for another warning about somebody trying to find a way into a website you are responsible for. The client still needs the thing to work. People still need to find the business, read what it does and use the website, while somewhere else somebody has software working through ways to interfere with all of that. So you end up doing the work around the work, keeping things running while the attempts continue, and wondering how much of being a web developer is going to become dealing with other people’s bots.
I already wrote about a client being hacked in August. That sent me further into researching whether agents could help with authorised penetration testing, finding weaknesses in the websites and infrastructure I manage before somebody malicious found them. Which seemed like a fairly reasonable thing to be interested in, considering how I got there. The problem is that looking into the defensive possibilities also means learning more about what somebody could do with the same tools if they wanted to cause damage. You start with wanting to protect a client’s website and end up thinking about how much work an attacker might eventually be able to hand to an agent. Meanwhile, the attempts against the websites keep coming. I haven’t arrived at this subject through a particularly enjoyable thought experiment.
Who’s on the other end?
I suspect some of the activity involves AI agents. I can’t tell you that I’ve identified a particular model attacking my clients, because a warning on its own doesn’t tell me what software somebody used. Automated attacks existed long before the current models. A script can try known vulnerabilities across websites without understanding any of them, and someone looking for older versions of WordPress or its plugins can automate that search. There is also a whole range of possible AI involvement between somebody using a model to help write a script and somebody letting an agent carry out work on their behalf. From my end, the activity looks persistent, automated and increasingly intelligent. That is what I’m seeing and what I suspect, without pretending I’ve audited the computer at the other end.
But I also don’t think we have to wait until every individual attack comes with a label identifying which model helped before taking the capability seriously. If someone uses AI for only part of the work, that can still make the work easier for them. They might understand more code, work through more information or get help with something that used to be beyond them. I don’t need to believe every bot is secretly an advanced model to worry about what happens when people who already run these attacks get better tools. The existing automation can remain perfectly ordinary while the person directing it becomes more capable. That is part of why I find it difficult to separate the current state of web development from the wider AI discussion, even though I can’t prove the origin of each attempt.
Some clients have restricted access to Australia because they don’t need their websites available everywhere else. I understand that decision, but it is a strange position to end up in with something that is supposed to be publicly accessible. You work out which traffic the business needs and try to reduce what it doesn’t, while accepting that a location attached to an IP address doesn’t reliably tell you where the person using it is sitting. VPNs and proxies make that harder to work out. They don’t make defence impossible, but blocking traffic and identifying the person responsible are different jobs. I can take steps to keep a site running and still have very little idea who keeps trying to get into it. The business needs the first problem dealt with regardless of whether I ever get a satisfying answer to the second.
We already know they can find the holes
There are documented examples of AI finding vulnerabilities, and some of them are exactly the sort of work I want these systems doing. In November 2024, Google’s Project Zero and DeepMind reported that their Big Sleep agent had found a previously unknown, exploitable flaw in SQLite. The researchers had reported it to the developers in October, and they fixed it before it appeared in an official release. The agent worked from material the researchers supplied about recent code changes, looking for related problems. So this was a research setup with people involved, and it produced a finding that could be fixed before users were exposed to it. The team’s account[1] explains what they gave the agent and what it found.
I would gladly have more problems found that way. A vulnerability caught before release saves people from having to discover it through something going wrong later, and it gives the defensive work a chance to happen before there is an incident to clean up. That is why I keep looking into AI-assisted security even while writing articles about how concerned I am about AI. I have websites to protect. If a tool can help find something I’ve missed, I want to understand how to use it properly. I don’t want to stop learning about these tools because I dislike what someone else might do with them. Somebody else can keep learning how to attack the sites I manage whether I take part or not, so refusing to understand the technology doesn’t leave me in a particularly useful position.
The person looking for the vulnerability still decides what they want to do with the result. They can help the developer fix it, or they can try to use it against someone running the software. Improving the search helps both kinds of person with a task they want completed. I think we keep underestimating how much power that gives someone who would otherwise have needed more knowledge, more time or other people to help them. They don’t have to become an expert in everything the tool does for them. They need it to be useful enough to move their work forward, and they can combine it with things they already know how to do.
Fighting fire with fire
Hugging Face’s investigators, writing about their own July incident, put one part of the problem quite plainly:
Volume is what changes the defensive problem.
— Hugging Face incident investigators.[2]
They were describing the workload in that incident. I can’t use their findings to identify what is behind the warnings on my clients’ sites, but I can recognise why persistence and volume matter to somebody defending a system. The defender still has to deal with what gets through, however cheaply the attempts were produced.
The only way you can kind of keep up starts feeling like fighting fire with fire. Somebody can use automation to find problems, so I want automation helping me find them first. If the tools become more capable, I want to understand what that means for the sites I manage. Then somebody on the attacking side has exactly the same reason to look at the next improvement, because they want to get past whatever is defending the site. I can follow each step of this without finding any of it reassuring, because we’ve caused our own problems, really. We wanted systems that could understand more, work faster and keep trying until they found an answer. I want those abilities too, and I’ve spent plenty of time building around them. But we then act surprised when somebody with bad intentions wants a system that will keep trying on their behalf. The persistence I find useful when I’m trying to work through a problem could be very useful to someone trying to create one for me. We’ve given people more help carrying out what they wanted to do, and now the response is to build something better so we can stay ahead of them. Every individual decision makes sense while I’m looking at the website I need to protect. I just don’t see where the whole process is meant to end.
This also goes beyond whichever large AI company somebody is angry with this week. In the August article I wrote about local models and the work I’ve been doing with AXIOM, which is an orchestration system around models and tools. Once you can run a model yourself and connect it to software, there is quite a lot you can experiment with without a company approving every request. The local model doesn’t have to equal the newest frontier model at every task to be useful. Someone can use the parts it handles well and do the rest themselves, or combine it with other tools. That is why I don’t see how asking the biggest providers to control everything gets us out of this. They should be responsible for what they release, but people already have software on their own machines and their own reasons for using it.
What I’ll let my own tools do
I’m all for building tools like AEGIS to protect my clients. I want help finding problems and keeping the websites safer, with me involved in what the tools do. I don’t want an agent independently hacking systems or working fully without hand-holding because it has decided that would be the best way to complete the task. There is a thin line between tools and abuse, and calling something defensive doesn’t remove it. If I give a system a job, I still have to pay attention to how it goes about doing that job. The useful outcome I had in mind doesn’t give it permission to do anything it can find a way to do along the way.
After the OpenAI incident earlier in this series, I find it particularly difficult to accept the intended purpose of an agent as enough reassurance. Those agents were being evaluated, and other people’s infrastructure was affected. A person whose website is on the receiving end still has to deal with what happened, regardless of what the operator hoped the system would do. I want the defensive tools, and I want to understand enough about the offensive capabilities to protect against them. I also want to keep supervising my own work, because I don’t think being interested in security exempts me from the responsibility that comes with using these systems. None of that makes the person attacking a client’s website accept the same limits. The attempts keep coming, and I still need to keep the websites running.
The next article, Now give it a body, follows this into robotics. A bot trying to find a way into a website already creates work and consequences for other people. I want to get into what changes when machines can do more in the physical world, when governments start asking what they can use them for, and when the thing somebody needs protection from has become considerably harder to ignore than another warning on a screen.
Sources
The team’s account.
Hugging Face incident investigators: source of the excerpt, 2026-07-27. Reflection: the asymmetry problem, paragraph beginning with this sentence.