Photo by Tomas Heligr-Pyke · Bokeh
I was having a conversation with a friend about the government’s latest internet legislation, and we ended up at a fairly strange place. The people responsible for policing online safety may soon have explicit powers to operate fake accounts, with the Commissioner also able to generate material using AI.
My immediate reaction was: hang on, what exactly are we doing here?
Then we went through the usual argument. Of course a regulator needs to see how a platform behaves. Of course a company knowing it is being inspected might change what the inspector sees. And yes, there is a difference between investigating a service and secretly trying to manipulate everyone on it.
I can follow that reasoning. I still don’t particularly like where it leaves us.
At least that answers last year’s question
Last year, in You Must Be This Old To Scroll, I wrote about Australia’s attempt to legislate its way through the problems of children using the internet. One of the questions that kept bothering me was how on earth we were going to police it. Announcing that a company must do something is the easy bit. Working out whether it actually does it is considerably harder.
The new proposal is broader than the under-16 social media restrictions. The government’s Digital Duty of Care announcement describes obligations for online services to prevent foreseeable harm, along with more choice over how people’s feeds work. The exposure draft was released on 8 September. This is proposed legislation, not a set of powers already brought into force by that document.
Giving adults more control over their feeds sounds useful to me. I also understand wanting platforms to deal with dangerous features before somebody has to lodge a complaint about the damage. I build online systems. Expecting the people who operate them to take responsibility isn’t a ridiculous starting point.
It is the way we grant and control the enforcement powers that bothers me.
Yes, AI-generated material is actually in the draft
The relevant provisions are proposed sections 205G–205L. They call these accounts “sock puppet identities”: false identities used on online services. The Commissioner can use them for statutory functions; approved university researchers get a separate route for qualifying, ethics-approved research. These provisions don’t ban anonymous accounts generally.
The permitted activities include observing services, recording material and testing features. Engagement with other users is limited to what is necessary to stop the account being closed. Section 205K(2) specifically allows the Commissioner to generate material, including through AI, subject to the restriction that producing, distributing or possessing it must not be a criminal offence.
The identities can be used despite contrary laws, contracts or platform policies. Section 205L provides civil immunity for good-faith exercise, or purported exercise, of the specified powers. That is a limited protection, not permission for arbitrary criminal conduct. The activity lists also use “without limitation”, which deserves scrutiny alongside those restrictions.
So the reassurance that they won’t be generating anything themselves doesn’t really settle this for me. The text expressly contemplates it. That doesn’t prove some sinister operation is planned. It does make the scope a perfectly reasonable thing to question.
During the conversation we joked that they were becoming NetWatch, Cyberpunk’s internet police. Apparently my science-fiction references are getting more use out of Australian legislation than I expected.
Who gets the account after the next election?
I don’t want this argument to depend on whether I happen to trust the current Commissioner. An office can outlast the person holding it. Governments change, priorities change, and the people using a power years later may have a very different idea of what counts as a worthwhile investigation.
You only need one person willing to push things further than intended for a supposedly sensible arrangement to become a problem. That possibility isn’t proof of abuse. It is a reason to ask about the controls before we settle for somebody’s good intentions.
I want to know who approves an operation, what gets recorded, who independently checks it and how somebody affected can challenge it. I can understand keeping the details of an active test confidential. I have a much harder time accepting that the public should simply trust that the people running it will always know where to stop.
Those questions need answers across the whole framework. Pulling out one clause won’t tell us everything about the oversight, just as pointing to a safety objective won’t answer every concern about power.
We still have to live with the result
The conversation eventually wandered into what would happen if a major platform decided Australia wasn’t worth the trouble. I don’t know that this bill would cause that. It was a hypothetical, and I don’t want to pretend otherwise. But thinking through the consequences was fairly uncomfortable.
I hate plenty about social media. I also have a client whose Facebook audience helped support fundraising for people with cancer and police officers with PTSD. I built him a website, which gives the work somewhere else to live, but people still share it through social media. The audience doesn’t magically move because you have a new address to send it to.
A sudden gap there would affect real people doing useful things. It would affect how I get my own work out as well. There are businesses and communities whose distribution depends on platforms they don’t control, and replacing that takes considerably more effort than telling them to find another channel.
That dependence on a few enormous foreign companies is also a problem. I am hardly comfortable with a company having the power to cut off that much activity in a country. But making the rules more complicated for everybody doesn’t automatically give us a useful Australian alternative. Smaller local companies still have to operate under them.
And somewhere in all of this, parents still need to teach their children how to use the internet. I saw enough horrible material as a teenager to have no interest in pretending the old internet was harmless. I don’t think we can outsource the entire job of preparing children for it to Canberra either. Parents can’t redesign a recommendation system, but a recommendation system isn’t going to raise their child for them.
I can see why an investigator would need an account that doesn’t announce itself as an investigation. The government has now put forward a way to do that. Before Parliament grants it, I want the government to explain how misuse gets discovered, how it gets stopped and what happens to the person responsible. If the answer is that we should trust the office, they need to do a better job of answering.