It’s interesting that #mainstreaming tech makes it easy to use you, while alternative tech makes it hard to use freedom. One is built for control – and works very well at it. The other is built for freedom – but often works badly at delivering it. That’s the real #techshit problem we face, we need to make freedom easier to use than control.

The latest #fashionista panic about AI scraping is exposing a real problem with how we understand and defend the #openweb. Faced with aggressive commercial data extraction by the #dotcons and #AI industry, the response from parts of the traditional media establishment and mainstreaming policymakers is to restrict automated access, demand identification and introduce new controls over who can collect #4opens available information. This is the wrong direction.
We have a genuine problem of #dotcons extracting open flows, enormous amounts of public information, imposing costs on the people who create, host and maintain it. They take the value generated by the #openweb, feed it into proprietary systems and then build closed commercial products that bypass the original sources. This is another round of enclosure.
But shutting down the tools that make the #openweb work is not a solution, it leads to destroying the infrastructure we need to challenge this enclosure. The native #openweb depends on open flows, automated crawling, scraping, indexing, archiving and redistribution are not some new abuse of the internet. They are fundamental parts of how the native #openweb develops.
Search engines, independent journalism, public-interest research, accessibility projects, security monitoring and archives all depend on the ability to collect and process publicly available information. The Internet Archive’s Wayback Machine has preserved more than a trillion web pages. Investigative journalists have used automated collection to expose conflicts of interest in government. Researchers use it to examine discrimination, misinformation and corporate behaviour.
These activities need the #KISS ability to follow information across the web without asking permission from every website owner. Sometimes anonymity is needed, if a journalist, researcher or activist investigating powerful interests has to identify themselves and explain what they are looking for, the investigation itself is compromised. An #openweb cannot depend on asking permission from every gatekeeper before we see, collect, preserve and share public information.
The problem is extraction, not openness – we need to distinguish between two very different things. There is grassroots, public-interest use of automated tools to make information accessible, preserve knowledge and connect communities. Then there is the #dotcons industrial-scale commercial extraction, the gray area were they take the collective output of the openweb, concentrate it into private databases and use it to build proprietary #closedweb services. These are not the same activity.
The problem is not that information flows freely, it is that the value created by those flows is being enclosed and privately captured. Making every crawler identify itself doesn’t solve this. A commercial scraper can identify itself and still overwhelm a server with requests. Meanwhile, an independent archivist or journalist could be deterred from doing legitimate work simply because they cannot safely disclose their identity. We need to regulate harmful behaviour, not criminalize the basic mechanisms of openness.
The #geekproblem is about trying to control the mess instead of fixing it. When faced with complex social and technical problem, the instinct is to introduce another layer of technical control, identification and permission. But the native #openweb was built on a different understanding: distributed participation, open protocols, independent servers and the ability to connect information across organizational boundaries.
We should be strengthening the infrastructure that allows independent websites, archives and communities to survive. What the #OMN adds is rebuilding the commons. The answer is not to retreat into closed platforms, private APIs, permission-based networks or another generation of corporate-controlled infrastructure. We need to rebuild the shared flows, it is not about pretending that every use of public information is harmless. It is about building the social and technical capacity to distinguish useful participation from destructive extraction.
Don’t repeat the mistake – for too long, we have allowed the #dotcons to take the value of the native #openweb while selling us increasingly closed replacements. Now, faced with the consequences, some people want to restrict the openness that made the internet useful in the first place. We cannot allow the response to commercial enclosure to become another round of enclosure. The native #openweb was never perfect, it has technical debt, weak governance, funding problems and social conflicts. But it gave us something we have largely lost, the possibility of building collective power through open information flows. We need to recover that.
The answer to bad bots is not to break the web, the answer to commercial extraction is not to close public information, the answer to platform power is not another platform with more controls. The path is simple, we need to reboot the #openweb, not shut it down. That means rebuilding the grassroots communities, shared infrastructure and collective governance that allow openness to grow real social value.
https://www.techpolicy.press/breaking-the-open-web-wont-stop-the-bad-bots















