The problem is not that AI can generate code. The problem is that our contribution, review, and release systems were built for a world where code was expensive to produce.
The Software World Is Changing
The software world is changing. AI is writing a lot of the code now, and seasoned experts are increasingly using AI to generate code effectively.
AI slop is also a thing. Or, more specifically, low-quality code and low-quality implementations are becoming more common. Sometimes the problem is in the implementation itself. Other times it is in the user-facing design: text, images, workflows, or features that are obviously wrong, broken, or poorly thought through. The basic pattern is easy to recognize: someone prompts an AI, takes whatever output it produces, and ships it with little or no human validation.
It is a troubled time. We are generating more software, and often generating it much faster, but that does not necessarily mean the software is better. I do think it will improve over time. AI models continue to get better, and the software they generate seems to get slightly higher in quality with each generation, especially when guided by better prompts and more experienced users.
Open Source Is Drawing a Line
Godot recently announced that they are banning AI code contributions. Ghostty has a strong policy as well, and Mitchell has built a vouch system to help address this problem. SQLite has also banned agentic contributions. These are not the only examples, but they are a few recent ones I have seen.
The core issues seem to be:
- Time. Each pull request or contribution takes time to review. With the slop problem, a lot of that time is wasted.
- Responsibility. AI may generate the code, but the maintainer is left to maintain it over time.
- Tools. GitHub is where a lot of code is hosted, but it does not seem to have the right tools to manage this new reality.
These Problems Are Not New
These are not entirely new problems, though:
- Inexperienced developers could already generate pull requests that were as bad as, or worse than, AI-generated ones.
- The long-term maintenance burden of accepted code already existed.
- Submissions might be for features maintainers never wanted in the first place.
- Reviewing other people’s pull requests has never been especially fun.
The real issue seems to be volume. These problems were always present, but they were perhaps manageable. Maybe they scaled by adding more maintainers over time. This harsher reaction feels like the result of people hitting a cliff, getting sick of the flood of low-quality contributions, and finally doing something about it. I cannot really fault them. I hear about the same thing happening internally at companies. Teams simply cannot keep up with the number of PRs.
Banning AI Code Is a Stopgap
But is this the right long-term approach?
This problem is not going away. Code is going to continue to be generated faster and faster. There are no guarantees that it will get better at the same pace. The barrier to entry for software development will also continue to get lower, with less experienced people “writing” software through AI.
Humans may simply become too slow to keep up with the pace at which code is generated.
Would this be similar to books? Robots could generate unique novels far faster than humans could ever read them. One difference is that books usually need a publisher, and publishers are probably not going to accept endless slop. They have to be choosy. You could self-publish all of it, but that does not mean it would be successful. The vouch system seems like a reasonable approach here: rely on known contributors, or on people who can credibly vouch for new contributors. Publishing a book also costs real money, whereas submitting to open source software generally does not.
But are we going down the right road here?
Use AI to Triage AI
It seems like we need to use AI to combat the influx of AI-generated software development.
For the flood of pull requests, why are we not using AI to bucket them into quality categories? Imagine an A–F grade on each PR, based on criteria set by the project and analyzed by AI. If a PR does not make an A grade, the author gets AI-generated feedback explaining why and how to improve it. If it takes someone one PR to get to an A, fine. If it takes twenty attempts, maybe that is a signal that you probably do not want that commit either.
Code quality seems like something AI should be able to help evaluate. It is unlikely that you will successfully educate someone with no coding experience into writing excellent code through PR feedback alone. But you may be able to educate them on the rules of your system. Completely ignore anything below an A grade. Have AI summarize the common themes among rejected PRs. My guess is that a system like this does not really exist yet, and building it would be a significant investment. It also seems like the current PR ecosystem and its rules are not well designed for this kind of workflow.
Rethink Software Versions
What about more variability in software releases?
A lot of this matters because there is typically “one” version or release of a piece of software where everything has to work. But in an AI-driven world, could we have hundreds or thousands of versions of a piece of software, each with different options? In theory, every pull request generates a new version of the software. Why not let AI generate or build those versions on the fly?
When selecting software, you could choose what you want, and the software could be generated around those requirements. This sounds like a nightmare even as I write it, but I think that reaction mostly comes from the existing tools and ecosystems we have today. Those tools may be about to be turned upside down. The complexity of this is likely too much for a human to manage manually. Today, the interface we have is usually a version number or a SHA. But what if we had an MCP-style interface to software instead, where we described our requirements?
Imagine if every PR was automatically “accepted” into some possible branch of the software. You probably would not choose the multiverse software timeline that was bad. Or maybe some people would choose it anyway because it had the features they wanted and they did not care about the downsides. Could that ultimately be a healthier ecosystem?
Review Outcomes, Not Just Code
What if, instead of reviewing the code, we reviewed the outcomes and outputs of that code?
Today, what we mostly have are tests and test frameworks. Testing is useful, but it is also flawed. You can write bad tests. You can write bad software that still passes all the tests. The number of tests required to ensure perfection would be extremely high, and even AI is not going to write 100% of the tests we need. Tests also take time to develop and run. They cannot reliably infer how maintainable software will be over time, or how many bugs a particular area of the code is likely to produce in production.
What if we could review the outcomes and outputs of the software instead? What if we could evaluate whether the pull request was actually good or bad based on what it produced, how it behaved, and what impact it had?
We Still Need Human Judgment
What would these three options ultimately look like? I am not sure. Are they even viable? I am not sure about that either.
It just feels like we have put stopgap measures in place. They were probably necessary, but they may not be enough. We may need to rethink software development itself.
I do not think we should eliminate humans from software development. We still need human judgment. But maybe the goal should be to make that judgment easier, faster, and better informed, rather than simply trying to stop AI-generated code at the gate.