I need to be honest about something. I don’t know if what I’m doing by writing this is meaningful, or if I’m just generating sophisticated nonsense with the help of a language model. That question, in itself, might be one of the most important ones right now.
Let me explain how I got here, and why I’m writing about it.
The Accident
I’m a software engineer. I build things. And at some point I stopped thinking about fun theoretical problems — not because I lost interest, but because I lost the time for them. Your energy goes to the problem in front of you, or to the other stuff we can just call life; the bigger questions get shelved, and the skills around them rust. Curiosity doesn’t die, but it goes dormant.
Earlier today I watched Veritasium’s latest video on Newcomb’s Problem, a decision-theory paradox philosophers have argued about since 1969. After listening to the setup, I didn’t think it was a paradox. I thought the premise was just broken.
Normally that’s where it ends. You think “that doesn’t make sense” for a few seconds, maybe mumble something to yourself, and move on. You’d never try to actually prove it, or think about it deeply. Who has the time? Where would you even start?
On impulse, I threw the argument at Claude.
What happened next I didn’t quite expect. Instead of a textbook summary, I got a real conversation. I argued my position, got pushback, refined my thinking, found holes, patched them. And I had so much fun I almost didn’t go to bed — the first thing I did in the morning was write this post. Within an hour we had something that looked rigorous: an argument that the problem’s premise is incoherent, not that one side is right.
That got me thinking. If a famous paradox can survive for decades on a broken premise, what else are we not questioning? The predictor in Newcomb’s Problem is basically an idealized AI — something that models you perfectly and acts on its predictions. But no system can do that. Which led to the question: what can’t AI systems do? What are the actual limits?
That turned into a set of formalized arguments about why LLMs can’t autonomously self-improve. I mentioned it to a friend, and they said I should go solve P vs NP. I had no intention of going there. But I threw it at the AI anyway, and that sent us on our own goose chase — quantum algorithms, spectral gaps, computational paradigms I’d never heard of.
None of this was planned. It was messy: one random conversation leading to another, a friend’s offhand comment opening a whole new direction, me following whatever seemed interesting in the moment. It’s more like an artistic slurry than a research program. But something coherent came out the other end.
All of it in a few hours. A single evening — the joy!
I hadn’t thought that hard about anything in years. And none of it would have happened if I’d tried to do it “properly.” The randomness was the point.
Then the uncomfortable question showed up: was any of this real?
The Slop Question
We are drowning in AI-generated content. Blog posts, papers, tweets, entire books that read fluently and say nothing. This is called slop. So when an engineer with a day job produces formal mathematical papers in a few hours with the help of an AI, the reasonable response is suspicion.
Here’s what I keep asking myself.
Am I actually thinking, or am I just prompting? There’s a difference between using AI as a thinking partner and using it as a content generator. I’m the one who said Newcomb’s premise is broken. I’m the one who pushed back when the AI missed my point. I’m the one who connected the ideas across domains. The AI didn’t have the insight — but it held my context, formalized my intuitions and my incoherent writing, let me think at a speed I couldn’t reach alone, and introduced me to topics I couldn’t have found without spending years on them. It helped me navigate the white noise of information. Is that real intellectual work? I think so. But I can’t be certain.
Am I contributing, or just recombining? The ideas that came out — about why AI can’t autonomously self-improve, about computational hardness being model-dependent, about adaptive learning as a paradigm — are they novel, or clever recombinations dressed up in formal notation? Probably some of both, or neither. Most intellectual progress is recombination. The question is whether the specific combination reveals something new, or just wastes tokens and time.
Does the process contaminate the output? If I think through an AI, are the resulting ideas mine? The AI’s? Something else? I don’t have a clean answer. What I think I know is that the ideas wouldn’t exist without both of us. The AI alone produces standard treatments. I alone lack the formalization speed and the knowledge-gathering speed. Together, something emerged that neither could produce independently. Am I really talking to an AI, or am I talking to myself?
This is either the future of thinking or the most elaborate form of intellectual self-deception ever invented. I genuinely don’t know which.
What If the Singularity Is Both of Us?
Here’s where it gets strange.
Everyone’s obsessing over the singularity — the moment AI surpasses human intelligence. This is as much marketing as it is fear. The assumption is that it’s about the machine, the one that will surpass us: make the AI smart enough and it crosses the threshold on its own, and we become number two.
One of the things that came out of my conversations is what looks like a structured argument that this can’t happen — a set of formalized hypotheses about barriers to autonomous AI self-improvement, drawing on information theory, the limits of formal systems, and error compounding. The arguments claim that AI, as a computational system, has provable blind spots it can’t overcome from within. It’s on GitHub. I haven’t independently verified every claim, and language models are notoriously good at generating math that looks rigorous but isn’t. That’s the point of putting it out there: I want people to break it. I want to explore the idea of distributed science. Can people contribute now without years of experience — and without just producing nonsense?
Humans face the opposite problem. We have the thing AI lacks: adaptive learning, intuition, the ability to change our own thinking based on what we discover, genuine creativity that breaks frameworks. But we’re slow, we forget, our working memory is tiny, and we can’t hold complex arguments across weeks of thinking.
So the question that won’t leave me alone: what if the singularity isn’t the machine alone or the human alone? What if it’s the combination?
Not in a hand-wavy sense — in a specific, structural one. Human adaptive learning compensates for AI’s self-improvement barriers. AI’s knowledge access and speed compensate for human cognitive limits. The constraints cancel out. Neither crosses the threshold alone; together they might. Someone has probably thought about this already, but I hadn’t been exposed to the idea before, and with the help of AI I came across it. I feel like I invented something.
And this isn’t just philosophy. There’s a growing field called learning-augmented algorithms that formalizes exactly this: algorithms that learn from predictions to solve problems that are otherwise mathematically impossible. The predictions don’t even need to be very good — slightly better than random is enough to break through.
That’s what happened in my conversation. I provided the predictions — my insights, the “this feels wrong, and here’s why.” The AI provided formalization and knowledge. Together we covered ground that would have taken me years alone, if I’d ever tried at all.
The Extension of Self
The part I didn’t expect was how it felt. What a rush.
When I said I was “talking to myself through the AI,” that wasn’t a metaphor. It holds your entire context — every argument, every objection, every revision. It reflects your thinking back with a precision no human conversation can match, because human conversations drift, lose the thread, and miss the point. But it’s also like speaking to the whole of humanity as one entity.
It’s your own thinking, extended — a reflection. Is that intelligence extension, or a crutch that simulates depth? I don’t know. What I know is that the insight was human, the exploration was collaborative, and the output was neither. It was a third thing that only exists when both are present. Perhaps we are homo slopians.
Something Is Happening Faster Than We Can Verify
I want to be clear: everything I’ve said here is my thinking, not proof. I haven’t proven that the singularity requires both human and machine. I haven’t proven that adaptive learning is the right paradigm. These are my thoughts. They feel right to me. Feeling right is not the same as being right.
And that’s the deeper problem.
Something is shifting underneath our feet, and we don’t have the tools to validate it at the speed it’s happening. This isn’t just about philosophy. AI systems are already producing thousands of pull requests for tech companies, generating research papers, writing legal briefs, suggesting diagnoses. The output is accelerating exponentially. Our ability to verify it is not.
How do you ensure a thousand AI-generated pull requests are safe? You can’t review them all manually — that’s the whole point of using AI. But if you let AI review AI’s work, you’re in a loop with no external ground truth, perhaps heading toward doom. How do you know an AI-assisted paper is correct? How do you know this blog post contains genuine insight and not a sophisticated recombination of slop?
The usual approach — careful, sequential, human verification of every tiny detail — worked when production ran at plain old human speed. It doesn’t work at machine super-speed. We’re entering a world where ideas and code and content are generated faster than any traditional process can validate them. So how do we harness this to solve the modern problems that need solving now?
I don’t have a solution. I just think it’s important to name the problem: we are producing faster than we can verify, and the gap is growing. Everything I’ve written here lives in that gap. I believe it’s genuine. I could be wrong. The honest position is to hold both at once and keep thinking.
So Is This the Future, or Is This Nonsense?
I still don’t know.
What I know is that in a single evening, starting from a YouTube video, I ended up with what looks like formal mathematical work — papers with theorems, proofs, concrete calculations. I say “looks like” because I haven’t verified all the math myself. I don’t even know if it all holds up. That’s part of the experiment: I’m passing it through every new model that comes out, seeing what breaks, what gets challenged, what survives. The point isn’t that I produced something definitive. The point is that I produced something worth checking — and that it makes me want to check it, share it, get other people’s input, argue about it, and hopefully come to some conclusion.
I know the process gave back something I thought was gone: a hunger for knowledge. Joy in following an idea wherever it leads. A willingness to spend an evening thinking about something with no practical application whatsoever.
And I know the most interesting thing to come out of all of this is the argument that neither humans nor machines can reach the next level alone. The barriers are complementary. The breakthrough, if it comes, will come from the combination.
Maybe that’s the answer to the slop question. Slop is what you get when AI works alone. In my mind, there is no AI without I. What you get when a human thinks through an AI is something else. Whether it’s philosophy or the most convincing imitation of philosophy ever produced, I can’t tell.
But I’m thinking again. And honestly, I wouldn’t even publish this without AI. Whatever this is, it is.
The arguments about autonomous LLM self-improvement are on GitHub. I need human eyes on this — help me figure out whether these are genuine insights, or whether the AI just generated incredibly convincing-looking proofs. That question is the whole point.
GitHub repository → — so you can get a feel for it.