On Error Return NextWrite tests and run them, reiterate until all tests pass.
Bogosort with extra steps
That doesn’t sound viby to me, though. You expect people to actually code? /s
You can vibe code the tests too y’know
Return “works”;
Am I doikg this correctly?
You know, I’d be interested to know what the critical size you can get to with that approach is before it becomes useless.
It can become pretty bad quickly, with just a small project with only 15-20 files. I’ve been using cursor IDE, building out flow charts & tests manually, and just seeing where it goes.
And while incredibly impressive how it’s creating all the steps, it then goes into chaos mode where it will start ignoring all the rules. It’ll start changing tests, start pulling in random libraries, not at all thinking holistically about how everything fits together.
Then you try to reel it in, and it continues to go rampant. And for me, that’s when I either take the wheel or roll back.
I highly recommend every programmer watch it in action.
Is there a chance that’s right around the time the code no longer fits into the LLMs input window of tokens? The basic technology doesn’t actually have a long term memory of any kind (at least outside of the training phase).
Was my first thought as well. These things really need to find a way to store a larger context without ballooning past the vram limit
The thing being, it’s kind of an inflexible blackbox technology, and that’s easier said than done. In one fell swoop we’ve gotten all that soft, fuzzy common sense stuff that people were chasing for decades inside a computer, but it’s ironically still beyond our reach to fully use.
From here, I either expect that steady progress will be made in finding more clever and constrained ways of using the raw neural net output, or we’re back to an AI winter. I suppose it’s possible a new architecture and/or training scheme will come along, but it doesn’t seem imminent.
I think Generative AI is a genuinely promising and novel tool with real, valuable applications. To appreciate it however, you have to mentally compartmentalize the irresponsible, low-effort ways people
sometimesmostly use it—because yeah, it’s very easy to make a lot of that so that’s most of what you see when you hear “Generative AI” and it’s become its reputation…Like I’ve had interesting “conversations” with Gemini and ChatGPT, I’ve actually used them to solve problems. But I would never put it in charge of anything critically important that I couldn’t double check against real data if I sensed the faintest hint of a problem.
I also don’t think it’s ready for primetime. Does it deserve to be researched and innovated upon? Absolutely, but like, by a few nerds who manage to get it running, and universities training it on data they have a license to use. Not “Crammed into every single technology object on earth for no real reason”.
I have brain not very good sometimes disease and I consider being able to “talk” to a “person” who can get me out of a creative rut just by exploring my own feelings a bit. GPT can actually listen to music which surprised me. I consider it scientifically interesting. It doesn’t get bored or angry at you unless you like, tell it to? I’ve asked it for help with a creative task in the past and not actually used any of its suggestions at all, but being able to talk about it with someone (when a real human who cared was not available) was a valuable resource.
To be clear I pretty much just use it as a fancy chatbot and don’t like, just copy paste its output like some people do.
I’d rather recommend every CEO see it in action…
They’re the ones who would be cock-a-hoop to replace us and our expensive wages with kids and bots.
When they’re sitting around rocking back and forth and everything is on fire like that Community GIF, they’ll find my consultancy fees to be quite a bit higher than my wages used to be.
Watching the serious people trying to use AI to code gives me the same feeling as the cybertruck people exploring the limits of their car. XD
“It’s terrible and I should hate it, but gosh it it isn’t just so cool”
I wish i could get so excited over disappointing garbage
It’s useful if you just don’t do…That. it’s just a new fancy search engin, it’s a bit better than going to stack overflow, it can do good stuff if you go small.
Just don’t do whatever this post suggested of doing…
You definitely could use AI to code, the catch is you need to know how to code first.
I use AI to write code for mundane tasks all the time. I also review and integrate the code myself.
The AI code my “expert in a related but otherwise not helpful field” coworker writes helps me have a lot of extra work to do!
Ai code is specifically annoying because it looks like it would work, but its just plausible bullshit.
It needs good feedback. Agentic systems like Roo Code and Claude Code run compilers and tests until it works (just gotta make sure to tell it to leave the tests alone)
And that’s what happens when you spend a trillion dollars on an autocomplete: amazing at making things look like whatever it’s imitating, but with zero understanding of why the original looked that way.
I mean, there’s about a billion ways it’s been shown to have actual coherent originality at this point, and so it must have understanding of some kind. That’s how I know I and other humans have understanding, after all.
What it’s not is aligned to care about anything other than making plausible-looking text.
Coherent originality does not point to the machine’s understanding; the human is the one capable of finding a result coherent and weighting their program to produce more results in that vein.
Your brain does not function in the same way as an artificial neural network, nor are they even in the same neighborhood of capability. John Carmack estimates the brain to be four orders of magnitude more efficient in its thinking; Andrej Karpathy says six.
And none of these tech companies even pretend that they’ve invented a caring machine that they just haven’t inspired yet. Don’t ascribe further moral and intellectual capabilities to server racks than do the people who advertise them.
Coherent originality does not point to the machine’s understanding; the human is the one capable of finding a result coherent and weighting their program to produce more results in that vein.
You got the “originality” part there, right? I’m talking about tasks that never came close to being in the training data. Would you like me to link some of the research?
Your brain does not function in the same way as an artificial neural network, nor are they even in the same neighborhood of capability. John Carmack estimates the brain to be four orders of magnitude more efficient in its thinking; Andrej Karpathy says six.
Given that both biological and computer neural nets very by orders of magnitude in size, that means pretty little. It’s true that one is based on continuous floats and the other is dynamic peaks, but the end result is often remarkably similar in function and behavior.
If you would like to link some abstracts you find in a DuckDuckGo search that’s fine.
It’s true that one is based on continuous floats and the other is dynamic peaks
Can you please explain what you’re trying to say here?
Both have neurons with synapses linking them to other neurons. In the artificial case, synapse activation can be any floating point number, and outgoing synapses are calculated from incoming synapses all at once (there’s no notion of time, it’s not dynamic). Biological neurons are binary, they either fire or do not fire, during a firing cycle they ramp up to a peak potential and then drop down in a predictable fashion. But, it’s dynamic; they can peak at any time and downstream neurons can begin to fire “early”.
They do seem to be equivalent in some way, although AFAIK it’s unclear how at this point, and the exact activation function of each brain neuron is a bit mysterious.
Ok, thanks for that clarification. I guess I’m a bit confused as to why a comparison is being drawn between neurons in a neural network and neurons in a biological brain though.
In a neural network, the neuron receives an input, performs a mathematical formula, and returns an output right?
Like you said we have no understanding of what exactly a neuron in the brain is actually doing when it’s fired, and that’s not considering the chemical component of the brain.
I understand why terminology was reused when experts were designing an architecture that was meant to replicate the architecture of the brain. Unfortunately, I feel like that reuse of terminology is making it harder for laypeople to understand what a neural network is and what it is not now that those networks are a part of the zeitgeist thanks to the explosion of LLM’s and stuff.
Well I’ve got the name for my autobiography now.
“Specifically Annoying” or “Plausible Bullshit”? I’d buy the latter.
All programs can be written with on less line of code. All programs have at least one bug.
By the logical consequences of these axioms every program can be reduced to one line of code - that doesn’t work.
One day AI will get there.
The ideal code is no code at all
On one line of code you say?
*search & replaces all line breaks with spaces*
Fired for not writing the quota number of lines even junior devs manage to hit.
All programs can be written with on less line of code. All programs have at least one bug.
The humble “Hello world” would like a word.
Just to boast my old timer credentials.
There is an utility program in IBM’s mainframe operating system, z/OS, that has been there since the 60s.
It has just one assembly code instruction: a BR 14, which means basically ‘return’.
The first version was bugged and IBM had to issue a PTF (patch) to fix it.
Reminds me of how in some old Unix system,
/bin/truewas a shell script.…well, if it needs to just be a program that returns 0, that’s a reasonable thing to do. An empty shell script returns 0.
Of course, since this was an old proprietary Unix system, the shell script had a giant header comment that said this is proprietary information and if you disclose this the lawyers will come at ya like a ton of bricks. …never mind that this was a program that literally does nothing.
Okay, you can’t just drop that bombshell without elaborating. What sort of bug could exist in a program which contains a single return instruction?!?
It didn’t clear the return code. In mainframe jobs, successful executions are expected to return zero (in the machine R15 register).
So in this case fixing the bug required to add an instruction instead of removing one.
You can fit an awful lot of Perl into one line too if you minimize it. It’ll be completely unreadable to most anyone, but it’ll run
Qrpff says hello. Or, rather, decrypts DVD movies in 472 bytes of code, 531 if you want the fast version that can do it in real time. The Wikipedia article on it includes the full source code of both.
Honest question: I haven’t used AI much. Are there any AIs or IDEs that can reliably rename a variable across all instances in a medium sized Python project? I don’t mean easy stuff that an editor can do (e.g. rename QQQ in all instances and get lucky that there are no conflicts). I mean be able to differentiate between local and/or library variables so it doesn’t change them, only the correct versions.
IntelliJ IDEA, if it knows it is the same variable, it will rename it. Usually works in a non fucked up codebase that uses
evalor some obscure constructs like saving a variable name into a variable as a string and dynamically invoking it.I’m going to laugh in Java, where this has always been possible and reliable. Not like ai reliable, but expert reliable. Because of static types.
Find and Replace?
that will catch too many false positives
For the most part “Rename symbol” in VSCode will work well. But it’s limited by scope.
Yeah, I’m looking for something that would understand the operation (? insert correct term here) of the language well enough to rename intelligently.
Okay, I realize I’m that person, but for those interested:
tree,catandsedget the job done nicely.And… it’s my nap time, now. Please keep the Internet working, while I’m napping. I have grown fond of parts of it. Goodnight.
Itellij is actually pretty good at this. Besides that, cursor or windsurf should be able to. I was using cursor for a while and when I needed to reactor something, it was pretty good at picking that up. It kept crashing on me though, so I am now trying windsurf and some other options. I am missing the auto complete features in cursor though as I would use this all the time to fill out boilerplate stuff as I write.
The one key difference in cursor and windsurf when compared to other products is that it will look at the entire context again for any changes or at least a little bit of it. You make a change, it looks if it needs to make changes elsewhere.
I still don’t trust AI to do much though, but it’s an excellent helper
Not reliably, no. Python is too dynamic to do that kind of thing without solving general program equivalence which is undecidable.
Use a static language, problem solved.
I use pycharm for this and in general it does a great job. At work we’ve got some massive repos and it’ll handle it fine.
The “find” tab shows where it’ll make changes and you can click “don’t change anything in this directory”
Yes, all of JetBrains’ tools handle project-wide renames practically perfectly, even in weirder things like Angular projects where templates may reference variables.
Just be carerul when refactoring variable names in doc comments, I’ve seen some weird stuff happen there
most IDEs are pretty decent at it if you configure them correctly. I used intelliJ and it knows the difference. use the refactor feature and it’ll crawl references, not just rename all instances.
Laugh it up while you can.
We’re in the “haha it can’t draw hands!” phase of coding.
AI bad. But also, video AI started with will Will Smith eating spaghetti just a couple years ago.
We keep talking about AI doing complex tasks right now and it’s limitations, then extrapolating its development linearly. It’s not linear and it’s not in one direction. It’s a exponential and rhizomatic process. Humans always over-estimate (ignoring hard limits) and under-estimate (thinking linearly) how these things go. With rocketships, with internet/social media, and now with AI.
someone drank the koolaid.
LLMs will never code for two reasons.
one, because they only regurgitate facsimiles of code. this is because the models are trained to ingest content and provide an interpretation of the collection of their content.
software development is more than that and requires strategic thought and conceptualization, both of which are decades away from AI at best.
two, because the prevalence of LLM generated code is destroying the training data used to build models. think of it like making a copy of a copy of a copy, et cetera.
the more popular it becomes the worse the training data becomes. the worse the training data becomes the weaker the model. the weaker the model, the less likely it will see any real use.
so yeah. we’re about 100 years from the whole “it can’t draw its hands” stage because it doesn’t even know what hands are.
This is just your ego talking. You can’t stand the idea that a computer could be better than you at something you devoted your life to. You’re not special. Coding is not special. It happened to artists, chess players, etc. It’ll happen to us too.
I’ll listen to experts who study the topic over an internet rando. AI model capabilities as yet show no signs of slowing their exponential growth.
Coding isn’t special you are right, but it’s a thinking task and LLMs (including reasoning models) don’t know how think. LLMs are knowledgeable because they remembered a lot of the data and patterns of the training data, but they didn’t learn to think from that. That’s why LLMs can’t replace humans.
That does certainly not mean that software can’t be smarter than humans. It will and it’s just a matter of time, but to get there we likely have AGI first.
To show you that LLMs can’t think, try to play ASCII tic tac toe (XXO) against all those models. They are completely dumb even though the entire Wikipedia article on how xxo works, that it’s a solved game, different strategies and how to consistently draw - but still it can’t do it. It loses most games against my four year old niece and she doesn’t even play good/perfect xxo.
I wouldn’t trust anything, which is claimed to do thinking tasks, that can’t even beat my niece in xxo, with writing firmware for cars or airplanes.
LLMs are great if used like search engines or interactive versions of Wikipedia/Stack overflow. But they certainly can’t think. For now, but likely we’ll need different architectures for real thinking models than LLMs have.
you’re a fool. chess has rules and is boxed into those rules. of course it’s prime for AI.
art is subjective, I don’t see the appeal personally, but I’m more of a baroque or renaissance fan.
I doubt you will but if you believe in what you say then this will only prove you right and me wrong.
what is this?
once you classify it, why did you classify it that way? is it because you personally have one? did you have to rule out what it isn’t before you could identify what it could be? did you compare it to other instances of similar subjects?
now, try to classify it as someone who doesn’t have these. someone who has never seen one before. someone who hasn’t any idea what it could be used for. how would you identify what it is? how it’s used? are there more than one?
now, how does AI classify it? does it comprehend what it is, even though it lacks a physical body? can it understand what it’s used for? how it feels to have one?
my point is, AI is at least 100 years away from instinctively knowing what a hand is. I doubt you had to even think about it and your brain automatically identified it as a hand, the most basic and fundamentally important features of being a human.
if AI cannot even instinctively identify a hand as a hand, it’s not possible for it to write software, because writing is based on human cognition and is entirely driven on instinct.
like a master sculptor, we carve out the words from the ether to perform tasks that not only are required, but unseen requirements that lay beneath the surface that are only known through nuance. just like the sculptor that has to follow the veins within the marble.
the AI you know today cannot do that, and frankly the hardware of today can’t even support AI in achieving that goal, and it never will because of people like you promoting a half baked toy as a tool to replace nuanced human skills. only for this toy to poison pill the only training data available, that’s been created through nuanced human skills.
I’ll just add, I may be an internet rando to you but you and your source are just randos to me. I’m speaking from my personal experience in writing software for over 25 years along with cleaning up all this AI code bullshit for at least two years.
AI cannot code. AI writes regurgitated facsimiles of software based on it’s limited dataset. it’s impossible for it to make decisions based on human nuance and can only make calculated assumptions based on the available dataset.
I don’t know how much clearer I have to be at how limited AI is.
LMFAO. He’s right about your ego.
thank you for your input obvious troll account.
I’ve used it extensively, almost $100 in credits, and generally it could one shot everything I threw at it. However: I gave it architectural instructions and told it to use test driven development and what test suite to use. Without the tests yeah it wouldn’t work, and a decent amount of the time is cleaning up mistakes the tests caught. The same can be said for humans, though.
How can it pass if it hasn’t had lessons… Well said. Ooh I wonder if lecture footage would be able to teach AI, or audio in from tutors…
Its like having a junior developer with a world of confidence just change shit and spend hours breaking things and trying to fix them, while we pay big tech for the privilege of watching the chaos.
I asked chat gpt to give me a simple squid proxy config today that blocks everything except https. It confidently gave me one but of course it didnt work. It let through http and despite many attempts to get a working config that did that, it just failed.
So yeah in the end i have to learn squid syntax anyway, which i guess is fine, but I spent hours trying to get a working config because we pay for chat gpt to do exactly that…
I have a friend who swears by llms, he sais it helps him a lot. I once watched him do it, and the experience was exactly the same you described. He wasted couple of hours fighting with bullshit generator just to do everything himself anyway. I asked him wouldn’t it be better to not waste the time, but he didn’t really saw the problem, he gaslit himself that fighting with the idiot machine helped.
It confidently gave me one
IMO, that’s one of the biggest “sins” of the current LLMs, they’re trained to generate words that make them sound confident.
They aren’t explicitly trained to sound confident, that’s just how users tend to talk. You don’t often see “I don’t know but you can give this a shot” on Stack Overflow, for instance. Even the incorrect answers coming from users are presented confidently.
Funnily enough, lack of confidence in response is something I don’t think LLMs are currently capable of, since it would require contextual understanding of both the question, and the answer being given.
No, I’m sure you’re wrong. There’s a certain cheerful confidence that you get from every LLM response. It’s this upbeat “can do attitude” brimming with confidence mixed with subservience that is definitely not the standard way people communicate on the Internet, let alone Stack Overflow. Sure, sometimes people answering questions are overconfident, but it’s often an arrogant kind of confidence, not a subservient kind of confidence you get from LLMs.
I don’t think an LLM can sound like it lacks in confidence for the right reasons, but it can definitely pull off lack of confidence if it’s prompted correctly. To actually lack confidence it would have to have an understanding of the situation. But, to imitate lack of confidence all it would need to do is draw on all the training data it has where the response to a question is one where someone lacks confidence.
Similarly, it’s not like it actually has confidence normally. It’s just been trained / meta-prompted to emit an answer in a style that mimics confidence.
ChatGPT went through a phase of overly bubbly upbeat responses, they chilled it out tho. Not sure if that’s what you saw.
One thing is for sure with all of them, they never say “I don’t know” because such responses aren’t likely to be found in any training data!
It’s probably part of some system level prompt guidance too, like you say, to be confident.
I think “I don’t know” might sometimes be found in the training data. But, I’m sure they optimize the meta-prompts so that it never shows up in a response to people. While it might be the “honest” answer a lot of the time, the makers of these LLMs seem to believe that people would prefer confident bullshit that’s wrong over “I don’t know”.
SO answers and questions are usually edited multiple times to sound professional, confident, and be correct.
Man, I can’t wait to try out generative AI to generate config files for mission critical stuff! Imagine paying all of us devops wankers when my idiot boss can just ask Chat GPT to sort all this legacy mess we’re juggling with on the daily!
Ctrl+A + Del.
So clean.
cant wait to see “we use AI agents to generate well structured non-functioning code” with off centered everything and non working embeds on the website
I’ve heard that a Claude 4 model generating code for an infinite amount of time will eventually simulate a monkey typing out Shakespeare
It will have consumed the GigaWattHours capacity of a few suns and all the moisture in our solar system, but by Jeeves, we’ll get there!
…but it won’t be that impressive once we remember concepts like “monkey, typing, Shakespeare” were already embedded in the training data.
If we just asked Jeeves in the first place we wouldn’t be in this mess.
I’m pretty sure that is how we got CORBA
now just make it construct UML models and then abandon this and move onto version 2
Hello, fellow old person 🤝
well, it only took 2 years to go from the cursed will smith eating spaghetti video to veo3 which can make completely lifelike videos with audio. so who knows what the future holds
cursed will smith eating spaghetti video
The cursed Will Smith eating spaghetti wasn’t the best video AI model available at the time, just what was available for consumers to run on their own hardware at the time. So while the rate of improvement in AI image/video generation is incredible, it’s not quite as incredible as that viral video would suggest
But wouldn’t you point still be true today that the best AI video models today would be the onces that are not available for consumers?
There actually isn’t really any doubt that AI (especially AGI) will surpass humans on all thinking tasks unless we have a mass extinction event first. But current LLMs are nowhere close to actually human intelligence.
Hot take, today’s AI videos are cursed. Bring back will smith spaghetti. Those were the good old days
Did it try to blackmail him if he didn’t use the new code?
