> but every time I try I find it borderline impossible to get functional code even for pretty straightforward prompts.
I work in 9 different projects now and I would say that around 80% of functional code comes from Sonnet (like GP) for these projects. These are not (all) trivial either; there is a very niche (for banking) key/value store written in Go for instance which has a lot of edge cases etc, all the plumbing (x,err = etc aka stuff people find annoying) comes from sonnet and works one-shot. A lot of business logic comes from sonnet too; it works but usually needs a little tweaking to make it correct.
Tests are all done by Sonnet. I think 80% is low balling it on Go code really.
We have a lot of complex code generator stuff and DSLs in TS which also works well often. Sometimes it gets some edge cases wrong, but either re-prompting with more details or fixing it ourselves, will do it. At a fraction of the time/money of what a fully human team would deliver.
I wrote a 3d editor for fun with Sonnet in a day.
I have terrible results with gpt/copilot (copilot is good for editing instead of complete files/functions; chatgpt is not good much compared with sonnet); it doesn't get close at all; it simply keeps giving me the same code over and over again when I say it's wrong; it hardcodes things specifically asked to make flexible etc. Not sure why the difference is so massive all of sudden.
Note: I use the sonnet API, not the web interface, but same for gpt so...
It was a few hours work so it was not 'done properly', but the point was, it is working code and, as I know nothing about 3d or game programming, I didn't do any of it. People here claim it (current LLMs) cannot produce working code for entire software's without a human doing a lot of it; it clearly can and for non trivial stuff. I would say the real point is ; it cannot do complete code for non trivial programs without human doing most of the coding IF the person prompting is not an experienced coder. I am, so I can review the generated code, prompt in English what it should change and then it works. It is often faster to write code instead of that (so the thing blurbs 100% working code but with 20% wrong/bugs which I fix as a human) but in domains I know nothing about, English is faster for me.
I work in 9 different projects now and I would say that around 80% of functional code comes from Sonnet (like GP) for these projects. These are not (all) trivial either; there is a very niche (for banking) key/value store written in Go for instance which has a lot of edge cases etc, all the plumbing (x,err = etc aka stuff people find annoying) comes from sonnet and works one-shot. A lot of business logic comes from sonnet too; it works but usually needs a little tweaking to make it correct.
Tests are all done by Sonnet. I think 80% is low balling it on Go code really.
We have a lot of complex code generator stuff and DSLs in TS which also works well often. Sometimes it gets some edge cases wrong, but either re-prompting with more details or fixing it ourselves, will do it. At a fraction of the time/money of what a fully human team would deliver.
I wrote a 3d editor for fun with Sonnet in a day.
I have terrible results with gpt/copilot (copilot is good for editing instead of complete files/functions; chatgpt is not good much compared with sonnet); it doesn't get close at all; it simply keeps giving me the same code over and over again when I say it's wrong; it hardcodes things specifically asked to make flexible etc. Not sure why the difference is so massive all of sudden.
Note: I use the sonnet API, not the web interface, but same for gpt so...