Some people have asked me why I'm not super bullish on AI.

A rough timeline non specific

• ChatGPT 3.5ish - I remember asking it to generate some code. It didn't work. It wouldn't compile. But I was surprised how far it got.
• Copilot - At some point I installed the Copilot plugin for JetBrains and spent a night building a web app. I remember fondly that it handled about 70% of the work for it me. Plenty of compile errors, but what a great start. I was especially happy it handled the annoying bits of CSS and Javascript for me.
  • At work eventually it got to a point where Copilot was so wrong so often that I stopped paying attention to it. I mostly rolled my eyes at the PR comments it generated.
• ChatGPT 4ish - This is when I started paying for ChatGPT. This felt like a real watershed moment. The code it generated was about 90% correct, but it was mostly constrained to 1-3 functions at a time.
• Claude - Wow, where do I begin
  • My previous architect at work recommended it to me because he was leaving.
  • At first I mostly used to help me generate summarizations of repos I didn't understand but managed to inherit. I let it do a few changes here and there.
  • Around this time Junie came out for Jetbrains, which was mostly great but also limited and expensive. I moved on fast.
  • At some point in time, I just didn't have anything left in me. Call it burn out. I tasked claude with figuring out something mildly complex and it suceeded admirably. I began to trust it.
  • Over time, I tried trusting Claude with more and more. Eventually the ratio was like 90% of the time it could handle small things, but every 10% of the time it would just fuck up horribly. Often it would spend hours on a problem I was able to solve within just a few moments of thinking about the problem.
  • Claude started getting worse - it would comment out tests and functions, refuse to work on things, and I'd hit my time limits often.
  • Now I mostly keep Claude in plan mode. I might let it help tackle a quick 1 min bug fix.

So basically, today, when people ask me "What I think about AI?" my answer is something like.

Meh