- AI can do amazing things. It also fails at basic stuff, all the time.
- That’s not a temporary state of things. We’ll be dealing with that dichotomy for a long time.
- Which makes predictions about what AI is going to do to work, and everything else, very hard to make.
Sometimes I use ChatGPT and it seems stunningly obvious that AI is going to have a transformative effect on my life. I use it more every day.
And other times I find myself yelling at ChatGPT in ALL CAPS, because it can’t do basic, simple tasks — ones I could reasonably farm out to a 5th grader. Or even worse: It can’t do basic tasks but won’t tell me it can’t do them, and tries to fudge a result instead. And that makes me wary of using it again.
Does this sound familiar?
It turns out that the AI business has a great term for this dichotomy: “The jagged frontier,” coined in a 2023 research paper. Here’s another way of putting it, via Reuters:
“It might be a Ferrari in math but a donkey at putting things in your calendar,” said Anastasios Angelopoulos, the CEO and cofounder of LMArena, a popular benchmarking tool.”
That quote comes from a report looking at the struggles various businesses have had implementing AI in their work. It’s a theme we’ve been hearing a lot about over the last few months, like the MIT study that found that 95% of companies were getting “zero return” on their AI investment.
This issue is core to the “Is AI a bubble and when will it pop?” question, of course. Which is a very important question, with some $2 trillion in investment in play.
But I think it’s not the only question: The tech isn’t going away, so many of us are unquestionably going to be using AI in all kinds of ways, no matter what.
So a more practical question is: What kind of tasks can AI do reliably well today — reliably enough that businesses (and the rest of us) can use it day in and day out — and which ones are going to take a while to sort out? And which ones may never be something we can hand over to AI?
This is a pretty good summary of the ongoing experiments we’re working out in real time, right now.
