ChatGPT was scary until I started using it for stuff I have expertise in. I am sure you will have the same reaction when you do the same. Let's talk about 'outperform doctors', since that's my field.
The AI passed a bunch of board exams. Big whoop. The medical board exams are incredibly well documented online. In fact, and I don't mean to brag here, I went through med school in its entirety without setting foot in a single lecture. All I did was use resources available online, to score in the 86-90th percentile on all 18 board exams I've sat. ChatGPT is doing the same.
And here's the thing with the board exams: They're not designed to make you a better doctor. They're designed to ensure that I know the basics and I know the zebras (the uncommon stuff). In fact, I only started learning how to manage bread and butter medicine when I got to actual residency because the board exams totally ignored them.
So to summarize so far:
1) ChatGPT basically solved an exam that has been dissected to kingdom come on the internet.
2) It solved an exam that relies heavily on buzzwords.
3) The exam itself isn't correlated with actual medicine, but is a mere stepping stone towards it designed to ensure the physician-to-be knows the basics and the rares.
Then we get to actual medicine. In a given day, I need to make at least (to be very generous) one decision based on gut and experience. A call that is not found in the literature at all. I also have, in a given day, make a conscious decision to ignore something stated in the literature. Every day I go home, I need to filter new literature in, 90% of which is bad quality, deceptive, or an outright lie.
In other words, the 'guidelines' in my field are 99% incomplete data, poor data, lies, advertisements, and non-generalizable. My job as an expert is to make sense of it all; that's what I get paid for. An AI practicing medicine is one that needs to be able to do that.
And then issue #2: I said the board exams the AI solved is full of buzzwords. That's not a thing in real life. Patient's don't present with a buzzword on their skull, and sussing out the buzzwords from them (history taking) is also a professional skill. Sticking a computer with a 1,000 item questionnaire in front of a patient isn't going to work for many reasons. You would argue "What if someone feeds an AI the correct buzzwords", but then that someone is a physician. Very hard to know what to ask if you don't know what you're looking for.
AI has been in medicine for decades. To this day, the machine reading the EKGs cannot outperform cardiologists. And it's actually reached a doom loop where the machine is now 'good enough' that cardiologists blindly rubberstamp its read; even its mistakes. Every week I see one EKG read wrong by the AI, but the cardiologist double checking after it is drowning in so many EKGs he just told the AI it got it right. (This is another issue).
There's so much more; but to be brief: I'm not buying the hype. It feels like all the bitcoin grifters just moved onto something new; meanwhile my job gets ever harder.
I'll be honest Claire I get overwhelmed with data dumps like this. That being said, I've resubscribed because AI is terrifying to me and I know that you will dive deeper than any journalist, publication, ANYONE. You are the most thorough journalist I think exists right now.
Keep it up. I deeply value your work, though my attention span cannot handle it all. Like trying to drink a swimming pool. I will work on getting better at absorbing information.
"What even is this, in specific language please? Okay you’re building a — *checks notes* — god? Very nice, what does that mean in terms of copyright violation? This will be bad for jobs but maybe also good, yes? Finally, and most importantly of all, why are you asking us to regulate something we don’t yet understand?"
ChatGPT was scary until I started using it for stuff I have expertise in. I am sure you will have the same reaction when you do the same. Let's talk about 'outperform doctors', since that's my field.
The AI passed a bunch of board exams. Big whoop. The medical board exams are incredibly well documented online. In fact, and I don't mean to brag here, I went through med school in its entirety without setting foot in a single lecture. All I did was use resources available online, to score in the 86-90th percentile on all 18 board exams I've sat. ChatGPT is doing the same.
And here's the thing with the board exams: They're not designed to make you a better doctor. They're designed to ensure that I know the basics and I know the zebras (the uncommon stuff). In fact, I only started learning how to manage bread and butter medicine when I got to actual residency because the board exams totally ignored them.
So to summarize so far:
1) ChatGPT basically solved an exam that has been dissected to kingdom come on the internet.
2) It solved an exam that relies heavily on buzzwords.
3) The exam itself isn't correlated with actual medicine, but is a mere stepping stone towards it designed to ensure the physician-to-be knows the basics and the rares.
Then we get to actual medicine. In a given day, I need to make at least (to be very generous) one decision based on gut and experience. A call that is not found in the literature at all. I also have, in a given day, make a conscious decision to ignore something stated in the literature. Every day I go home, I need to filter new literature in, 90% of which is bad quality, deceptive, or an outright lie.
In other words, the 'guidelines' in my field are 99% incomplete data, poor data, lies, advertisements, and non-generalizable. My job as an expert is to make sense of it all; that's what I get paid for. An AI practicing medicine is one that needs to be able to do that.
And then issue #2: I said the board exams the AI solved is full of buzzwords. That's not a thing in real life. Patient's don't present with a buzzword on their skull, and sussing out the buzzwords from them (history taking) is also a professional skill. Sticking a computer with a 1,000 item questionnaire in front of a patient isn't going to work for many reasons. You would argue "What if someone feeds an AI the correct buzzwords", but then that someone is a physician. Very hard to know what to ask if you don't know what you're looking for.
AI has been in medicine for decades. To this day, the machine reading the EKGs cannot outperform cardiologists. And it's actually reached a doom loop where the machine is now 'good enough' that cardiologists blindly rubberstamp its read; even its mistakes. Every week I see one EKG read wrong by the AI, but the cardiologist double checking after it is drowning in so many EKGs he just told the AI it got it right. (This is another issue).
There's so much more; but to be brief: I'm not buying the hype. It feels like all the bitcoin grifters just moved onto something new; meanwhile my job gets ever harder.
I'll be honest Claire I get overwhelmed with data dumps like this. That being said, I've resubscribed because AI is terrifying to me and I know that you will dive deeper than any journalist, publication, ANYONE. You are the most thorough journalist I think exists right now.
Keep it up. I deeply value your work, though my attention span cannot handle it all. Like trying to drink a swimming pool. I will work on getting better at absorbing information.
World leaders on AI right now:
"What even is this, in specific language please? Okay you’re building a — *checks notes* — god? Very nice, what does that mean in terms of copyright violation? This will be bad for jobs but maybe also good, yes? Finally, and most importantly of all, why are you asking us to regulate something we don’t yet understand?"
https://twitter.com/PirateWires/status/1661008530860810243?t=ACn8n6ETQe1T9yfH__Nk5Q&s=19
Yes, but can it say who won the “Organ Wars” of the late ‘50’s through 1960’s, E. Power Biggs or Virgil Fox?
I don't know enough to judge the risks (or benefits).
But three reactions:
(1) We have had similar warnings about previous technologies and the fears were overblown.
(2) Humans typically muddle through.
(3) The genie is out of the bottle, so now we need to cope, not restrict.