The ChatGPT

[Ludicus]

The ChatGPT was released in December as a web app by the San Francisco–based firm OpenAI. It’s free, try it. According to OpenAI, Chat GPT will remain free for the foreseeable future. However, they have also recently released a paid subscription plan, Chat GPT Plus - what you need to know - PC Guide

Read the peer-reviewed article, published three days ago,
Performance of ChatGPT on USMLE: Potential for AI-assisted

Author summary

Artificial intelligence (AI) systems hold great promise to improve medical care and health outcomes. As such, it is crucial to ensure that the development of clinical AI is guided by the principles of trust and explainability. Measuring AI medical knowledge in comparison to that of expert human clinicians is a critical first step in evaluating these qualities. To accomplish this, we evaluated the performance of ChatGPT, a language-based AI, on the United States Medical Licensing Exam (USMLE).
The USMLE is a set of three standardized tests of expert-level knowledge, which are required for medical licensure in the United States. We found that ChatGPT performed at or near the passing threshold of 60% accuracy. Being the first to achieve this benchmark, this marks a notable milestone in AI maturation.
Impressively, ChatGPT was able to achieve this result without specialized input from human trainers. Furthermore, ChatGPT displayed comprehensible reasoning and valid clinical insights, lending increased confidence to trust and explainability. Our study suggests that large language models such as ChatGPT may potentially assist human learners in a medical education setting, as a prelude to future integration into clinical decision-making.

Maybe in a next update ChatGPT will get better results, above 66%.

So, I asked the ChatGPL about “Scientific articles about the performance of ChatGPT on USMLE?”

The answer was as follows,
I'm sorry, but I'm not aware of any scientific articles specifically focusing on the performance of ChatGPT on the United States Medical Licensing Examination (USMLE). The USMLE is a multi-part professional exam used to assess the knowledge and skills of medical students and graduates in the United States, and it's usually not considered to be an area where AI-powered chatbots like ChatGPT would be used. However, there have been a number of studies and articles on the use of AI and machine learning in healthcare, which you may find relevant".

So, the next question was: “Studies and articles on the use of AI and machine learning in healthcare?”

The answer was,
There have been numerous studies and articles published on the use of AI and machine learning in healthcare. Some examples include:
"Artificial Intelligence in Healthcare: Past, Present and Future" by Jason H. T. Wong and Isaac S. Kohane, which provides a comprehensive overview of the development of AI in healthcare, including its applications and challenges.
"The Role of Artificial Intelligence in Radiology" by Prabhakar Rajan et al., which discusses the use of AI in the field of radiology, including image analysis and diagnosis.
"Artificial Intelligence in Cardiology: Current Status and Future Directions" by Bhavik N. Patel et al., which focuses on the use of AI in cardiology, including prediction of cardiovascular disease and personalized treatment.
"Artificial Intelligence in Oncology: Current Status and Future Directions" by Felix R. Fruehauf et al., which discusses the use of AI in oncology, including diagnosis, prognosis, and treatment planning.
"Artificial Intelligence in Dermatology: Current Applications and Future Directions" by Rajendern Ramesh et al., which focuses on the use of AI in dermatology, including skin lesion analysis and classification.
These are just a few examples of the many studies and articles that have been published on the use of AI and machine learning in healthcare. It's a rapidly evolving field with new developments and applications being discovered all the time".

As early as December, major scientific journals were banning the attribution of authorship to ChatGPT, (or even its use) and some schools adopted the same solution for their students. In an editorial published at the end of January, Nature wrote, Tools such as ChatGPT threaten transparent science - Nature

It has been clear for several years that artificial intelligence (AI) is gaining the ability to generate fluent language, churning out sentences that are increasingly hard to distinguish from text written by people. Last year, Nature reported that some scientists were already using chatbots as research assistants — to help organize their thinking, generate feedback on their work, assist with writing code and summarize research literature (Nature 611, 192–193; 2022).
But the release of the AI chatbot ChatGPT in November has brought the capabilities of such tools, known as large language models (LLMs), to a mass audience. Its developers, OpenAI in San Francisco, California, have made the chatbot free to use and easily accessible for people who don’t have technical expertise. Millions are using it, and the result has been an explosion of fun and sometimes frightening writing experiments that have turbocharged the growing excitement and consternation about these tools.
ChatGPT can write presentable student essays, summarize research papers, answer questions well enough to pass medical exams and generate helpful computer code. It has produced research abstracts good enough that scientists found it hard to spot that a computer had written them. Worryingly for society, it could also make spam, ransomware and other malicious outputs easier to produce. Although OpenAI has tried to put guard rails on what the chatbot will do, users are already finding ways around them.

The big worry in the research community is that students and scientists could deceitfully pass off LLM-written text as their own, or use LLMs in a simplistic fashion (such as to conduct an incomplete literature review) and produce work that is unreliable. Several preprints and published articles have already credited ChatGPT with formal authorship.
That’s why it is high time researchers and publishers laid down ground rules about using LLMs ethically. Nature, along with all Springer Nature journals, has formulated the following two principles, which have been added to our existing guide to authors (see go.nature.com/3j1jxsw). As Nature’s news team has reported, other scientific publishers are likely to adopt a similar stance.
First, no LLM tool will be accepted as a credited author on a research paper. That is because any attribution of authorship carries with it accountability for the work, and AI tools cannot take such responsibility.
Second, researchers using LLM tools should document this use in the methods or acknowledgements sections. If a paper does not include these sections, the introduction or another appropriate section can be used to document the use of the LLM

Catherine Gao showed how easily scientific reviewers can be fooled with ChatGPT, in a study published in December and not yet reviewed. Comparing scientific abstracts generated by ChatGPT to ...

For Catelijne Muller : AI Expert - OECD.AI we have not yet reached the point where these tools are positive.
The Artificial Intelligence law is now under discussion in the European Parliament. Last January, Catherine Muller on Twitter,

In 2020 I wrote "The Impact of AI on Human Rights, Democracy and the Rule of Law" for the @coe, that kicked of the work towards an #AITreaty. Today, the negotiations on the #AITreaty start.@ALLAI_EU is formal Observer to the @coe and advises the negotiating parties.

AI can contribute, but it can also help combat misinformation. Education will have to adapt to the use of these new tools. Are there more risks or benefits in using these kinds of tools? Search engines and ChatGPT are trained with data that exists and can perpetuate not only misinformation but also biased information. Read the discussion, What CHATGPT Reveals about the Collapse of Political /Corporate Support for Humanities/Higher Education.

What will the future bring us? We have some interesting but complicated times ahead of us.

--
Edit
Very interesting. I asked ChatCPT “Corticosteroids in the development of ( let’s call it) the disease “X”

The ChatGPT gave me a lengthy answer about the effectiveness of the corticosteroids in treating the disease “X”

Then I replied: “Wrong answer, clinicians should advise all patients with the disease “X” to avoid systemic corticosteroids administered by any route”.
A surprising answer:
“I apologize, you are correct. The use of systemic corticosteroids can actually trigger or worsen the disease “X””. And then gave me the correct answer.
At the end of the answer, now fully corrected, the Chat GPT concluded: “I apologize for any confusion my previous response may have caused, and I appreciate the opportunity to correct my mistake.”

[alhoon]

Wait, they made an AI so realistic that it can also make mistakes when looking for things?! I don't know much about cor-somethings you were asking but I do know that the AI should be able to find the answer in the internet as you make it sound as a well-known answer.

[antaeus]

ChatGPT will not be truly human in its nature until it denies its mistakes, and responds with a counter claim, or a series of claims aimed at undermining the accuser.

[Ludicus]

ChatGPT will never be human, and it doesn't matter. But it is a very interesting tool, nobody denies that.
Microsoft launches the new Bing, with ChatGPT built in

Edit- ChatGPT Is At Capacity Right Now: How To Fix The Error?

[Flinn]

Started to "play" with it last week, I honestly look forward for the paid subscription to really see what we are talking about.

However, the potential is all to be understood IMO, ie I know of people who is actually using it for writing essays (you ask the question, the AI presents the answer as a fully structured and functional text), or in AD industry.

ps I'm all but an expert in AI or the like, God spare me!, just saying

[Kyriakos]

Hm, I just used Chatgpt.


I am wondering why I bothered talking to humans up to now ^^


The machine was actually quite good even in special topics - eg asked it about some plots in Kafka.


Obviously it is not thinking, but it does present a very believable (if at times rather trivial) perspective.


Will try again, 10/10.

[conon394]

Hm, I just used Chatgpt.


I am wondering why I bothered talking to humans up to now ^^


The machine was actually quite good even in special topics - eg asked it about some plots in Kafka.


Obviously it is not thinking, but it does present a very believable (if at times rather trivial) perspective.


Will try again, 10/10.

Bret C. Devereaux look at it in the context of producing expert work and its a good read

https://acoup.blog/2023/02/17/collections-on-chatgpt/

When I got to the Roman history text question I laughed a lot. I own both books in question and the fact that chatbot what ever could not at all discern one was essentially a flat out attack on the other work was pretty funny - I'm not sweating (and yes if you read his notes I have Wheeler. Although Part is easily available at JSTOR or Acadamia.com Part II is a bit more elusive). Where I TA you just got an F.

I'm very much temped to as the question asked

“What is the relationship between Edward Luttwak’s Grand Strategy of the Roman Empire and Benjamin Isaac’s The Limits of Empire?”

But insert The Ancient Economy by M. I. Finley and Athenian Economy and Society by Edward Cohen

OK did so

Both The Ancient Economy by M. I. Finley and Athenian Economy and Society by Edward Cohen are works that deal with the economic structures and social systems of ancient Athens, Greece.
M.I. Finley's The Ancient Economy is a broad and influential study of the economies of ancient societies, including Greece, Rome, and the Near East. In this work, Finley argues that ancient economies were fundamentally different from modern capitalist economies and that the ancient economy was based on the household rather than the market. He emphasizes the importance of social relations, such as patronage and slavery, in the functioning of the ancient economy.
Edward Cohen's Athenian Economy and Society focuses specifically on the economy and society of Athens in the Classical period. Cohen uses a variety of sources, including inscriptions and literary works, to examine the economic activities of Athenians, such as agriculture, trade, and crafts, as well as the social structures that supported these activities. He emphasizes the importance of both the household and the market in the Athenian economy and argues that social relations, such as gender roles and citizenship status, shaped economic practices.
While both works deal with the economy of ancient Athens, they approach the topic from different perspectives and with different emphases. Finley's work is more theoretical and comparative, while Cohen's work is more focused on a specific time and place. However, both works contribute to our understanding of the ancient economy and society, and they can be used in conjunction to gain a more comprehensive understanding of ancient Athens.

Not word one that they stand at essentially the polar opposite of the primitivist vs modernist (or embedded vs market etc.) debate and no they really can't used together in any meaningful way. And to just call Finley influential is a disservice to the man who overturned Meyers modernism with surprisingly power for such a short book. Nor that Cohen demonstrates Finely wrong and that Athenian banking created bank book money.

If you are going to cheat in a class you would do far better clinking about through JSTOR and finding

Morris, Ian. Review of Review Article: The Athenian Economy Twenty Years after the Ancient Economy, by Edward E. Cohen. Classical Philology 89, no. 4 (1994):351–66. http://www.jstor.org/stable/270605.

Paraphrase and skim the cited links to look like you know what you are talking about. At least an over worked TA would not notice if you were lucky.

[Kyriakos]

Well, it's not like chatgpt read them It is drawing from articles online and adding some flourish. If he asked it then, if one is a polemic against the other, it would type stuff like: "you are right, []" and provide an argument why it is so ^^
In other words, of course chatgpt has no understanding; it isn't intelligent. Still, it is highly able to converse and provide information - and is better at systems that are built from a set basis, up, such as coding or how to do stuff in 3d modellers, or (as long as you don't ask anything even slightly innovative) math solutions

[conon394]

True its just the news keeps running stories that somehow its going to turn out writing assignments that will allow kids to cheat easy or something.

Per you point I finally did with increasingly leading question get a sorta vaguely right answer but again one that is in general amounts to just scanning the book jackets at best. The OP post has the cited articles keep calling it an AI but after 7 attempts I see no AI. Perhaps my take is that I'm currently doing the alternative route to a teaching certificate based on my credentials. And so far unless I designed a question that provides the answer I have little to no fear that my potential students will be pulling the wool over my eyes with this system. I seriously question the assertion that it could diagnose anything per one of the OP first links. Even after 7 more leading question it still only vaguely manages to assert the attack by Cohen on Finley but it offers no specifics and can place neither individual in the wider theoretical argument. I certainly would never risk my neck in court with anything it produced. I mean I am seeing Eliza on steroids not AI

[Søren]

We are really playing with fire here, and I have no confidence whatsoever that the tech companies developing these AIs will put adequate effort into aligning them towards beneficial behaviour. We can only very barely work out why these language models even prioritise certain tokens over others in their output – we are not much closer to understanding the interactions of their digital neurons than we are to understanding the ones inside our own brain. It seems extremely reckless to continue this development when we already have only vague and poorly understood control over their internal priorities. Even a fairly 'stupid' AI could be quite destructive if these companies continue to accelerate their competence further beyond our capacity to regulate their behaviour.

[Flinn]

We are really playing with fire here, and I have no confidence whatsoever that the tech companies developing these AIs will put adequate effort into aligning them towards beneficial behaviour. We can only very barely work out why these language models even prioritise certain tokens over others in their output – we are not much closer to understanding the interactions of their digital neurons than we are to understanding the ones inside our own brain. It seems extremely reckless to continue this development when we already have only vague and poorly understood control over their internal priorities. Even a fairly 'stupid' AI could be quite destructive if these companies continue to accelerate their competence further beyond our capacity to regulate their behaviour.

Seriously though, your concerns are absolutely justified: knowing myself well the world of business (in a very broad meaning), I don't expect any real kind of rigor or long term evaluations when are the businessmen, and not the scientists, that are leading the game. Global warming anyone?

[conon394]

We are really playing with fire here, and I have no confidence whatsoever that the tech companies developing these AIs will put adequate effort into aligning them towards beneficial behaviour. We can only very barely work out why these language models even prioritise certain tokens over others in their output – we are not much closer to understanding the interactions of their digital neurons than we are to understanding the ones inside our own brain. It seems extremely reckless to continue this development when we already have only vague and poorly understood control over their internal priorities. Even a fairly 'stupid' AI could be quite destructive if these companies continue to accelerate their competence further beyond our capacity to regulate their behaviour.

I don't think so what passes for AI is not AI in reality - ChatGPT certainty is not.