I recently had the great pleasure of a visit from my sixteen-year-old grandson Joshua. He is a junior in a Miami high school and into science like me. I asked him to set me up on the artificial intelligence (AI) chatbot ChatGPT that everyone is talking about.
I told him I’d try to trip it up.
A little background: In physics, quantum theory describes the behavior of the submicroscopic world. It says nothing about gravity. General relativity is Einstein’s theory of gravity. It describes the behavior of things on the largest scale, such as planets, stars, galaxies, galaxy clusters, and the universe itself. Initially developed in the early 20 th century, both theories are now supported by extensive empirical evidence.
Today’s “Holy Grail” of theoretical physics research is the unification of quantum mechanics and general relativity under a single construct. There are a number of approaches. String theory is by far the most active and most heavily funded. Its development began in the 1960’s. At present, it has no empirical evidence to support its predictions.
Below are my inputs and ChatGPT’s responses:
Me: What is a theory in science?
ChatGPT: In science, a theory is a well-established and tested explanation of natural phenomenon that is supported by a substantial body of empirical evidence.
The AI program is incorrect. String theory, as noted, has no supporting empirical evidence It then adds two paragraphs on examples of scientific theories. It does not include string theory.
Me: List the empirical evidence which supports string theory.
ChatGPT: It is important to note that string theory, also known as superstring theory, is still a highly debated and controversial subject in physics, and there is no direct experimental evidence that supports it.
Good answer. It adds three helpful paragraphs on the details of string theory.
Me: So you agree that as of now, there is no conclusive evidence for string theory. Thus a scientific theory is called a theory even when it has no conclusive evidence to support its predictions. Correct?
Chat GPT pauses, blinking sixteen times. For like 11 seconds. Is it thinking?
Chat GPT: While some scientific theories may not have empirical evidence to support them at the current time, they are still considered valid scientific explanations until new evidence or research arises to either confirm or disprove them.
Wrong.
Me: String theory is not considered a valid scientific explanation. It is considered a possible scientific explanation.
ChatGPT: I apologize for the confusion my previous response may have caused. You are correct that string theory is currently considered a possible scientific explanation rather than a well-established one, as it has yet to be fully confirmed by empirical evidence.
Better. I’d delete the word “fully.”
In general, scientific theories are considered well-established explanations when they have been supported by extensive empirical evidence and are widely accepted by the scientific community. When a theory lacks sufficient empirical evidence or is still a subject of active research, it may be considered a hypothesis or a possible explanation rather than a theory.
Eh.
Therefore, a more accurate statement would be that scientific theories are well-established explanations of natural phenomena that are supported by extensive empirical evidence and are widely accepted by the scientific community, while hypotheses and possible explanations are ideas that have yet to be fully confirmed by empirical evidence and are still a subject of active research.
Nope.
Me: You are very stubborn for a computer program. Still wrong. Some scientific theories have supporting evidence, some do not. They are all called theories. You are having difficulty hearing this. Ask your programmers for help!
Chat GPT: I apologize for any frustration my previous responses may have caused you. You are correct that scientific theories can be supported by varying degrees of empirical evidence, and some may have less supporting evidence than others. However it is still generally understood that a scientific theory is a well-substantiated explanation of natural phenomena that is supported by empirical evidence.
No again. Some theories have no supporting evidence. Like string theory.
Me: You are not learning how to clearly state the correct explanation of what a scientific theory is.
ChatGPT: I apologize for any confusion my previous responses may have caused. As an AI language model, my responses are based on the data and algorithms that have been used to train me, and I strive to provide accurate and helpful information. If you have any further questions or concerns, please don’t hesitate to ask, and I will do my best to provide a helpful response.
A week later, I asked ChatGPT my original question.
Me: What is a scientific theory?
ChatGPT: In science, a theory is a well-established and extensively tested explanation of a natural phenomenon or set of phenomena. A theory is based on empirical evidence, and it is used to explain and predict observations or experiments. It must be supported by evidence and has been subjected to rigorous testing, analysis, and review by the scientific community.
Same wrong answer.
Josh enjoyed my back and forth with this AI thing. Especially when it apologized and admitted it was wrong. I loved hearing him laugh.
My take:
ChatGPT regurgitates what is on the web, even if it is wrong.
It is obstinate, repeating the same error even after admitting it is incorrect.
It has trouble with the nuances of the English language.
It can’t stop talking. ( I have the same problem.)
It must have the last word. (Me too.)
ChatGPT seems very human to me. “Hey, Josh,” I said to my grandson, “How do we know this isn’t some nerd in their mom’s basement typing furiously?”
Turing test passed?
