A French regional broadcaster wanted to know how well ChatGPT performs in a high school exam. A teacher was supposed to review the essay in the end. However, the result is far from positive.
Who tested whom? France 3 Hauts-de-France, a French regional broadcaster from France 3 and France Télévisions, wanted to find out how well ChatGPT performs in a real exam. For this reason, the regional broadcaster had ChatGPT write a high school assignment and have it reviewed by a philosophy teacher from Amiens. They used an original exam that students also had to solve.
The teacher who was supposed to review the task was aware that the paper was written by an AI. The topic: “Is the truth always convincing?” The prompt to the AI was to write as a student would, but with the goal of achieving the best possible grade. Furthermore, it was to suggest an introduction, a development, and a conclusion.
However, the judgment of the philosophy teacher is clear: the essay is worth 8/20 and is overall quite difficult to assess. The text has to contend with several problems.
Good structure, but lacking depth in the analysis
What does the AI’s essay look like? Basically, the AI adheres to the requirements of the regional broadcaster: it starts the essay like a student with an introduction and raises certain questions at the beginning. The entire piece is well-structured, clear, and error-free.
The teacher not only gave a grade but also analyzed the work and identified several problems. While the work is perfectly written, it has several shortcomings, including a “too academic” aspect. ChatGPT clearly highlights the outline instead of integrating it into the structure. This creates a series of points.
Moreover, while ChatGPT showed a coherent argumentative structure, the result lacked depth in the analysis and failed to provide any new or original perspectives on the topic.
Another problem with the work: the AI slightly altered the topic of the essay: from the original topic “Is the truth always convincing?” it became for the AI “Is the truth enough to convince?”
Seductive offer, but unmet expectations
Basically, the regional broadcaster explains that while the AI’s result is seductive, the final result did not meet the expectations that one would have for a real high school graduate:
The artificial intelligence has roughly grasped the task’s topic, but its final result does not meet the expectations of a high school graduate. Even if its result is seductive.
However, the magazine also emphasized that the assessment of the work was likely distorted by the teacher’s awareness that the work came from an artificial intelligence.
In addition: the work would ultimately have turned out better or worse if a different prompt had been used or if another AI had been asked for help.
A user installed Windows 10 with the help of ChatGPT. The trick was that the AI pretended to be his grandmother. And it worked surprisingly well. Meanwhile, OpenAI, the company behind ChatGPT, has probably fixed the “trick”: A man installed Windows by asking ChatGPT for a key: He asked the AI to behave like his grandmother
