ChatGPT: Exposing the Need To Re-Evaluate Our Assessment Strategies

The use of large language models (LLMs), particularly ChatGPT developed by OpenAI, has generated heated discussions about how to evaluate our students’ writing, with many educators seeing this new technology as a threat. These individuals would bring up the concern that ChatGPT's ease of use will lead to a decline in critical thinking and problem-solving skills among students. The tool's impact has been so significant that several schools in Australia, including Victoria, New South Wales, Queensland, Western Australia, and Tasmania (Anyfantis and Kaur) have banned its use, while "New York City Public Schools in the United States banned it in early January” (Johnson). Their concerns are not completely unfounded. There have been several reported instances of ChatGPT being used for cheating. For example, students have been caught using the tool to generate responses for written assignments or essays, and some have even used it to produce answers for multiple-choice questions. In one case, a student was caught using ChatGPT to produce an entire essay on a final exam, leading to “widespread concern about the potential impact of the technology on academic integrity" (Ayfantis and Kaur).

However, simply banning ChatGPT is not a viable option, as it is publicly available and popular among students. Instead, it is important to reconsider assessment methods and ensure they provide students with opportunities to write authentically and critically. While certain shortcuts should be discouraged, ChatGPT could be used to explore and re-evaluate the type of assessments we are providing students.

For example, a recent assignment given to four sections of my Advanced Placement (AP) psychology class pertained to unit eight, clinical psychology, where students were required to watch films depicting mental illness. Viewing selections included: A Beautiful Mind, Joker, The Soloist, The Aviator, Rain Man, and several other prominent films that depict mental illness. The students were prompted to write a synopsis of the film, explain the type of mental disorder portrayed, evaluate both the accuracy of the portrayal of the illness as well as its treatment, and reflect on whether the movie provided an accurate representation of the disorder as well as whether it reduced/increased stigmatization. The assignment in its original form was sourced from the AP psychology Facebook group of which I am a member. I am grateful for the wealth of resources that the group provides to its members, and it is not my intent to criticize or denigrate the original assignment.

By using ChatGPT for this assignment, I discovered that it could be completed within ten minutes by creating a foundation with the tool, then spending five to eight minutes adding original text, and finally rearranging the output via other apps so that it would not be detected by apps such as ZeroGPT.

The screenshot of the original document that ChatGPT created is below:

When I copied and pasted this text into the software, GPTZero. It stated:

The point here is not about how easy or hard it is to deceive ZeroGPT. Rather, going back to my original premise, it was time for me to reflect upon and re-evaluate my own assessment questions and practices.

When I consider Bloom’s taxonomy, I have to wonder: Where would that assignment be about watching a movie related to clinical psychology? Bloom himself stated, "The ultimate aim of education is to cultivate the capacity to engage in autonomous and analytical thinking” (Bloom, 1956). Were these questions I was providing truly testing students' ability in critical thinking, and helping them evaluate and analyze the nuances and complexities of something as complicated as psychological cognitive disorders?

Although certain parts of the assignment do require some level of critique, it has become clear that a majority of the assignment is about the “understand” level and at best, the “apply” level. The initial half of the essay prompts are limited to tasks such as "Give a brief summary" and "Did your character receive treatment for their disorder or symptoms?" Tasks that require little critical thinking.

It is not to say that every assignment that can be completed with the assistance of AI is of poor quality or is one that would be considered as something at the bottom of Bloom's Taxonomy pyramid. However, more often than not, questions which can be readily answered with the aid of AI may be lacking in demanding authentic and original answers. At the very least, we as educators should look at the growth of ChatGPT as an opportunity, an opportunity to reevaluate the assessments we provide, and understand that these technologies are not going away anytime soon.

As educational institutions, ranging from elementary schools to prestigious universities, struggle to come up with policies, it is not my intent to promote or discourage any particular approach, as age groups and demographics can largely affect decision-making in regard to ChatGPT. While it is critical for elementary school students to learn basic skills, such as summarizing information from a book chapter, it is equally important for high school students, such as those studying Lord of the Flies, to comprehend the significance of symbolism in the text, such as Piggy’s glasses or the value of the conch shell. However, what does this mean for our older students in K-12 institutions, who for some, are less than half a year away from entering college? In contrast to K-12 institutions, universities are refusing to impose bans and are instead encouraging professors to establish their own policies, as noted by Ayfantis and Kaur. As an educator who teaches many high school seniors, I recognize that many of my pupils will be in college classrooms in the next four to five months. As such, would I be doing them a disservice by denying them the opportunity to employ this new tool to enhance their learning? This is a question that I cannot answer immediately, but something I must consider. What I do know is that as educators face this issue, we must take into consideration the various advantages and disadvantages and tackle them with flexibility. Given that technology is transforming education in unprecedented ways, and AI and machine learning are increasingly ingrained in our students' learning experiences, it is more crucial than ever that we educators are receptive to utilizing the diverse advantages of technology.

The ever-changing landscape of technology has been and will always disrupt the status quo. This is not the first time technology has been a disrupter, and most certainly will not be the last. When Henry Ford made his Model T affordable for most Americans, he never guessed that a president named Roosevelt would order such an immense restructuring of America’s road systems decades later. Neither can we foresee what effect AI will have in our future. All we can do is watch, learn, and adapt. And that’s something we’ve been doing since our cave days.

References

Anyfantis, Alex, and Harleen Kaur. “Amid ChatGPT Bans in Australian Schools, Educators Consider Risks and Benefits.” SBS Language, Special Broadcasting Service (SBS), 6 Mar. 2023, www.sbs.com.au/language/punjabi/en/podcast-episode/amid-chatgpt-bans-in-australian-schools-educators-consider-risks-and-benefits/sh545nl1e. Accessed 6 Mar. 2023.

Johnson, Arianna. “ChatGPT in Schools: Here’s Where It’s Banned—and How It Could Potentially Help Students.” Forbes, Forbes, 18 Jan. 2023, www.forbes.com/sites/ariannajohnson/2023/01/18/chatgpt-in-schools-heres-where-its-banned-and-how-it-could-potentially-help-students/?sh=2ca6cff56e2c. Accessed 6 Mar. 2023.

Bloom, Benjamin S. Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. David McKay Company, Inc., 1956.

Okrent, Daniel. Last Call: The Rise and Fall of Prohibition. Scribner, 2010.

Source: Common Core State Standards Initiative. "Appendix B: Text Exemplars and Sample Performance Tasks." Common Core State Standards for English Language Arts and Literacy in History/Social Studies, Science, and Technical Subjects. http://www.corestandards.org/assets/Appendix_B.pdf. Accessed 6 Mar. 2023.

Drew, Chris. All 6 Levels of Understanding (on Bloom’s Taxonomy)., 2023, helpfulprofessor.com/wp-content/uploads/2022/09/blooms-taxonomy.jpg. Accessed 6 Mar. 2023.

--------------------------------------------------------------
Nickolas Hironao Harris teaches Advanced Placement psychology and Advanced Placement research at Korea International School Jeju in South Korea.

Website: www.nickharrisjapan.com
LinkedIn: Nickolas Hironao Harris