History
In 2015, a group of entrepreneurs and companies in Silicon Valley pooled together a $1 billion fund for the formation of OpenAI, an organization dedicated to researching and developing Artificial Intelligence (AI) technology, “to ensure that artificial general intelligence benefits all of humanity.”
One of OpenAI’s flagship projects is the Generative Pre-trained Transformer (GPT), which is now on its 3rd version (GPT-3). GPT is a language-processing AI architecture that solves one of the AI field’s long standing problems, Natural Language Processing (NLP). NLP is a class of problems that require an AI to be able to correctly interpret language, from analyzing meaning and connotations to generating texts based on questions. GPT-3 is the largest iteration to date, with 175 billion parameters that need to be trained by inputting truly vast amounts of data from across the internet.
ChatGPT is a chatbot built using the latest GPT-3 version, GPT-3.5. It is accessible to the general public in a free research preview. It can answer a variety of questions asked through it, although it does have some limitations.
Technical Details
GPT-3, the model that ChatGPT is based on, uses a transformer architecture. Transformers are a type of AI model uniquely well-suited to do NLP, selectively weights different parts of the previous data (the parts of the sentence before the current location) differently according to a self-attention metric. This allows transformer models to only focus on the parts of the sentence that influence the meaning of the current text token. OpenAI’s innovation with the development of GPT-3 was the sheer number of parameters involved (175 billion), with 10x more parameters than the previous standard.
GPT-3 was pre-trained on a large database of text, then it can be fine-tuned to adjust to the specifics of the particular task at hand (answering human-formulated questions, in the case of ChatGPT). OpenAI specifically trained GPT-3 on the Common Crawl dataset, a culmination of years of bots collecting data from the web amassed into a single petabyte-sized dataset of text.
Application
In the words of ChatGPT itself, the prototype chatbot is primarily used for “answering questions, providing information, translating languages, and engaging in conversation” as a user-friendly virtual assistant. On top of answering your complex questions with surgical precision, the model is constantly learning, processing, and evolving as it accumulates more response data.
Perhaps one of ChatGPT’s most impressive features is its ability to write and debug code seamlessly. At the user’s request, the AI can synthesize entry-level code, specified by programming language, and also offer suggestions for troubleshooting errors. For providing general information or explaining programming concepts, the general-purpose language model is well-equipped to provide accessible, comprehensible answers.
Controversy
With the vast skill set of ChatGPT, many schools around the country are wary of foul play. In New York City, school districts are already beginning to restrict access to OpenAI out of fear of cheating. As an automated writing tool, the chatbot is capable of answering essay questions and homework questions, albeit with certain limitations. Students can easily use ChatGPT to forge well-written essays that can pass as original work.
Considering that ChatGPT is, by and large, a language model, can it really be considered a threat to academic integrity? In the words of OpenAI Sam Altman: “ChatGPT is incredibly limited, but good enough at some things to create a misleading impression of greatness.”
Ultimately, neglecting the limitations of ChatGPT to take short-cuts will prove to be a pitfall; insufficient answers may be synthesized and bad habits will undoubtedly be reinforced. Students overly reliant on ChatGPT could lose essential skills. Yet, regardless of these limitations, with the promising future of ChatGPT and AI in general, ChatGPT could signify a permanent change in academia’s relationship with technology.
Sources
