The scientific community is dissatisfied with OpenAI’s secrecy about how and what data the model was trained on, as well as how it works.
OpenAI’s chatbot ChatGPT, GPT-4, unveiled this week. It has the capacity to generate text that appears human-like, images, and computer code from almost any prompt. However, scientists are unable to access the technology, source code, or details about how it was trained, raising questions about its security and its usefulness for research.
The March 14th release of GPT-4 now supports both text and images, and Open AI, a company with headquarters in San Francisco, California, passed the US bar legal exam with scores in the ninetieth percentile. However, the technology is only available to ChatGPT subscribers who have paid for access.
Evi-Anne van Dis, a psychologist at the University of Amsterdam, says that Chatbot GPT-4 is currently on a waiting list, making it impossible to use it right now. However, she has seen demonstrations of its abilities, such as using a hand-drawn website doodle to generate computer code. This demonstrates its capability to handle images as inputs.
The scientific community, on the other hand, is dissatisfied with OpenAI’s secrecy about how and what data the model was trained on, as well as how it works.
OpenAI’s closed-source models are seen as dead-ends in science due to their secrecy and lack of transparency. Sasha Luccioni, a research scientist specializing in climate at HuggingFace, argues that OpenAI can keep building upon their research, but it is a dead end for the community.
Andrew White, a chemical engineer at the University of Rochester, has had special access to GPT-4 as a “red-teamer,” or someone hired by OpenAI to test the platform in order to make it do something bad. He claims that he has had access to GPT-4 for the last six months. According to him, it didn’t seem all that different from previous iterations early on in the process.
He asked the bot about the chemical reactions steps required to make a compound, predicting the reaction yield, and selecting a catalyst. “At first, I wasn’t overly impressed,” White admits.”It was quite surprising because it appeared so real, but it would hallucinate an atom here. It would skip that step “he continues.
Things changed dramatically when he gave GPT-4 access to scientific papers as part of his red-team work. “It made us realise that these models might not be so great on their own. When you connect them to the Internet and use tools like a retrosynthesis planner or a calculator, new kinds of abilities emerge. Concerns accompany those abilities. For example, could GPT-4 allow the production of hazardous chemicals?”
According to White, OpenAI engineers fed back into their model with input from people like him to discourage GPT-4 from creating dangerous, illegal, or damaging content. Another issue is the generation of false data.
According to Luccioni, models like GPT-4, which exist to predict the next word in a sentence, cannot be cured of producing false facts — a condition known as hallucinating. “You can’t trust these models because there’s so much hallucination,” she says. This is still a concern in the latest version, she says, despite OpenAI’s claims that it has improved safety in GPT-4.
Without access to the training data, OpenAI’s assurances of safety fall short for Luccioni. “You have no idea what the data is. So you can’t make it better. It’s simply impossible to conduct science with a model like this “she claims.
The mystery about how GPT-4 was trained is also a concern for van Dis’s colleague at Amsterdam, psychologist Claudi Bockting. “It’s very difficult as a human being to be accountable for something you can’t control,” she says. “One of the concerns is that they could be far more biassed than, say, the bias that humans have.”
Luccioni explains that without access to the code underlying GPT-4, it is impossible to determine where the bias may have originated or to correct it. Bockting and van Dis are also concerned that large tech companies are increasingly controlling these AI systems.
They want to ensure that the technology is thoroughly tested and validated by scientists. “This is also an opportunity because, of course, collaboration with big tech can speed up processes,” she adds.
Van Dis, Bockting, and colleagues argued earlier this year that a set of “living” guidelines to govern how AI and tools like GPT-4 are used and developed is urgently needed. They are concerned that any legislation pertaining to AI technologies will be unable to keep up with the rate of development.
Bockting and van Dis have invited representatives from organisations such as UNESCO‘s science-ethics committee, the Organization for Economic Cooperation and Development, and the World Economic Forum to an invitational summit at the University of Amsterdam on April 11 to discuss these concerns.
Despite the concerns, White believes that Chatbot GPT-4 and its future iterations will shake up science. “I believe it will be a huge infrastructure change in science, similar to how the internet was a big change,” he says. It will not replace scientists, he adds, but it may assist with some tasks. “I believe we will begin to recognise that we can connect papers, data programmes, libraries that we use, computational work, and even robotic experiments.”