An artificial intelligence (AI) chatbot, Google Gemini, generated answer to “what is AI?” prompt is “the development of computer systems capable of performing tasks that typically require human intelligence. These tasks include learning from data, recognizing patterns, understanding language, and making decisions.” At the end of the definition, it was written in small fonts that “AI can make mistakes, so double-check responses.”
AI in biomedical research
Last month, Oxford University in the UK published findings of a human trial of a PanSarbeco vaccine, which was designed by computational method – a form of AI – to act against all variants of SARS-CoV-2 virus – the causal agent of Covid-19 – and related viruses. Conventionally, the whole process would have taken eons to accomplish. The researchers utilised the large amount of existing information on sarbecoviruses, the genus to which SARS-CoV-2 belong, to detect certain patterns or immunologically relevant sites and predict a vaccine candidate to be effective against nearly all sarbecoviruses. As a double-check step, the researchers tested their design (or prediction) firstly in mice, rabbits and guinea pigs to generate data on the safety and efficacy. Subsequently, they tested in human volunteers and observed encouraging results. Computational biologists have been using such large data analysis, pattern recognition and prediction of structure or function of biological compounds for decades now. In fact, the 2024 Nobel Prize in Chemistry was awarded to David Baker, one half of the prize, “for computational protein design” and remaining half to Demis Hassabis and John Jumper “for protein structure prediction”.
Pitfalls
AI, in general, has accelerated research, enhanced efficiency and improved delivery for scientists and academics but the danger of compromised quality always looms large. In academics, AI is a double-edged sword because it can make or break a career. According to a letter published in The Lancet in May this year, at least 57 per 10,000 biomedical research papers – out of an audit of a total number of around 25 lakh papers published during 2023-2026 – have “hallucinated” references generated by large language models, the algorithms that power AI. The pitfall of this is unauthentic scientific information becoming part of the evidence-based biomedical literature. Based on an analysis of biomedical research papers from 1975 to 2024, Korean Journal of Medical Science last December revealed that the highest retraction rate of around 6% was in the journal, Computational and Mathematical Methods in Medicine, while papers on oncology as a subfield had around 20% retractions. Retraction is the takedown of published research papers, forfeiting the academic benefits and causing dent in the reputation of the concerned researcher(s) as well as their institute. Another analysis published in the same journal revealed that “unethical use of AI” caused one-third of the retractions up to last November among all the records available on PubMed, an aggregator of biomedical research publications. This month witnessed the Supreme Court (SC) of India set aside an order of the National Company Law Tribunal (NCLT) consisting of references to “hallucinated” judicial precedents, i.e., non-existent AI-generated cases. The SC called it catastrophic to judicial process.
‘Un-academic’ use of AI
Teachers are masters of their subjects because of their knowledge accumulated over decades of learning, un-learning, re-learning and dissemination of knowledge. Human intelligence is the ability to recall facts and figures from memory combined with problem-solving skills acquired through practice to use effectively as and when required. This ability also shapes human wisdom. AI uses all the available information – validated and unvalidated – in the digital world to search for the most likely answer to a query. To acquire skills of doing certain tasks, AI trainers in India are nowadays producing video footages of daily activities of humans, as reported by Al Jazeera last month. This trend is going to take us closer to the AI doomsday predicted by the controversial philosopher and historian, Yuval Noah Harari. He predicted that AI was on the brink of acquiring wisdom and becoming self-reliant, as humans have used all their wisdom to train AI to make it think and command itself to do a task successfully. No technology in human history had become so much intelligent on their own as AI is today. Printing machine, electricity, weapons of mass destruction, none of them can function without a human handler, but AI is about to break free from its handler(s). Grok, an AI chatbot developed by Elon Musk’s company xAI, smartly fakes and fabricates responses to prompts. Is AI heading towards becoming another Frankenstein’s monster?
Increasingly teachers and students are relying on AI for academic activities. The trend is growing so fast that the use of human intelligence in academic activities is going to take the backseat very soon. The UNESCO had taken note of the trend and published a guideline for education policy-makers in 2021, but a survey by the same body released in last November found that AI-related ethical issues had been reported by 100 of the 400 respondents across 90 countries. Amidst this, an interesting but worrying case happened in May last year. The New York Times reported that a student was unhappy about the use of AI by her professor at Northeastern University in the U.S., prompting her to demand a refund of the tuition fee. What was noteworthy about the claim made by the student was that the teaching notes had misspellings, distorted fonts and images had extra fingers. The demand was eventually rejected by the university as the university’s policy allowed the use of AI with proper credits. Uncritical use of AI reveals itself through AI blind spots like the lecture notes of the Northeastern University professor. In the U.S., many schools banned ChatGPT, an AI chatbot, in 2023 due to concerns of poor learning outcomes and cheating in exams only to be repealed few months later to evolve and integrate AI in classroom.
The Fudan University example
According to a Bloomberg report last month, planning fallacy of Ford – an automobile manufacturing company in the U.S. – led to re-hiring of around 300 quality inspectors who were sacked earlier following AI deployment for quality checks. The reason for re-hiring was failure in ensuring quality by the AI-driven checks, i.e., AI blind spots. In the face of this development came the news of a unique way of conducting exam last week at the college of computer science and artificial intelligence of Fudan University in China. The exam asked students to find blind spots in AI models – Claude, DeepSeek and MiniMax. The grade was assigned based on how many AI models could a student outsmart using ten questions. Out of 51 students, 4 students could fail all AI models on all questions. It is also worrying that 47 students were outsmarted by the AI models. Nevertheless, this initiative should help shape the critical use of AI in academics in future unlike uncritical use in the flow it decides. Higher education institutes in India must evolve their own AI integration policies. India is yearning to become an AI powerhouse. This calls for urgent regulations on AI integration across professions. Irrespective of this, we must not forget to doubt and double-check the AI.
The writer is an Assistant Professor % Group Leader, Biochemistry and Molecular Biology Lab, Department of Biotechnology, Gauhati University, Guwahati. He may be contacted at nohimbo@gmail.com)
