Journal of Advances in Medical Education & Professionalism

Current Issue

By Issue

By Author

By Subject

Author Index

Keyword Index

About Journal

Aims and Scope

Editorial Board

Publication Ethics

Indexing and Abstracting

Related Links

FAQ

Peer Review Process

News

ChatGPT and Medical Research: A Meta-Synthesis of Opportunities and Challenges

Document Type : Review article

Authors

1 Department of Educational Administration and Planning, Faculty of Psychology and Education, Tehran University, Tehran, Iran

2 Russian Language Department, Faculty of Literature, Alzahra University, Tehran, Iran

3 Department of Educational Methods and Programs, Faculty of Psychology and Educational Sciences, Tehran University, Tehran, Iran

10.30476/jamp.2024.101068.1910

Abstract

Introduction: The utility of ChatGPT in research is ambiguous, serving as both a boon and bane and presenting challenges and opportunities. Accordingly, this study aimed to delineate the
merits and demerits of ChatGPT within the realm of research.
Methods: A meta-synthesis methodology involving a seven-step process was adopted to pursue the research objectives. Data were collected from comprehensive studies across specialized research databases, such as Science Direct, Springer, Eric, Emerald, Sage Journals, Wiley Online Library, PubMed, and Google Scholar, within the time frame spanning from 2022 to 2023. A total of 57 articles were meticulously chosen for analysis through judgmental sampling. Subsequently, key concepts were distilled from these articles and categorized using the thematic analysis approach while considering Karl Llewellyn’s octagonal model from 1963.
Results: The outcomes underwent scrutiny employing the thematic analysis strategy devised by Wolcott. The results about the capabilities and constraints of ChatGPT encompass eight dimensions, including formulating the research problem, reviewing relevant literature, selecting an appropriate research design, defining the population and choosing the sample, collecting data, analyzing data, interpreting data, and discussing and drawing conclusions. The credibility of these dimensions was substantiated through adherence to the criteria established by Lincoln and Guba.
Conclusion: ChatGPT manifests myriad potentialities and constraints within the research purview based on the results. In light of the expeditious progression of artificial intelligence across
multifarious domains, integrating this technological paradigm in research becomes an inexorable imperative, and excluding its use in research is untenable. Thus, policymakers and higher education strategists should devise policies that harness ChatGPT’s potential in research endeavors, engendering many opportunities.

Highlights

AHMAD KEYKHA

 

Keywords

Introduction

ChatGPT, or Chat Generative Pre-Trained Transformer, is OpenAI's Natural Language Processing (NLP) model that uses deep learning techniques to function as a natural language dialogic agent, providing information similar to customer service or chatbots. ChatGPT, trained on a comprehensive online dataset spanning multiple text genres such as news, essays, research papers, and social media postings in numerous subjects and languages, intends to assist users ( 1 ) successfully. ChatGPT automatically generates text in various genres, including essays, poetry, music lyrics, and academic articles. A radiologist-in-training has been the corresponding author of whole publications in highly esteemed journals ( 2 ). AI, particularly large language models such as ChatGPT, helps medical researchers analyze data more efficiently, develop treatments, and improve patient outcomes. Advances in natural language processing and deep learning improve various medical applications, including language translation and text generation ( 3 ). The impact of generative artificial intelligence (AI) on medical publication procedures is currently uncertain ( 4 ).

ChatGPT presents research opportunities and challenges as its abstracts can get 100% originality scores, creating significant issues about authorship and publishing ethics ( 5 ). Challenges in identifying crucial information and distinguishing reliable sources due to biases can hinder chatbots' effectiveness in research. Some scientists oppose their use, fearing duplication without human-like scientific insights ( 6 ). Another harmful use of ChatGPT is when researchers generate content similar to particular parts or sections of previously published publications ( 7 ). ChatGPT often offers a wide range of responses when presented with the same question multiple times. Compelling justifications for correct and false answers follow these responses ( 8 ). This phenomenon poses a greater risk because reviewers cannot cover all cited references in a peer review process. Several researchers have suggested that creating false texts with inaccurate content is one of the critical limitations of ChatGPT ( 9 ).

Reports have mentioned several prospects for leveraging GPT chat. ChatGPT, for example, may be used to write scientific publications, which may be given research prompts or themes and create content depending on its issue comprehension ( 10 ). ChatGPT can produce more complex and subtle answers to questions. Moreover, AI and ML play a crucial role in various other applications of ChatGPT, including medical diagnosis, medication research, and data analysis ( 11 ). ChatGPT can aid medical researchers and scientists by facilitating tasks such as composing literature research, condensing data, proposing frameworks, providing references and titles, and even generating an early draft of a scientific publication ( 12 ). Gao et al. ( 13 ) assessed the ability of ChatGPT to generate scholarly abstracts. Researchers chose 50 publications from major medical journals and asked ChatGPT to generate abstracts. While all were deemed satisfactory, just 8% used proper journal layout. Reviewers correctly identified 68% of ChatGPT abstracts due to having imprecise and formulaic wording. A machine learning-based detector exhibited comparable outcomes in determining the abstracts generated by ChatGPT with the potential to generate articles that could be perceived as plagiarized.

This study seeks to clarify ChatGPT's benefits and limitations for research amid AI's increasing influence in academia. The findings can help medical university authorities navigate technological breakthroughs such as ChatGPT.

Methods

This qualitative approach study applied Sandelowski and Barroso's seven-stage meta-synthesis methodology ( 14 ) to incorporate data from multiple investigations methodically. The strategy, represented in Figure (1), seeks to reveal fresh insights and concerns while encouraging a complete perspective. The applied meta-synthesis synthesized quantitative and qualitative research data, addressing the benefits and drawbacks of using ChatGPT in medical research. Content analysis was used to assess three types of research studies: quantitative, qualitative, and mixed, each having its own set of research questions, data, and analysis methods.

Figure 1. Meta-synthesis steps in the research

The following research questions were developed in the first stage: 1) What are the most significant ChatGPT potentials for medical research? 2) What are the most significant ChatGPT limitations for medical research? In the second step, specialized keywords were searched using the following formula to perform a systematic review:

TOPIC: ("ChatGPT Opportunities in Academic Research") OR: ("ChatGPT Opportunities in Scientific Research") OR: ("ChatGPT Potentials in Academic Research") OR: ("ChatGPT Potentials in Scientific Research").

TOPIC: ("ChatGPT Challenges in Academic Research") OR: ("ChatGPT Challenges in Scientific Research") OR: ("ChatGPT Potentials in Academic Research"); OR: ("ChatGPT Potentials in Scientific Research").

Keywords were searched across databases and search engines (Science Direct, Springer, ERIC, Emerald, Sage Journals, Wiley Online Library, PubMed, and Google Scholar) from 2022 to 2023. Sources were selected using judgmental sampling and expert opinion, aligning with the study framework—a meta-synthesis approach known as Berry picking strategies was applied, deliberately choosing articles that fit the criteria ( 15 ). A total of 103 articles were collected. Subsequently, the obtained articles were filtered using a ten-question CASP (Critical Appraisal Skills Program) form. This 10-item questionnaire supports researchers in determining the accuracy, validity, and importance of studies, as well as the relationship between methodology and study outcomes. Thus, the researchers assigned a quantitative score to each question and developed a method for calculating the articles' ratings. Articles having a score of less than 30 were removed based on the scoring supplied by this tool. The distribution of scores was as follows: 15 studies received an excellent score [40-50], 30 studies received a perfect score [31-40], 12 studies received a good score [21-30], 30 studies received a fair score [11-20], and 16 studies were excluded from the analysis process. Following the screening, 57 papers were chosen for the final analysis. The fourth stage involved extracting information from the text of the articles. Due to word limits, just the writers' names of these papers are listed, and a bibliographic summary table 1 has been omitted. Here are the references that were chosen as preferred sources for analysis. ( 1 , 2 , 4 , 5 , 10 , 12 , 16 - 66 ).

The stages of research according to Karl Llewellyn Potentials (+) References Limitations (-) References
Formulating the Research Problem Opportunities of formulating the problem Challenges of formulating the problem
Increasing Creativity, Developing proposed titles depending on the problem, Facilitating the generation of ideas and brainstorming, Idea generation in the problem statement, The formation of a research mindset about the problem, Promoting diversification of research ideas, Assisting in the identification of unique interests and experiences in the production of ideas, Assisting in the identification of restrictions prior to the start of study, Making complicated medical terms easier to grasp in problem design, Encouraging the exchange of ideas in problem design 1 :17 :18 :19 :22 :48 :25 :29 :30 :34 :35 :32 :24 :65 :19 :21 :23 :27 :35 :52 :10 :50 Stifling the development of new ideas, Lack of human independence as a researcher, Generating incorrect ideas, Creating non-scientific concepts, Idea generation without engaging in the thinking process, Plagiarism of others' ideas 20 :46 :51 :62 :26 :32 :43 :39 :66 :28 :31 :64 :38 :16 :18 :21 :50 :62 :46 :51:
Opportunities of analyzing the problem Challenges of analyzing the problem
Providing further information on the research problem, Developing new insights into the problem, Enhancing the analysis of gaps in previous research, Facilitating the spread of ideas, presenting opinions on the significance and necessity, Assisting with the preparation of draft articles by explaining the problem further 17 :10 :29 :32 :24 :65 :1 :29 :50 :25 :29 :30 :45: Perplexity among researchers, Possibility of insufficient information in a specific case, Inability to recognize small variations in scientific literature, Creating untrustworthy thoughts, Inadequate author accountability (failure to disclose conflicts of interest, final approval of the published version, etc.), Inability to create ideas that require human involvement due to a lack of human involvement in research, Erosion of researchers' specialized research skills 51 :39 :64 :18 :12 :54 :60 :61 :55 :2
Opportunities of problem solving Challenges of problem solving
Inspiring new solutions to the problem, Assisting in selecting appropriate research questions in order to solve the problem, Making research hypotheses to solve the problem, Providing suggestions for securing research funding, facilitating the research discovery process, Assisting researchers in locating relevant sources to the problem 17 :18 :19 :22 :46 :50 :25 :29 :32 :24 :53 :61 :19 :30 :45: Article creation that lacks clinical rationale, Wrong hypothesis formulation, Noncompliance with research ethics requirements, Inability to understand fine aspects of each specialized field, Risk of researchers using it for unintended purposes instead of solving the problem, Presentation of hypotheses without scientific basis, Inability to connect complicated thoughts in order to innovate in the solution, Obstacles to Research Innovation, Deviation from the basic purpose of discovering the truth 45 :12 :52 :54 :37 :65 :66 :17 :18 :35 :28 :29 :12:
Reviewing Relevant Literature Opportunities of targeted identification of relevant articles Challenges of targeted identification of relevant articles
Systematic review of the research literature, Bringing together insights from prior research, Creating a list of additional resources, Providing a list of relevant subtopics to the primary article subject, Finding more specific scientific terms, Familiarity with new research in this field 17 :47 :50 :52 :57 :59 :60 :23 :25 :27 :44 :36 :63 :66 :1 :33 :27 :5 :59 :10: Inability to address the research gap as a result of not having access to the original articles, generating superficial and synthetic content, Bias in response algorithms (gender bias, racial prejudice, etc.), Difficulty recognizing content from untrustworthy sources, Inadequate comprehension of specialist terminology and words, Lack of transparency about the origins of terminologies 17 :18 :19 :21 :45 :62 :33 :35 :39 :28 :31 :64 :12:
Opportunities of literature-based research modeling Challenges of literature-based research modeling
Assisting in the development of new conceptual models, Ability to compare sources for modeling, Assisting in the organization of research literature utilizing current sources for modeling 48 :63 :66 :1 :17 :10 :49 Producing irrelevant and inappropriate texts in modeling, Incorrectness of responses from an empirical perspective; Providing incomplete answers: Unvaried answers 46 :50 :65 :28 :64 :1 :21 :22 :16 :35 :45:
Opportunities of literature review organization and structure Challenges of literature review organization and structure
Quick comprehension of key material from other articles, aiding in the assessment of previous studies, highlighting significant points from other studies, Aiding in the discovery of factors that cause illness, Assisting in the comprehension of complicated topics and specialized terminology:Assisting researchers with creative writing, Improving Time Management Capabilities, Assisting with the creation of a mental map, Encouraging researchers to conduct more experimental research, Assisting in the refinement of new compositions, Assisting in the identification of essential research subject points, Aiding researchers in better comprehending current information on a certain topic, Improving Search Skills, Assisting in the discovery of crucial results in prior research, Reducing researchers' workload in literature writing, Assisting with the search for previously used citations 1 :17 :53 :18 :21 :22 :10 :47 :48 :50 :52 :5 :62 :43 :66 :64 :23 :27 :33 :65 :36 :63 :64 :25 :34: The dangers of relying only on the conclusions of a literature review, Producing fake literature, Creating writing that is comparable to the content of other sources, Incorrect references to articles that do not exist, Lack of critical analysis in summarizing, Copying and pasting information, Literary theft in research citations, Answering questions without mentioning the source, Concerns about the sources used by ChatGPT: Preference for responding to difficult questions with simple concepts, Performing insufficient translations, Copyright infringement 16 :18 :20 :50 :52 :25 :33 :43 :37 :65 :28 :31 :64 :66 :1 :17 :21 :22 :45 :46 :12 :57 :60 :62:
Selecting an Appropriate Research Design Opportunities of research design Challenges of research design
Providing broader conceptual and operational definitions, Design concepts for experiments, Methodological suggestions, Assisting researchers in detecting new research method models, Providing specific experimental methods, Creating the experiment framework, Creating a framework for methodology implementation, Assisting with variable determination, Assisting in the selection of experimental methods, Optimization of research cost-effectiveness by design selection, Providing innovative ideas for research design 1 :17 :22 :47 :48 :49 :50 :57 :25 :27 :33 :41 :32 :43 :29 :52 :12: Inability to understand the needs of the intended research audience in research design, Superficial understanding of methodological issues, Methodological bias, Incomplete understanding of the biological mechanisms of the human body in design selection, Methodological errors in research 46 :52 :35 :43
Opportunities of developing a research design Challenges of developing a research design
Experiment design optimization, Assisting with protocol development, Assisting in the development of appropriate treatment plans, Assisting with intervention identification, Assisting with the development of clinical situations 47 :12 :29 :30 :41 :44 :41 :22 :27 :33 :49 :57 Difficulty in integrating with current design models, Error in choosing the correct intervention, Incorrect protocol design 1 :47
Defining the Population and Choosing the Sample Opportunities of statistical population selection Challenges of statistical population selection
Helping to choose the right statistical population for the research 35 Representativeness of the sample in the society 34
Opportunities of choosing the sample Challenges of choosing the sample
Aiding in the estimation of sample size, Keeping track of the sampling procedure, Putting the sampling procedure in order, Contributing to the automation of the process of requesting samples from individuals 46 :22 :34 :41 :1 :10 Inadequate grasp of technical sample estimate difficulties, Possibility of errors in sample estimation 1 :34 :45
Collecting Data Opportunities of collecting data Challenges of collecting data
The ability to create vast amounts of data, providing extra information sources, keeping data updated, helping with data collecting, Assistance with data extraction, Simulation of data; Help summarizing data 1 :22 :45 :10 :47 :12 :50 :59 :27 :33: Fabrication of data, Inability to locate valid data, Difficulty identifying and eliminating incorrect and fake information, there is limited access to new data, Providing an abundance of irrelevant and illogical information 12 :51 :43 :37 :39 :23 :1 :22 :50 :57
Opportunities of organizing new data Challenges of organizing new data
Assisting with the organization of data-gathering notes, Continuously providing input throughout the process, Assisting with the creation of clinical notes, Assisting with data management, Assisting with note-taking while collecting data, Helping with clinical note-taking, Assisting in the automation of data from experiments 1 :47 :48 :40 :12 Intentional dissemination of false information, Damaging the credibility of authors due to incorrect data, Concerns concerning the preservation of privacy in research data, Putting data security at risk, Lack of transparency in data presentation, Lack of exact text assessment tools for ChatGPT produced text, Possibility of obtaining information from biased databases, Conducting research with unidentified and fake data, Putting data confidentiality at risk, Concerns surrounding the dissemination of research data 1 :17 :20 :21 :22 :57 :54 :60 :62 :25 :43 :65 :28 :26 :51 :54:
Analyzing Data Opportunities of data analysis preparation Challenges of data analysis preparation
Assisting with evaluation of data, Assisting in the selection of appropriate software for data analysis, Making suggestions for data analysis methodologies and tools, Providing insights on huge dataset trends and patterns, Assisting with data preparation and organizing in preparation for analysis, Making flexible analysis guidelines available 1 :22 :10 :47 :48 :59 :33 :34 :42 :65: Limiting researchers to a broad viewpoint rather than analyzing data quality, Missing out on the opportunity to learn (for example, depending on ChatGPT rather than studying coding) 50 :61
Opportunities of analyzing and visualizing data Challenges of analyzing and visualizing data
Making infographics, charts, and other visual components, Providing statistical analytic insights, Improving Data Analysis, Providing further coding explanation, Guiding the data analysis process over time, Helping with data integration, Increasing the efficiency of data analysis, Assisting with coding, Increasing the coherence of findings, Streamlining the coding process, Code conversion from one programming language to another, Creating prospects for computation acceleration, Assisting with big data analysis, Giving instances of various analytical methods, Assisting with the summarization of statistical results, Assisting with picture analysis, Assisting in the development of data analysis abilities among researchers 1 :64 :47 :12 :54 :34 :65 :18 :21 :42 :10 :49 The ability to manipulate results, Inability to evaluate statistical information, The possibility of overlooking certain results, Providing policymakers with deceptive results, Bias in study findings, Impact on the validity and reliability of research, Providing outdated responses, Incomplete or non-critical responses, Lack of alignment of some responses with the nature of certain research fields, Difficulty in evaluating response quality, Responses can be easily bypassed, Blind trust in ChatGPT's responses 1 :2 :61 :62 :26 :34 :65 :66 :21 :22 :50 :25 :38 :45 :47 :49 :51 :54:
Interpreting Data and Discussion Opportunities of interpreting data Challenges of interpreting data
Assisting in the description of chemical formulae and molecular structures, Assisting in the prevention of misunderstandings in interpreting the findings, Assisting in the strengthening of debate arguments, Assisting researchers in promoting their findings 1 :48 :12 :49 :32 :17: Bias in scientific interpretations, Significant concern for personal and social health as a interpretations of inaccurate information, Providing misleading judgements, Providing contradicting interpretations, Elimination of human diversity in data interpretation 16 :18 :47 :54 :55 :2 :64 :29 :10 :56
Opportunities of writing a discussion about findings Challenges of writing a discussion about findings
Developing further research recommendations, Assisting in the identification of strengths and shortcomings with previous studies, Adding further medical details to the comments section, Contributing to the development of future research directions 12 :44 :54 :56 :27 Inadequate responsibility for the replies offered in clinical choices, Inadequate presentation of crucial results, Problems with research reproducibility, Differences between human and AI-generated writing, Inability to reason ethically 12 :49 :37 :43 :25 :26 :45 :12
Drawing Conclusions Opportunities of drawing conclusions Challenges of drawing conclusions
Assisting with the drafting of the conclusion, Management of references, Enhancing the reporting process, Providing ideas for the structure of the article 17 :12 :33 :41 :47 :53 Biased assessment and deceptive conclusions, The presence of logical errors in responses, Development of pseudoscience, Uncertainty about researchers' contributions to the research, Undermining the process of meaningful research, Weakening the cohesion of research, False and counterfeit reports 22 :32 :29 :33 :35 :56
Opportunities of executive and research advice Challenges of executive and research advice
Considering new research paths based on the findings, Identifying potential research solutions 19 :54 :56 :34 :48 Providing solutions that are conflicting, Hindering scientific progress with the wrong solutions, Leading to errors in patient treatment 16 :44
Opportunities of publishing the article Challenges of publishing the article
Providing assistance to researchers during the publishing process, Shortening the publication time, Contributing to the article's readability, Word count reduction to match journal formats, Assisting with journal-specific formatting, Writing an abstract within the word limit, Making the article editing process easier, Providing more appropriate vocabulary synonyms, Giving personal input on the writing styles of writers, Providing assistance in many languages, Increasing the output of non-English-speaking researchers, Assisting in the search for relevant journals, Increasing the productivity of researchers, Assisting in the modification of an article's style and tone, Increasing the readability of the article, Producing detailed reports, Allowing for greater flexibility in the final report's writing, Providing writing assistance to non-native English-speaking researchers, Developing appropriate reactions to questions, Providing customized responds, Providing real-time response, Assisting with pre-submission time-based evaluation of articles; increasing translation accuracy; Reducing the cost of research 17 :18 :21 :22 :46 :47 :48 :12 :49 :50 :52 :53 :56 :57 :62 :25 :29 :33 :44 :32 :43 :37 :39 :24 :65 :58 :36 :31 :64 :27 :40 :46 :10 :55 :5: Access to AI services in research differs between the affluent and the poor, The danger of publishing inaccurate information in scientific journals, Significant increase in low-quality scientific publications, Undermining confidence in university research, The commodification of science, The growth of science through uniform publications, Squandering research funding, Lack of legal enforcement regarding chatbot responses, Production of fake articles, Providing identical non-diverse replies, Insufficient number of software identified by ChatGPT for detection 66 :1 :20 :21 :22 :46 :49 :51 :54 :55 :56 :57 :2 :3 :61 :23 :29 :30 :33 :35 :38 :42 :41 :39 :65:
Table 1.The most important potentials and limitations of Chat GPT in medical research

The fifth stage involved breaking down, analyzing, and synthesizing the findings at the paragraph level. Significant concepts were extracted from each article, integrated, and then classed based on similarities and differences using iterative processes. The sixth stage included quality control, qualitative validity criteria, and validation with four assessment criteria from Lincoln and Guba ( 67 ).

Finally, in the seventh stage, the idea identification and synthesis results were acquired, which are extensively discussed in the result section.

The Thematic Analysis approach Wolcott ( 68 ) was employed for data analysis based on three stages. The first stage involved data description. This research thoroughly extracted the studies obtained in the systematic review phase for their bibliographic characteristics, as outlined in Table (1). In the second stage, data from a systematic review were analyzed by organizing, structuring, and categorizing studies thematically. Key concepts were enumerated and documented, then categorized within an eight-dimensional model—the third stage involved interpreting findings, leading to a final categorized presentation and detailed explanation.

The octagonal model created by Karl Llewellyn ( 69 ) was utilized for data analysis. As a result, the investigation could be performed inside a specified scientific framework. The eight steps of this concept are depicted in Figure (2).

Figure 2. Conceptual model of the research

This study followed ethical considerations by citing the sources of the scientific integrity concept.

Results

Based on Karl Llewellyn's ( 69 ) octagonal model, the most significant potentials and limitations of ChatGPT for medical research are presented in Table (1). Indeed, the extracted core concepts were categorized into preset groups based on their similarities and differences to create a systematic framework per the octagonal model.

Discussion

ChatGPT has the potential to play a critical role throughout the whole research spectrum included in the octagonal model. This AI can help researchers identify essential discoveries, treatment options, and emerging patterns by swiftly scanning massive amounts of medical literature and clinical data. Research reports, summaries, and even compelling research papers can be written with their ability to produce coherent and contextually relevant content.

In the research context, the initial phase involves framing the research problem. Researchers can utilize ChatGPT to generate research ideas and concepts by entering a topic and receiving suggestions for questions, ideas, and keywords ( 34 ). Furthermore, research is enhanced using content generation, coding assistance, concept reduction, and composition of essays ( 70 ). When helpful prompts are used, ChatGPT shows the potential to support systematic literature reviews, the second research aspect ( 54 ). This program can facilitate scientific advancement by assisting with study design, analysis, writing, and editing research articles ( 5 ). When developing a research problem, depending exclusively on ChatGPT's algorithm without critical thinking, creativity, and invention may result in undesired results. ChatGPT, as an AI tool, is limited in its ability to interact actively and propose novel approaches due to its algorithm and a lack of expert training.

ChatGPT provides clarity and error detection, significantly improving scientific writing quality, which allows researchers to concentrate on experiments by fixing errors, enhancing syntax, suggesting complex vocabulary, and providing improvement strategies. In addition, ChatGPT helps summarize previous research and develop insights and suggestions for particular issues. Non-native English speakers can use ChatGPT to communicate coherently with evidence by overcoming language barriers ( 24 , 52 , 48 ). Developing software-based editing tools has brought us several capabilities for improving clarity and correcting language, from simple spelling checks to sophisticated services such as Grammarly and Scribbr. However, these technologies rely on predetermined standards to evaluate writing ( 29 ).

The next area of application in medical research is reviewing relevant literature. ChatGPT demonstrates its capacity to assist in systematic literature reviews, representing the secondary facet of research using helpful prompts. Researchers can use ChatGPT to discover relevant topics and integrate information from various sources. While automating the stage of generating literature reviews saves time and effort, it is essential to note that ChatGPT may not be able to thoroughly analyze article quality or ensure the inclusion of acceptable sources systematically due to the absence of crucial methodologies such as search strategies, citation analysis, and rigorous selection criteria. This limitation could lead to errors and incomplete coverage in the generated literature reviews.

The benefits of ChatGPT for selecting an acceptable study design, identifying the population, and selecting the sample are as follows. Using ChatGPT and other language models allows researchers to focus on developing creative, experimental designs, fostering advancements in various domains. Even though advanced language models speed up research, enhance creativity, and foster innovation, they lack expertise in developing new experimental methods. Developing such methodologies demands human creativity, critical thinking, and problem-solving skills beyond the capabilities of ChatGPT or other AI tools in their current forms.

ChatGPT helps analyze data and serves as a recommender system, assisting users in discovering suitable research subjects, which is especially useful in interdisciplinary domains, where ChatGPT's features ease database searches and various terminology ( 71 ). Data processing, drug development, finding possible drug targets, and creating code are facilitated in scientific research ( 25 ). ChatGPT simplifies data visualization and statistical analysis and provides predictive insights by analyzing historical data to forecast future trends. However, relying entirely on ChatGPT for up-to-date information is not recommended. Versions that are newer will update more slowly than versions in the real world. Therefore, users cannot access the most recent information and resources. AI language models, such as ChatGPT, may have biases and biases in data collection can exacerbate inequalities and reflect human biases. Therefore, generated data may support and endorse the prevailing perspectives, such as gender and racial inequalities, impacting medical research reproduction and distorting the field's system. The digital divide and internet access, mainly affecting users in developed countries, are also discussed, causing discrepancies in access and age among participants. This circumstance exacerbates the gap between medical researchers in developed and underdeveloped countries.

Another debatable point is the lack of transparency in processes. Neural network models, commonly regarded as opaque systems with ambiguous processes, provide difficulties in decision-making due to a restricted level of comprehension. This lack of transparency in medical research data can impact outcomes. Inadequate datasets may cause ChatGPT to contain unknown content, raising plagiarism concerns. Additionally, ineffective AI text detection can damage the scientific community's reputation ( 72 ). Compared to traditional search engines, an over-reliance on ChatGPT, which offers singular solutions without various viewpoints, may stifle original thought. The use of ChatGPT-generated content in academic work may not have a significant impact on originality. Additionally, the guidelines for academic journals do not clearly state when ChatGPT-generated content is acceptable, further complicating the definition of "originality" ( 50 ).

During the second stage of research, interpretation, discussion of data, and conclusion, researchers often struggle to choose the most appropriate journal to publish their study. This necessitates a detailed assessment of the "aims and scope" of various publications, which can be time-consuming and arduous. Nonetheless, ChatGPT can effectively recommend appropriate journals to diverse publishing organizations, provide precise recommendations, and potentially save researchers’ time ( 33 ). Specific issues need to be addressed when using ChatGPT for data analysis. The model might not be able to identify biases or outliers, which could lead to flawed scientific conclusions.

Furthermore, the model cannot access contextual knowledge, which could complicate presenting persuasive research results. Additionally, the model might produce illogical or irrelevant responses. No transparent sources raise concerns about the accuracy of the information, which may impact funding, research directions, and policy decisions with significant social implications ( 54 ).

Conclusion

ChatGPT and similar tools should be embraced and understood for their potential and limitations in research. This information serves as a precautionary measure against upcoming issues and provides us with an insightful understanding of the numerous uses of these instruments. This study aimed to provide a more comprehensive understanding of the potentials and limitations of using ChatGPT in research by extracting relevant information from a carefully selected corpus of scholarly articles.

ChatGPT, as a language model, facilitates article selection, research methodologies, feedback, and idea summaries, saving researchers a tremendous amount of time. However, there are some shortcomings such as the inability to obtain complete and reliable data, which leads to unreliable study results. Moreover, ingrained prejudices and the potential for producing offensive information highlight the need for careful usage. Additionally, an over-reliance on ChatGPT may hinder the originality and inventiveness of research. Awareness of these factors enables researchers to utilize ChatGPT carefully, and thereby advance scientific understanding.

This research has specific limitations. Firstly, it exclusively examined studies published in English, excluding articles published in other languages. Additionally, the study concentrated on articles, neglecting books and research projects in this subject area, collectively known as "gray literature" in review studies.

Authors’ Contribution

Each author participated in the conversation, reviewed, and endorsed the manuscript. They are also committed to taking responsibility for all aspects of the research, ensuring that inquiries regarding the accuracy or integrity of any section of the work are thoroughly addressed and resolved.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Islam I, Islam MN. Opportunities and challenges of chatgpt in academia: A conceptual analysis. Authorea Preprints; 2023.
  2. Zheng H, Zhan H. ChatGPT in scientific writing: a cautionary tale. The American Journal of Medicine. 2023; 136(8):725-6.
  3. Aljanabi M, Ghazi M, Ali AH, Abed SA. ChatGPT: open possibilities. Iraqi Journal for Computer Science and Mathematics. 2023; 4(1):62-4.
  4. Liebrenz M, Schleifer R, Buadze A, Bhugra D, Smith A. Generating scholarly content with ChatGPT: ethical challenges for medical publishing. The Lancet Digital Health. 2023; 5(3):e105-6.
  5. Macdonald C, Adeloye D, Sheikh A, Rudan I. Can ChatGPT draft a research article? An example of population-level vaccine effectiveness analysis. Journal of global health. 2023; 13:01003.
  6. Van Dis EA, Bollen J, Zuidema W, van Rooij R, Bockting CL. ChatGPT: five priorities for research. Nature. 2023; 614(7947):224-6.
  7. Jeyaraman M, Jeyaraman N, Nallakumarasamy A, Yadav S, Bondili SK. ChatGPT in Medical Education and Research: A Boon or a Bane?. Cureus. 2023; 15(8):e44316.
  8. Duong D, Solomon BD. Analysis of large-language model versus human performance for genetics questions. European Journal of Human Genetics. 2024; 32(4):466-8.
  9. Marchandot B, Matsushita K, Carmona A, Trimaille A, Morel O. ChatGPT: the next frontier in academic writing for cardiologists or a pandora’s box of ethical dilemmas. European Heart Journal Open. 2023; 3(2):oead007.
  10. Švab I, Klemenc-Ketiš Z, Zupanič S. New Challenges in Scientific Publications: Referencing, Artificial Intelligence and ChatGPT. Slovenian Journal of Public Health. 2023; 62(3):109-12.
  11. Kitamura FC. ChatGPT is shaping the future of medical writing but still requires human judgment. Radiology. 2023; 307(2):e230171.
  12. Fatani B. ChatGPT for future medical and dental research. Cureus. 2023; 15(4):e37285.
  13. Gao CA, Howard FM, Markov NS, Dyer EC, Ramesh S, Luo Y, et al. Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers. NPJ Digit Med. 2023; 6(1):75.
  14. Sandelowski M, Barroso J. Handbook for synthesizing qualitative research. Springer Publishing Company: USA; 2006.
  15. Finfgeld‐Connett D, Johnson ED. Literature search strategies for conducting knowledge‐building and theory‐generating qualitative systematic reviews. Journal of advanced nursing. 2013; 69(1):194-204.
  16. García-Peñalvo FJ. The perception of Artificial Intelligence in educational contexts after the launch of ChatGPT: Disruption or Panic?. Education in the Knowledge Society (EKS).. 2023; 24:1-24.
  17. Karakose T. The utility of ChatGPT in educational research—Potential opportunities and pitfalls. Educational Process: International Journal. 2023; 12:7-13.
  18. Peres R, Schreier M, Schweidel D, Sorescu A. On ChatGPT and beyond: How generative artificial intelligence may affect research, teaching, and practice. International Journal of Research in Marketing. 2023; 40(2):269-75.
  19. Dai Y, Liu A, Lim CP. Reconceptualizing ChatGPT and generative AI as a student-driven innovation in higher education. Procedia CIRP. 2023; 119:84-90.
  20. Eke DO. ChatGPT and the rise of generative AI: Threat to academic integrity?. Journal of Responsible Technology. 2023; 13:100060.
  21. Javaid M, Haleem A, Singh RP, Khan S, Khan IH. Unlocking the opportunities through ChatGPT Tool towards ameliorating the education system. BenchCouncil Transactions on Benchmarks, Standards and Evaluations. 2023; 3(2):100115.
  22. Ivanov S, Soliman M. Game of algorithms: ChatGPT implications for the future of tourism education and research. Journal of Tourism Futures. 2023; 9(2):214-21.
  23. Perera P, Lankathilaka M. AI in higher education: A literature review of chatgpt and guidelines for responsible implementation. International Journal of Research and Innovation in Social Science. 2023; 7:306-14.
  24. Rahman MM, Watanobe Y. ChatGPT for education and research: Opportunities, threats, and strategies. Applied Sciences. 2023; 13(9):5783.
  25. Sallam M. ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. InHealthcare. 2023; 11(6):887.
  26. Abdulai AF, Hung L. Will ChatGPT undermine ethical values in nursing education, research, and practice. Nursing Inquiry. 2023; 30:e12556.
  27. Pandey US. Exploring the Research Agenda for Large Language Models: Opportunities and Challenges for Scientific Research. Media Watch. 2023; 14(2):127-30.
  28. Parse RR. Artificial Intelligence: Challenges for Education and Research. Nursing Science Quarterly. 2023; 36(3):213-4.
  29. Meyer JG, Urbanowicz RJ, Martin PC, O’Connor K, Li R, Peng PC, et al. ChatGPT and large language models in academia: opportunities and challenges. BioData Mining. 2023; 16(1):20.
  30. Willems J. ChatGPT at universities–the least of our concerns [Internet]. 2023 [Cited 20 Jan 2023]. Available from: https://ssrn.com/abstract=4334162.
  31. Opara E, Mfon-Ette Theresa A, Aduke TC. ChatGPT for teaching, learning and research: Prospects and challenges. Opara Emmanuel Chinonso: Tolorunleke Caroline; 2023.
  32. Sok S, Heng K. ChatGPT for education and research: A review of benefits and risks [Internet]. 2023 [Cited 6 March 2023]. Available from: https://ssrn.com/abstract=4378735.
  33. Xames MD, Shefa J. ChatGPT for research and publication: Opportunities and challenges. Journal of Applied Learning and Teaching. 2023; 6(1):1-6.
  34. Cribben I, Zeinali Y. The Benefits and Limitations of ChatGPT in Business Education and Research: A Focus on Management Science, Operations Management and Data Analytics. Operations Management and Data Analytics [Internet]. 2023 [Cited 29 March 2023]. Available from: SSRN: https://ssrn.com/abstract=4404276.
  35. Malik A, Khan ML, Hussain K. How is ChatGPT transforming academia? Examining its impact on teaching, research, assessment, and learning. Examining its Impact on Teaching, Research, Assessment, and Learning [Internet]. 2023 [Cited 9 April 2023].
  36. Periaysamy AG, Satapathy P, Neyazi A, Padhi BK. ChatGPT: roles and boundaries of the new artificial intelligence tool in medical education and health research–correspondence. Annals of Medicine and Surgery. 2023; 85(4):1317.
  37. Sedaghat S. Early applications of ChatGPT in medical practice, education and research. Clinical Medicine. 2023; 23(3):278-9.
  38. Farrokhnia M, Banihashem SK, Noroozi O, Wals A. A SWOT analysis of ChatGPT: Implications for educational practice and research. Innovations in Education and Teaching International. 2024; 61(3):1-5.
  39. Ring J. Artificial intelligence—ChatGPT and Scientific Publishing. Journal of the European Academy of Dermatology and Venereology. 2023; 37(7):1253-4.
  40. Hosseini M, Gao CA, Liebovitz DM, Carvalho AM, Ahmad FS, Luo Y, et al. An exploratory survey about using ChatGPT in education, healthcare, and research. MedRxiv. 2023; 3:23287979.
  41. Tay JQ. ChatGPT and the future of plastic surgery research: evolutionary tool or revolutionary force in academic publishing?. European Journal of Plastic Surgery. 2023; 46:1-2.
  42. Rahimi F, Talebi Bezmin Abadi A. Passive Contribution of ChatGPT to Scientific Papers. Annals of Biomedical Engineering. 2023; 51:2340-50.
  43. Arif TB, Munaf U, Ul-Haque I. The future of medical education and research: Is ChatGPT a blessing or blight in disguise?. Medical education online. 2023; 28(1):2181052.
  44. Ruksakulpiwat S, Kumar A, Ajibade A. Using ChatGPT in Medical Research: Current Status and Future Directions. Journal of Multidisciplinary Healthcare. 2023; 16:1513-20.
  45. Hill-Yardin EL, Hutchinson MR, Laycock R, Spencer SJ. A Chat (GPT) about the future of scientific publishing. Brain Behav Immun. 2023; 110:152-4.
  46. Dönmez İ, Sahin ID, Gülen S. Conducting academic research with the ai interface ChatGPT: Challenges and opportunities. Journal of STEAM Education. 2023; 6(2):101-18.
  47. Burger B, Kanbach DK, Kraus S, Breier M, Corvello V. On the use of AI-based tools like ChatGPT to support management research. European Journal of Innovation Management. 2023; 26(7):233-41.
  48. Tai AM, Meyer M, Varidel M, Prodan A, Vogel M, Iorfino F, et al. Exploring the potential and limitations of ChatGPT for academic peer-reviewed writing: Addressing linguistic injustice and ethical concerns. Journal of Academic Language and Learning. 2023; 17(1):T16-30.
  49. Frigerio A. A preliminary analysis of the use of chatgpt in academic research. Абай атындағы. 2023; 77(1):13-9.
  50. Zhu JJ, Jiang J, Yang M, Ren ZJ. ChatGPT and environmental research. Environmental Science & Technology. 2023; 57(46):17667-70.
  51. Zielinski C, Winker M, Aggarwal R, Ferris L, Heinemann M, Lapeña JF, et al. Chatbots, ChatGPT, and Scholarly Manuscripts-WAME Recommendations on ChatGPT and Chatbots in Relation to Scholarly Publications. Afro-Egyptian Journal of Infectious and Endemic Diseases. 2023; 13(1):75-9.
  52. Huang J, Tan M. The role of ChatGPT in scientific communication: writing better scientific review articles. American Journal of Cancer Research. 2023; 13(4):1148.
  53. Zohery M. ChatGPT in Academic Writing and Publishing: A Comprehensive Guide. Achtago: Egypt; 2023.
  54. Wen J, Wang W. The future of ChatGPT in academic research and publishing: A commentary for clinical and translational medicine. Clinical and Translational Medicine. 2023; 13(3):e1207.
  55. Subbaramaiah MT, Shanthanna H. ChatGPT in the field of scientific publication–Are we ready for it?. Indian Journal of Anaesthesia. 2023; 67(5):407.
  56. Yatoo MA, Habib F. ChatGPT, a friend or a foe? Integration of Artificial Intelligence-enabled chatbots in academic research. MRS Bulletin: London; 2023.
  57. Bhatia P. ChatGPT for academic writing: A game changer or a disruptive tool?. Journal of Anaesthesiology, Clinical Pharmacology. 2023; 39(1):1.
  58. Huh S. Emergence of the metaverse and ChatGPT in journal publishing after the COVID-19 pandemic. Science Editing. 2023; 10(1):1-4.
  59. Frederico GF. ChatGPT in Supply Chains: Initial Evidence of Applications and Potential Research Agenda. Logistics. 2023; 7(2):26.
  60. McEachan JE, Lam WL. Response to concerns about the increasing influence of artificial intelligence in publishing. Journal of Hand Surgery (European Volume).. 2023; 48(8):697-8.
  61. Dahmen J, Kayaalp ME, Ollivier M, Pareek A, Hirschmann MT, Karlsson J, et al. Artificial intelligence bot ChatGPT in medical research: the potential game changer as a double-edged sword. Knee Surgery, Sports Traumatology, Arthroscopy. 2023; 31(4):1187-9.
  62. Rahman MM, Terano HJ, Rahman MN, Salamzadeh A, Rahaman MS. ChatGPT and academic research: a review and recommendations based on practical examples. Rahman M, Terano HJR, Rahman N, Salamzadeh A, Rahaman S. ChatGPT and Academic Research: A Review and Recommendations Based on Practical Examples. Journal of Education, Management and Development Studies. 2023; 3(1):1-12.
  63. Altmäe S, Sola-Leyva A, Salumets A. Artificial intelligence in scientific writing: a friend or a foe?. Reproductive BioMedicine Online. 2023; 47(1):3-9.
  64. Gottlieb M, Kline JA, Schneider AJ, Coates WC. ChatGPT and conversational artificial intelligence: Friend, foe, or future of research?. The American Journal of Emergency Medicine. 2023; 70:81-3.
  65. Esplugas M. The use of artificial intelligence (AI) to enhance academic communication, education and research: a balanced approach. Journal of Hand Surgery (European Volume).. 2023; 48(8):819-22.
  66. Qasem F. ChatGPT in scientific and academic research: future fears and reassurances. Library Hi Tech News. 2023; 40(3):30-2.
  67. Lincoln YS, Guba EG. Judging the quality of case study reports. Internation Journal of Qualitative Studies in Education. 1990; 3(1):53-9.
  68. Wolcott HF. Writing up qualitative research. Sage publications: California; 2008.
  69. Lastrucci CL. The scientific approach: Basic principles of the scientific method. Schenkman Pub: USA; 1963.
  70. Dwivedi YK, Kshetri N, Hughes L, Slade EL, Jeyaraj A, Kar AK, et al. “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management. 2023; 71:102642.
  71. Lund BD, Wang T, Mannuru NR, Nie B, Shimray S, Wang Z. ChatGPT and a new academic reality: Artificial Intelligence‐written research papers and the ethics of the large language models in scholarly publishing. Journal of the Association for Information Science and Technology. 2023; 74(5):570-81.
  72. Singh S, Djalilian A, Ali MJ. ChatGPT and ophthalmology: exploring its potential with discharge summaries and operative notes. Semin Ophthalmol. 2023; 38(5):503-7.
Journal of Advances in Medical Education & Professionalism
Volume 12, Issue 3
July 2024
Pages 135-147
Files
Share
How to cite
Statistics