1. Research Backgrounds and Purpose

  intelligence(AI), like Chat GPT, and develop a research methodology applicable to the Korea Institute of Criminology and Justice(KICJ)’s study of crime and criminal justice policy. Moreover, it seeks to develop a research platform that utilizes readily available AI to increase efficiency in the study of criminal justice policy. Additionally, the study aims to develop a research method that uates the contribution of KICJ research to criminal justice issues, enabling effective responses to newly launched government initiatives. To achieve these objectives, keywords (topic words) were extracted from research reports published by KICJ from 2002 to 2022. AI which utilizes pre-built learning models for natural language processing, was chosen due to its convenience and accessibility. With AI software hosted in the cloud, users can easily use this technology online.

  This study conducted a relevance analysis between KICJ’s research reports and criminal justice issues addressed by criminal justice organizations such as the Ministry of Justice and the National Police Agency to test the utility of AI in the field of criminology and criminal justice. The research also analyzed trends to determine how well KICJ’s reports covered topics aligned with the institute’s objectives. Additionally, KICJ researchers were surveyed to uate the validity of the use of AI in the study of crime and criminal justice policy.

2. Findings

  The relevance analysis between KICJ’s research reports and criminal justice issues found that the reports were classified into seven topics: ‘Crime Survey,’ ‘Victim Protection and Support System,’ ‘Police System/Juvenile Law,’ ‘Criminal Law Reform,’ ‘Organized crime, Drug trafficking, and Cybercrime,’ ‘Advancement of Criminal Justice Policy,’ and ‘Correctional Treatment and Recidivism Prevention.’ The further analysis revealed that these topics were evenly distributed, with no significant overrepresentation of any specific topic. Secondly, the composition of topics varied across different presidential terms. The KICJ report topics centered around more on ‘criminal law reform’ during the 16th presidential term while it was ‘crime survey,’ during the 17th presidential term and ‘criminal justice policy advancement’ during the 18th and 19th presidential term. Thirdly, the report topics showed the highest similarity to the objectives of the organizations, including the Ministry of Justice, National Police Agency, Legal Affairs Ministry, and Ministry of Gender Equality and Family. However, in recent years, the similarity with the goals of the Ministry of Justice, National Police Agency, and Legal Affairs Ministry has notably increased. Lastly, a rising number of reports are found to closely align with the objectives of the Ministry of Justice and National Police Agency in recent years. These findings quantitatively demonstrate that KICJ maintains a balance between the research addressing criminal justice issues related to the criminal justice agencies and those on contemporary issues.

  The results derived from the text mining techniques and network analysis on the KICJ research trends of are as follows. First, the network analysis, focusing on research topic words, revealed that the most active research topic in the field of criminology was “Causes and Characteristics of Crime.” The topics, which are close to the most active one, included “Construction and Utilization of Crime Statistics,” “Crime Trends,” and “Crime Victimization Survey (including Juveniles).” These findings indicate that KICJ makes a consistent effort towards gathering empirical data on crime phenomena, such as crime victimization surveys, as well as examining and analyzing the causes of crime. Research was also conducted in the areas of ‘crime prevention policies and programs’ and ‘recidivism prevention and re-socialization,’ implying that the institution is fulfilling its established purpose.

  Second, a community detection algorithm identified 12 communities (research areas). The largest areas were centered around the nature of crime, encompassing ‘causes and characteristics of crime’ and ‘crime trends.’ Additionally, the fields of corrections, sexual violence and crime prevention, juvenile justice, and crime victimization played significant roles in the flow of criminological research. Similar to the findings from the topic modeling analysis, these results demonstrate that KICJ research reports cover a wide range of topics within the context of criminal justice policy rather than focusing on specific subjects.

  Third, the analysis of the betweenness centrality of research topics indicated that ‘Causes and Characteristics of Crime’ plays a key role in connecting other research topics in the network. However, no other topic exhibited high betweenness centrality, suggesting relative independence between research topics. 

  Lastly, the degree centrality analysis per each member revealed that researchers with longer work experience consistently demonstrated greater research topic diversity. Each researcher maintains expertise in a specific research area while actively engaging in related areas of study.

  The analysis of KICJ researchers’ uations of AI-generated keywords and summaries of research reports showed that the overall satisfaction with AI-generated summaries was at a “moderate” level. The researchers view the AI-generated research report summaries as clear and concise but not accurate in terms of expression. 

  Second, the factors influencing the satisfaction were found to be conciseness and similarity to summaries written by researchers. The result may imply that the efficiency of transforming lengthy content into relatively shorter summaries and the narratives not significantly differing from those written by researchers might have influenced satisfaction. Third, uations of research report summaries tended to be more positive among social science majors rather than law majors, co-researchers rather than principal investigators, and those studying criminal justice policy rather than legal policy. Fourth, positive ratings for the keywords extracted by AI outweighed negative ratings by more than threefold. The findings suggest that KICJ researchers are receptive to the use of AI for summarizing reports or extracting keywords to some extent. However, perceptions of the application of new technology differ across the respondent characteristics. Specifically, law majors, who use complex legal terms and concepts, may perceive that the current level of natural language processing capabilities of AI requires significant improvement as opposed to social science majors, who often deal with everyday terms related to criminal behavior. These observations highlight the limitations associated with the use of service-oriented AI technologies, which are developed for general purposes, in specialized fields like research reports and legal s.

3. Conclusion and Suggestions

  This study can be considered the first attempt to apply a scientific methodology to KICJ research reports, and it is expected to serve as a foundation for future project development and the expansion of research areas. The various case studies explored in this study hold significance in enhancing the efficiency of KICJ research activities through the use of AI technology and statistical techniques.

  However, it is worth noting that there are limitations on the use of AI in this study. Firstly, due to the constraints of AI inputs (tokens, etc), some texts were not utilized in their original form, potentially leading to distortions in the results of keyword extraction. Secondly, since service-oriented AI is trained on general-purpose data, the study found that some extracted keywords included idiomatic expressions commonly used in research reports, along with duplication of similar words and those unrelated to the research domain. For example, words like ‘suspect’ and ‘crime’ might be keywords in regular sentences but may not carry the same significance within the context of research report summaries. Therefore, it is necessary to create a dictionary of unused keywords and refine unnecessary words to enhance accuracy. Thirdly, due to the lack of a theoretical foundation for AI, there are no criteria for determining the weight (importance) value of keyword similarity. In this study, certain percentiles were used as thresholds. These issues highlight the need for ongoing improvement through continuous keyword extraction and validation work in the future.

  Although the use of service-oriented AI has partially overcome the absence of AI experts and technological limitations to a certain extent, continuous research and improvement efforts are requisite to ensure optimal accuracy and service quality. Firstly, standardizing the length of KICJ research report summary is necessary to enhance natural language processing and user convenience. It was noted that similar words or expressions with similar meanings were identified as different keywords, leading to the problem of categorizing keywords representing the same concept into different categories within research reports. For example, ‘murder’ and ‘homicide’ were sometimes classified as different keywords, while ‘drug possession’ and ‘drug distribution’ were grouped together, causing inaccuracies in keyword extraction for drug-related crimes. It is imperative to construct a thesaurus tailored to the field of criminal justice, and legal policy to address the issue of similar words or expressions being recognized as distinct keywords. Finally, continuous monitoring and management are required to adapt to changes in keywords and the emergence of new terminology. Systematic management activities for implementing AI technology enhance the accuracy in analyzing KICJ research reports and ensure that the information is effectively conveyed to users without distortion. This long-term approach will not only serve the purpose of providing KICJ research reports to users but also contribute to improving the efficiency of the research and management activities of KICJ and enhance its function as the public policy research institution in the field of criminology and justice, supporting criminal justice agencies.