Unveiling the Potential Concealed Intentions in Journalistic Articles Through AI Analysis
In a groundbreaking development, a new system called JOA-ICL (Journalism-guided Agentic In-Context Learning) has been created for detecting the stance of long-form news articles. According to its creators, JOA-ICL outperforms existing stance detection methods, offering a significant contribution to the analysis of full-length content.
The system, presented in a paper titled "Journalism-Guided Agentic In-Context Learning for News Stance Detection," was tested on a Korean dataset containing 2,000 news articles annotated for both article-level and segment-level stance. Each article was annotated twice: once for its overall stance toward a given issue, and again for individual segments (headline, lead, conclusion, and direct quotations).
JOA-ICL breaks news articles into structural parts and assigns a smaller model to label each one. The local predictions from each agent are passed to a larger model, which uses them to determine the article's overall stance. This segment-based method showed a notable edge in detecting supportive stances, a common weakness in similar models.
The dataset used for testing, called K-NEWS-STANCE, was compiled from Korean news coverage between June 2022 and June 2024. Notably, the new work and the stance prediction corpus in general lack consideration for hyperlinks, which can potentially be subjective and political.
The method was also tested on the CheeSE German dataset and outperformed fine-tuned baselines and zero-shot prompting. JOA-ICL outperformed both prompting-based and fine-tuned baselines, demonstrating particular strength in detecting supportive stances.
Experiments demonstrate JOA-ICL’s superiority over various existing methods such as chain-of-thought prompting, few-shot prompting, and fine-tuned baselines. The local predictions from each agent are passed to a second agent that decides the article's overall stance, with the two agents coordinated by a controller.
The results are impressive, with JOA-ICL achieving the best overall performance across both accuracy and macro F1, outperforming all other approaches tested. This suggests that JOA-ICL's journalism-informed segmentation and agentic in-context learning provide better precision and robustness in article-level stance detection than traditional approaches on both Korean and German news corpora.
In conclusion, the JoA-ICL system offers a promising avenue for improving the accuracy and nuance of stance detection in long-form news articles, demonstrating the value of segment-level analyses within large language models for understanding complex news content.
1) The advancement of JOA-ICL has potential applications beyond stance detection in journalism, such as promoting health-and-wellness by providing mental-health articles tailored to individuals based on their perceived stance.
2) As artificial-intelligence continues to progress, systems like JOA-ICL could revolutionize technology and science, improving the understanding and analysis of long-form content, including articles related to various scientific research and discoveries.
