Web Science and Digital Libraries Research Group

Figure 1 Cheng et al.: RELIC allows users to search for information from large language models (A), view the model’s top response, and understand the variations between response samples to verify the correctness of the generated information (B). For long-form generated text, users can inspect the consistency of each individual claim (C) and find contradicting or supporting evidence from other samples (D). Steps (1-6) illustrate the user’s verification process of InstructGPT’s response regarding Don Featherstone.(Figure 1 in original paper ) Large Language Models (LLMs) have revolutionized natural language processing, achieving remarkable performance across various tasks, such as translation, summarization, and question-answering. Despite their success, understanding and evaluating their responses remains a significant challenge. In the paper " RELIC (Response Evaluation through Linguistic Insight and Consistency)," published in CHI 2024 by Furui Cheng , Vilém Zouhar , Si...