Leveraging Open Large Language Models for Historical Named Entity Recognition - Laboratoire LI, équipe BDTLN
Conference Papers Year : 2024

Leveraging Open Large Language Models for Historical Named Entity Recognition

Abstract

The efficacy of large-scale language models (LLMs) as few-shot learners has dominated the field of natural language processing, achieving state-of-the-art performance in most tasks, including named entity recognition (NER) for contemporary texts. However, exploration of NER in historical collections (e.g., historical newspapers and classical commentaries) remains limited. This presents a greater challenge as historical texts are often noisy due to storage conditions, OCR extraction, and spelling variation. In this paper, we conduct an empirical evaluation comparing different Instruct variants of open-access and open-sourced LLMs using prompt engineering through deductive (with guidelines) and inductive (without guidelines) approaches against the fully supervised benchmarks. In addition, we study how the interaction between the Instruct model and the user impacts the entity prediction. We conduct reproducible experiments using an easy-to-implement mechanism on publicly available historical collections covering three languages (i.e., English, French, and German) with code-switching on Ancient Greek and four open Instruct models. The results show that Instruct models encounter multiple difficulties handling the noisy input documents, scoring lower than fine-tuned dedicated NER systems, yet the resulting predictions provide entities that can be used in further tagging processes by human annotators.
Fichier principal
Vignette du fichier
gonzalez2024leveraging.pdf (641.42 Ko) Télécharger le fichier
Origin Files produced by the author(s)
licence

Dates and versions

hal-04662000 , version 1 (25-07-2024)

Licence

Identifiers

  • HAL Id : hal-04662000 , version 1

Cite

Carlos-Emiliano González-Gallardo, Tran Thi Hong Hanh, Ahmed Hamdi, Antoine Doucet. Leveraging Open Large Language Models for Historical Named Entity Recognition. The 28th International Conference on Theory and Practice of Digital Libraries, Sep 2024, Ljubljana, Slovenia. ⟨hal-04662000⟩
215 View
203 Download

Share

More