From 0255ec879382b4d9dd889868fa343e61b76d1f78 Mon Sep 17 00:00:00 2001 From: David Ponce Date: Tue, 10 Sep 2024 11:11:25 +0200 Subject: [PATCH 1/2] Paper Metadata: 2024.acl-long.622 --- data/xml/2024.acl.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/data/xml/2024.acl.xml b/data/xml/2024.acl.xml index e0a20a98fa..ca189c1eab 100644 --- a/data/xml/2024.acl.xml +++ b/data/xml/2024.acl.xml @@ -8088,7 +8088,7 @@ 11588-11607 The Split and Rephrase (SPRP) task, which consists in splitting complex sentences into a sequence of shorter grammatical sentences, while preserving the original meaning, can facilitate the processing of complex texts for humans and machines alike. It is also a valuable testbed to evaluate natural language processing models, as it requires modelling complex grammatical aspects. In this work, we evaluate large language models on the task, showing that they can provide large improvements over the state of the art on the main metrics, although still lagging in terms of splitting compliance. Results from two human evaluations further support the conclusions drawn from automated metric results. We provide a comprehensive study that includes prompting variants, domain shift, fine-tuned pretrained language models of varying parameter size and training data volumes, contrasted with both zero-shot and few-shot approaches on instruction-tuned language models. Although the latter were markedly outperformed by fine-tuned models, they may constitute a reasonable off-the-shelf alternative. Our results provide a fine-grained analysis of the potential and limitations of large language models for SPRP, with significant improvements achievable using relatively small amounts of training data and model parameters overall, and remaining limitations for all models on the task. 2024.acl-long.622 - ponce-martinez-etal-2024-split + ponce-etal-2024-split <fixed-case>C</fixed-case>hunk<fixed-case>A</fixed-case>ttention: Efficient Self-Attention with Prefix-Aware <fixed-case>KV</fixed-case> Cache and Two-Phase Partition From 746cfa24e3147c50e5ff528243ccbe30ec465e0e Mon Sep 17 00:00:00 2001 From: David Ponce Date: Wed, 11 Sep 2024 09:26:08 +0200 Subject: [PATCH 2/2] Paper Metadata: 2024.acl-long.622 --- data/xml/2024.acl.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/data/xml/2024.acl.xml b/data/xml/2024.acl.xml index ca189c1eab..bc18be8839 100644 --- a/data/xml/2024.acl.xml +++ b/data/xml/2024.acl.xml @@ -8083,7 +8083,7 @@ Split and Rephrase with Large Language Models DavidPonceVicomtech ThierryEtchegoyhenVicomtech - Jesus JavierCalleja PerezUniversidad del País Vasco and Vicomtech + JesúsCallejaUniversidad del País Vasco and Vicomtech HarritxuGeteUniversity of the Basque Country and Vicomtech Foundation 11588-11607 The Split and Rephrase (SPRP) task, which consists in splitting complex sentences into a sequence of shorter grammatical sentences, while preserving the original meaning, can facilitate the processing of complex texts for humans and machines alike. It is also a valuable testbed to evaluate natural language processing models, as it requires modelling complex grammatical aspects. In this work, we evaluate large language models on the task, showing that they can provide large improvements over the state of the art on the main metrics, although still lagging in terms of splitting compliance. Results from two human evaluations further support the conclusions drawn from automated metric results. We provide a comprehensive study that includes prompting variants, domain shift, fine-tuned pretrained language models of varying parameter size and training data volumes, contrasted with both zero-shot and few-shot approaches on instruction-tuned language models. Although the latter were markedly outperformed by fine-tuned models, they may constitute a reasonable off-the-shelf alternative. Our results provide a fine-grained analysis of the potential and limitations of large language models for SPRP, with significant improvements achievable using relatively small amounts of training data and model parameters overall, and remaining limitations for all models on the task.