LAW XX | Schedule

Location: La Jolla, Manchester Grand Hyatt San Diego

Underline page (see for Zoom and Gather Town links): https://underline.io/events/530/sessions?eventSessionId=22989

Time	Session
8:45–9:00	Opening Remarks
9:00–9:45	Keynote: Separating signal from noise: Annotation errors and alternative interpretations (Marie-Catherine de Marneffe)
9:45–10:30	Oral Session 1: Annotating with LLMs — Errors & Disagreement (Special Theme) Human-AI Annotation Error Auditing for Hebrew Diacritization with Frontier LLMs LLMs as annotators of credibility assessment in Danish asylum decisions: evaluating classification performance and errors beyond aggregated metrics (virtual) Revisiting Faithfulness Annotations for Long-form Summaries
10:30–11:15	Coffee Break
11:15–12:00	Oral Session 2: Corpora, Resources & Annotation Quality UD-CHILDES-BG: a dependency treebank of Bulgarian child and child-directed speech (virtual) When Ground Truth Disagrees: A Human-in-the-Loop Audit of Annotation Errors in High-Stakes Crash Narratives Rules-based system for Czech legal text readability (virtual)
12:00–13:30	Lunch
13:30–14:15	Oral Session 3: Annotation Guidelines & Methodology TalkTag: Fine-Grained Morphosyntactic Error Annotation for Transcribed Speech Semantic-pragmatic Annotations in the Prague Dependency Treebank (Findings) Not Worth Mentioning? A Pilot Study on Salient Proposition Annotation
14:15–14:30	Short Break
14:30–15:30	Poster Session (Hybrid in Grand Hall and Gather Town) Designing Annotation Guidelines for Trait-Based Arabic Automated Essay Scoring: A Systematic Methodology Cracks in the Bridge—or A Bridge Too Far? Comparing Human and LLM Errors in the Annotation of Bridging Anaphora Prompts in the Wild: A Large Analyzed Collection of Transactional Prompts in Code When LLMs Disagree with Human Experts: Understanding LLM Annotation Failures in Nutrition Misinformation through Hierarchical Error Analysis using Seed Oil Narratives Math-DB: A Discourse Framework for Mathematical Word Problems to Enhance LLM Reasoning Parser agreement and disagreement in L2 Korean UD: Implications for human-in-the-loop annotation Completing and Validating the Re-Aligned Switchboard Dialog Act Corpus Clustering Analysis for Error Detection in Named Entity Recognition Datasets Annotating Clinical Risk and Variation in Haitian Creole Medical Translation EVADE: LLM-Based Explanation Generation and Validation for Error Detection in NLI (Findings) IndiAnn: A Web-based Annotation Platform for Indic Languages (virtual) Beyond Annotator Disagreement: Guideline-Induced Errors in Arabic Hate Speech Annotation (virtual) Cross-Linguistic Situation Entity Segmentation for Discourse Analysis in Diachronic English and German Text (virtual)
15:30–16:00	Coffee Break
16:00–16:45	Keynote: Like Nailing Jello to the Wall? Annotation Challenges for Common Sense and Cultural Consensus (Rachel Rudinger)
16:45–17:15	Roundtable Discussion: Open Questions & Future Directions
17:15–17:30	Closing Remarks

Time

Session

8:45–9:00

Opening Remarks

9:00–9:45

Keynote: Separating signal from noise: Annotation errors and alternative interpretations (Marie-Catherine de Marneffe)

9:45–10:30

Oral Session 1: Annotating with LLMs — Errors & Disagreement (Special Theme)

Human-AI Annotation Error Auditing for Hebrew Diacritization with Frontier LLMs
LLMs as annotators of credibility assessment in Danish asylum decisions: evaluating classification performance and errors beyond aggregated metrics (virtual)
Revisiting Faithfulness Annotations for Long-form Summaries

10:30–11:15

Coffee Break

11:15–12:00

Oral Session 2: Corpora, Resources & Annotation Quality

12:00–13:30

Lunch

13:30–14:15

Oral Session 3: Annotation Guidelines & Methodology

14:15–14:30

Short Break

14:30–15:30

Poster Session (Hybrid in Grand Hall and Gather Town)

15:30–16:00

Coffee Break

16:00–16:45

Keynote: Like Nailing Jello to the Wall? Annotation Challenges for Common Sense and Cultural Consensus (Rachel Rudinger)

16:45–17:15

Roundtable Discussion: Open Questions & Future Directions

17:15–17:30

Closing Remarks