Metadata harmonisation: align spoken-data fields with UniDive naming conventions
Back to English GENTLE · Back to index
Repo: https://github.com/UniversalDependencies/UD_English-GENTLE
Cross-posting from the UniDive WG1 T1.5 (spoken language guidelines) metadata harmonisation review. We compared UD_English-GENTLE’s current CoNLL-U metadata against the proposed naming conventions (see also the full treebank status table). This is a suggestion for maintainers to review - feel free to push back on anything that doesn’t fit the corpus. The comparison was carried out semi-automatically with the help of Claude (Anthropic); errors or misunderstandings are possible, so please double-check anything unclear.
1. Mark spoken documents with # modality
The spoken portion of this treebank is identifiable via meta::genre = esports, which points to exactly 2 of the 26 documents:
document_id |
Title | Source |
|---|---|---|
GENTLE_esports_fifa |
CHAMPIONS LEAGUE SEMI-FINAL!!! FIFA 20 AC MILAN CAREER MODE #16 | YouTube |
GENTLE_esports_fortnite |
WORLD CUP SOLO’S FINAL MATCH | Fortnite World Cup Game 6 | Live commentary | YouTube |
Both are live esports commentary videos - the other 6 meta::genre values (poetry, threat, medical, dictionary, proof, legal, syllabus) are written or mixed registers.
Suggestion: Add # modality = spoken to GENTLE_esports_fifa and GENTLE_esports_fortnite, and # modality = written to the rest.
2. Document-level (naming conventions)
| Field | Suggestion |
|---|---|
document_id |
make tags: doc_id |
meta::genre |
make tags: genre |
3. Speaker-level (naming conventions)
| Field | Suggestion |
|---|---|
speaker |
make tags: speaker_id |
4. Sentence-level (naming conventions)
| Field | Suggestion |
|---|---|
meta |
corpus-specific (sentence-level) - verify against metadata.html |
Implementation notes
- Quick search & replace:
document_id→tagdoc_id,meta::genre→taggenre,speaker→tagspeaker_idare plain renames:python3 workgroups/spoken-data/scripts/harmonize_metadata.py rename-comment DIR --map meta::genre=genre,speaker=speaker_id --write(thedocument_id→doc_id“tag” is a UD tagset addition, not a comment rename - handle via the repo’s tagset file). - Needs a small script: the modality tag is mechanical once the genre→modality mapping is fixed - verified against the real corpus (
en_gentle-ud-test.conllu):python3 workgroups/spoken-data/scripts/harmonize_metadata.py tag-modality DIR --spoken-if '_esports_' --written-if '.*' --writecorrectly isolatesGENTLE_esports_fifa/GENTLE_esports_fortniteas the only 2 matches out of 26 documents, matching the draft’s finding exactly. - Needs manual input from maintainers: the
metasentence-level field is corpus-specific with no proposed name yet - needs a maintainer decision on what it holds before it can be classified or scripted.
This issue was prepared as part of the UniDive WG1 T1.5 spoken language guidelines effort. Happy to help implement these changes ourselves if that’s easier than doing it on your end - just let us know.