Metadata harmonisation: align spoken-data fields with UniDive naming conventions
Back to Greek GDT · Back to index
Repo: https://github.com/UniversalDependencies/UD_Greek-GDT
Cross-posting from the UniDive WG1 T1.5 (spoken language guidelines) metadata harmonisation review. We compared UD_Greek-GDT’s current CoNLL-U metadata against the proposed naming conventions (see also the full treebank status table). This is a suggestion for maintainers to review - feel free to push back on anything that doesn’t fit the corpus. The comparison was carried out semi-automatically with the help of Claude (Anthropic); errors or misunderstandings are possible, so please double-check anything unclear.
1. Is the spoken portion identifiable?
This treebank mixes spoken and written material but its .conllu files don’t explicitly mark which sentences are spoken. It’s identifiable via the source-outlet component embedded in document_id (gdt-<date>-<source>-<docname>).
Finding: Sentences with ep as the source (e.g. gdt-20020204-ep-sessions_*-*) are transcripts of European Parliament plenary sessions (45 docs); ert/ertonline (Greek public broadcaster, 23 docs) are also spoken/broadcast material. voa (Voice of America, 8 docs) is written news text, grouped with elwikinews/wikipedia/apogevmatini (~94 docs total).
Suggestion: Add # modality = spoken to documents whose document_id source component is ep, ert, or ertonline.
2. Document-level (naming conventions)
| Field | Suggestion |
|---|---|
document_id |
make tags: doc_id |
| — | add # genre = speech on ep-sourced documents (European Parliament plenary session transcripts) |
| — | add # genre = news on ert/ertonline-sourced documents (broadcast news) |
| — | interaction-parameter classification for ep documents (please confirm): degree_of_spontaneity = planned, number_of_participants = monologic, context = professional, setting = broadcast, symmetry = symmetric |
| — | interaction-parameter classification for ert/ertonline documents (please confirm): degree_of_spontaneity = planned, number_of_participants = monologic (unless interview segments are present), context = public, setting = broadcast, symmetry = symmetric |
Implementation notes
- Needs a small script: the modality tag is mechanical and verified against the real corpus (all three release files):
python3 workgroups/spoken-data/scripts/harmonize_metadata.py tag-modality DIR --spoken-if '\-(ep|ert|ertonline)\-' --written-if '.*' --writegives 68 spoken documents (11 dev + 12 test + 45 train) - matching the draft’s “45ep+ 23ert/ertonline= 68” total exactly. The# genre = speech(onepdocs) and# genre = news(onert/ertonlinedocs) additions follow the same pattern but with a fixed value instead ofspoken/written- not yet a dedicated subcommand, but a ~10-line variant oftag-modalityusing the same regex groups would cover it. - Quick search & replace:
document_id→tagdoc_idis a tagset addition, not a text rename - handle via the repo’s tagset file rather thanrename-comment. - Needs manual input from maintainers: the proposed interaction-parameter values for
epandert/ertonlinedocuments (degree_of_spontaneity,number_of_participants,context,setting,symmetry) are genre-level defaults, not verified per document - need sign-off before scripting the inserts.
This issue was prepared as part of the UniDive WG1 T1.5 spoken language guidelines effort. Happy to help implement these changes ourselves if that’s easier than doing it on your end - just let us know.