home edit page issue tracker

This page pertains to UD version 2.

It appears that you have Javascript disabled. Please consider enabling Javascript for this page to see the visualizations.

Metadata harmonisation: align spoken-data fields with UniDive naming conventions

Back to Persian Seraji · Back to index

Repo: https://github.com/UniversalDependencies/UD_Persian-Seraji

Cross-posting from the UniDive WG1 T1.5 (spoken language guidelines) metadata harmonisation review. We compared UD_Persian-Seraji’s current CoNLL-U metadata against the proposed naming conventions (see also the full treebank status table). This is a suggestion for maintainers to review - feel free to push back on anything that doesn’t fit the corpus. The comparison was carried out semi-automatically with the help of Claude (Anthropic); errors or misunderstandings are possible, so please double-check anything unclear.

1. Is the spoken portion identifiable?

This treebank’s genre metadata lists spoken alongside news, fiction, medical, legal, social, nonfiction, but its README only mentions written data, and its .conllu files don’t explicitly mark which sentences are spoken.

Finding: No genre-like field or document_id found at all, and no comment-level metadata beyond sent_id/text detected.

Suggestion: Could you confirm whether any part of this corpus is actually spoken, and if so, point us to which sentences/documents? Otherwise, please consider dropping spoken from the genre metadata.

2. Sentence-level (naming conventions)

Field	Suggestion
`translit`	corpus-specific (sentence-level) - verify against metadata.html
`text_en`	rename to `text_eng` (ISO 639-3)

Implementation notes

Quick search & replace

text_en → text_eng: python3 workgroups/spoken-data/scripts/harmonize_metadata.py rename-comment DIR --map text_en=text_eng --write

Needs manual input from maintainers

Whether any part of the corpus is actually spoken, or spoken should be dropped from genre - this is a factual question, no script can resolve it.
translit - corpus-specific field, needs confirmation of what it represents before proposing a name.

This issue was prepared as part of the UniDive WG1 T1.5 spoken language guidelines effort. Happy to help implement these changes ourselves if that’s easier than doing it on your end - just let us know.