home edit page issue tracker

This page pertains to UD version 2.

It appears that you have Javascript disabled. Please consider enabling Javascript for this page to see the visualizations.

Metadata harmonisation: align spoken-data fields with UniDive naming conventions

Back to Turkish_German SAGT · Back to index

Repo: https://github.com/UniversalDependencies/UD_Turkish_German-SAGT

Cross-posting from the UniDive WG1 T1.5 (spoken language guidelines) metadata harmonisation review. We compared UD_Turkish_German-SAGT’s current CoNLL-U metadata against the proposed naming conventions (see also the full treebank status table). This is a suggestion for maintainers to review - feel free to push back on anything that doesn’t fit the corpus. The comparison was carried out semi-automatically with the help of Claude (Anthropic); errors or misunderstandings are possible, so please double-check anything unclear.

1. Document-level (naming conventions)

No document_id exists, but it can be derived directly from the sent_id prefix: sent_id follows <doc-id>-<number> (e.g. TRDE-CS-C15-0001), with 48 distinct document ids (TRDE-CS-C01 … TRDE-CS-V06).

Field	Suggestion
—	derive `# document_id` from the `sent_id` prefix (everything before the trailing `-<number>`)

2. Sentence-level (naming conventions)

Field	Suggestion
`NOTE`	corpus-specific (sentence-level) - verify against metadata.html
`annotated with partaxis`	corpus-specific (sentence-level) - verify against metadata.html

3. Token-level (naming conventions)

Field	Suggestion
`Lang`	rename to `Lang`

Implementation notes

Needs a small script: derive # document_id from the sent_id prefix:
```
python3 workgroups/spoken-data/scripts/harmonize_metadata.py derive-document-id \
    UD_Turkish_German-SAGT --pattern '^(?P<doc>.+)-\d+$' --write
```
Verified against the local clone (dry-run, all three splits): derives 17/16/15 documents (dev/test/train) with no unmatched sent_ids.
Needs manual input from maintainers: NOTE and annotated with partaxis (sentence-level, corpus-specific) - need a name decision against the naming conventions before any rename can be scripted.
No action needed: Lang is already using the standard name.

This issue was prepared as part of the UniDive WG1 T1.5 spoken language guidelines effort. Happy to help implement these changes ourselves if that’s easier than doing it on your end - just let us know.