A curated list of resources for the processing of Slovak language.
- Slovak resources by Essential Data
- Slovak speech and language processing at KEMT FEI TUKE with tools, demos and language resources.
- Slovak National Corpus
- Created for training mT5 model.
- Contains 67GB Slovak part. bv
- Available in Tensorflow format and HuggingFace JSON format.
- Can be downloaded from allenai/allennlp#5056 using git LFS.
- automatic POS (SNK)
- source: web
- can be downloaded from Clarin
- deduplicated
- source: Common Crawl
- automatic lemmatization, MSD, POS (AUT, ensemble lemmatization+POS)
- source: web
- no annotattion
- twitter part
- web corpora
- no annotation
- no deduplication
- lemmatized, POS+MSD+syntactically tagged, deduplicated corpus: https://www.juls.savba.sk/hpltskcorp.html
- Manually annotated clone of SQUAD 2.0
- Contains "unanswerable questions"
- 92k items
- Machine translation of SQUAD 2.0 Database
- 140k annotated items
- Slovak version of the Question to Declarative Sentence (QA2D).
- Machine-translated using DeepL service.
- https://arxiv.org/abs/2312.10171
- 70k questions and answers
- 5 000 yes-no questions
- Each example is a triplet of (question, passage, answer), with the title of the page as optional additional context.
- Machine translated
- Can be used also for summarization
- Korpus obsahuje texty právnych predpisov (aktuálnych aj minulých) v slovenčine. Okrem automatickej lematizácie a morfologickej anotácie je korpus anotovaný aj syntakticky
- Citácia: GARABÍK, Radovan: Corpus of Slovak legislative documents. Jazykovedný časopis, 2022, Vol. 73, No 2, pp. 175-189.
- 45 miliónov tokenov
- form, lemma, POS+MSD (SNK)
- source: SNK
- form, lemma, POS+MSD (SNK)
- source: SNK
- form, lemma, POS+MSD
- source: JÚĽŠ SAV
- form, lemma, POS+MSD
- source: JÚĽŠ SAV
- form, lemma, POS+MSD
- source: JÚĽŠ SAV
- tokenization, segmentation, UPOS, XPOS (SNK), UD, lemma
- manual annotation
- format: conllu, PDT tagset
- source: SNK
Reference:
- Gajdošová, Katarína; Šimková, Mária and et al., 2016, Slovak Dependency Treebank, LINDAT/CLARIAH-CZ digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University, http://hdl.handle.net/11234/1-1822.
A conversion of the Slovak Dependency Treebank into Universal Dependency tagset.
- GitHub page
- tokenization, segmentation, UPOS, XPOS (SNK), UD, lemma
- manual annotation
- format: conllu, UD tagset
- source: SNK
Reference:
- Zeman, Daniel. (2017). Slovak Dependency Treebank in Universal Dependencies. Journal of Linguistics/Jazykovedný casopis. 68. 10.1515/jazcas-2017-0048.
- tokenization, segmentation, UPOS, XPOS (SNK), UD, lemma
- format: conllu
- source: Slovak UD, SNK
- form, lemma, POS (Multext East)
- Corpus of the Šariš dialect
- 4.7k examples.
- authors: Viktória Ondrejová and Marek Šuppa
- 62 languages, 1,782 bitexts
- Slovak part contains 100 mil. tokens
- source: Europarl
- speech, vectors, language
- automatic POS (SNK)
- source: Acquis, Europarl, EU-journal, EC-Europa, OPUS
- automatic POS (SNK)
- source: Acquis, Europarl, EU-journal, EC-Europa, OPUS
- sentence aligned, POS
- Bulgarian, Czech, English, Estonian, Hungarian, Macedonian, Persian, Polish, Romanian, Serbian, Slovak, Slovenian
- source: "1984" novel
Reference:
Erjavec, Tomaž; et al., 2010, MULTEXT-East "1984" annotated corpus 4.0, Slovenian language resource repository CLARIN.SI, ISSN 2820-4042, http://hdl.handle.net/11356/1043.
- Parallel web Corpus with Slovak Part
- 3.3 mil sentences English-Slovak
- Unsupervised processing of Wikipedia to obtain parallel corpora
- Used LASER embeddings.
- 85 different languages, 1620 language pairs, 134M parallel sentences, out of which 34M are aligned with English
- Machine translated by OPUS-en-sk model
- Sentence similarity dataset contains two sentences with a floating-point number between 0 and 5 as a target, where the highest number means higher similarity. The dataset contains train: 5 749, validation: 1 500 and test: 1 379 examples.
- Referenced from this report by J. Agarský.
- 5k items
- positive and negative class
- Reference: Samuel Pecar, Marian Simko, and Maria Bielikova. 2019. Improving Sentiment Classification in Slovak Language. In Proceedings of the 7th Workshop on Balto-Slavic Natural Language Processing, pages 114–119, Florence, Italy. Association for Computational Linguistics.
- Unknown/undocumented source
- positive/negative
- source: Twitter
- 3 categories - positive, negative, neutral
- Dataset contains totally 1 588 comments in Slovak language from various Facebook pages. The texts are annotated by 5 categories.
- Machine translated
- Sentiment analysis dataset, binary classification task: positive sentiment, negative sentiment. It includes reviews from 7 categories with positive, neutral and negative sentiment labels.
- Source: Slovakbert auxiliary repository BY Matúš Pikuliak, Štefan Grivalský, Martin Konôpka, Miroslav Blšták, Martin Tamajka, Viktor Bachratý, Marián Šimko, Pavol Balážik, Michal Trnka, and Filip Uhlárik. , 2021
- Referenced from this report by J. Agarský.
- CSFD Movie Reviews
- 25k items
- 13k items
- Crowdsourced hate and offensive speech in Facebook comments
- binary classification
- Multilingual fact checking database with Slovak part
- Contains 28k posts in 27 languages from social media, 206k fact-checks in 39 languages written by professional fact-checkers, as well as 31k connections between these two groups.
9.1k Czech, 2.8k Polish and 12.6k Slovak labeled claims with reasoning: demagog.zip (~16.5 MB)
- Machine translated facts with evidence representend as references to Wikipedia pages.
- 350k items
- Machine translation of the Stanford Alpaca
- 40k annotations
- CNEC 2.0 Czech model machine translated to Slovak & filtered
- CNEC entity hierarchy
- source: JÚĽŠ SAV
- models: CNEC 2.0 cs2sk, morphodita SNK
- source: JÚĽŠ SAV, SNK
- 8,48k sentences
- Annotated by a large langauge model
- PER, ORG, LOC annotations
- 8,48k train, 1k dev and 2k test sentences from Universal Dependencies
- Human annotated by 2 annotators as part of the Universal NER project
- PER, ORG, LOC annotations
- 10k manually annotated items from Wikipedia
translated version of the original CONLL2003 dataset (translated from English to Slovak via Google translate
- automatically annotated Wikipedia for Named Entities
- massively multilingual
- Slovak part has 500k sentences.
- Reference: Al-Rfou, Rami, et al. "Polyglot-NER: Massive multilingual named entity recognition." Proceedings of the 2015 SIAM International Conference on Data Mining. Society for Industrial and Applied Mathematics, 2015.
- download data
- automatic annotation
- source: Wikipedia
- Manually annotated set
- Diploma thesis at Commeius University
- PER, ORG, LOC, MISC annotations
- cca 7k sentences.
- corpus of spelling errors created from edits in Wikipedia
- spelling errors are sorted into 5 categories,
- Multilingual Summarization Dataset
- Slovak part has 1.3k rows.
- 200k of news article summaries
- Reference: Marek Suppa and Jergus Adamec. 2020. A Summarization Dataset of Slovak News Articles. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 6725–6730, Marseille, France. European Language Resources Association.