CLARIN-NL - sentence splitting

Country

Germany

CMDI File Link

License

not specified

Inventory Scope

global

Distribution Type

not specified

Tokenizer/Sentences - Alpino

root — Mon, 08 Apr 2019 08:39:02 +0000

Description

Tokenizer and sentence splitter from Alpino.

CLARIN National Project

not specified

CLARIN centre

SfS: Uni-Tuebingen

Research domain

Linguistics

Linguistic Subject

Morpho-syntax

Syntax

Tool task

tokenisation

Country

Germany

Tool Type

web service

Research Phase

Enriching Data

Tool status

development

Input Language

Dutch

Access Contact

/mailto:/

Project Contact

/mailto:/

Creator Contact

/mailto:/

Source code

http://portal.clarin.nl/not%20specified

Resource

CMDI File Link

License

not specified

Inventory Scope

global

Distribution Type

not specified

Tokenizer/Sentences - OpenNLP Project

root — Mon, 08 Apr 2019 08:39:02 +0000

Description

Tokenizer/sentences from the OpenNLP project. The 'newlineBounds' parameter treats newlines as a hard break (a sentence boundary).

CLARIN National Project

not specified

CLARIN centre

SfS: Uni-Tuebingen

Research domain

Linguistics

Linguistic Subject

Morpho-syntax

Syntax

Tool task

tokenisation

Country

Germany

Tool Type

web service

Research Phase

Enriching Data

Tool status

production

Input Language

unclear

Access Contact

/mailto:/

Project Contact

/mailto:/

Creator Contact

/mailto:/

Source code

http://portal.clarin.nl/not%20specified

Resource

CMDI File Link

License

not specified

Inventory Scope

global

Distribution Type

not specified

Tokenizer

root — Mon, 08 Apr 2019 08:38:55 +0000

CLARIN National Project

not specified

CLARIN centre

IMS: University of Stuttgart

Research domain

Linguistics

Linguistic Subject

Morpho-syntax

Syntax

Tool task

tokenisation

Country

Germany

Tool Type

web service

Research Phase

Enriching Data

Tool status

production

Input Language

Czech

Slovenian

Hungarian

Italian

German

English

Access Contact

/mailto:/

Project Contact

/mailto:/

Creator Contact

/mailto:/

Source code

http://portal.clarin.nl/not%20specified

Resource

CMDI File Link

License

not specified

Inventory Scope

global

Distribution Type

not specified

Ucto

root — Mon, 08 Apr 2019 08:38:55 +0000

https://raw.githubusercontent.com/LanguageMachines/ucto/master/logo.svg

Title

Ucto Tokeniser

Description

Ucto tokenizes text files: it separates words from punctuation, and splits sentences. This is one of the first tasks for almost any Natural Language Processing application. Ucto offers several other basic preprocessing steps such as changing case that you can all use to make your text suited for further processing such as indexing, part-of-speech tagging, or machine translation. The tokeniser engine is language independent. By supplying language-specific tokenisation rules in an external configuration file a tokeniser can be created for a specific language. Ucto comes with tokenization rules for English, Dutch, French, Italian, and Swedish; it is easily extendible to other languages. It recognizes dates, times, units, currencies, abbreviations. It recognizes paired quote spans, sentences, and paragraphs. It produces UTF8 encoding and NFC output normalization, optionally accepts other encodings as input. Optional conversion to all lowercase or uppercase. Ucto supports FoLiA XML.

Project

CLARIN-NL

CLARIAH-CORE

CLARIN National Project

CLARIN-NL

CLARIAH-CORE

CLARIN centre

none yet

Research domain

Linguistics

Linguistic Subject

Syntax

Tool task

tokenisation

Country

Netherlands

Tool Type

local desktop

Research Phase

Enriching Data

Tool status

published

Input format

application/pdf

application/msword

text/folia+xml

text/plain

Output format

text/plain

Input Language

Dutch

Swedish

Russian

Spanish

Portuguese

English

German

French

Italian

Version

v0.13

Access Contact

Project Contact

/mailto:/

Creator Contact

Maarten van Gompel, Ko van der Sloot and Antal van den Bosch. 2017. Ucto: Unicode Tokenizer. version 0.9.6. Reference Guide. Technical Report, Jan 23, 2017.

Ko van der Sloot

Documentation

Maarten van Gompel, Ko van der Sloot, Antal van den Bosch (2012). Ucto: Unicode Tokeniser. Reference Guide. ILK Technical Report 12-05.

readme

releaseNotes

issueTracker

contIntegration

Source code

https://github.com/LanguageMachines/ucto

Resource

CMDI File Link

License

GNU GPL

Inventory Scope

local

Ucto Engine

root — Mon, 08 Apr 2019 08:38:55 +0000

https://raw.githubusercontent.com/LanguageMachines/ucto/master/logo.svg

Title

Ucto Tokeniser Engine

Description

The Ucto tokenisation engine is a language-independent engine that, given an external configuration file with tokenisation rules for a specifc language ,yields a tokenizer for that language that tokenizes text files: it separates words from punctuation, and splits sentences. This is one of the first tasks for almost any Natural Language Processing application. Ucto offers several other basic preprocessing steps such as changing case that you can all use to make your text suited for further processing such as indexing, part-of-speech tagging, or machine translation. The tokeniser engine is language independent. By supplying language-specific tokenisation rules in an external configuration file a tokeniser can be created for a specific language. Ucto comes with tokenization rules for English, Dutch, French, Italian, and Swedish; it is easily extensible to other languages. It recognizes dates, times, units, currencies, abbreviations. It recognizes paired quote spans, sentences, and paragraphs. It produces UTF8 encoding and NFC output normalization, optionally accepts other encodings as input. Optional conversion to all lowercase or uppercase. Ucto supports FoLiA XML.

Project

CLARIN-NL

CLARIAH-CORE

CLARIN National Project

CLARIN-NL

CLARIAH-CORE

CLARIN centre

none yet

Research domain

Linguistics

Linguistic Subject

Syntax

Tool task

tokenisation

Country

Netherlands

Tool Type

local desktop

Research Phase

Enriching Data

Tool status

published

Input format

application/pdf

application/msword

text/folia+xml

text/plain

Output format

text/plain

Input Language

Language independent

Version

v0.13

Access Contact

Project Contact

/mailto:/

Creator Contact

Maarten van Gompel, Ko van der Sloot and Antal van den Bosch. 2017. Ucto: Unicode Tokenizer. version 0.9.6. Reference Guide. Technical Report, Jan 23, 2017.

Ko van der Sloot

Documentation

Maarten van Gompel, Ko van der Sloot, Antal van den Bosch (2012). Ucto: Unicode Tokeniser. Reference Guide. ILK Technical Report 12-05.

readme

releaseNotes

issueTracker

contIntegration

Source code

https://github.com/LanguageMachines/ucto

Resource

CMDI File Link

License

GNU GPL

Inventory Scope

local

PICCL

root — Mon, 08 Apr 2019 08:38:55 +0000

Title

PICCL: Philosophical Integrator of Computational and Corpus Libraries

Description

PICCL is a set of workflows for corpus building through OCR, post-correction, modernization of historic language and Natural Language Processing. It combines Tesseract Optical Character Recognition, TICCL functionality and Frog functionality in a single pipeline. Tesseract offers Open Source software for optical character recognition. TICCL (Text Induced Corpus Clean-up) is a system that is designed to search a corpus for all existing variants of (potentially) all words occurring in the corpus. This corpus can be one text, or several, in one or more directories, located on one or more machines. TICCL creates word frequency lists, listing for each word type how often the word occurs in the corpus. These frequencies of the normalized word forms are the sum of the frequencies of the actual word forms found in the corpus. TICCL is a system that is intended to detect and correct typographical errors (misprints) and OCR errors (optical character recognition) in texts. When books or other texts are scanned from paper by a machine, that then turns these scans, i.e. images, into digital text files, errors occur. For instance, the letter combination `in' can be read as `m', and so the word `regeering' is incorrectly reproduced as `regeermg'. TICCL can be used to detect these errors and to suggest a correct form. Frog enriches textual documents with various linguistic annotations.

Project

CLARIN-NL

CLARIAH-CORE

Nederlab

CLARIN National Project

CLARIN-NL

CLARIAH-CORE

CLARIN centre

none yet

Research domain

Linguistics

Philosophy

Literary Studies

Religion Studies

History

Linguistic Subject

Orthography

Morphology

Syntax

Tool task

optical character recognition

orthographic normalisation

tokenisation

shallow parsing

lemmatisation

Country

Netherlands

Tool Type

local desktop

Research Phase

Enriching Data

Tool status

published

Input format

application/pdf

image/tiff

text/plain

text/folia+xml

image/vnd.djvu

Output format

text/folia+xml

Input Language

Dutch

Swedish

Russian

Spanish

Portuguese

English

German

French

Italian

Finnish

Modern Greek

Classical Greek

Icelandic

German (Fraktur)

Latin

Romanian

Version

v0.6.4

Access Contact

Martin Reynaert

Project Contact

/mailto:/

Creator Contact

Martin Reynaert

Ko van der Sloot

Documentation

Information Page

readme

releaseNotes

issueTracker

contIntegration

Source code

https://github.com/LanguageMachines/piccl

Publications

Martin Reynaert, Maarten van Gompel, Ko van der Sloot and Antal van den Bosch. 2015. PICCL: Philosophical Integrator of Computational and Corpus Libraries. Proceedings of CLARIN Annual Conference 2015, pp. 75-79. Wrocław, Poland. http://www.nederlab.nl/cms/wp-content/uploads/2015/10/Reynaert_PICCL-Philosophical-Integrator-of-Computational-and-Corpus-Libraries.pdf

Resource

LandingPage

CMDI File Link

License

GNU GPL

Inventory Scope

local

Alpino

root — Mon, 08 Apr 2019 08:38:55 +0000

Title

Alpino: a dependency parser for Dutch

Description

Alpino is a dependency parser for Dutch, developed in the context of the PIONIER Project Algorithms for Linguistic Processing.

Project

ALP

CLARIAH-CORE

CLARIN National Project

CLARIAH-CORE

CLARIN centre

none yet

Research domain

Linguistics

Linguistic Subject

Syntax

Tool task

parsing

lemmatisation

tokenisation

Country

Netherlands

Tool Type

web application

Research Phase

Enriching Data

Tool status

published

Input format

text/plain

Output format

text/xml

Input Language

Dutch

Version

unknown

Access Contact

Gertjan van Noord

Project Contact

Gertjan van Noord

/mailto:/

Creator Contact

Gertjan van Noord

Gosse Bouma

Robert Malouf

Geert Kloosterman

Documentation

Alpino User Guide (HTML)

Alpino User Guide (PDF)

Alpino Annotation Guide

Alpino Treebank Tools User Guide (HTML)

Alpino Treebank Tools User Guide (PDF)

Syntactic Annotation Reference Manual

Documentation for ADT (Abstract Dependency Trees)

Source code

git://urd.let.rug.nl/Alpino.git

https://github.com/rug-compling/alpino-docker

Publications

Bouma, G., van Noord, G. J. M. and Malouf, R. 2001.Alpino: Wide-coverage computational analysis of Dutch. in Daelemans, W., Simaan, K., Veenstra, J. and Zavrel, J. (eds.). Computational Linguistics in the Netherlands 2000. Amsterdam: Rodopi, p. 45-59 15 p. (LANGUAGE AND COMPUTERS : STUDIES IN PRACTICAL LINGUISTICS)

Robert Malouf and Gertjan van Noord. Wide Coverage Parsing with Stochastic Attribute Value Grammars. In: IJCNLP-04 Workshop Beyond Shallow Analyses - Formalisms and statistical modeling for deep analyses.

Leonoor van der Beek, Gosse Bouma, and Gertjan van Noord. Een brede computationele grammatica voor het Nederlands. 2002. Nederlandse Taalkunde. https://www.let.rug.nl/~vannoord/papers/taalkunde.pdf .

Resource

LandingPage

Resource

CMDI File Link

License

GNU LGPL

Inventory Scope

local

Alpino (CLST web service and application)

root — Mon, 08 Apr 2019 08:38:55 +0000

Title

Alpino: a dependency parser for Dutch (CLST web service and application)

Description

This is a web service and web application to the Alpino dependency parser for Dutch, developed in the context of the PIONIER Project Algorithms for Linguistic Processing.

Project

CLARIAH-CORE

CLARIN National Project

CLARIAH-CORE

CLARIN centre

none yet

Research domain

Linguistics

Linguistic Subject

Syntax

Tool task

parsing

lemmatisation

tokenisation

Country

Netherlands

Tool Type

web application

Research Phase

Enriching Data

Tool status

published

Input format

text/plain

Output format

text/xml

Input Language

Dutch

Version

unknown

Access Contact

Project Contact

/mailto:/

Creator Contact

Documentation

Info Page of the RUN Web application and service for Alpino

Source code

view-source:https://webservices-lst.science.ru.nl/alpino/info/

Resource

CMDI File Link

License

GNU LGPL

Inventory Scope

local

Frog

root — Mon, 08 Apr 2019 08:38:48 +0000

http://dev.clarin.nl/sites/default/files/froglogo.svg

Title

Frog: An advanced Natural Language Processing Suite for Dutch (Web Service and Application)

Description

Frog is an integration of memory-based natural language processing (NLP) modules developed for Dutch. It performs automatic linguistic enrichment such as part of speech tagging, lemmatisation, named entity recognition, shallow parsing, dependency parsing and morphological analysis. All NLP modules are based on TiMBL.

Project

CGN

CLARIN-NL

CLARIAH-CORE

CLARIN National Project

CLARIN-NL

CLARIAH-CORE

CLARIN centre

none yet

Research domain

Linguistics

Linguistic Subject

Syntax

Tool task

shallow parsing

lemmatisation

tokenisation

Country

Netherlands

Tool Type

web application

Research Phase

Enriching Data

Tool status

published

Input format

application/pdf

application/msword

text/folia+xml

text/plain

Output format

text/csv

Input Language

Dutch

Version

v0.15

Access Contact

Project Contact

/mailto:/

Creator Contact