CLARIN-NL - text processing

Stylene

Jan Odijk — Fri, 26 Aug 2016 13:30:52 +0000

Stylene

Summary

Stylene is a robust, modular system for stylometry and readability research on the basis of existing techniques for automatic text analysis and machine learning, and the development of a web service that allows researchers in the humanities and social sciences to analyze texts with this system. In this way, the project will make available to researchers recent advances in research on the computational modeling of style and readability.

Background

Stylene consists of an educational demonstration interface and tools for stylometry (authorship attribution and profiling) and readability research for Dutch. The Stylene system consists of a popularization interface for learning to understand stylometric analysis, and web-based interfaces to software for readability and stylometry research aimed at researchers from the humanities and social sciences who don’t want to develop or install such software themselves. Stylene has been created in the context of CLARIN Flanders.

Contacts

Project leader: Prof. dr. Walter Daelemans, CLIPS, Antwerp University  
CLARIN center: n.a.
Help contact : walter.daelemans@uantwerpen.be

Links

Web-sites: http://www.stylene.be/
User scenario's (screencasts, screenshots): n.a.
Manual: [ref to end-report]
Tool/Service link: writing style analysis demo: http://www.clips.ua.ac.be/cgi-bin/stylenedemo.html
Publications:

Research domain

Linguistics

Readability Studies

Discourse

Stylistics

Language

Dutch

Resource tags

service

web-application

web-service

mono-lingual

Tool task

analysis

Authorship Attribution

stylistic analysis

Profiling

Readability Analysis

text processing

Country

Netherlands

Nederlab

Erica Renckens — Mon, 10 Aug 2015 08:07:39 +0000

Nederlab, online laboratory for humanities research on Dutch text collections

Summary

A user-friendly and tool-enriched open access web interface that that aims at containing all digitized texts relevant for the Dutch national heritage and the history of Dutch language and culture (c. 800 - present).

Background

The Nederlab project aims to bring together all digitized texts relevant to Dutch national heritage, the history of Dutch language and culture (c. 800 - present) in one user-friendly and tool-enriched open access web interface, allowing scholars to simultaneously search and analyze data from texts spanning the full recorded history of the Netherlands, its language and culture. The project builds on various initiatives: for corpora Nederlab collaborates with the scientific libraries and institutions, for infrastructure with CLARIN (and CLARIAH), for tools with eHumanities programmes such as Catch, IMPACT and CLARIN (TICCL, frog).
Nederlab will offer a large number of search options with which researchers can find the occurrence of a particular term in a particular corpus or subcorpus. It'll also offer visualization of search results through line graphs, bar graphs, circle graphs, or scatter graphs. Furthermore, this online lab will offer a large set of tools, like tokenization tools, tools for spelling normalization, PoS-tagging tools, lemmatization tools, a computational historical lexicon and indices. Also, the use of (semi-) automatic syntactic parsing, tools for text mining, data mining and sentiment mining, Named Entity Recognition tools, coreference resolution tools, plagiarism detection tools, paraphrase detection tools and cartographical tools is offered
The first version of Nederlab was launched in early 2015, it’ll be expanded until the end of 2017. Nederlab is financed by NWO, KNAW, CLARIAH and CLARIN-NL.

Contacts

Project leader:  prof. dr. Hans Bennis, Meertens Institute
CLARIN center: Meertens Institute
Help contact: hennie.brugman@meertens.knaw.nl

Links

Web-sites: http://www.nederlab.nl/
User scenario's (screencasts, screenshots): n.a.
Manual: http://www.nederlab.nl/onderzoeksportaal/?action=help
Tool/Service link: http://www.nederlab.nl/onderzoeksportaal/
Publications: http://www.nederlab.nl/wp/?page_id=12

Research domain

Cultural Sciences

History

Linguistics

Language

Dutch

Resource tags

data

corpus

text corpus

service

web-application

mono-lingual

Tool task

semantic registry & mapping

analysis

evaluation

annotation

corpus exploration

browse

visualisation

text processing

POS tagging

multiword unit identification

lemmatisation

parsing

co-reference assignment

tokenization

CLARIN centre

Meertens/HuC

Country

Netherlands

OpenConvert

Jan Odijk — Tue, 21 Apr 2015 12:29:10 +0000

Title

Summary

The OpenConvert tools convert to TEI or FOLiA from a number of input formats (alto, text, word, HTML, ePub). The tools are available as a Java command line tool, a web service and a web application.

Background

The OpenConvert Tools were created by INL in the OpenConvert project. The OpenConvert tools convert to TEI or FOLiA from a number of input formats (alto, text, word, HTML, ePub). The tools are available as a Java command line tool, a web service and a web application. Furthermore, as a proof of concept, the website currently provides two annotation tools: a simple Tokenizer for TEI files and a modern Dutch part of speech tagger.

Contacts

Project leader: Jan Theo Bakker 
CLARIN center:
Help contact : helpdesk@clarin.nl

Links

Web-sites: n.a.
User scenario's (screencasts, screenshots): n.a.
Manual: http://openconvert.clarin.inl.nl/openconvert/web/help.html
Tool/Service link:
- web application: http://openconvert.clarin.inl.nl/
- web services: see http://openconvert.clarin.inl.nl/openconvert/web/help.html
- command line tools: https://github.com/INL/OpenConvert
Publications: n.a.

Research domain

Religion Studies

Communication & Media Studies

Computational Linguistics

Cultural Sciences

History

Literary Studies

Philosophy

Political Studies

Linguistics

Language

Dutch

Language independent

Country

Netherlands

@PhilosTEI

Erica Renckens — Wed, 21 Jan 2015 15:05:45 +0000

@philosTEI - TICCLing Philosophy

Summary

An open source, web-based, user-friendly workflow from digital images of text to TEI, which enables building digital philosophy corpora. This workflow uses a combination of an OCRopus / Tesseract webservice for text layout analysis and Optical Character Recognition (OCR) and a multilingual version of TICCL available as webservice TICCLops.

Background

Philosophy and philosophy-informed intellectual history can profit immensely from teaming up with computer scientists and applying computational methods. A new empirical, datadriven and collaborative computational methodology can scale up our research and help making it more efficient and objective. The tool is able to build appropriate high quality, easily accessible, large-scale corpora of multi-script texts in a sustainable format (TEI) which are multi-language and from different historical periods in a reliable, efficient, user-friendly, sustainable and cost-cutting way.

This is demonstrated on a multi-lingual, multi-script corpus of 18th-20th century philosophical texts. This test case corpus contains at least the following books and articles (maybe more will follow):

Wolff (1740) (Latin)
Bolzano (1837) (German)
Frege (1879) (German)
Tarski (1936) (Polish)

Contacts

Project leader:  Dr. Arianna Betti (VU University Amsterdam)
CLARIN center: Huygens ING
Help contact : t.b.a.
Twitter: https://twitter.com/philostei

Links

Web-sites: http://www.axiom.humanities.uva.nl/PhilosTEI.html
User scenario's (screencasts, screenshots): See ? demo menu item on http://ticclops.clarin.inl.nl/philostei/
Manual: http://ticclops.clarin.inl.nl/philostei/about.html
Tool/Service link: http://ticclops.clarin.inl.nl/philostei/
Publications: t.b.a.

Research domain

History

Philosophy

Language

German

Multiple languages

Resource tags

data

corpus

service

web-application

multi-lingual

Tool task

text processing

orthographic normalisation

CLARIN centre

Huygens ING

Country

Netherlands

PaQu

Erica Renckens — Thu, 15 Jan 2015 13:00:04 +0000

PaQu - Parse and Query

Summary

The PaQu web service makes it possible to search in syntactically annotated corpora in Dutch. You can parse your own Dutch text corpus or use one of two corpora provided by the developers.

Background

Please note: This is a preliminary version of this application. Use Lassy (http://dev.clarin.nl/node/1966) for searching the Lassy Treebanks.

PaQu uses the Alpino parser to make treebanks of your own text corpus, and to search in these treebanks using an interface based on the LASSY Word Relations Search interface (http://dev.clarin.nl/node/1966). Two treebanks are already available in the application: Lassy Klein (1M words, manually checked syntactic analysis) and Lassy Groot (700M words, syntactic analysis automatically assigned by Alpino).
PaQu offers two ways to search through the syntactically annotated texts. The first option is to use the search bar to look for word pairs, optionally complemented by their syntactic relationship. The second search option is to use the query language XPath.

Contacts

Project leader: prof.dr. Gertjan van Noord  
Developer: Drs. P. Kleiweg (Groningen University)
CLARIN center: none
Help contact : p.c.j.kleiweg@rug.nl

Links

Web-sites: n.a.
User scenario's (screencasts, screenshots): n.a.
Manual: http://www.let.rug.nl/alfa/paqu/info.html (http://zardoz.service.rug.nl:8067/info.html )
Tool/Service link: http://www.let.rug.nl/alfa/paqu (http://zardoz.service.rug.nl:8067/)
Publications: n.a.

Research domain

Computational Linguistics

Linguistics

Syntax

Language

Dutch

Resource tags

data

corpus

text corpus

service

text processing

mono-lingual

Tool task

annotation

corpus exploration

browse

text processing

grammatical relation assignment

parsing

Country

Netherlands

TTNWW

root — Sun, 15 Jun 2014 13:01:54 +0000

TTNWW - TST Tools voor het Nederlands als Webservices in een Workflow

Summary

TTNWW integrates and makes available existing Language Technology (LT) software components for the Dutch language that have been developed in the STEVIN and CGN projects. The LT components (for text and speech) are made available as web-services in a simplified workflow system that enables researchers without much technical background to use standard LT workflow recipes.

Background

The web services are available in two separate domains: "Text" and "Speech" processing. For "Text", workflows for the following functionality is offered by TTNWW:

Orthographic Normalisation using TICCLops (version CLARIN-NL 1.0)
Part of Speech Tagging, Lemmatisation, Chunking, limited Multiword Unit Recognition, and Grammatical Relation Assignment by Frog (Version 012.012)
Syntactic Parsing (including grammatical relation assignment, limited named entity recognition, and limited multiword unit recognition) by the Alpino Parser (version 1.3)
Semantic Annotation
Named Entity Recognition
Co-reference Assignment

For "Speech", the following workflows are offered:

Automatic Transcription of speech files using a Netherlands Dutch acoustic model
Automatic Transcription of speech files using a Flemish Dutch acoustic model
Conversion of the input speech file to the required sampling rate, followed by automatic transcription

The TTNWW services have been created in a Dutch and Flemish collaboration project building on the results of past Dutch and Flemish projects. The web services are partly deployed in the SURF-SARA BiG-Grid cloud or at CLARIN centres in the Netherlands and at CLARIN VL University partners.

The architecture of the TTNWW portal consists out of several components and follows the principles of Service Oriented Architecture (SOA). The TTNWW GUI front-end is a Flex module that communicates with the TTNWW web-application which keeps track of the different sessions and knows which LT recipes are available. TTNWW communicates assigments (workflow specifications) to the WorkflowService that evaluates the requested workflow and requests the DeploymentSevice to start the required LT web-services. After initialization of the LT web-services, the workflow specification is sent to the Taverna Server, that takes further care of the workflow.

To facilitate the process of wrapping applications that were originally designed as standalone applications into web services, the CLAM (Computational Linguistics Application Mediator) wrapper software allows for easy and transparent transformation of applications into RESTful web services. The CLAM software has extensively been used in the TTNWW project for both text and speech processing tools. With the exception of Alpino and MBSRL all web services work operate on CLAM wrappers.

Given the number of web services involved in the TTNWW project and possibilities offered by the cloud environment the preferred method of delivering the web service installations was delivery of complete virtual machine images by the LT providers. These could be directly uploaded into the cloud environment and thus relieving the CLARIN centres nd LT providers from the original foreseen task of running the webservices themselves. A potential advantage of this method, that has not been exploited in the project yet, is that these images may be also be delivered directly to the end user so these can be run in a local configuration using virtualization software such as VMWare of VirtualBox.

The workflow engine used in the project was Taverna. But build on top of this was a a number of selectable task recipes, following a task oriented approach in line with the premises that users with no or little technical expertise should be able to use the system. In this context, tasks are understood in terms of end results of processes such as semantic role labelling, pos tagging or syntactic analysis and ready-made workflows are constructed that can be readily used by the end user. Contacts

Project leader: Marc Kemps-Snijders (NL), Ineke Schuurman (VL)  
CLARIN center: Meertens Institute (portal) and others (web-services).
Help contact : n.a.

Links

Web-sites: n.a.
User scenario's (screencasts, screenshots): n.a.
Manual: http://yago.meertens.knaw.nl/apache/TTNWW/assets/TTNWW.pdf
Tool/Service link: http://yago.meertens.knaw.nl/apache/TTNWW/
Publications: n.a.

Research domain

Communication & Media Studies

History

Oral History

Linguistics

Morpho-syntax

Orthography

Discourse

Semantics

Syntax

Language

Dutch

Resource tags

service

speech processing

text processing

web-application

web-service

Tool task

parsing

audio-visual processing

speech recognition

speech transcription

up/down sampling

text processing

chunking

co-reference assignment

grammatical relation assignment

multiword unit identification

orthographic normalisation

POS tagging

tokenization

lemmatisation

NE recognition

CLARIN centre

Meertens/HuC

Country

Netherlands

GrNe

root — Sun, 27 Apr 2014 10:14:38 +0000

GrNe: Greek-Dutch dictionary

Summary

Online dictionary (ancient) Greek - Dutch for the letter Pi. Search functions include searches for Greek lemmata, search of Greek declined or conjugated word-forms that lead to the correct lemma (‘lemmatizer’), searches for Dutch words leading to different Greek lemmata, and etymological searches. The dictionary is linked to Logeion, the international website of Greek dictionaries at the University of Chicago. The developers estimate that a complete version of the dictionary will be finished by the end of 2016 and that it will be published by the end of 2017.

Background

A new dictionary ancient Greek – Dutch is currently under construction at Leiden University. The dictionary is being financed through the 2010 Spinoza award of project director Ineke Sluiter. CLARIN funding enabled the digital production of the letter Pi. Currently, the letters beta, gamma, zeta, pi and sigma are available onlineas well, as parts of the letter alpha. The developers estimate that a complete first version of the dictionary will be finished by the end of 2015 and that it will be published by the end of 2016.
The corpus that is being covered by this dictionary covers Greek literature from its beginnings (Homer) and consists of ca. 3.680.000 words (tokens); it includes all classical authors from the 5th and 4th centuries BCE, and a selection of later Greek (selection based on the likelihood that the text will be used by our target groups), but all of the New Testament, Lucian and Plutarch. The dictionary will eventually contain ca. 52.500 headwords. It is based on a thorough comparison of state of the art dictionaries, supplemented with the help of the material from the Thesaurus Linguae Graecae.
Greek morphology is complicated. In order to use a dictionary effectively, a rather high level of initial language competence is necessary for the user to be able to relate the word-form s/he finds in a text to the correct basic lemma form, where the definition of the word can be found. This digital dictionary however has an added ‘lemmatizer’ function, which enables the user to type in the word as found in the text and to be redirected to the correct lemma. The digital resource enables both Greek-Dutch searches and searches for the possible Greek equivalents of Dutch terms. This also makes it possible to explore the relation of semantic fields in Dutch and Greek. E.g., it is possible to locate all Greek words that have ‘courage’ as part of their definition. Furthermore, the digital resource makes it possible to locate different Greek words with the same etymological roots. And finally, the dictionary is linked to the website of the University of Chicago, where a comparison of all Greek-x dictionaries is supported. Here, one can enter a Greek word and be provided with the equivalents and definitions in all the dictionaries that are linked on this website.

Contacts

Project leader:  Prof. dr. Ineke Sluiter (Leiden University)
CLARIN center: University Library of the University of Amsterdam
Help contact: woordenboekgrieks@hum.leidenuniv.nl

Links

Web-sites: http://www.woordenboekgrieks.nl/
User scenario's (screencasts, screenshots): http://www.woordenboekgrieks.nl/hulp.html
Manual: http://www.woordenboekgrieks.nl/hulp.html
Tool/Service link: http://www.woordenboekgrieks.nl/
Publications: n.a.

Research domain

Literary Studies

Linguistics

historical linguistics

Language

Classical Greek

Dutch

Resource tags

data

lexicon

service

web-application

multi-lingual

Tool task

browse

text processing

lemmatisation

CLARIN centre

DANS

Country

Netherlands

NameScape

root — Sun, 27 Apr 2014 07:52:07 +0000

NameScape: Mapping the Landscape of Names in Modern Dutch Literature

Summary

Searching and visualizing Named Entities in modern Dutch novels.

Background

The named entity (NE) tagging and resolution in NameScape enables quantitative and repeatable research where previously only guesswork and anecdotal evidence was feasible. The visualisation module enables researchers with a less technical background to draw conclusions about functions of names in literary work and help them to explore the material in search of more interesting questions (and answers). Users from other communities (sociolinguistics, sentiment analysis, …) also benefit from the NE tagged data, especially since the NE recognizer is available as a web service, enabling researchers to annotate their own research data. Datasets in NameScape (total of 1.129 books):

Corpus Sanders: A corpus of 582 Dutch novels written and published between 1970 and 2009 will.
Corpus Huygens: Consists of 22 novels manually tagged with detailed named entity information. IPR for this corpus do not allow distribution.
Corpus eBooks: Consists of 7000+ Dutch eBooks tagged automatically with basic NER features and person name Part information. IPR for this corpus do not allow distribution.
Corpus SoNaR Books: 105 Dutch books; NE tagged.
Corpus Gutenberg Dutch: Consists of 530 NE tagged TEI files converted from the Epub versions of the corresponding Gutenberg documents.

Contacts

Project leader:  dr. Karina van Dalen-Oskam (Huygens ING)
CLARIN center: Huygens ING
Help contact : karina.van.dalen@huygens.knaw.nl

Links

Web-sites: http://www.namescape.nl/
User scenario's (demo): http://dev.clarin.nl/sites/default/files/Namescape_demonstration_scenari...
Manual: http://ner.namescape.nl/namescape/web/help.html
Metadata (VLO): http://catalog.clarin.eu/vlo/search?fq=collection:Namescape:+mapping+the...
Tool/Service link:
- Named entity tagger: http://ner.namescape.nl/namescape/tagger
- Search: http://search.namescape.nl/
- Barcode browser: http://barcode-browser.namescape.nl/index.xql
- Visualiser: http://visualizer.namescape.nl/
Publications: Karina van Dalen-Oskam (2013), Nordic Noir: a background check on Inspector Van Veeteren, 31 May 2012, http://blog.namescape.nl/?p=47

Research domain

Computational Linguistics

Literary Studies

Language

Dutch

Resource tags

corpus

text corpus

service

text processing

web-application

web-service

mono-lingual

Tool task

text processing

NE recognition

CLARIN centre

Huygens ING

Country

Netherlands

Adelheid

root — Sun, 20 Apr 2014 15:34:46 +0000

Adelheid: A Distributed Lemmatizer for Historical Dutch

Summary

With this web-application an end user can have historical Dutch texts tokenized, lemmatized and part-of-speech tagged, using the most appropriate resources (such as lexica) for the text in question. For each specific text, the user can select the best resources from those available in CLARIN, wherever they might reside, and where necessary supplemented by own lexica.

Background

Research on transcribed manuscripts from the early centuries of the use of the Dutch language, say up to the middle of the 19th century, is seriously hampered by a lack of standardized orthography. As an example, in the corpus Van Reenen-Mulder, the 1055 occurrences of the lemma ‘penning’ (which would be important for the last of the research questions) show 56 different spellings, all of which a researcher would have to guess in order to find the corresponding text passages. The accessibility of the material is enhanced enormously by adding annotation layers showing normalized representations for word boundaries, lemmas and preferably also part-of-speech tags. A fully manual annotation is not only expensive, but also runs the risk of deviations from the agreed upon normalizations, e.g. using the frequent form ‘penninc’ as lemma instead of ‘penning’. Any annotation should therefore in principle be done semi-automatically, with an automatic system providing the most likely options and a human specialist providing further disambiguation only when deemed necessary.

The Adelheid web-application calls upon web-services performing tokenization, tagging and lemmatization. An input text is first split into tokens by a tokenizing component, then all possible lemma-tag combinations for each token are derived in a lexical component, after which the likelihood of each lemma-tag combination is estimated by a contextual component. Each of these three components accesses specific data resources and can be expected to work best if the resources are tuned to the text’s domain and time period. The tokenizing and contextual component will be atomic web-services. The lexical component, however, will be a web-service which allows for a selection of compatible lexical resources, in the form of lexica relating specific word forms to lemma-tag combinations that might underlie them, each possibly with a numerical indication of its relative likelihood. The main lexica for the demonstrator scenario, tuned for 14th century charters, have been created by Hans van Halteren on the basis of the Corpus van Reenen-Mulder, basing the likelihood indication on counts in the corpus and statistics on spelling variation. It’s also possible for the end user to provide an own lexicon, e.g. concerning lists of specific terms or proper names, but this will have to conform to the format of the current lexicons. Metadata describing the format and provenance of the lexicons will be created in IMDI/CMDI MD formats.

Contacts

Project leader: Dr. Hans van Halteren (Radboud University)  
CLARIN center: Max Planck Institute for Psycholinguistics (MPI)
Help contact : http://tla.mpi.nl/contact (service)

Links

Web-sites: http://adelheid.ruhosting.nl/
User scenario's (screencasts, screenshots): http://adelheid.ruhosting.nl/download/Adelheid1_DemoScenario.pdf
Manual:
- Tagger-Lemmatizer: http://adelheid.ruhosting.nl/download/Adelheid1_TaggerManual.pdf
- Tagset: http://adelheid.ruhosting.nl/download/Adelheid1_TagsetManual.pdf
- Annotation Tool: http://adelheid.ruhosting.nl/download/Adelheid1_AnntoolManual.pdf
Tool/Service link:
- Tagger-Lemmatizer: http://hdl.handle.net/1839/00-SERV-0000-0000-0006-4
- Visualisation tool: http://hdl.handle.net/1839/00-SERV-0000-0000-0009-D
Publications: n.a.

Research domain

Linguistics

Syntax

historical linguistics

Language

Dutch

Resource tags

service

web-application

web-service

Tool task

annotation

visualisation

text processing

POS tagging

tokenization

lemmatisation

CLARIN centre

MPI for Psycholinguistics

Country

Netherlands

TICClops

root — Sun, 20 Apr 2014 14:24:40 +0000

TiCCLops: Text-Induced Corpus Clean-up online processing system

Summary

TICCL (Text Induced Corpus Clean-up) is a system that is designed to search a corpus for all existing variants of (potentially) all words occurring in the corpus. This corpus can be one text, or several, in one or more directories, located on one or more machines. TICCL creates word frequency lists, listing for each word type how often the word occurs in the corpus. These frequencies of the normalized word forms are the sum of the frequencies of the actual word forms found in the corpus. TICCL is a system that is intended to detect and correct typographical errors (misprints) and OCR errors (optical character recognition) in texts. When books or other texts are scanned from paper by a machine, that then turns these scans, i.e. images, into digital text files, errors occur. For instance, the letter combination `in' can be read as `m', and so the word `regeering' is incorrectly reproduced as `regeermg'. TICCL can be used to detect these errors and to suggest a correct form.

Background

Text-Induced Corpus Clean-up (TICCL) was developed first as a prototype at the request of the Koninklijke Bibliotheek - The Hague (KB) and reworked into a production tool according to KB specifications (currently at production version 2.0) mainly during the second half of 2008. It is a fully functional environment for processing possibly very large corpora in order to largely remove the undesirable lexical variation in them. It has provisions for various input and output formats, is flexible and robust and has very high recall and acceptable precision. As a spelling variation detection system it is to the developer’s knowledge unique in making principled use of the input text as possible source for target output canonical forms. As such it is far less domain-sensitive than other approaches: the domain is largely covered by the input text collection.

Contacts

Project leader: dr. Martin Reynaert (Tilburg University)  
CLARIN center: The Institute for Dutch Lexicology (INL)
Help contact: n.a.

Links

Web-sites: n.a.
User scenario's (screencasts, screenshots): n.a.
Manual: http://ticclops.uvt.nl/ticclops_manual.v101.pdf
Tool/Service link: (classic CLAM interface) http://ticclops.clarin.inl.nl/ticclops/
Tool/Service link: (@PHILOSTEI interface) http://ticclops.clarin.inl.nl/philostei/
Publications:
- Reynaert, M. (2008). All, and only, the errors: More complete and consistent spelling and OCR-error correction evaluation. In: Proceedings of the Sixth International Language Resources and Evaluation (LREC’08), Marrakech, Morocco.
- Reynaert, M. (2010). Character confusion versus focus word-based correction of spelling and ocr variants in corpora. International Journal on Document Analysis and Recognition, pp 1-15, URL http://dx.doi.org/10.1007/s10032-010-0133-5, 10.1007/s10032-010-0133-5