bigcloudsmallcloud smallcloud
server

There is an increasing demand in being able to effectively access, evaluate, compare, visualize and integrate multiple text mining systems in order to process natural language document collections.

Starting with the BioCreative MetaServer (BCMS), the first distributed prototype platform requesting, retrieving and unifying biomedical textual annotations developed under the umbrella of the BioCreative II protein-protein interaction task, several BioCreative tasks have tried to promote the development of online text annotation servers.

BCMS metaserver
conceptual Framework

Challenges to address

Despite the relevance of previous efforts to create an annotation metaserver, several crucial aspects have not sufficiently or only partially been addressed. Some of the issues include:

  • Continuous evaluation of text mining systems to promote more stable annotation tools
  • Facilitate the interoperability of multiple text annotation systems at the technical level and through the design of compatible annotation schemas
  • Extraction of textual content from heterogeneous sources, such as scientific literature (PubMed abstracts) or patent abstracts
  • Visualization and detailed comparative evaluation assessment of automatic and manual annotations
  • Harmonization of multiple biomedical text annotations
  • Usage of standard evaluation metrics for the performance evaluation of component-level tasks
  • Management of users corresponding to developers of text annotation systems

Aspects like continuous evaluation, interoperability or harmonization should be gradually addressed by the BioCreative-BeCalm tasks, with the aim of using community-driven evaluation challenges as the foundation towards the implementation of an open, web-based infrastructure for biomedical text annotations. The BeCalm infrastructure will gradually include additional annotation services based on the biomedical end user community requests (researches, database annotators, data providers, etc).

BeCalm  task

Task

Developer

Text miners and developers

BeCalm Benchmarking

User community

Policy and strategy

BeCalm regulation

Regulation

Before using BeCalm platform it is important for the users read the terms and conditions of the web page, due to the privacy of their data as a team or an annotation server and the documents they will work with.

BeCalm Feedback

Feedback

We always wanted to let the users know that we were able to use their feedback to make better the BeCalm platform. We really appreaciate the time the users take to help us improve, so, feel free to send us your opinion!