Implementación de técnicas para minería de texto usando modelos de tópicos.

Abstract

Text mining allows the extraction of relevant information from large volumes of data. To carry out this process, text mining exploits techniques and methodologies in the areas of information retrieval, information extraction and corpus-based computational linguistics. The main input for text mining is semi-structured and unstructured documents. This input adds complications to the analysis process, since the rules of natural language must be considered. There are probabilistic methods of analysis for the extraction of relevant characteristics of text documents, which are not affected by the context in which the words are found, compared to the methods of topic analysis that are based on linguistic analysis. that these models are based on the criteria that documents are a "bag of words". One of the models of the mentioned type is LDA (Latent Dirichlet Allocation), which assumes that a document is a random mix of latent themes, where each theme is characterized by a distribution over words; This distribution of words is linked to random variables. LDA intends to find these variables to determine the existing topics in a set of documents.