Abstract:
With the emergence of Web 2.0 and social medias, huge amounts of content
data is being generated as we speak. The data piles up in a such degree
that it is not feasible for humans to filter the data manually. As a
consequence, a lot of data is never being used, even though it has
potential value for being used as input for innovating the organization
and its product offerings. A special type of data that is being stored
is ideas coming from users that engage in conversations online. Examples
of places were such conversations at taking place are online message
boards, blogs, facebook fanpages etc. With my research I wish to show
how to use text mining and machine learning to detect ideas. I argue
that even though an idea is an abstract concept, we humans should have
a common way of communicating ideas to each other. In order to test this,
I have set up an experiment where I first show that two independent human
judges are able to agree on what is an idea. I measure the reliability by
Cohens Kappa and we achieve kappa=0.55 ±0.06 at alpha=0.05. Next I show that one
can use our training set as input data for a model committee of linear
support vector machines, and make the committee predict which texts
contains ideas and which do not. I achieve F = 0.625 on my test set.
I conclude that it is possible to detect ideas in text by text mining
machine learning techniques.