Mining Web Data for Image Semantic Annotation
Roberto Basili, Riccardo Petitti and Dario Saracino
The 10th Congress of the Italian Association for Artificial Intelligence - Special Track on Intelligent Access to Multimedia Information
Roma, Italy, September 10-13, 2007
Abstract
Multimedia documents as they can be generally found in the Web are characterized by visual and linguistic properties expressed by images (or video) and texts combined into structured pages. The intrinsic properties characterizing the individual media levels are often insufficient to fully describe the overall conveyed information. While technologies have been widely investigated to mine Web information at the individual levels, the suitable combinations that better generalize semantic and applicative aspects of the overall multimedia material are the focus of on-going research. In this paper unsupervised techniques for the discovery of concepts, topics and analogies among multimedia documents are proposed able to combine different media levels. In particular geometrical models of visual features are here integrated with textual descriptions derived through Information Extraction processes from Web pages. The purpose is to exploit regularities, similarities or analogies at both levels and bootstrap the automatic discovery as an unsupervised clustering process within the resulting richer information space.
While the higher expressivity of the combined individual descriptions increases the complexity of the clustering algorithms, methods for dimensionality reduction (e.g. PCA) can be still applied effectively. Evaluation of the methodology is carried out on large collections in heterogeneous domains with different degrees of complexity. Early experimental results confirm that the proposed model outperforms other methods acting on the individual levels. The resulting combination of Information Extraction engines for the derivation of linguistic features and image processing tools for the extraction of visual properties is thus a promising direction to flexible Web Mining systems for Security and Intelligence applications.