Ανάλυση πολυμέσων με χρήση γνώσης: ισορροπώντας ανάμεσα στην πολυπλοκότητα και την εγκυρότητα

Abstract

This thesis introduces tools for the semantic analysis of multimedia documents based on prior knowledge and its main goal is to turn the computational complexity into a controllable parameter of such systems. Entities are divided into (i) directly measurable quantities (syntactic entities) and (ii) high-level concepts, closer to human perception (semantic entities) and organized within a hierarchical fuzzy model. Moreover, appropriate metrics for quantifying the semantic search procedure and its results are proposed. The methodology is equipped with Inference mechanisms that fit various scenarios, while appropriate methods for computing the fuzzy weights of the knowledge model are also described. Although the proposed expressivity is limited w.r.t. Description Logics, it is fully adequate and compatible with the way classifiers treat multimedia documents. On these grounds, this thesis combines the results of other measurement methods (e.g. classifiers), by using a knowledge model that does not require complicated computations during inference. This can be achieved because the truth factors of the entities under examination are computed using closed mathematical expressions that stem directly from knowledge, eliminating the need for ABox reasoning. Furthermore, through the proposed methodology, semantic search can be efficiently used under any restrictions posed by computational complexity, by selecting optimal subsets of the available measurements. The subset selection problem is efficiently solved using dynamic programming, minimizing the extra computational burden it may pose. Experiments demonstrate that the proposed method can achieve very good accuracy while searching for and retrieving new entities, together with improving the scores given by existing classifiers. This method can be adapted to various domains/datasets through a process of fuzzy weight re-computation. An extra application scenario is presented, where the mathematical tools provided here are used for software agent evaluation. Finally, we theoretically prove that, even in the case of a more expressive language, the execution of a fuzzy tableau algorithm on the measurements (i.e. using the instantiated ABox) yields results identical with the ones our method can achieve using closed-form mathematical expressions. Corresponding experiments illustrate the virtues of this language, while also indicate that the performance of our methodology can be estimated using the development set.