Rewiring peer connections over semantic overlay networks

Abstract

Today's content providers are naturally distributed and produce large amounts of information every day, making peer-to-peer data management a promis- ing approach o®ering scalability, adaptivity to dynamics and failure resilience. The management of large volumes of data in peer-to-peer networks has generated ad- ditional interest in methods for e®ective network organisation based on providers' contents and consequently, in methods supporting information retrieval. Overlay architectures (or overlay networks) are being introduced as tools for the organisa- tion and sharing of information residing in a network of peers. Moreover, semantic overlay networks partition the overlay layer by clustering the data sources. This is achieved through a rewiring protocol that is executed independently by each peer. The purpose of this protocol is to establish connections among peers being semanti- cally, thematically or socially similar (i.e., peers sharing similar interests). By this, the problem of ?nding the most relevant resources is reduced to one of locating the clusters similar to the query. In this thesis, we present iCluster, a generic architecture for supporting full- °edged information retrieval in large-scale peer-to-peer networks, along with its associated organisation and query forwarding protocols. The main focus of this work is on rewiring. We study the functional issues related to peer rewiring and discuss a number of choices in designing the rewiring strategy. In addition, we introduce the concept of clustering e±ciency, a measure that quanti?es the quality of network organisation by exploiting the underlying network structure. Clustering e±ciency is used for evaluating the performance of rewiring. We study the system performance and identify the rewiring protocol that proves e±cient under peer churn. Finally, we apply our architecture to a digital library use case and support searching and - xiii - ?ltering functionality using the same infrastructure. The experimental evaluation with real-word data and queries demonstrates that iCluster achieves signi?cant performance improvements (in terms of communication load and retrieval accuracy) over a well-known state-of-the-art peer-to-peer clus- tering method. Compared to exhaustive search by °ooding, iCluster exchanged a small loss in retrieval accuracy for much less message °ow. In addition, the proposed distributed protocols are proven e±cient under peer churn, shown to achieve high retrieval accuracy and scale up well for large networks. Our experimental results con?rm the dependency between rewiring strategies and retrieval performance, and give insights on the trade-o®s involved in the selection of a rewiring strategy. - x