Leveraging P2P overlays for Large-scale and Highly Robust Content Distribution and Search

Overview & Synopsis

In the last decade, there has been a tendency of shifting content distribution towards peer-to-peer (P2P) technology. The reason behind this is the self-scalability of P2P systems provided by the principles of communal collaboration and resource sharing in P2P systems. By building a P2P Content Distribution Network}(CDN), peers collaborate to distribute the content of under-provisioned websites and to serve queries for large audiences on behalf of the websites. When designing a P2P CDN, the main challenge is to actually maintain an acceptable level of performance in terms of client-perceived latency and hit ratio while minimizing the incurred overhead. This is not a straightforward endeavor given that the P2P CDN relies on autonomous and dynamic peers rather than a dedicated infrastructure. Indeed, the distribution of duties and content over peers should take into account their interests in order to give them proper incentives to cooperate. Moreover, the P2P-CDN should adapt to increasing numbers of participants and provide robust algorithms under high levels of churn because these issues have a key impact on performance. Finally, the routing of queries should aim peers close in locality and serve content from close-by providers to achieve short latencies. This paper gives an overview of our contributions in designing and maintaining a P2P CDN that tackles the issues identified above. First, we present Flower-CDN, a P2P content distribution network (CDN) that tackles some of these issues. Peers store only content of websites they are interested in and serve them to others. Furthermore, peers can find close-by content providers by a locality aware P2P directory structure. Secondly, we present a highly scalable approach of Flower-CDN called PetalUp-CDN which dynamically adjusts the directory structure in order to avoid overload situations and to keep the index information any peer must maintain at an acceptable level. Thirdly, we discuss maintenance protocols for Flower-CDN and PetalUp-CDN to cope with the worst scenarios of churn. The performance evaluation wrt. scalability and churn management shows that our generic approach enhances hit ratio by 40% and reduces response time by a factor of 12, compared to a well-known P2P-CDN.