Bandwidth Issues White Papers

Design of a Crawler With Bounded Bandwidth

Overview This paper presents an algorithm to bound the bandwidth of a Web crawler. The crawler collects statistics on the transfer rate of each server to predict the expected bandwidth use for future downloads. The prediction allows to activate the optimal number of fetcher threads in order to exploit the assigned bandwidth. The experimental results show the effectiveness of the proposed technique.

Further White Paper Details
PublisherUniversita degli Studi di Siena File FormatPDF
Date PublishedMay 2004
FormatWhite Papers   
Topics
E4 embraces web 2.0 audience

E4 embraces web 2.0 audience

Case study: How the Channel 4's teen channel put its mind to building a community website... more

Danone on health kick with Itil

Danone on health kick with Itil

Case study: Food company making IT easier to manage more

Cheat Sheet: Cloud computing

Cheat Sheet: Cloud computing

A tech storm is brewing...  more


Quick Sitemap Links: