Overlay network management for scheduling tasks on the grid

In this paper, we address the problem of building and maintaining dynamic overlay networks on top of physical networks for the autonomous scheduling of divisible load Grid applications. While autonomous scheduling protocols exist to maximize steady-state throughputs for given overlay networks, not much work has been done on building the most efficient overlay. In our work, nodes use the bandwidth-centric principle to select other nodes with which overlay edges must be formed dynamically. The node which has the entire dataset initially (the scheduler) starts forming the overlay and the nodes which receive tasks at rates greater than their task execution rates further expand it. We use simulation studies to illustrate the functioning of our overlay forming mechanism, and its robustness to changes in the characteristics of the system resources. © Springer-Verlag Berlin Heidelberg 2007.