Abstract
Stream-based overlay networks (SBONs) are one approach to implementing large-scale stream processing systems. A fundamental consideration in an SBON is that of service placement, which determines the physical location of in-network processing services or operators, in such a way that network resources are used efficiently. Service placement consists of two components: node discovery, which selects a candidate set of nodes on which services might be placed, and node selection, which chooses the particular node to host a service. By viewing the placement problem as the composition of these two processes we can trade-off quality and efficiency between them.
We evaluate the appropriateness of using DHT routing paths for service placement in an SBON, when aiming to minimize network usage. For this, we consider two DHT-based algorithms for node discovery, which use either the union or intersection of DHT routing paths in the SBON, and compare their performance to other techniques. We show that current DHT-based schemes are actually rather poor node discovery algorithms, when minimizing network utilization. An efficient DHT may not traverse enough hops to obtain a sufficiently large candidate set for placement. The union of DHT routes may result in a low-quality set of discovered nodes that requires an expensive node selection algorithm. Finally, the intersection of DHT routes relies on route convergence, which prevents the placement of services with a large fan-in.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Abadi, D., Carney, D., Cetintemel, U., et al.: Aurora: A New Model and Architecture for Data Stream Management. In: VLDB (2003)
Abadi, D., Ahmad, Y., Balakrishnan, H., et al.: The Design of the Borealis Stream Processing Engine. Technical Report CS-04-08, Brown University (2004)
Huebsch, R., Hellerstein, J.M., Lanham, N., et al.: Querying the Internet with PIER. In: VLDB, Berlin, Germany (2003)
Gibbons, P.B., Karp, B., Ke, Y., Nath, S., Seshan, S.: IrisNet: An Architecture for a World-Wide Sensor Web. IEEE Pervasive Computing 2 (2003)
Pietzuch, P., Shneidman, J., Welsh, M., Seltzer, M., Roussopoulos, M.: Path Optimization in Stream-Based Overlay Networks. Tr, Harvard University (2004)
Dabek, F., Cox, R., Kaashoek, F., Morris, R.: Vivaldi: A Decentralized Network Coordinate System. In: Proc. of ACM SIGCOMM 2004, Portland, OR (2004)
The Planetlab Consortium (2004), http://www.planet-lab.org
Stribling, J.: All-Pairs-Pings for PlanetLab (2004)
Zegura, E.W., Calvert, K.L., Bhattacharjee, S.: How to Model an Internetwork. In: Proc of IEEE Infocom 1996, San Francisco, CA, vol. 2, pp. 594–602 (1996)
Karp, B., Ratnasamy, S., Rhea, S., Shenker, S.: Spurring Adoption of DHTs with OpenHash, a Public DHT Service. In: Voelker, G.M., Shenker, S. (eds.) IPTPS 2004. LNCS, vol. 3279, pp. 195–205. Springer, Heidelberg (2005)
Rhea, S., Geels, D., Roscoe, T., Kubiatowicz, J.: Handling Churn in a DHT. In: USENIX 2004, Boston, MA (2004)
Spring, N., Wetherall, D., Anderson, T.: Scriptroute. In: USITS 2002 (2003)
Castro, M., Druschel, P., Kermarrec, A.M., Rowstron, A.: Scribe: A Large-scale and Decentralized Application-level Multicast Infrastructure. JSAC 20 (2002)
Shneidman, J., Pietzuch, P., Ledlie, J., Roussopoulos, M., Seltzer, M., Welsh, M.: Hourglass: An Infrastructure for Connecting Sensor Networks and Applications. Technical report, Harvard University (2004)
Chen, L., Reddy, K., Agrawal, G.: GATES: A Grid-Based Middleware for Processing Distributed Data Streams. In: HPDC-13, Honolulu, Hawaii (2004)
Ahmad, Y., Çetintemel, U.: Network-Aware Query Processing for Stream-based Applications. In: VLDB (2004)
Zhuang, S.Q., Zhao, B.Y., Joseph, A.D., Katz, R.H., Kubiatowicz, J.: Bayeux: An Architecture for Scalable and Fault-tolerant Wide-Area Data Dissemination. In: NOSSDAV (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Pietzuch, P., Shneidman, J., Ledlie, J., Welsh, M., Seltzer, M., Roussopoulos, M. (2005). Evaluating DHT-Based Service Placement for Stream-Based Overlays. In: Castro, M., van Renesse, R. (eds) Peer-to-Peer Systems IV. IPTPS 2005. Lecture Notes in Computer Science, vol 3640. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11558989_25
Download citation
DOI: https://doi.org/10.1007/11558989_25
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29068-1
Online ISBN: 978-3-540-31906-1
eBook Packages: Computer ScienceComputer Science (R0)