Optimal matrix transposition and bit reversal on hypercubes: All-to-all personalized communication

doi:10.1016/0743-7315(91)90039-C

Journal of Parallel and Distributed Computing

Volume 11, Issue 4, April 1991, Pages 328-331

https://doi.org/10.1016/0743-7315(91)90039-C Get rights and content

Abstract

In a hypercube multiprocessor with distributed memory, messages have a street address and an apartment number, i.e., a hypercube node address and a local memory address. Here we describe an optimal algorithm for performing the communication described by exchanging the bits of the node address with that of the local address. These exchanges occur typically in both matrix transposition and bit reversal for the fast Fourier transform.

References (9)

P.N. Swarztrauber
Multiprocessor FFTs
Parallel Comput.
(1987)
Bertsekas, D. P., Ozveren, C., Stamoulis, G. D., Tseng, P., and Tsitsiklis, J. N. Optimal communication algorithms for...
P.M. Flanders
A unified approach to a class of data movements on an array processor
IEEE Trans. Comput.
(1982)
D. Fraser
Array permutation by index-digit permutation
J. Assoc. Comput. Mach.
(1976)

There are more references available in the full text version of this article.

Cited by (26)

All-port total exchange in cartesian product networks
2004, Journal of Parallel and Distributed Computing
We present a general solution to the total exchange (TE) communication problem for any homogeneous multidimensional network under the all-port assumption. More specifically, we consider cartesian product networks where every dimension is the same graph (e.g. hypercubes, square meshes, n-ary d-cubes) and where each node is able to communicate simultaneously with all its neighbors. We show that if we are given an algorithm for a single n-node dimension which requires T steps, we can construct an algorithm for d-dimensions and running time of n^d−1T steps, which is provably optimal for many popular topologies. Our scheme, in effect, generalizes the TE algorithm given by Bertsekas et al. (J. Parallel Distrib. Comput. 11 (1991) 263–275) for the hypercubes and complements our theory (IEEE Trans. Parallel Distrib. Systems 9(7) (1998) 639) for the single-port model.
A comparison of optimal FFTs on torus and hypercube multicomputers
2001, Parallel Computing
In this paper we compare the optimum performance of the fast Fourier transform (FFT) on torus and hypercube multicomputers. The optimal number of floating point operations is architecturally invariant so the relative performance is determined by communication time. We compare the performance of known multiport communication algorithms that utilize the full bandwidth of the interconnection network on every cycle. Furthermore, data is routed unblocked over paths with minimum length. Specifically we compare FFT performance on the hypercube as well as two-, three- and four-dimensional torus architectures. On the hypercube we observe the somewhat surprising result that, in the limit, communication is ultimately negligible compared to computation. While the opposite is true of the torus, it is nevertheless possible to obtain comparable performance over a broad range of processors and problem sizes. As computers are built with increasing numbers of processors, torus performance can still be made comparable to the hypercube by gradually increasing the dimension of the interconnect. Any number of processors can be used to compute the FFT with efficiency that is theoretically bounded from zero.
Transposing Arrays on Multicomputers Using de Bruijn Sequences
1998, Journal of Parallel and Distributed Computing
Transposing anN×Narray that is distributed row- or columnwise acrossP=Nprocessors is a fundamental communication task that requires time- consuming interprocessor communication. It is the underlying communication task for the fast Fourier transform of long sequences and multidimensional arrays. It is also the key communication task for certain weather and climate models. A parallel transposition algorithm is presented for hypercube and mesh connected multicomputers with programmable networks. The optimal scheduling of network transmissions is not unique and is known to be nontrivial. Here, scheduling is determined by a single de Bruijn sequence ofNbits. The elements in each processor are first preordered and then, in groups of log₂ Padjacent elements, either transmitted or not transmitted, depending on the corresponding bit in the de Bruijn sequence. The algorithm is optimal both in overall time and the time that any individual element is in the network. The results are extended to other communication tasks including shuffles, bit reversal, index reversal, and general index-digit permutation. The caseP≠Nand rectangular arrays with non-power-of-two dimensions are also discussed. Algorithms for mesh connected multicomputers are developed by embedding the hypercube in the mesh. The optimal implementation of the algorithms requires certain architectural features that are not currently available in the marketplace.
Optimal communication algorithms for Manhattan Street networks
1998, Discrete Applied Mathematics
We evaluate the mean internodal distance and the saturation throughput of the Manhattan Street network in the general case, and present a simple shortest path routing algorithm for this topology. We also propose efficient routing algorithms to execute generic communication tasks, such as the total exchange and the multinode broadcast, in a Manhattan Street network of processors. The proposed algorithms execute these tasks in an optimal number of steps, while achieving full (equal to 100%) utilization of the network links. Finally, we construct edge-disjoint Hamiltonian cycles for the Manhattan Street network.
Transposition of banded matrices in hypercubes: A nearly isotropic task
1995, Parallel Computing
A class of communication tasks, called isotropic, was introduced in [15], and minimum completion time algorithms for all tasks in this class were found. Isotropic tasks are characterized by a type of symmetry with respect to origin node. In this paper we consider the problem of transposing a sparse matrix of size N × N with a diagonal band of size 2^{β + 1} + 1, which is stored by columns in a hypercube network of N = 2^d processors. We propose an assignment of matrix columns to hypercube nodes such that the transposition becomes a ‘nearly isotropic’ task, that is, it looks ‘almost identical’ to all nodes. Under this assignment, we give an algorithm to transpose the matrix in 2^β steps. We prove that the algorithm given is optimal over all affine assignments of columns to processors. We also derive a lower bound on the minimum number of steps required to transpose a banded matrix, which holds for any possible assignment of matrix columns to hypercube processors. In the case that 2^{β + 1} + 1 = Θ(N^c), for some constant c ϵ (0, 1], we prove that the completion time of our transposition algorithm is of the same order of magnitude with the lower bound. We further show that [ $d β$ ] banded matrices, each of bandwidth 2^{β + 1} + 1, can be stored by columns in a hypercube so that all of them can be concurrently transposed in 2^{β + 1} steps. Finally, we modify our algorithms so that they apply to arbitrary matrix bandwidths and multiple column storage by each processor, while maintaining their efficiency.
Partial multinode broadcast and partial exchange algorithms for d-dimensional meshes
1994, Journal of Parallel and Distributed Computing
In this paper we consider the partial multinode broadcast and the partial exchange communication tasks in d-dimensional meshes. The partial multinode broadcast in an N-processor network is the task in which each of M ≤ N arbitrary nodes broadcasts a packet to all the remaining N − 1 nodes. Correspondingly, in the partial exchange there are M ≤ N nodes that wish to send a separate, personalized packet to each of the other nodes. We propose algorithms for the d-dimensional mesh network that execute the partial multinode broadcast and the partial exchange communication tasks in near-optimal time. No assumption is made concerning the locations of the M source nodes. The communication algorithms proposed are "on line" and distributed. We further look at a dynamic version of the broadcasting problem, where broadcast requests are generated at random times. In particular, we assume that the broadcast requests are generated at each node of the mesh according to a Poisson distribution with rate λ. Based on the partial multinode broadcast algorithm, we propose a dynamic decentralized scheme to execute the broadcasts in this dynamic environment. We find an upper bound on the average delay required to serve each broadcast. We prove that the algorithm is stable for network utilization ρ close to 1, and the average delay is of the order of the diameter for any load in the stability region.

View all citing articles on Scopus

View full text

ArticleOptimal matrix transposition and bit reversal on hypercubes: All-to-all personalized communication

Abstract

Parallel Comput.

A unified approach to a class of data movements on an array processor

IEEE Trans. Comput.

Array permutation by index-digit permutation

J. Assoc. Comput. Mach.

Article
Optimal matrix transposition and bit reversal on hypercubes: All-to-all personalized communication