Parallel and distributed computation

The book:

covers the fundamental convergence, rate of convergence, communication, and synchronization issues associated with parallel and distributed algorithms, with an emphasis on numerical computation and asynchronous methods.

A much shorter treatment is in:

Communication and coordination in distributed computing

J.N. Tsitsiklis and G.D. Stamoulis, "On the Average Communication Complexity of Asynchronous Distributed Algorithms", Journal of the ACM, Vol. 42, No. 2, March 1995, pp. 382-400.

G. D. Stamoulis and J.N. Tsitsiklis, "The Efficiency of Greedy Routing in Hypercubes and Butterflies", IEEE Transactions on Communications, Vol. 42, No. 11, November 1994, pp. 3051-3061.

G.D. Stamoulis and J.N. Tsitsiklis, "Efficient Routing Schemes for Multiple Broadcasts in Hypercubes", IEEE Transactions on Parallel and Distributed Systems, Vol. 4, No. 7, July 1993, pp. 725-739.

G.D. Stamoulis and J.N. Tsitsiklis, "An Efficient Algorithm for Multiple Simultaneous Broadcasts in the Hypercube", Information Processing Letters, Vol. 46, 1993, pp. 219-224.

D.P. Bertsekas, C. Ozveren, G.D. Stamoulis, P. Tseng, and J.N. Tsitsiklis, "Optimal Communication Algorithms for Hypercubes" Journal of Parallel and Distributed Computing, Vol. 11, 1991, pp. 263-275.

J.N. Tsitsiklis, "On the Use of Random Numbers in Asynchronous Simulation via Rollback", Information Processing Letters, Vol. 31, 1989, pp. 139-144.

Z.Q. Luo and J.N. Tsitsiklis, "Lower Bounds on the Time to Compute a Simple Boolean Function on a Parallel Random Access Machine," Technical Report LIDS-P-1616, Laboratory for Information and Decision Systems, M.I.T., October 1986.

Asynchronous computation

P. Tseng, D.P. Bertsekas, and J.N. Tsitsiklis, "Partially Asynchronous, Parallel Algorithms for Network Flow and other Problems", SIAM Journal on Control and Optimization, Vol. 28, No. 3, May 1990, pp. 678-710.

J.N. Tsitsiklis, "A Comparison of Jacobi and Gauss-Seidel Parallel Iterations", Applied Mathematics Letters, Vol. 2, No. 2, 1989, pp. 167-170.

D. P. Bertsekas and J. N. Tsitsiklis, "Convergence Rate and Termination of Asynchronous Iterative Algorithms", in Proceedings of the 1989 International Conference on Supercomputing, Irakleion, Greece, June 1989, pp. 461-470.

J.N. Tsitsiklis, "On the Stability of Asynchronous Iterative Processes", Mathematical Systems Theory, Vol. 20, No. 2-3, 1987, pp. 137-153.

J. N. Tsitsiklis, D. P. Bertsekas and M. Athans, "Distributed Asynchronous Deterministic and Stochastic Gradient Optimization Algorithms," IEEE Transactions on Automatic Control, Vol. 31, No. 9, 1986, pp. 803-812.

J.N. Tsitsiklis and D.P. Bertsekas, "Distributed Asynchronous Optimal Routing in Data Networks", IEEE Transactions on Automatic Control, Vol. 31, No. 4, 1986, pp. 325-332.