Skip to main content
Log in

A linear programming primer: from Fourier to Karmarkar

Annals of Operations Research Aims and scope Submit manuscript

Abstract

The story of linear programming is one with all the elements of a grand historical drama. The original idea of testing if a polyhedron is non-empty by using a variable elimination to project down one dimension at a time until a tautology emerges dates back to a paper by Fourier in 1823. This gets re-invented in the 1930s by Motzkin. The real interest in linear programming happens during World War II when mathematicians ponder best ways of utilising resources at a time when they are constrained. The problem of optimising a linear function over a set of linear inequalities becomes the focus of the effort. Dantzig’s Simplex Method is announced and the Rand Corporation becomes a hot bed of computational mathematics. The range of applications of this modelling approach grows and the powerful machinery of numerical analysis and numerical linear algebra becomes a major driver for the advancement of computing machines. In the 1970s, constructs of theoretical computer science indicate that linear programming may in fact define the frontier of tractable problems that can be solved effectively on large instances. This raised a series of questions and answers: Is the Simplex Method a polynomial-time method and if not can we construct novel polynomial time methods, etc. And that is how the Ellipsoid Method from the Soviet Union and the Interior Point Method from Bell Labs make their way into this story as the heroics of Khachiyan and Karmarkar. We have called this paper a primer on linear programming since it only gives the reader a quick narrative of the grand historical drama. Hopefully it motivates a young reader to delve deeper and add another chapter.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Notes

  1. This method is sometimes referred to as Fourier–Motzkin Elimination in the literature as there was presumably an independent rediscovery (Fourier 1826) of the method over a 100 years later by Motzkin (1936), Dantzig and Eaves (1973) and extended by Cerkinov (1961) and Chandru (1993).

References

  • Adler, I., & Cosares, S. (1991). A strongly polynomial algorithm for a special class of linear programs. Operations Research, 39, 955–960.

    Article  Google Scholar 

  • Bixby, R. E. (1994). Progress in linear programming. ORSA Journal on Computing, 6(1), 15–22.

    Article  Google Scholar 

  • Borgwardt, K. H. (1987). The simplex method: A probabilistic analysis. Berlin: Springer.

    Book  Google Scholar 

  • Cerkinov, R. N. (1961). The solution of linear programming problems by elimination of unknowns. Doklady Akademii Nauk, 139, 1314–1317.

    Google Scholar 

  • Chandru, V., & Kochar, B. S. (1985). A class of algorithms for linear programming, Research Memorandum No. 85-14, Purdue University.

  • Chandru, V., & Kochar, B. S. (1986). Exploiting special structures using a variant of Karmarkar’s algorithm, Research Memorandum No. 86-10, School of Industrial Engineering, Purdue University.

  • Chandru, V. (1993). Variable elimination in linear constraints. The Computer Journal, 36(5), 463–472.

    Article  Google Scholar 

  • Charnes, A. (1952). Optimality and degeneracy in linear programming. Econometrica, 20, 160–170.

    Article  Google Scholar 

  • Cohen, E., & Megiddo, N. (1991). Improved algorithms for linear inequalities with two variables per inequality. In Proceedings of the twenty third symposium on theory of computing, New Orleans (pp. 145–155).

  • Cohen, E., & Megiddo, N. (1993). New algorithms for generalized network flows, Revised. In D. Dolev, Z. Galil, & M. Rodeh (Eds.), Proceedings of the 1st Israeli symposium on the theory of computing and systems (pp. 103–114).

  • Cohen, E., & Megiddo, N. (1994). Improved algorithms for linear inequalities with two variables per inequality, Extended Abstract. SIAM Journal of Computing, 23, 1313–1347.

    Article  Google Scholar 

  • Cottle, R. W., & Veinott, A. F, Jr. (1972). Polyhedral sets having a least element. Mathematical Programming, 3, 238–249.

    Article  Google Scholar 

  • Dantzig, G. B. (1951). Maximization of a linear function of variables subject to linear inequalities. In C. Koopmans (Ed.), Activity analysis of production and allocation (pp. 339–347). New York: Wiley.

    Google Scholar 

  • Dantzig, G. B. (1963). Linear programming and extensions. Princeton: Princeton University Press.

    Book  Google Scholar 

  • Dantzig, G. B., & Eaves, B. C. (1973). Fourier–Motzkin elimination and its dual. Journal of Combinatorial Theory (A), 14, 288–297.

    Article  Google Scholar 

  • Dantzig, G. B., Orden, A., & Wolfe, P. (1955). The generalized simplex method for minimizing a linear form under linear inequality restraints. Pacific Journal of Mathematics, 5, 183–195.

    Article  Google Scholar 

  • Fourier, L. B. J. (1826). Reported in: Analyse des travaux de l’Academic Royale des Sciences. pendant l’annee (1823), Partie mathematique, Histoire de l’Academie Royale des Sciences de l’Institut de France (Vol. 6, pp. xxix–xli).

  • Fourier, L. B. J. (1827). Reported in: Analyse des travaux de l’Academic Royale des Sciences. pendant l’annee (1824), Partie mathematique, Histoire de l’Academie Royale des Sciences de l’Institut de France (Vol. 7, pp. xlviii–lv) (Partial English Translation in: D.A. Kohler, Translation of a report by Fourier on his work on linear inequalities. Opsearch, Vol. 10, pp. 38–42, 1973)

  • Grötschel, M., Lovász, L., & Schrijver, A. (1982). The ellipsoid method and its consequences in combinatorial optimization. Combinatorica, 1, 169–197.

    Article  Google Scholar 

  • Grötschel, M., Lovász, L., & Schrijver, A. (1988). Geometric algorithms and combinatorial optimization. Berlin: Springer.

    Book  Google Scholar 

  • Haĉijan, L. G. (1979). A polynomial algorithm in linear programming. Soviet Mathematics Doklady, 20, 191–194.

    Google Scholar 

  • Haimovich, M. (1983). The simplex method is very good! On the expected number of pivot steps and related properties of random linear programs. Unpublished Manuscript.

  • Hochbaum, D. S., & Naor, J. (1994). Simple and Fast Algorithms for Linear and Integer Programs with two variables per inequality. SIAM Journal on Computing, 23(6), 1179–1192.

    Article  Google Scholar 

  • Karmarkar, N. K. (1984). A new polynomial-time algorithm for linear programming. Combinatorica, 4, 373–395.

    Article  Google Scholar 

  • Karp, R. M., & Papadimitriou, C. H. (1982). On linear characterizations of combinatorial optimization problems. SIAM Journal on Computing, 11, 620–632.

    Article  Google Scholar 

  • Klee, V., & Minty, G. J. (1972). How good is the simplex algorithm? In O. Shisha (Ed.), Inequalities III. Cambridge: Academic Press.

    Google Scholar 

  • Lassez, J.-L. (1991). From LP to LP: Programming with constraints. In Proceedings of theoretical aspects of computer software, Sendai.

  • Lassez, J.-L., & Maher, M. J. (1988). On Fourier’s algorithm for linear arithmetic constraints, IBM Research Report, T Watson Research Center.

  • Megiddo, N. (1983). Towards a genuinely polynomial algorithm for linear programming. SIAM Journal on Computing, 12(2), 347–353.

    Article  Google Scholar 

  • Megiddo, N. (1991). On finding primal- and dual-optimal bases. ORSA Journal on Computing, 3, 63–65.

    Article  Google Scholar 

  • Motzkin, T. S. (1936). Beitrage zur theorie der linearen Ungleichungen. Doctoral thesis, University of Base.

  • Padberg, M. W., & Rao, M. R. (1981). The Russian method for linear inequalities, Part III, Bounded integer programming. New York: New York University. (preprint).

    Google Scholar 

  • Saigal, R. (1995). Linear programming: A modern integrated analysis. Alphen aan den Rijn: Kluwer Press.

    Book  Google Scholar 

  • Schrijver, A. (1986). Theory of linear and integer programming. Hoboken: Wiley.

    Google Scholar 

  • Shor, N. Z. (1970). Convergence rate of the gradient descent method with dilation of the space. Cybernetics, 6, 102–108.

    Article  Google Scholar 

  • Stone, Richard E., & Tovey, Craig A. (1991). The simplex and projective scaling algorithms as iteratively reweighted least squares methods. SIAM Review, 33(2), 220–237.

    Article  Google Scholar 

  • Todd, M. J. (1986). Exploiting special structure in Karmarkar’s algorithm for linear programming. Technical Report 707, School of Operations Research and Industrial Engineering, Cornell University.

  • Todd, M. J. (1994). Theory and practice for interior-point methods. ORSA Journal on Computing, 6(1), 28–31.

    Article  Google Scholar 

  • Vaidya, P. M. (1989). Speeding-up linear programming using fast matrix multiplication. In Proceedings of the 30th IEEE annual symposium on foundations of computer science (pp. 332–337).

  • Vanderbei, R. J. (2015). Linear programming: Foundations and extensions (4th ed.). Berlin: Springer.

    Google Scholar 

  • Williams, H. P. (1986). Fourier’s method of linear programming and its dual. The America Mathematical Monthly, 93, 681–695.

    Article  Google Scholar 

  • Wolfe, P. (1963). A technique for resolving degeneracy in linear programming. SIAM Journal on Applied Mathematics, 11(205), 211.

    Google Scholar 

  • Wright, S. J. (1997). Primal-dual interior-point methods. Cambridge: SIAM Press.

    Book  Google Scholar 

  • Ziegler, M. (1995). Convex polytopes. Berlin: Springer.

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank the anonymous reviewers for their detailed comments and suggestions for improvement of the original submission. The authors have incorporated most of their suggestions and of course take responsibility for any remaining flaws.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to M. R. Rao.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This paper is an extended version of the original paper “The French Connection” by the authors Vijay Chandru and M R Rao as a part of the series “175 Years of Linear Programming” published in Resonance- Journal of Science Education, Vol 3, Issue 10, October 1998, pp 26–40. This paper also draws on some material in “Linear Programming,” Chapter 31 by V. Chandru and M.R.Rao in Handbook of Algorithms and Theory of Computing, edited by M.J.Atallah, CRC Press 1999, 31-1 to 31-37.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chakraborty, A., Chandru, V. & Rao, M.R. A linear programming primer: from Fourier to Karmarkar. Ann Oper Res 287, 593–616 (2020). https://doi.org/10.1007/s10479-019-03186-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10479-019-03186-2

Keywords

Navigation