Andrew Farmer
Christian Hoener zu Siederdissen
ABSTRACT
Stream Fusion, a popular deforestation technique in the Haskell community, cannot fuse the concatMap combinator. This is a serious limitation, as concatMap represents computations on nested streams. The original implementation of Stream Fusion used the Glasgow Haskell Compiler's user-directed rewriting system. A transformation which allows the compiler to fuse many uses of concatMap has previously been proposed, but never implemented, because the host rewrite system was not expressive enough to implement the proposed transformation. In this paper, we develop a custom optimization plugin which implements the proposed concatMap transformation, and study the effectiveness of the transformation in practice. We also provide a new translation scheme for list comprehensions which enables them to be optimized. Within this framework, we extend the transformation to monadic streams. Code featuring uses of concatMap experiences significant speedup when compiled with this optimization. This allows Stream Fusion to outperform its rival, foldr/build, on many list computations, and enables performance-sensitive code to be expressed at a higher level of abstraction.
- D. Burkett, J. Blitzer, and D. Klein. Joint Parsing and Alignment with Weakly Synchronized Grammars. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 127--135. Association for Computational Linguistics, 2010. Google Scholar
Digital Library
- D. Coutts. phStream Fusion: Practical Shortcut Fusion for Coinductive Sequence Types. PhD thesis, University of Oxford, 2010.Google Scholar
- D. Coutts, R. Leshchinskiy, and D. Stewart. Stream fusion: From lists to streams to nothing at all. In Proceedings of the 12th ACM SIGPLAN International Conference on Functional Programming, pages 315--326, Freiburg, Germany, 2007. ACM. Google ScholarDigital Library
- A. Farmer, A. Gill, E. Komp, and N. Sculthorpe. The HERMIT in the machine: A plugin for the interactive transformation of GHC core language programs. In Proceedings of the ACM SIGPLAN Haskell Symposium, Haskell '12, pages 1--12. ACM, 2012. 10.1145/2364506.2364508. Google ScholarDigital Library
- A. Gill. Cheap deforestation for non-strict functional languages. PhD thesis, The University of Glasgow, January 1996.Google Scholar
- A. Gill. A Haskell hosted DSL for writing transformation systems. In Proceedings of the IFIP TC 2 Working Conference on Domain-Specific Languages, DSL '09, pages 285--309. Springer-Verlag, July 2009. Google ScholarDigital Library
- A. Gill, J. Launchbury, and S. L. Peyton Jones. A short cut to deforestation. pages 223--232. ACM Press, 1993. Google ScholarDigital Library
- J.-Y. Girard. Interprétation fonctionelle et élimination des coupures de l'arithmétique d'ordre supérieur. PhD thesis, Université Paris VII, 1972.Google Scholar
- T. Grosser, H. Zheng, R. Aloor, A. Simbürger, A. Größlinger, and L.-N. Pouchet. Polly - Polyhedral optimization in LLVM. In First International Workshop on Polyhedral Compilation Techniques, 2011.Google Scholar
- D. Grune and C. J. Jacobs. Parsing Techniques: A Practical Guide. Springer-Verlag New York Inc, 2008. Google ScholarDigital Library
- R. Hinze, T. Harper, and D. W. James. Theory and Practice of Fusion. Implementation and Application of Functional Languages, pages 19--37, 2011. Google ScholarDigital Library
- C. Höner zu Siederdissen. Sneaking around concatMap: Efficient combinators for dynamic programming. In Proceedings of the 17th ACM SIGPLAN International Conference on Functional Programming, pages 215--226, Copenhagen, Denmark, 2012. ACM. Google ScholarDigital Library
- C. Höner zu Siederdissen, I. L. Hofacker, and P. F. Stadler. How to Multiply Dynamic Programming Algorithms. In Brazilian Symposium on Bioinformatics (BSB 2013), Lecture Notes in Bioinformatics, volume 8213. Springer, Heidelberg, 2013.Google ScholarDigital Library
- F. W. Huang, J. Qin, C. M. Reidys, and P. F. Stadler. Partition function and base pairing probabilities for RNA--RNA interaction prediction. Bioinformatics, 25 (20): 2646--2654, 2009. Google ScholarDigital Library
- C. Lattner and V. Adve. LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation. In Code Generation and Optimization, 2004. CGO 2004. International Symposium on, pages 75--86. IEEE, 2004. Google ScholarDigital Library
- B. Lippmeier, M. M. Chakravarty, G. Keller, and A. Robinson. Data Flow Fusion with Series Expressions in Haskell. In Proceedings of the 2013 ACM SIGPLAN Haskell Symposium, Haskell '13, pages 93--104. ACM, 2013. Google ScholarDigital Library
- G. Mainland, R. Leshchinskiy, and S. Peyton Jones. Exploiting Vector Instructions with Generalized Stream Fusion. In Proceedings of the 18th ACM SIGPLAN International Conference on Functional Programming, pages 37--48, 2013. Google ScholarDigital Library
- B. O'Sullivan. http://hackage.haskell.org/package/criterion.Google Scholar
- W. Partain. The nofib Benchmark Suite of Haskell Programs. In Proceedings of the 1992 Glasgow Workshop on Functional Programming, pages 195--202, 1993. Google ScholarDigital Library
- S. Peyton Jones, editor. phHaskell 98 Language and Libraries -- The Revised Report. Cambridge University Press, Cambridge, England, 2003.Google Scholar
- S. Peyton Jones. Call-pattern Specialisation for Haskell Programs. In Proceedings of the 12th ACM SIGPLAN International Conference on Functional Programming, ICFP '07, pages 327--337. ACM, 2007. Google ScholarDigital Library
- J. C. Reynolds. Towards a theory of type structure. In Colloque sur la Programmation, volume 19 of Lecture Notes in Computer Science, pages 408--423. Springer-Verlag, 1974. Google ScholarDigital Library
- A. Santos. Compilation by Transformation in Non-Strict Functional Languages. PhD thesis, University of Glasgow, 1995.Google Scholar
- N. Sculthorpe, A. Farmer, and A. Gill. The HERMIT in the tree: Mechanizing program transformations in the GHC core language. In Implementation and Application of Functional Languages 2012, volume 8241 of Lecture Notes in Computer Science. Springer, 2013.Google Scholar
- N. Sculthorpe, N. Frisby, and A. Gill. The Kansas University Rewrite Engine: A Haskell-embedded strategic programming language with custom closed universes. Submitted to the Journal of Functional Programming, 2013.Google Scholar
- M. Sulzmann, M. M. T. Chakravarty, S. Peyton Jones, and K. Donnelly. System F with type equality coercions. In Types in Language Design and Implementaion, pages 53--66. ACM, 2007. Google ScholarDigital Library
- J. Svenningsson. Shortcut Fusion for Accumulating Parameters Zip-like Functions. PhD thesis, 2002.Google Scholar
- D. A. Terei and M. M. Chakravarty. An LLVM Backend for GHC. In Proceedings of the third ACM Haskell Symposium, pages 109--120. ACM, 2010. Google ScholarDigital Library
- P. Wadler. List comprehensions. In S. Peyton Jones, editor, The Implementation of Functional Programming Languages, chapter 7. Prentice Hall International, 1987.Google Scholar
- P. Wadler. Deforestation: Transforming programs to eliminate trees. In Proceedings of the 2nd European Symposium on Programming, pages 344--358, London, UK, UK, 1988. Springer-Verlag. Google ScholarDigital Library
- R. C. Waters. Automatic transformation of series expressions into loops. ACM Trans. Program. Lang. Syst., 13 (1): 52--98, Jan. 1991. Google ScholarDigital Library
- B. A. Yorgey, S. Weirich, J. Cretin, S. Peyton Jones, D. Vytiniotis, and J. P. Magalhaes. Giving Haskell a promotion. In Types in Language Design and Implementation, pages 53--66. ACM, 2012. Google ScholarDigital Library
Index Terms
- The HERMIT in the stream: fusing stream fusion's concatMap
Recommendations
Stream fusion: from lists to streams to nothing at all
Proceedings of the ICFP '07 conferenceThis paper presents an automatic deforestation system, stream fusion, based on equational transformations, that fuses a wider range of functions than existing short-cut fusion systems. In particular, stream fusion is able to fuse zips, left folds and ...
Stream fusion: from lists to streams to nothing at all
ICFP '07: Proceedings of the 12th ACM SIGPLAN international conference on Functional programmingThis paper presents an automatic deforestation system, stream fusion, based on equational transformations, that fuses a wider range of functions than existing short-cut fusion systems. In particular, stream fusion is able to fuse zips, left folds and ...
A library writer's guide to shortcut fusion
Haskell '11: Proceedings of the 4th ACM symposium on HaskellThere are now a variety of shortcut fusion techniques in the wild for removing intermediate data structures in Haskell. They are often presented, however, specialised to a specific data structure and interface. This can make it difficult to transfer ...
Comments