The tensor algebra compiler

Fredrik Kjolstad, Shoaib Kamil, Stephen Chou, David Lugato, Saman Amarasinghe
[doi] [Google Scholar] [DBLP] [Citeseer] [url]
Read: 04 October 2021

Proc. ACM Program. Lang. 1(OOPSLA)
Association for Computing Machinery
New York, NY, USA
October 2017
Note(s): tensor, tensorflow, neural network, sparse model

The Tensor Algebra COmpiler (TACO) is a compiler for tensor index notation for sparse tensor code (with dense tensors as an easy special case). This early paper about the system introduces a format that generalizes existing sparse formats such as CSR, CSC or DCSR; introduces iteration graphs for capturing the structure of the loop nest that will be generated; and the use of merge lattices for capturing the constraints on iteration due to use of addition or multiplication, etc.

TACO generated code is fast (better than most of the libraries/tools it is compared against).

Around 10 further papers and theses by the same group improve representation, distributed computing, transposition and format conversion.

Later tools by other groups claim significant improvements.