summarisebig: Grouped Summaries for Large 'arrow' Datasets

Provides a 'dplyr'-like interface for grouped summaries on large 'arrow' datasets. Complete summaries are attempted in 'arrow' first. When a result can be reconstructed from 'arrow'-computable sufficient statistics, an explicit MapReduce-style reduction and 'R' finalization strategy is available. For arbitrary 'R' functions that require raw group observations, complete groups are materialized in bounded chunks, with optional parallel execution and shared-memory processing. The MapReduce strategy follows the programming model described by Dean and Ghemawat (2008) doi:10.1145/1327452.1327492 .

See also

Useful links:

• https://github.com/larry77/summarisebig

• https://larry77.github.io/summarisebig/

• Report bugs at https://github.com/larry77/summarisebig/issues

Author

Maintainer: Lorenzo Isella lorenzo.isella@gmail.com

Authors:

• Lorenzo Isella lorenzo.isella@gmail.com