CachedNDArray - data structure that allows to deal with large N-dimensional arrays through caching method:

  • Allows to avoid Matlab out-of-memory error by caching large array into several files on hard disk and then reading the necessary chunks using memmapfile function.
  • The data structure is inherited from handle abstract class which avoids parameter-by-value and supports parameter-by-reference.
  • Supports two types of movements - continuous (very slow) and discrete (fast); the former might have no more than two file to represent a chunk; while the latter means the data is processed chunk-after-chunk, and each chunk is represented strictly as a single file.
  • Caching flag can be set to either manual or automatic mode. If no caching is needed to perform, the CachedNDArray is treated like a normal Matlab array.
  • Automatic or manual dimension breakage into number of chunks.

View on github