You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It's probably reasonable to expect a user to be able to hold (1) the particle data of the source and (2) the datacube in memory simultaneously as a minimum memory requirement. As currently implemented, however, martini is more memory-hungry than this because it pre-computes and stores the spectra of all particles simultaneously, requiring nchannels * nparticles floating point values, which for some setups could be a very large number. Pre-computing the spectra is done for speed, but it would be possible to cut down on memory by segmenting the source into a series of chunks of ~equal numbers of particles. The simplest implementation would be to run the source insertion loop once for each such chunk, which is a significant hit in terms of cpu time cost. Giving some flexibility to keep memory requirements under control would be good though. Could explore other options like optionally using a memmap to store the spectra on disk rather than in memory. Costs a bit of access speed but maybe better than running the main loop more than once.
The text was updated successfully, but these errors were encountered:
It's probably reasonable to expect a user to be able to hold (1) the particle data of the source and (2) the datacube in memory simultaneously as a minimum memory requirement. As currently implemented, however, martini is more memory-hungry than this because it pre-computes and stores the spectra of all particles simultaneously, requiring nchannels * nparticles floating point values, which for some setups could be a very large number. Pre-computing the spectra is done for speed, but it would be possible to cut down on memory by segmenting the source into a series of chunks of ~equal numbers of particles. The simplest implementation would be to run the source insertion loop once for each such chunk, which is a significant hit in terms of cpu time cost. Giving some flexibility to keep memory requirements under control would be good though. Could explore other options like optionally using a memmap to store the spectra on disk rather than in memory. Costs a bit of access speed but maybe better than running the main loop more than once.
The text was updated successfully, but these errors were encountered: