PartialLoads¶

Partial Loading of Datasets¶

The VizSchema database reader plugin for VisIt allows the user to specify a stride to be used when loading data. In this manner, an extremely large dataset may be loaded on lower powered machines.

Accessing Plugin Settings

When opening a data file with the VizSchema plugin, do not load the file by double-clicking in the browser window. Instead, use the open file command from the File menu. The open file dialog box allows the user to choose the plugin to use, and once chosen, allows the user to specify settings for that plugin. To specify settings for a plugin, click on the "Set default open options". The VizSchema plugin allows the user to enter three stride values, one for each data axis. By default these values are all "1", which specifies that the plugin should load all data.

Compatible datasets

At the moment only a subset of the possible meshes and variables are capable of being partially loaded with stride values. The allowable meshes are:

• Uniform Meshes & Variables
• Structured Meshes & Variables
• Particle Data (Variable With Mesh)

Reductions in Memory Requirements

VizSchema leverages hdf5 hyperslabs to load data without exceeding the allowable memory footprint. At no time is extra data loaded into memory, thus ensuring maximum efficiency on low-memory machines. Using stride values, a representative sample of even a few gigabytes of data may be loaded with only a few megabytes of memory. For example, using a stride value of 2 on all three axes will reduce the required memory by 87.5%.
Note that memory reductions differ depending on the dimensionality of the data. A 3-d dataset will reduce more rapidly than a 2-d dataset, simply because the 3-d dataset is being filtered by three stride values while the 2-d dataset is only being filtered by two stride values.

Examples
The first image is from FACETS data and was loaded with stride = 1.

The second image is the same FACETS data, but loaded with a stride of 3, giving a reduction in memory footprint of 96.3%.