VsTests¶

How to Regression-Test the Vs Plugin¶

VsTests is the regression test suite for the Vs database reader plugin for VisIt. When making any changes to the Vs source code, it is absolutely required to run vstests on at least one system prior to committing the changes. There are many subtleties and odd cases in the Vs plugin!

Repositories

In order to run vsTests, one must check out two separate repositories.
The first is the vsTests repository itself, which contains the scripts, data files, and VisIt session files necessary to run the tests. It is located at

https://ice.txcorp.com/svnrepos/code/vstests/trunk

The other is the set of accepted results. Results vary from machine to machine, so the correct results repository to check out depends on which machine one is using to run the tests. Our "standard" test machine is iter, so that is a good place to start if the development machine does not have a specific results directory. The iter test results are located at

https://ice.txcorp.com/svnrepos/results/vsresults/trunk/iter

Other Software
VsTests depends on a third-party utility named "compare" in order to compare the generated images with the known-good results images. If you are working on iter, you should not worry. But if not, you better prepare your machine for bilder as described at the Bilder Page

"Compare" is a part of the ImageMagick software suite, which is an open-source project. In order for vsTests to run properly, the "compare" utility must be in your path. In addition, "compare" requires that the LD_LIBRARY_PATH environment variable be set and include the "lib" directory of ImageMagick. On Mac, the DYLD_LIBRARY_PATH variable should be set as well. As an example:

export PATH=$PATH:/Applications/ImageMagick/bin
export LD_LIBRARY_PATH=/Applications/ImageMagick/lib
export DYLD_LIBRARY_PATH=/Applications/ImageMagick/lib

In some cases, it may be required to set in addition

export LD_LIBRARY_PATH=/home/pletzer/software/hdf5-1.8.7-ser/lib:$LD_LIBRARY_PATH

(adapt to your location and platform) in order to resolve a dlopen error associated with the libMNETCDFDatabase library.

Configuring VsTests
VsTests relies on autoconf for configuration. To start from a clean checkout, first run "./config/cleanconf.sh". This will generate the "configure" script for the next step.
The configure script requires four arguments, of which only two are actually used. The other two are required, but not used. The two required & used arguments are --with-visit-dir and --with-results-dir. The two required but ignored arguments are --with-source-dir and --with-serial-dir.
--with-visit-dir must give the path to the root directory of the VisIt install.
--with-results-dir must give the path to the vsresults directory
--with-serial-dir and --with-source-dir must be given paths to existing directories, but nothing will be done with them.
For example, On iter, you might want to run

config/cleanconf.sh
./configure --with-source-dir=/scr_iter/sveta/projects/vizschema --with-serial-dir=/scr_iter/sveta/projects/vizschema --with-results-dir=/scr_iter/sveta/projects/vsresults/iter --with-visit-dir=/sfbilder/volatile/visit_trunk

Running VsTests
The command to run the tests is "make check". This will produce a huge amount of output to the command line, so it is recommended to pipe it to a file like so:

  make check > check.txt

When the tests complete, a file will be created with the name check.log and check.failures. If any tests failed, check.failures will contain the names of the failed tests, one to a line. You can run and study individual tests by:

cd tests
cd rectilinear
make check
vi nodal3d.out