Test utilities for comparing rendered results with expected image files.
Procedure for unit-testing with images:
Run unit tests at least once; this initializes a git clone of vispy/test-data in config[‘test_data_path’]. This path is ~/.vispy/test-data unless the config variable has been modified. The config file is located at vispy/vispy/util/config.py
Run individual test scripts with the –vispy-audit flag:
$ python vispy/visuals/tests/test_ellipse.py –vispy-audit
Any failing tests will display the test results, standard image, and the differences between the two. If the test result is bad, then press (f)ail. If the test result is good, then press (p)ass and the new image will be saved to the test-data directory.
After adding or changing test images, create a new commit:
$ cd ~/.vispy/test-data $ git add … $ git commit -a
Look up the most recent tag name from the test_data_tag variable in get_test_data_repo() below. Increment the tag name by 1 in the function and create a new tag in the test-data repository:
$ git tag test-data-NNN $ git push –tags origin main
This tag is used to ensure that each vispy commit is linked to a specific commit in the test-data repository. This makes it possible to push new commits to the test-data repository without interfering with existing tests, and also allows unit tests to continue working on older vispy versions.
Finally, update the tag name in
get_test_data_repoto the new name.
- class vispy.testing.image_tester.ImageTester#
Graphical interface for auditing image comparison tests.
- test(im1, im2, message)#
- vispy.testing.image_tester.assert_image_approved(image, standard_file, message=None, **kwargs)#
Check that an image test result matches a pre-approved standard.
If the result does not match, then the user can optionally invoke a GUI to compare the images and decide whether to fail the test or save the new image as the standard.
This function will automatically clone the test-data repository into ~/.vispy/test-data. However, it is up to the user to ensure this repository is kept up to date and to commit/push new images after they are saved.
Run the test with python <test-path> –vispy-audit-tests to bring up the auditing GUI.
- image(h, w, 4) ndarray or ‘screenshot’
The test result to check
The name of the approved test image to check against. This file name is relative to the root of the vispy test-data repository and will be automatically fetched.
A string description of the image. It is recommended to describe specific features that an auditor should look for when deciding whether to fail a test.
- Extra keyword arguments are used to set the thresholds for automatic image
- comparison (see ``assert_image_match()``).
- vispy.testing.image_tester.assert_image_match(im1, im2, min_corr=0.9, px_threshold=50.0, px_count=None, max_px_diff=None, avg_px_diff=None, img_diff=None)#
Check that two images match.
Images that differ in shape or dtype will fail unconditionally. Further tests for similarity depend on the arguments supplied.
- im1(h, w, 4) ndarray
Test output image
- im2(h, w, 4) ndarray
Test standard image
- min_corrfloat or None
Minimum allowed correlation coefficient between corresponding image values (see numpy.corrcoef)
Minimum value difference at which two pixels are considered different
- px_countint or None
Maximum number of pixels that may differ
- max_px_difffloat or None
Maximum allowed difference between pixels
- avg_px_difffloat or None
Average allowed difference between pixels
- img_difffloat or None
Maximum allowed summed difference between images
- vispy.testing.image_tester.downsample(data, n, axis=0)#
Downsample by averaging points together across axis. If multiple axes are specified, runs once per axis.
Return the path to a git repository with the required commit checked out.
If the repository does not exist, then it is cloned from vispy/test-data. If the repository already exists then the required commit is checked out.
- vispy.testing.image_tester.git_commit_id(path, ref)#
Return the commit id of ref in the git repository at path.
Return a string listing all changes to the working tree in a git repository.
- vispy.testing.image_tester.make_diff_image(im1, im2)#
Return image array showing the differences between im1 and im2.
Handles images of different shape. Alpha channels are not compared.