1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
|
Running regression tests:
-------------------------
Prerequisites: cmake, python, ImageMagick 6.5.9.3 or newer
A) Building test environment
Linux, Mac:
$ cd tests
$ cmake .
$ make
Windows:
First, get a normal build working by following instructions at
http://en.wikibooks.org/wiki/OpenSCAD_User_Manual/Building_on_Windows
Then, from the QT command prompt:
> cd tests
> cmake . -DCMAKE_BUILD_TYPE=Release
> sed -i s/\/MD/\/MT/ CMakeCache.txt
> cmake .
> nmake -f Makefile
B) Running tests
$ ctest Runs tests enabled by default
$ ctest -R <regex> Runs only matching tests, e.g. ctest -R dxf
$ ctest -C <configs> Adds extended tests belonging to configs.
Valid configs:
Default - Run default tests
Heavy - Run more time consuming tests (> ~10 seconds)
Examples - test all examples
All - test everything
Adding a new regression test:
------------------------------
1) create a test file at an appropriate location under testdata/
2) if the test is non-obvious, create a human readable description of the test in the same directory (e.g testdata/scad/mytest.txt)
3) if a new test app was written, this must be added to tests/CMakeLists.txt
4) Add the tests to the test apps for which you want them to run (in tests/CMakeLists.txt)
5) run the test with the environment variable TEST_GENERATE=1, e.g.:
$ TEST_GENERATE=1 ctest -R mytest
(this will generate a mytest-expected.txt file which is used for regression testing)
6) manually verify that the output is correct (tests/regression/<testapp>/mytest-expected.<suffix>)
7) run the test normally and verify that it passes:
$ ctest -R mytest
Troubleshooting:
------------------------------
0. Headless unix servers (no X11)
$ Xvfb :5 -screen 0 800x600x24 &
$ DISPLAY=:5 ctest
1. Trouble finding libraries
To help CMAKE find eigen2, OpenCSG, CGAL, Boost, and GLEW, you can use
environment variables, just like for the main qmake & openscad.pro. Examples:
OPENCSGDIR=~/OpenCSG-1.3.2 EIGEN2DIR=~/eigen2 cmake .
Valid variables are as follows (see CMakeLists.txt for more info):
BOOSTDIR, CGALDIR, EIGEN2DIR, GLEWDIR, OPENCSGDIR, OPENSCAD_LIBRARIES
2. Logs
Logs of test runs are found in tests/build/Testing/Temporary
Pretty-printed index.html is in a subdir of tests/build/Testing/Temporary
Expected results are found in tests/regression/*
Actual results are found in tests/build/testname-output/*
3. Cross-compiling
Cross-compiling of tests has not been automated nor tested
4. Image-based tests takes a long time, they fail, and it says 'return -11'
Imagemagick may have crashed. You can try using the alternate IM comparator
based on Normalized Cross Correlation. Pass -DCOMPARATOR=ncc to cmake
5. Testing images fails with 'morphology' not found for ImageMagick
Your version of imagemagick is old. Upgrade, or pass -DCOMPARATOR=old to
cmake. The comparison will be of lowered reliability.
6. Unexplained or bizarre errors.
This can happen on dynamic-library systems (linux) where you try to use
your own version of a library while the system still has another version
under the system paths. You can diagnose this by looking at your cmake
log as well as your sysinfo.txt file, as well as running 'ldd' against
your binaries, to make sure that the proper versions of libraries are
getting compiled and linked with the test binaries.
|
