summaryrefslogtreecommitdiff
path: root/doc/testing.txt
blob: 9391832e5fea00af3f4f7285d137354bad41ee89 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
Running regression tests:
-------------------------

Prerequisites: cmake, python, ImageMagick 6.5.9.3 or newer

A) Building test environment

Linux, Mac:
$ cd tests
$ cmake .
$ make

Windows:

First, get a normal build working by following instructions at 
http://en.wikibooks.org/wiki/OpenSCAD_User_Manual/Building_on_Windows
Then, from the QT command prompt:

> cd tests
> cmake . -DCMAKE_BUILD_TYPE=Release
> sed -i s/\/MD/\/MT/ CMakeCache.txt
> cmake .
> nmake -f Makefile

B) Running tests

$ ctest               Runs tests enabled by default
$ ctest -R <regex>    Runs only matching tests, e.g. ctest -R dxf
$ ctest -C <configs>  Adds extended tests belonging to configs.
                      Valid configs:
                      Default  - Run default tests
                      Heavy    - Run more time consuming tests (> ~10 seconds)
                      Examples - test all examples
                      All      - test everything

Headless unix servers (no X11):
$ Xvfb :5 -screen 0 800x600x24 &
$ DISPLAY=:5 ctest

Adding  a new regression test:
------------------------------

1) create a test file at an appropriate location under testdata/
2) if the test is non-obvious, create a human readable description of the test in the same directory (e.g testdata/scad/mytest.txt)
3) if a new test app was written, this must be added to tests/CMakeLists.txt
4) Add the tests to the test apps for which you want them to run (in tests/CMakeLists.txt)
5) run the test with the environment variable TEST_GENERATE=1, e.g.:
   $ TEST_GENERATE=1 ctest -R mytest
   (this will generate a mytest-expected.txt file which is used for regression testing)
6) manually verify that the output is correct (tests/regression/<testapp>/mytest-expected.<suffix>)
7) run the test normally and verify that it passes:
  $ ctest -R mytest

Troubleshooting:
------------------------------

To helping CMAKE find eigen2, OpenCSG, CGAL, Boost, and GLEW, you can use the
-D option. Here are some examples:

 cmake . -DOPENCSG_DIR=~/OpenCSG-1.3.2
 cmake . -DCGAL_DIR=c:\CGAL-3.7 -DBOOST_ROOT=c:\boost_1_46_0
 
Logs of test runs are found in tests/build/Testing/Temporary
Pretty-printed html output is in a subdir of tests/build/Testing/Temporary
Expected results are found in tests/regression/*
Actual results are found in tests/build/testname-output/*
contact: Jan Huwald // Impressum