7bfaee9aef
In --enable-valgrind mode, this commit adds the ability to compare current test
result output with any previous test suite execution. Whenever valgrind flags
a new function or if an existing flagged function has a greater number of
calls, then the final valgrind test will fail. This allows a greater level of
valgrind validation to take place for new code in an automated fashion. For
example, if a change to a piece of code introduces a memory handling problem of
the sort that valgrind can detect, then the final test will fail like so:
# ./test-fwknop.pl --include "complete cycle.*HMAC" --enable-valgrind --test-limit 1
[+] Starting the fwknop test suite...
args: --include complete cycle.*HMAC --enable-valgrind --test-limit 1
Saved results from previous run to: output.last/
[Rijndael SPA] [client+server] complete cycle + HMAC (tcp/22 ssh)...pass (1)
[valgrind output] [flagged functions] ..............................fail (2)
[+] 1/1/2 tests passed/failed/executed
The newly flagged functions will be written to the corresponding test file:
# cat output/2.test
[+] TEST: [valgrind output] [flagged functions]~
[-] 1.test (client) '[+] TEST: [Rijndael SPA] [client+server] complete cycle + HMAC (tcp/22 ssh)' --> NEW valgrind flagged function: main
[-] 1.test (client) '[+] TEST: [Rijndael SPA] [client+server] complete cycle + HMAC (tcp/22 ssh)' --> NEW valgrind flagged function: fko_spa_data_final
[-] 1.test (client) '[+] TEST: [Rijndael SPA] [client+server] complete cycle + HMAC (tcp/22 ssh)' --> NEW valgrind flagged function: strdup
[-] 1.test (client) '[+] TEST: [Rijndael SPA] [client+server] complete cycle + HMAC (tcp/22 ssh)' --> NEW valgrind flagged function: fko_new
[-] 1.test (client) '[+] TEST: [Rijndael SPA] [client+server] complete cycle + HMAC (tcp/22 ssh)' --> NEW valgrind flagged function: fko_encrypt_spa_data
[-] 1.test (client) '[+] TEST: [Rijndael SPA] [client+server] complete cycle + HMAC (tcp/22 ssh)' --> NEW valgrind flagged function: fko_encode_spa_data
[-] 1.test (client) '[+] TEST: [Rijndael SPA] [client+server] complete cycle + HMAC (tcp/22 ssh)' --> NEW valgrind flagged function: fko_calculate_hmac
[-] 1.test (client) '[+] TEST: [Rijndael SPA] [client+server] complete cycle + HMAC (tcp/22 ssh)' --> NEW valgrind flagged function: fko_set_username
[-] 1.test (client) '[+] TEST: [Rijndael SPA] [client+server] complete cycle + HMAC (tcp/22 ssh)' --> NEW valgrind flagged function: fko_set_rand_value
[-] 1.test (client) '[+] TEST: [Rijndael SPA] [client+server] complete cycle + HMAC (tcp/22 ssh)' --> NEW valgrind flagged function: fko_set_spa_message
[-] 1.test (client) '[+] TEST: [Rijndael SPA] [client+server] complete cycle + HMAC (tcp/22 ssh)' --> NEW valgrind flagged function: set_digest
[-] 1.test New and/or greater number of valgrind flagged function calls
This directory contains the fwknop test suite. After compiling the fwknop sources in the parent directory of test/ the test suite can be executed. One of the most comprehensive ways of executing the test suite is as follows: # ./run-test-suite.sh --enable-all This mode enables IP resolution tests, so you will need Internet access in this case. If this is not possible or desirable, then another comprehensive testing mode can be achieved with: # ./run-test-suite.sh --enable-recompile --enable-perl-module-checks --enable-distcheck Additional detail on the test suite (including --diff and --enable-valgrind modes) may be found in the fwknop tutorial available here: http://www.cipherdyne.org/fwknop/docs/fwknop-tutorial.html#3.5