Commit Graph

129 Commits

Author SHA1 Message Date
Joe Ranweiler
d4dc9eaa01 Add struct for info about single test runs, update in global setters
Instead of exit codes, we will use the global instance of this (shared
with a parent process) to determine the result of forked test runs.
2018-02-21 18:49:42 -08:00
Joe Ranweiler
9d47f21f5a Hide mutation of test globals behind internal functions 2018-02-21 17:17:43 -08:00
Joe Ranweiler
341c2da0f8 Fix type of --take_over flag 2018-02-21 16:56:12 -08:00
Joe Ranweiler
454ee11a6e Merge pull request #44 from trailofbits/take-over-api
Take-over API
2018-02-20 16:06:21 -08:00
Joe Ranweiler
c09feec114 Call DeepState_Begin() for each forked take-over test case 2018-02-20 15:52:14 -08:00
Joe Ranweiler
23dbbbdc57 Log all test run result cases in native take-over
Warning: this does not work correctly with tests that soft fail, e.g.
via a `CHECK` assertion. This is because the soft failures only update
the child's `DeepState_TestFailed` global variable, but do not exit.
What we will soon do is share memory with the child process, and derive
the "test result" from that shared memeory.
2018-02-20 15:43:34 -08:00
Joe Ranweiler
a81f816d89 If in take-over mode, exit on fatal error
In take-over mode, the "test" is the entire binary being executed as a
child process. So, we want to exit, rather than trying to `longjmp()` to
`DeepState_ReturnToRun`, which was never initialized.
2018-02-20 15:43:34 -08:00
Joe Ranweiler
065c97c2c0 Add --take_over flag to native binaries 2018-02-20 15:43:34 -08:00
Joe Ranweiler
661c600d2b Add impl of native takeover test case replay 2018-02-20 15:43:34 -08:00
Joe Ranweiler
93a95b0e96 Use native fake test and file name that matches executors 2018-02-20 15:43:34 -08:00
Joe Ranweiler
4d5e390c15 Factor out native runs of saved cases for a single test 2018-02-20 15:43:34 -08:00
Joe Ranweiler
2ea978999b Replace DeepState_TakeOver() with Manticore function model
Now we have API parity with the angr executor, and return 1 from
`DeepState_TakeOver()` when it is hooked for symbolic execution under
the Manticore backend.
2018-02-20 15:43:34 -08:00
Joe Ranweiler
9e7266399c Add TakeOver impl to Manticore executor 2018-02-20 15:43:34 -08:00
Joe Ranweiler
353bed10ee Factor out unit test running in Manticore executor 2018-02-20 15:43:34 -08:00
Joe Ranweiler
76965704b4 Add TakeOver impl to angr executor
- Add `--take_over` flag
- Allow running tests from non-function instruction addr
- Hook `DeepState_TakeOver()` with a `SimProcedure` that returns 1
- Make a fake test case that starts after `TakeOver` returns
2018-02-20 15:43:34 -08:00
Joe Ranweiler
281c5117ed Add example for DeepState_TakeOver() 2018-02-20 15:43:34 -08:00
Alex Groce
dfd027e8ff Fix to #47 (missing -) 2018-02-20 14:11:01 -07:00
Joe Ranweiler
45fcfe0921 Add DeepState_TakeOver() interface 2018-02-20 10:31:29 -08:00
Joe Ranweiler
0a746ca078 Factor out deepstate-angr API hook setup, unit test exec 2018-02-20 10:31:24 -08:00
Joe Ranweiler
c2b227481f Merge pull request #46 from trailofbits/default-out-dir
Add default output directory for saved test cases
2018-02-20 10:26:32 -08:00
Joe Ranweiler
20781f91d0 Add default output directory for saved test cases 2018-02-20 10:01:45 -08:00
Joe Ranweiler
df6ffd7dbd Merge pull request #45 from trailofbits/deepstate-manticore
Add explicit `deepstate-manticore` executor script
2018-02-19 18:28:37 -08:00
Joe Ranweiler
b7aca2196c Add explicit deepstate-manticore executor script
This matches the command-line API described in the paper, and lets users
invoke the Manticore backend without relying on the fact that it is the
default (which might change).
2018-02-19 18:23:58 -08:00
Alex Groce
a4cb4f4127 First stab at "why DeepState" 2018-02-19 18:21:15 -07:00
Alex Groce
ff3ae3debd remove unused variable i 2018-02-19 14:52:08 -07:00
Peter Goodman
956ee3261e Update main_manticore.py 2018-02-19 11:45:23 +08:00
Dan Guido
1294def677 Update README.md (#36)
Clarify the readme:
* incorporated the paper reference into the first paragraph so it gets excerpted together
* removed a lot of extraneous guidance from the building and installing sections
* added a new contributing section that notes the "good first issue" label, bounties, and our Slack
* noted the the best usage guidance is available in the examples and readme for now
2018-02-18 17:54:35 -05:00
Alex Groce
c9fa425e4b Add link to BAR paper 2018-02-18 13:13:33 -07:00
Peter Goodman
7e69ee208f Merge pull request #33 from trailofbits/improve-mc-crash-saving
Be stricter about what terminated states are saved as crashes
2018-02-18 12:00:45 +08:00
alex
4cb2354759 Better failure when using Manticore but Z3 is not installed 2018-02-16 16:26:50 -08:00
Joe Ranweiler
39e57b6725 Be stricter about what terminated states are saved as crashes
We only want to save a `.crash` file if the input would cause a crash
when executing using the native harness. The old impl treated any state
termination as a crash. Now we add a predicate which checks the `reason`
and decides if the termination represents a crash in the program being
analyzed.

For now, we only flag `InvalidMemoryAccess` exceptions as crashes, as
identified by the `message` property on a `TerminateState` exception.
2018-02-16 12:31:33 -08:00
Joe Ranweiler
febe14ac89 Merge pull request #30 from trailofbits/save-crashes
Save crashes
2018-02-16 10:46:37 -08:00
Joe Ranweiler
226d0f5513 Add enum for test run results and exit codes 2018-02-15 12:43:12 -08:00
Joe Ranweiler
4814e8184b Flag unknown Manticore state terminations as crashes
This is an over-approximation, which we will tighten later.
2018-02-15 10:54:47 -08:00
Joe Ranweiler
c1b91432fd Fix executor docstrings 2018-02-14 14:26:24 -08:00
Joe Ranweiler
0b82d4e409 Run saved .crash test cases in native harness 2018-02-14 12:30:14 -08:00
Joe Ranweiler
7fbb966777 Detect, report crashes in deepstate-angr 2018-02-14 12:24:33 -08:00
Joe Ranweiler
23af5b562d Add DeepState_Crash() hook 2018-02-14 12:22:38 -08:00
Joe Ranweiler
8ede1e2ddc Remove stray whitespace 2018-02-14 12:06:26 -08:00
Joe Ranweiler
86d0fac206 Log error when test crashes 2018-02-14 12:06:26 -08:00
Joe Ranweiler
12dc706534 Add crashing example tests 2018-02-14 12:06:26 -08:00
Joe Ranweiler
dd9c16865d Merge pull request #29 from trailofbits/fork-test-runs
Fork test runs
2018-02-14 10:25:26 -08:00
Joe Ranweiler
ac7e57a833 Consistently stub test case save hooks 2018-02-14 10:24:15 -08:00
Joe Ranweiler
ab43f75361 Add hook to save crashing tests 2018-02-14 10:24:15 -08:00
Joe Ranweiler
68595a9493 Fix handling of test failures due to uncaught exceptions
We don't just want to exit here, but `longjmp()` back to the conditional
via a call to `DeepState_Fail()`. In doing so, we end up in the common
`DeepState_CatchFail()` branch, which exits with the same nonzero error
code, but also saves the test case if appropriate.
2018-02-14 10:10:51 -08:00
Joe Ranweiler
a68520642a Fully factor out forking and running a single test 2018-02-13 14:37:14 -08:00
Joe Ranweiler
3c5d5daeaf Use more accurate function name 2018-02-13 14:37:14 -08:00
Joe Ranweiler
b61b3e7f2d Fork for all test runs 2018-02-13 14:37:14 -08:00
Joe Ranweiler
8f39961b3f Save result of forked test run if flag set 2018-02-13 14:37:14 -08:00
Joe Ranweiler
8ff5f6cac6 Fork when running saved test cases
This anticipates support for crashing tests.
2018-02-13 14:23:44 -08:00