298 Commits

Author SHA1 Message Date
Mark Mossberg
3e81cba8fb Fix linter issues on cpu (#144)
* quick linter stuff

* Fix cpu init in tests

* update cpu name
2017-04-19 10:29:05 -04:00
JP Smith
e32701f978 Refactor logging (#140)
* begin refactoring logging to use manticore object

* fix verbosity things

* fix logging when in scripts

* remove m.log_file now that it's no longer used

* remove unnecessary init_logging usage and unused log_debug variable

* accidentally deleted the wrong line

* re-hide init_logging

* remove old log_debug

* remove unnecessary API stuff and refactor around that

* re-introduce logging PR

* fix Popen

* lost a line in merge

* fix test_binaries

* Log to stdout instead of stderr

* implement mark's changes
2017-04-19 01:21:14 -05:00
Mark Mossberg
65df1cbe35 Fix bad reference to args (#146) 2017-04-18 18:56:58 -04:00
feliam
a9711cf119 Graceful fallback if Z3 doesn't support optimization queries (#135)
* Configure Z3 depending on installed version

* Mocking for subprocess.check_output

* change get_value_fmt to private attribute _get_value_fmt

* Move memoized to utils

* Remove minimal version and simplify

* Re-add invalid versions/sanity checks

* Z3 version format checked on github

* New tests. Version is future proof

* Better logging
2017-04-18 18:40:06 -03:00
Mark Mossberg
f6f20b5210 Register stat64 for 32bit syscalls (#142) 2017-04-18 16:21:28 -04:00
Mark Mossberg
d8335dcd97 track sys_writev32 (#143) 2017-04-18 15:39:04 -04:00
Mark Mossberg
531f245817 Fix arm PUSH (#141)
If sp is in the list, it gets pushed incorrectly because it gets
updated by the stack_pushes for registers that occur before it
2017-04-18 14:00:13 -04:00
Yan
398e4d2510 README + Output updates (#139)
* Importing Dan's README
* Reintroduce etcd's pr
2017-04-17 16:58:34 -04:00
Mark Mossberg
05b7c1138b Fix memory._ceil, Fix linux.load (#134)
* Reimplement ceil so page aligned addresses are unaffected

* Lingering fix

* Fixup users of _ceil

* Correct memory tests
2017-04-13 18:45:18 -04:00
feliam
bbfb234cca Dev fix amd64 emulation bugs (#136)
* Failing tests

* Tha Fix

* Fix auto test

* PSLLDQ for 32b python?

* More descriptive temporary variable
2017-04-13 19:22:04 -03:00
Mark Mossberg
43d7669252 Create stderr file, also watch sys_writev (#129) 2017-04-11 14:21:18 -04:00
Mark Mossberg
ebd0a8be72 Fix linter bugs from model (#122)
* liux

* easy

* fix

* more

* import
2017-04-11 09:47:05 -04:00
Mark Mossberg
b02b551911 Update README/travis (#126)
* Update travis install to match README

* Fix README error

Placement of the -e is important, apparently
2017-04-10 17:22:02 -04:00
Mark Mossberg
5870211b1e More gracefully handle uninstalled z3 (#127)
* More gracefully handle uninstalled z3

* Use better error
2017-04-10 17:10:40 -04:00
Yan
801e20aa4c Update docstrings to sphinx format (#119)
* Update docstrings to sphinx format
* Update various docstrings
* Add more docstrings and update sphinx docs
* fix trailing colons on returns
2017-04-10 17:06:47 -04:00
Yan
5c112e56f3 Add brief example descriptions to README (#120)
* Added brief example descriptions to README
* Update config script
* Update sym.py API usage
* Remove individual example descriptions
* fix mcore init
* consistently document examples
* make basic_sym runnable
* More path fixes
* cleanup; update api
2017-04-10 14:31:51 -04:00
JP Smith
a41eb6feba Fix typo in linux.py (#116) 2017-04-10 12:40:45 -05:00
Dan Guido
99a3327b8d Condense some of the instructions, add slack badge 2017-04-08 16:24:45 -04:00
JP Smith
1bcadfed4d Add memory corruption example (#124)
* add memory corruption example

* put binary in proper folder
2017-04-06 10:39:45 -05:00
JP Smith
ccd473b88e add partial solves for multiple-styles (#123) 2017-04-06 10:39:07 -04:00
Yan
0b850dba40 Make unicorn be pull-based (#97)
* Enable simple ARM register concretization for Unicorn

* Add canonical_registers property to abstractcpu

* cpu to self

* Check for regs_access better

* Emulate a single instruction

* Bypass capstone 3.0.4 arm bug

* Dealing with capstone

* Temporary disable ASR and remobe BitVec.Bool from test

* WIP WIP debug prints WIP WIP

* Unicorn fallback working (using unicorn master)

* HAck to support unicorn 1.0.0

* WIP

* Unicorn hack to handle PC updates

* [WIP] do not do anything with this commit; for debugging only

* Adding before clean up

* emulation more or less works; need to work out more unicorn bugs

* clean up emulate() caller code

* move hooks to methods; cleanup

* Concretize memory when emulating

* Re-add Bool()

* Update tests to start at offset 4

 When an instruction branches to the previous instruction,
Unicorn attempts to dereference that memory. We'd like to use
unit tests to also make sure Unicorn emulation is in line with
our own semantics. If we start all tests at offset 4, we can
jump to a previous instruction and not fault when Unicorn
dereferences it.

* Fix concretization

* Clean up test imports; upper-case Cpu

* Unicorn tests

 * Add tests for all the ARM semantics, but make sure they're equivalent
   on unicorn.
 * Add a few tests to make sure unicorn correctly concretizes the memory
   it references

* Fix broken import

* Add symbolic register tests

* Re-introduce the unicorn hack

* Add the 'ONE' concretization policy

* Rm unused function

* Update concretization; add comments

* Add ONE policy test

* Create a base class for all concretization exceptions

* Remove Armv7Cpu._concretize_registers

* Check for enabled logging in a more idiomatic way

* [wip] intermediate testing commit

* Reimplement hooks and execution with unicorn

* Add a DMB (mem barrier) instruction; nop

* simplify instruction resolution

* improve unicorn error handling

* explicitly delete emu

* Handle ARM helpers inline

* map fetched memory

* Narrow exception handling

* Update DMB docs; make __kuser_dmb match real implementation

* Fix typo; add comment; remove extraneous parameter

* typos++
2017-04-03 16:00:49 -04:00
Mark Mossberg
0ad327522b Update readme (#117)
* Update readme

- eliminate duplicated install instructions
- document pip version
- explicitly add capstone/keystone workarounds

* Update

* spelling
2017-04-03 14:41:06 -04:00
Mark Mossberg
ffe758f5f3 Fix interpreter loader (#115)
* Correctly compute total interp size

* Rename

* Make staticmethod

* Add basic loading test
2017-03-31 13:22:59 -04:00
feliam
83cb4de825 Dev fix load interpreter (#96)
* Fix interpreter load

* turn off some debug

* cuack

* Correct loader condition (#111)

* Rm unnecessary condition

We always want to run this code if there is an interpreter present, not
just if the exe is ET_DYN

* Add correct interp header type check
2017-03-28 15:45:08 -03:00
feliam
dbb63cfa34 Dev stoping criteria (#108)
* no-mp not uset and redundant

* Add timeout

* Unused args reremoved timeout readded

* no-mp not uset and redundant

* Add timeout

* Unused args reremoved timeout readded

* Move import to the top op op op
2017-03-28 14:48:23 -03:00
JP Smith
db6370bf87 Remove redundant pstats import (#113) 2017-03-28 12:16:41 -05:00
JP Smith
1ad9a88032 Add basic benchmarking script (#83)
* add basic benchmarking script

* add results class, collect more information, and fix check for no args

* fix indentation

* we don't have logger here

* use argv[0] for name of program

* allow dumping of stats from the API and use .format()

* add ProfilingResults class

* bugfixes
2017-03-28 11:12:36 -05:00
Mark Mossberg
e3f6c8e4ba Force uninstall Manticore in case of cache (#112) 2017-03-27 17:52:36 -04:00
Mark Mossberg
1047fa42e3 Document state, cpu, issymbolic (#101)
* Add unstable warning

* verbosity docs

* Add State and Cpu to docs

* Add State docs

* Add cpu docs

* Add select cpu docs

* Add issymbolic helper doc

* Rm double docstring

* Update write_bytes docstring

* Update Cpu docstr

* Document cpu reg interfaces

* Fix rebase mistakes
2017-03-27 15:03:42 -04:00
feliam
6b1c69cb28 Dev fix decree (#98)
* Re enable skipped test

* Re enable skipped test

* Hack so we can use upstream Elftool

* pr comments

* pr comments - staticmethod

* pr comments - implicit > explicit
2017-03-24 19:10:33 -03:00
Yan
f0a6f66d77 Update MOV implementation (carry) (#105)
* Update MOV implementation wrt carry
* Remove intermediate flags dict
* Fix register reference
* Document MOV to conform with current standard
2017-03-24 17:55:24 -04:00
feliam
7dd0879918 Dev arm regfile refactor (#57)
* RegisterFile refactor everywhere

* Update arm_rf tests

* Refactor Operand in both x86 and arm

* Add APSR support (#53)

* Add APSR support

* #issuecomment-284826572

* APSR tests

* cspr to aspr

* RegisterFile refactor everywhere

* Update arm_rf tests

* Refactor Operand in both x86 and arm

* Rebased and NotImplementedError fix

* ARM register initialization compressed

* Fix merging bugs

* Assimilate PR comments

* Assimilate PR comments2

* The new style. future is now

* WIP

* Improve quick start ,add asciinema (#58)

* Towards a helloworld world

* Rm requirements.txt (#56)

* Print workspace dir by default (#55)

* Print workspace dir by default

* Change initial cli output

* refactor or or or

* WIP

* Add arm rf testcase

* ARM aliases fix

* debug print removed

* fix tests

* Remove unicorn script from travis build (#64)

* remove unicorn script from travis build

* remove unicorn script entirely

* Rename test -> tests (#66)

* rename test -> tests

* re-add ignored tests

* Switch instructions to prefer virtualenv (#65)

* switch instructions to prefer virtualenv

* document use of virtualenvwrapper

* Make cpuid more forgiving (#67)

* Make cpuid more forgiving

* error() to warning()

* Add CPUID EAX=0xd ECX=0,1

* Let logger handle the format string iff needed

* to hex

* Rm unused files (#76)

* Move state into own file (#75)

* Move State from executor.py

* Fix import

* relative import

* Rm unnecessary code (#80)

* Add example crackme (#77)

* add example crackme

* make filename and directory structure more accurate

* Fix state.branches (#74)

* init

* rm old code

pretty sure it's wrong

* Clean

* rm

* Rename record_fork, move to ConcretizeRegister exception handler

It doesn't need to be in fork() because it's only relevant for when
we fork due to PC.

* Add test for record_branches

* Add back record_branches which got removed in rebase

Rebase went oddly because State got moved to a new file

* unnecesary tuple()

* necesary tuple

* remove unnecessary keys()

* Use OrderedDict (#84)

It can be useful to know elements in `branches` towards the end are
more recently discovered branches

* arm: Fix broken dynamic APSR (#78)

* Refactor Register (#82)

* Fix register read bug

If we store a BitVec, we do NOT want to return bool() of it, which
returns True, which is totally incorrect. We do however, want to return
a symbolic Bool of it, if it is a 1 bit register (flag).

* Improve

* Raise error

* Fix register tests

Changed the interfaces:
- removed nbits param (never used)
- no longer raise AssertionError when overflowing a flag reg, just
  truncate
- rename test funcs to be more descriptive

* rm unused import

* Add symbolic tests

* Rm no longer applicable test

* Add docs (#61)

* Add sphinx-quickstart generated docs

* Add api.rst

* Edit index.rst

* Add autodocumenting Manticore class

* Update Manticore docstring

* Doc verbosity

* Doc hook and add_hook

* More docs

* Add Sphinx dev dep

For building docs

* RegisterFile refactor everywhere

* Update arm_rf tests

* Refactor Operand in both x86 and arm

* RegisterFile refactor everywhere

* Refactor Operand in both x86 and arm

* Rebased and NotImplementedError fix

* ARM register initialization compressed

* Fix merging bugs

* Assimilate PR comments

* Assimilate PR comments2

* The new style. future is now

* WIP

* Towards a helloworld world

* refactor or or or

* WIP

* Add arm rf testcase

* ARM aliases fix

* debug print removed

* fix tests

* unnecesary tuple()

* necesary tuple

* remove unnecessary keys()

* rebased

* Remove test

* https://github.com/trailofbits/manticore/pull/57#pullrequestreview-27971778

* https://github.com/trailofbits/manticore/pull/57#discussion_r107820815, https://github.com/trailofbits/manticore/pull/57#discussion_r107820331

* https://github.com/trailofbits/manticore/pull/57#discussion_r107821090

* https://github.com/trailofbits/manticore/pull/57#discussion_r107821066

* https://github.com/trailofbits/manticore/pull/57#discussion_r107821919

* OOps search and replace

* (invalid) -> None

* The (invalid) -> None

* None vs. invalid

* In armpy we know that STACK is SP so lets try to eliminate cpu.STACK in favor of x.SP

* remove ugly hex

* Removed redundant import and comment
2017-03-24 14:44:24 -03:00
Mark Mossberg
a12aae80d1 Move stats.py to scripts/ (#100) 2017-03-23 14:25:46 -04:00
JP Smith
053cc5ff8d Fix run_simple.py (#102)
`Manticore(None, path)` throws an error
2017-03-23 09:52:39 -05:00
feliam
f016b7635a Heads removed (#99) 2017-03-22 20:45:14 -03:00
Mark Mossberg
ebe7f8268b Add docs (#61)
* Add sphinx-quickstart generated docs

* Add api.rst

* Edit index.rst

* Add autodocumenting Manticore class

* Update Manticore docstring

* Doc verbosity

* Doc hook and add_hook

* More docs

* Add Sphinx dev dep

For building docs
2017-03-22 15:44:03 -04:00
Mark Mossberg
d0f2c0116e Refactor Register (#82)
* Fix register read bug

If we store a BitVec, we do NOT want to return bool() of it, which
returns True, which is totally incorrect. We do however, want to return
a symbolic Bool of it, if it is a 1 bit register (flag).

* Improve

* Raise error

* Fix register tests

Changed the interfaces:
- removed nbits param (never used)
- no longer raise AssertionError when overflowing a flag reg, just
  truncate
- rename test funcs to be more descriptive

* rm unused import

* Add symbolic tests

* Rm no longer applicable test
2017-03-21 16:50:39 -04:00
Mark Mossberg
d50b3da9a8 arm: Fix broken dynamic APSR (#78) 2017-03-21 16:47:52 -04:00
Mark Mossberg
ca396d8d13 Use OrderedDict (#84)
It can be useful to know elements in `branches` towards the end are
more recently discovered branches
2017-03-21 16:47:40 -04:00
Mark Mossberg
5368716e42 Fix state.branches (#74)
* init

* rm old code

pretty sure it's wrong

* Clean

* rm

* Rename record_fork, move to ConcretizeRegister exception handler

It doesn't need to be in fork() because it's only relevant for when
we fork due to PC.

* Add test for record_branches

* Add back record_branches which got removed in rebase

Rebase went oddly because State got moved to a new file
2017-03-20 17:56:25 -04:00
JP Smith
e77628bfe7 Add example crackme (#77)
* add example crackme

* make filename and directory structure more accurate
2017-03-17 14:43:06 -05:00
Mark Mossberg
7742df6b67 Rm unnecessary code (#80) 2017-03-17 14:10:48 -04:00
Mark Mossberg
e920b73e14 Move state into own file (#75)
* Move State from executor.py

* Fix import

* relative import
2017-03-16 14:43:17 -04:00
Mark Mossberg
4464b2e842 Rm unused files (#76) 2017-03-16 12:02:29 -04:00
feliam
963669f3f2 Make cpuid more forgiving (#67)
* Make cpuid more forgiving

* error() to warning()

* Add CPUID EAX=0xd ECX=0,1

* Let logger handle the format string iff needed

* to hex
2017-03-14 13:39:06 -03:00
JP Smith
5d2ca04957 Switch instructions to prefer virtualenv (#65)
* switch instructions to prefer virtualenv

* document use of virtualenvwrapper
2017-03-13 16:14:18 -05:00
JP Smith
ca0bee2377 Rename test -> tests (#66)
* rename test -> tests

* re-add ignored tests
2017-03-13 14:06:36 -05:00
JP Smith
dd24878724 Remove unicorn script from travis build (#64)
* remove unicorn script from travis build

* remove unicorn script entirely
2017-03-13 10:51:49 -05:00
Mark Mossberg
296f4abb34 Print workspace dir by default (#55)
* Print workspace dir by default

* Change initial cli output
2017-03-10 10:49:31 -05:00
Mark Mossberg
88456a4ef8 Rm requirements.txt (#56) 2017-03-09 13:01:26 -05:00