436 Commits

Author SHA1 Message Date
feliam
08e8c3b7d8 Fix coverage calculation vs. metadata (#529)
* Fixes symbolic reentrancy example

* Fix coverage Issue# 527

* Remove  debug unused code
2017-10-23 17:45:01 -03:00
feliam
a9ad183354 Fixes symbolic reentrancy example (#528) 2017-10-20 16:43:22 -03:00
Mark Mossberg
5aa681dc76 0.1.5 Release (#501)
* Update CHANGELOG.md

* Update 0.1.5 README

* Bump version/release date
2017-10-19 15:46:22 -04:00
feliam
23218d724e Simple event signature fixes (#525)
* dev-evm-event-bugfixes

* Better stringifierer
2017-10-18 19:17:26 -03:00
feliam
8591bff45f EVM support (#521)
* WIP New Policy class

* WIP pubsub

* Update Signal tests

* EVM support - Wip

* EVM support - dependencies fixed

* EVM support - fix decree merge

* fix decode instrucion event

* Fix small bugs in evm opcodes (too many arguments + wrong LOG name) (#380)

Fix wrong call parameters + typo

* Fix Create/Call

* Fix depth

* Default fixed point in arithmetic simplifier

* small fixes from github comments

* Fix event decode_instruction signature

* wip wip

* Auto tests for evm

* New EVM tests

* Ran 9556  FAILED (failures=166, errors=8, skipped=62)

* Fix some arithmetic instructions

* Ran 9556  FAILED (failures=136, errors=8, skipped=62)

* More instructions - Optimizing symbolic memory

* Added gas to opcodes description - FIX DELEGATECALL POPS

* Add wip wallet example

* The tests

* Solidity constructors need argument after bytecode

* Simple integer overflow working

* Good merge

* Good good merge

* WIP manticore refactor

* Fix default old-style initial state

* context now working

* Fix context serialization

* Fix test models.  Can not set a state constraints

* typo

* A few typos (constraints setter) and use of public properties in internal methods

* Fix init wallet example

* State __init__ needs to initialize platform constraints

* Internal methods use internal properties

* Better attack modeling

* Better example layout

* Storage backup on CALL is now faster .. and correct

* Add LOG support

* Minimal SE test

* Added examples

* Send ether bugfix

* EVM: Fix wrong balance destination on CALL + decrease caller balance on CREATE

* New balance management

* Trying to maintain known hashes

* Known hash concretization policy

* CALLDATA max size bugfix

* Minimal SE example

* Remove evm tests

* add -> enqueue

* @m.init

* Fix workspace url

* Some test skipped

* Ad Fixme to platform specific stuff in State

* add -> enqueue

* Enqueue created state

* Fix m.init

Use a messy hack to adhere to the spec (callback func receive 1 state argument)

* Add _coverage_file ivar to Manticore

* Fix symbolic files

* remove extra enqueue

* Fixing __main__

* comments

* Fix visitors oddity

* setup merged

* remove duplicates and add pysha3

* Remove EVMTests import

* Refactor platform specific code out of ManticoreOutput (#505)

* Initial moving work

* Clean

* Make linux.generate_workspace_files work

* Fix

* clean

* Add test

* Test workspace for platform files

* Skip EVM cpu pretty print

* Remove bad import

* Fix coverage.py for testing

* Clean comment

* Comment hack

* Print evm cpu

* pretty print evm world instead of platform

* delet old scripts/examples

* delet old tests

* Remove z3 install script

* Array.max_size can be None, include check for that

* Rm unused _symbolic_files

add_symbolic_files was moved to linux, so this is not needed

* Rm unused args

* Import evm

* Rm dup function

* Rm stray prints

* Add docs for new classmethod apis

* minimal

* minimal example

* fix minimal

* Fair symbolic SHA3 handling

* Simple mapping example

* coverage example

* fix tests

* fix minimal

* Some eko fixes

* New SETH

* integer_overflow refactored

* Fixing the examples

* init_bytecode -> init
'

* Concrete reentrancy exampole

* concrete reentrancy selfdestruct

* Update minimal.py

* It's a new Minimal

* Integer overflow example

* New minimal

* minimal fix

* Examples last minute fixes

* Remove debug print

* add plugin.py

* Fixing event subscription

* remove temp params

* Remove param

* Update uncovered will_exec callback prototype

* Clean up debug output

* Automatically generated intruction tests

* Uninplemented instruction test removed

* Unused concretization policy removed

* Fixes enabling default bplugins

* solc from PATH

* Removed unused import

* Logger name updated
2017-10-17 19:47:20 -03:00
feliam
e84c0513a2 Manticore plugins (#506)
* WIP New Policy class

* WIP pubsub

* Update Signal tests

* small fixes from github comments

* Fix event decode_instruction signature

* Good merge

* Good good merge

* WIP manticore refactor

* Fix default old-style initial state

* add -> enqueue

* @m.init

* Fix workspace url

* Some test skipped

* Ad Fixme to platform specific stuff in State

* add -> enqueue

* Enqueue created state

* Fix m.init

Use a messy hack to adhere to the spec (callback func receive 1 state argument)

* Add _coverage_file ivar to Manticore

* Fix symbolic files

* remove extra enqueue

* Fixing __main__

* comments

* Experimental plugin system

* tests fixed

* Fix plugins

* Some reporting moved to plugin

* Fix assertions test

* Add published events to classes that publish them

* Update how we verify callbacks

* Update Eventful._publish

* Dev plugins (#512)

* Yet another flavor for event name checking

* really it's a bunch of minimal bugfixes

* Remove get_all_event_names from Plugin

* Update where we get all events

* Use new metaclass-based event registry

* Define prefixes in one place

* remove debug print

* remove debug print
2017-10-04 15:51:37 -03:00
Yan
1edee15e5f Pythonic logger naming (#511)
* More generic logging

* Clean up logger name gen

* Refactor name summarization
2017-09-25 18:09:18 -04:00
JP Smith
b9333838ad install z3 with pip in Dockerfile (#502) 2017-09-14 11:52:15 -05:00
JP Smith
0efa37933a Fix stderr file creation and test for it going forwards (#495)
* fix stderr file creation and test for it going forwards

* consolidate tests

* clean up imports
2017-09-13 19:17:13 -05:00
Mark Mossberg
58cdee814c Update test_workspace.py (#497) 2017-09-13 18:34:45 -04:00
feliam
23c2585316 Refactor platform details out from Manticore (#421)
* WIP New Policy class

* WIP pubsub

* Update Signal tests

* small fixes from github comments

* Fix event decode_instruction signature

* Good merge

* Good good merge

* WIP manticore refactor

* Fix default old-style initial state

* add -> enqueue

* @m.init

* Fix workspace url

* Some test skipped

* Ad Fixme to platform specific stuff in State

* add -> enqueue

* Enqueue created state

* Fix m.init

Use a messy hack to adhere to the spec (callback func receive 1 state argument)

* Add _coverage_file ivar to Manticore

* Fix symbolic files

* remove extra enqueue

* Fixing __main__

* comments

* Refactor CLI, and Manticore high level interfaces (#498)

* Refactor main,

- classmethod for linux
- refactor manticore ctor - compat with old linux behavior
- changed verbosity API (to allow for this use case: what if you want to set verbosity for the stuff manticore does in its ctor?)

* rm old verbosity

* small

* Add decree classmethod

* Rm checks ; they are redundant anyway

* Misc

* Move add_symbolic_file to linux platform

* rm redundant checks

* Rm explicit args for deprecated interface

* Fix cli bug

* Allow for both linux and decree from cli

* Add back argv positional param for deprecated api compat
2017-09-13 18:37:42 -03:00
Mark Mossberg
f6fea10376 Use z3-solver package (#494)
* Add z3-solver pip dep for z3 install

* Update readme for z3
2017-09-12 15:21:48 -04:00
Mark Mossberg
50923974e2 Fix logger verbosity and test (#491)
* something liekt his

* `will_store_state` does not exist

* slightly better cli verbosity test

* Fix variable

* Rm unused callback fun
2017-09-12 15:08:53 -04:00
John F.X. Galea
a8ef3ecd9f Added implementation for sys_dup and sys_dup2 (#490)
* Added implementation for sys_dup and sys_dup2

* Fixed bug in close functionality.

* Removed unwanted white-space

* Updated implementation of sys_dup and sys_dup2

* Fixed is_open function.

* Handle a newfd in dup2 that extends beyond existing fd table size

* Add a minimal sys_getrlimit() implementation

Thank you @johnfxgalea!
2017-09-12 15:03:01 -04:00
Mark Mossberg
f8a8373e57 Request binaries that cause bugs in contributing.md (#486)
* Request binaries

* Update CONTRIBUTING.md
2017-09-11 13:05:51 -04:00
Mark Mossberg
8b66e58c7b Document cli better (#482)
* Update README.md

* Rewrite -h

* Clearer cli usage

* Fix forgotten variable name changes

* Fix more forgotten variables
2017-09-07 20:15:31 -04:00
John F.X. Galea
60c6813a80 Implemented getcwd sys call, along with a new helper function that wr… (#483)
* Implemented getcwd sys call, along with a new helper function that writes a NULL terminated string to memory.

The reason for this commit is that the sys call was not implemented.

* Fixed logging string for sys_getcwd imp

* Removed semicolon in sys_getcwd function

* Fixed logging string for sys_getcwd imp. The problem was that size is unsigned and the incorrect format symbol was used

* Fixed the following:
1) Included the string param in the doc of the write_string function.
2) Added less verbose code to append a NULL to a string in the write_string function
3) Removed the assert to check if the written string to memory matches with one that is read at a given address in the sys_getcwd function
4) Fixed bug in the return values returned by sys_getcwd

* Fixed error logs in the getcwd function.

* Fixed return values of sys_getcwd function.

* Fixed string format symbol in sys_getcwd function.

* Removed dir exists check from the sys_getcwd function.

* Arranged memory block check in the sys_getcwd function.

* Removed new line after write_string function.

(Thank you @johnfxgalea!)
2017-09-07 16:30:55 -04:00
Mark Mossberg
d83fa737a7 Create CONTRIBUTING.md (#481)
* Create CONTRIBUTING.md

* Update CONTRIBUTING.md

* Update CONTRIBUTING.md
2017-09-06 18:11:22 -04:00
Yan
7679773088 Update examples (#477)
* Use locked_context() in count_instructions example

 * We use this example on the front page, and it currently generates
   a flood of warnings.

* Update example indeces

* Remove dump_stats()

* Remove extra comments

* Remove redundant comments
2017-09-01 17:05:43 -04:00
Yan
9b8ef3cd76 Concretize stdout in state summary (#478) 2017-09-01 16:31:50 -04:00
JP Smith
0cd2560138 Update Manticore docstring (#476)
* update Manticore docstring

* fix type docstring
2017-09-01 11:43:29 -05:00
JP Smith
d5d22a8aaf Allow Manticore initialization with State object (#475) 2017-08-31 16:44:43 -05:00
feliam
f9b02bd194 New Policy class (#368)
* New Policy class

* Move visited to Policy - Fix contexts

* Add BranchLimited

* signals -> events for branchlimited
2017-08-31 12:48:55 -04:00
JP Smith
c6f8fd1291 initialize executor at manticore initialization (#471) 2017-08-31 10:16:52 -05:00
Yan
ca0d7b60d2 Add basic socket support (#428)
* Add a descriptor class for socket(2) return values

* Add basic socket(2) and bind(2)

* Basic accept(2)

* Basic recv(2) support

* Debug statements and accept(2)

* Add check for symbolic arg to open(2)

* Fix an example warning

* send/sendfile

* debug->info for some mcore output

* Produce socket data as part of state serialization

* Clean up SLinux.sys_open()

* Document socket(2) peculiarities
2017-08-29 17:00:17 -04:00
Theofilos Petsios
a025bda97a Updated visualization plugin with coverage info & live update option (#444)
* plugin rearrangement and UI options

* Live update and clear options

* coverage stats

* added check for xref address

* coverage in separate option as BackgroundTaskThread

* license cleanup
2017-08-23 17:47:23 -04:00
JP Smith
60a99d2f81 Seed random on policy init to make results deterministic (#466)
* seed random on policy init to make results deterministic

* document new random behavior
2017-08-23 16:37:26 -05:00
Theofilos Petsios
1b653be9b3 Basic Caching for Binja IL (#467)
* basic caching (wip)

* moved execute in binja and cleanup of self.instruction refs

* did_emulate_insn and get_current_llil_func renaming

* refactor for hasattr
2017-08-23 14:46:17 -04:00
Theofilos Petsios
b32379d3d4 did_execute_instruction nit (#468) 2017-08-21 23:03:47 -04:00
Mark Mossberg
d3d1e9c924 Manticore 0.1.4 (#462)
* Bump version number

* Changelog updates

* Include changes to undocumented functionality, but mark as experimental

* Function notation

* Update CHANGELOG.md

* thanks!!!
2017-08-18 13:48:33 -04:00
Mark Mossberg
604de56cfd Update readme (#461)
* No more partial arm

* windows unmaintained :(( and bye pip

* Move dev install insns to wiki, no need to clutter README
2017-08-18 11:12:14 -04:00
Theofilos Petsios
715cc436c7 Abstract Disassembler classes and Binja CPU + Platform (#364)
* adding abstract disasm class

* before adding abstract insn

* explicit capstone use

* (wip) removing capstone

* debugging nose

* removed disassembler from constructor

* nits

* capstone->cs and nits

* basic memory (wip)

* tmp

* added binja dissasm

* fixed execution pipeline - calling LLIL instructions

* updated parseargs to select disassembler

* unstable (disassembler at CPU constructor breaks tests)

* disasm fix for tests

* nit for aliases in x86

* added a flag to differentiate disassembly and IL-disassembly for binja

* before merging

* merging with dev-events master

* fixed signal forwarding for binja

* cleaner initialization

* Binja RegisterFile. Dropped Platform

* address fixup

* properly incrementing PC

* some work on XOR, SET_REG, REG, CONST, AND, PUSH, POP

* adding first CALL, JMP, LOAD (wip)

* JUMP fix

* GOTO and misc fixes in PC handling

* adding instructions (wip)

* fixing flags and sizes (wip)

* loading database if exists for faster analysis

* fix for register debug

* fixes for register sizes and flags

* FIX for multiple IL instructions sharing the same PC

* removed CONST_PTR and misc fixes

* RET, SHR, shift left & misc instructions, fixing insn sizes (wip)

* flag fixes

* ctypes 2's complement parsing

* JUMP_TO, SYSCALL

* syscall fixups and flag computation using binja il only

* FLAG, NEG, CMP(family) of LLIL, some unimplemented methods

* MUL, DIV

* cmpxchg

* MUL, IMUL, ROR, ROL

* fallback to capstone for all LLIL_UNIMPL and LLIL_UNIMPL_MEM

* fixes for registers and memory when switching CPUs

* check for binja to disable multiprocessing

* merging

* nit

* hack for serialization

* moved check for disassembler to Manticore from __main__

* removing __class__ refs

* cleanup from __class__.disasm

* size calculation from get_instruction_low_level_il

* fix for NEG, check for empty queue, execute refactor

* fixes for LLIL functions

* removal of redundant regfile writes

* nit fixes

* function overrides in Cpu classes and orphan printf cleanup

* nit

* incomplete merge

* verbosity temp mod

* cleanups of FIXMEs

* 2-stage constructor for disassembler

* cleanup binja refs in abstractcpu

* serialization for platform_cpu

* check for UNIMPL in all the il queue

* typo fix

* fix for arm

* typo correction and starting caching implementation

* restored register printing and fixed import in x86.py
2017-08-17 17:43:27 -04:00
Mark Mossberg
f9489d3249 Document Manticore.locked_context API (#460)
* Add locked_context to documented APIs

* Initial
2017-08-17 16:50:21 -04:00
Mark Mossberg
22974ca65e Add gotchas docs (#459)
* Add gotchas docs

* always
2017-08-17 16:48:59 -04:00
Garret Reece
5d87fc83b3 Instructions and tests to support issue binary for issue #182 (#403)
* orn instruction plus unit test
* uadd8 and it instructions implemented
* SEL instruction implementation
* GE flag set by UADD8
* IT instruction condition code properly no longer prevents its execution
* support for multiple instruction tests added to testing setup
* unit test for SEL instruction
* cleaned up the tests for thumb instructions
* implemented sxth and rev instructions
* implemented tests for sxth and rev instructions
* updated the add implementation to support two operand variant (i.e., add r4, #4)
* added test for itete ne instruction
2017-08-17 10:02:44 -05:00
JP Smith
7bb64fe26e fix trace-saving behavior (#454) 2017-08-15 10:50:46 -05:00
Theofilos Petsios
09bb702fac Implement logging presets (#445)
* logging template nits

* moved logging outside Manticore

* naming fix and exposed API call

* logging -> log renaming

* fix for verbosity test

* restored setter/getter for verbosity

* fixes for warnings in 0 mode and exteneed test_cli_verbosity

* removed print

* relative import, fork message to executor, init function

* verbosity docstring fix

* docstring nit
2017-08-14 16:05:29 -04:00
Mark Mossberg
1d8e051522 Fix state.generate_testcase (#451)
* Use correct event name

* Add test for state.generate_testcase

Test merely tests that this function publishes the 'will_generate_testcase' event

* Properly test; make sure the callback executes and error if not

This is the most convoluted unit test I've ever written.

* "better"
2017-08-14 14:44:53 -04:00
Yan
36a58810f9 Use the same workspace URI in Executor.__init__ (#443)
* Make sure we're using the same workspace

* create_store -> fromdescriptor

* classmethod -> cls variable

* Cache descriptor name

* join -> format

* Update Store docs
2017-08-10 15:07:53 -04:00
Mark Mossberg
e78ae8904e Add Manticore version to issue template (#447)
* Add Manticore version to issue template

* Update ISSUE_TEMPLATE.md
2017-08-10 15:06:51 -04:00
Mark Mossberg
eef3cd20e2 Implement state.generate_testcase (#438)
* ran

* Some dirty work

* Rm unnecessary sanitization

* Fix state.generate_inputs event publishing

We don't need to manually pass `self` because State is automatically
forwarded as an argument. Also correctly specify a message for the state
and expose to the API

* Rename

* Update test for new save_testcase args, make test more robust

* Simplify arguments related to generate_testcase_callback by removing default arg. Add test for this behavior

* Minor test fixup

* Add official API docs

* Remove unnecessary new event type
2017-08-08 18:12:55 -04:00
Mark Mossberg
9ae76a4430 Fix testcase generation output (#436) 2017-08-07 08:08:37 -07:00
Dushyanth Chowdary
68555f5234 Fixing raise issue #435 (#437) 2017-08-04 18:27:15 -07:00
JP Smith
af3020db95 log testcase generation (#430) 2017-08-04 17:18:45 -04:00
Mark Mossberg
7b832724a6 Implement symbolic branch tracking in core, add new fork event (#433)
* Add fork_state event for individual state forks

* Rename to forking_state, add branch tracking to state

* Remove unnecessary locking around event publishing
2017-08-03 14:13:40 -07:00
Yan
5ad18e736c Produce fork-related output at info level from mcore (#429) 2017-08-02 17:31:29 -04:00
feliam
3b57c0d502 Fix context serialization (#425)
* Fix context serialization

* Fix test models.  Can not set a state constraints

* typo

* A few typos (constraints setter) and use of public properties in internal methods

* State __init__ needs to initialize platform constraints

* Internal methods use internal properties

* test_record_branches removed
2017-08-01 14:08:54 -03:00
JP Smith
271400728c remove state.generate_inputs (#385) 2017-07-31 14:39:47 -04:00
Eric Hennenfent
51837df98b Add Taint Parameters (#414)
* Add taint args to buffer creation

Allows the `new_symbolic_buffer` and `symbolicate_buffer` functions to take keyword args for tainting. Defaults to frozenset in both cases.

* Add unit tests and ArrayProxy taint propery

Adds simple unit tests for tainted buffers. Added a property to the ArrayProxy class in smtlib.expression so that it's possible to access the taint of the proxied ArrayVariable.

* Updated docstrings
2017-07-28 11:58:25 -04:00
feliam
76357216da Pub-sub like events (#371)
* WIP New Policy class

* WIP pubsub

* Update Signal tests

* small fixes from github comments

* Fix event decode_instruction signature

* Good merge

* Good good merge

* Eventful class commented

* The million typos

* Code revision

* Fix tests for new mor strict Eventful
2017-07-27 19:41:08 -03:00