Transfer TUI Code to Dev Branch #1605
Conversation
|
|
|
As a general rule, generated files shouldn't be checked into the repo. They should be generated when the project is built. This way, there is no chance of forgetting to update them manually. |
Yeah point taken. Atm I think the Manticore dependency list has to be updated to account for protobuf and possibly npyscreen. |
|
Regarding the generation of protobuf files during build: Unfortunately, I don't know the best way of doing this for a python project, but I think taking a look at Google's own I can also help with taking a closer look if nothing helpful is turning up 👍 |
|
Actually looking into this right now, seems like a possible way to do this is to just run a post-install script via setuptools. I'll test and push if this ends up working. |
It might work, it might not. We'll let the CI sort it out.
Looks like
Personally I'd vote for the second option. |
Yeah, the user would have to (on ubuntu) I would probably go with the second option as well since I think doing so would save the user a step in the installation process, and that the protocol shouldn't change that often across builds. |
| @@ -191,7 +191,7 @@ def __init__(self, initial_state, workspace_url=None, policy="random", **kwargs) | |||
|
|
|||
| During exploration Manticore spawns a number of temporary states that are | |||
| maintained in different lists: | |||
|
|
|||
There was a problem hiding this comment.
Looks like some dangling whitespace made it in here somehow?
See if this makes the EVM tests pass
* Support for TUI (#1605) * Update worker thread for server creation * Add necessary files for TUI connectivity * Add necessary files for TUI connectivity * Update MonitorWorker * Update protocol * Blacken * Update setup.py dependencies * Remove state debugging messages * Update setup.py to build protobuf protocol upon install * Remove previously generated state_pb2.py * Change subprocess.Popen to subprocess.check_output * Remove extraneous output * First attempt at fixing protobuf installation It might work, it might not. We'll let the CI sort it out. * Can't forget the f-string * Error on missing protoc * Disable auto-generation of protobuf file * Ignore pb2_errors * Disable monitor start See if this makes the EVM tests pass Co-authored-by: Eric Hennenfent <ecapstone@gmail.com> * Add log monitoring * Log monitoring via TCP * Swittch to rendering state lists directly * Extraneous line * Switch log buffer to multiprocessing queue * Create state transition events Should make it possible to track movements between state lists * Plug new events into context This will break the state merging plugin (but I'll fix it eventually) * move most enums to their own module * Blacken * Add DaemonThread from TUI branch * Add interface for registering daemon threads * Timestamp StateDescriptor upon updates * Capture return value * Blacken * Add solver wrapper to StateBase * Add `solve` events to all instances of SelectedSolver.instance() * Remove executor constraints from WASM * Add solve events to memory.py * Add intermittent execution event * Be more generous with states whose initialization we missed * Add Native callback for updating state descriptor * Fix state killing * Blacken * codecov: Remove outdated 'yml' entry in CI From these commits codecov/codecov-action@ebea5ca codecov/codecov-action@49c86d6 * Add solve event to evm Make warning messages better Debug GH actions Revert "Debug GH actions" This reverts commit f575eea. Fix some pycharm-detected problems Make symbolic function error message more verbose Add solve to published events Loud errors in callbacks by default Trying to find out what's killing truffle Revert "Trying to find out what's killing truffle" This reverts commit 8bd0224. Revert "Make symbolic function error message more verbose" This reverts commit bd3e90c. Debugging Truffle Restore introspector Add try_except on every callback Unconditionally print error message Add traceback Update event.py Debug subscriptions Debug arguments to callbacks Different debug msg 1ast arg Print statement debugging... Pass in `None` as state Revert "Add try_except on every callback" This reverts commit 1c689dd. * Drop solve events outside of a state context Forgot did_solve Remove traceback * Fix must/cannot_be_null usage * Fix missing solve event * Partially restore old did_fork_state ABI * Called internally * Clone iterators instead of creating a list * Use isgenerator instead of checking if iterable * Fix snapshot restoration * Slightly improve Unicorn test API usage * Temporarily disable property verifier tests * improper skip arg * Add simple tests for introspection API * Add test for custom introspector, improve base introspection test * Add intermittent update timestamp * Only allow daemon registration and introspection registration at initialization * Add docs to manticore.py * Add docs for plugin, add update_state_descriptor to EVM * Fix renamed will_start_run --> will_run * Docstrings for DaemonThread and EventSolver * Docs for enums * Improve pretty printer, add some mypy fixes * Don't run daemon threads if run is called multiple times * If at first you don't succeed, destroy all the evidence you tried. * Test the pretty printer * Add StateDescriptor to RTD * Add newlines for RTD parsing * Update to work with new state introspection API * Add termination messages * Also capture killed state messages * Make info logs debug logs * Apply suggestions from code review Newlines for doc comments Co-authored-by: Eric Kilmer <eric.d.kilmer@gmail.com> * Add some type hints to manticore.py * Add some type hints to plugin.py * Fix type hint for get_state * Add termination message from TUI PR * Add example script * Add docstrings to the example script * Pass introspection plugin type as an argument * Unskip property verifier tests * Add mypy-requests type hints * Remove itertools.tee The problem with usign tee is that only the first callback to use the iterator can write to it. In `ready_states`, the `save_state` after the `yield` statement is ignored for all others. * Make generator cloning a little bit more robust Now Manticore will give up and return the original argument instead of blowing up if it can't clone the generator * Clean up invalidated unit tests We now fire `introspect` for the first time before we have any states * Debug missing Truffle & Examples coverage * Merge coverage from XML file * Switch coverage to JSON, ignore debug logging and NotImplemented code * Fix copy commands * Move .coverage files directly * Set examples to append coverage * FLAG_NAME doesn't work the way we'd like * Use plugin dict to store introspector * Appease mypy * Fix missing property on unique name * Grab EVM PC * Blacken * Run black on all files if the git diff command fails * Fix mypy errors * Make plugin logging even less verbose * Move log capture and state monitoring to daemon threads * Use the config module for host & port * Fix worker configuration and add test for TUI API * Fix log messages breaking native tests * Split up base Manticore tests and logging tests The verbosity changes seem to be taking hold when they shouldn't * Merge LogTCPHandler and MonitorTCPHandler * Confirm that logging tests return to base level * Fix mypy * Switch back to using a deque for log buffering in the default case * Fix deque API * Update state_pb2.py * Reformat programatically generated files * Drop max verbosity in logging tests Haven't been able to figure out why, but somehow other loggers get "stuck" at this high verbosity and the integration tests try to print out the values of every single register. * Fix duplicated code from bad merge * Remove is_main from state_monitor * Add comment about log buffer size * Remove vestigial is_main * Blacken Co-authored-by: Philip Wang <powdercheeze@gmail.com> Co-authored-by: Eric Kilmer <eric.d.kilmer@gmail.com>
Added an additional thread in manticore/core/worker.py to set up socket server and communicate with TUI via serialized protobuf messages. Currently more of a proof-of-concept than a robust implementation, but hopefully adding additional functionality won't be too difficult.
Current setup has hardcoded host and port (127.0.0.1, 1337), and successfully sends serialized states to the client TUI. In the future, the host and port should be saved by ansible upon provisioning a DO droplet.