Commit Graph

161 Commits

Author SHA1 Message Date
rocky
6d0f72f13b Fix bug in 2.6 tryelse get test_grammar working...
localize grammar rules
2017-12-02 01:35:43 -05:00
rocky
bf5a6237d8 Isolate listcomp rule in 2.7 2017-11-30 10:30:08 -05:00
rocky
2674ec893a Grammar reduction 2017-11-27 04:56:01 -05:00
rocky
17f5b35b1d More grammar isolation 2017-11-26 16:34:10 -05:00
rocky
b7003914c9 localize 2 and 3 argument BUILD_SLICE...
Nontermninal name matches AST anme now. Add test.
2017-11-25 21:10:11 -05:00
rocky
902941102f 2.7 continue-detection bug 2017-11-16 00:22:24 -05:00
rocky
dea17cd7f1 xdis _disassemble->disassemble 2017-11-06 00:38:22 -05:00
rocky
5df09540b5 Python 3.6-inspired instruction size cleanup
Revise and generalize for Python 3.6+ instructions vs < 3.6 instuctions.
Used more of the generalized methods in xdis and remove some (but not
all) of the magic numbers.

This is a lot of changes, but not all of the refactoring needed. Much
crap still remains. Also, there are still bugs in handling 3.6 bytecodes.
2017-10-29 11:46:28 -04:00
rocky
1d7a3c6444 Document hacky customize arg count better. 2017-09-30 18:02:35 -04:00
rocky
d138a01bf1 xdis's "exception match" is now "exception-match" 2017-07-17 22:42:57 -04:00
rocky
27ab6fe2f5 Use xdis 3.5.0's opcode sets 2017-07-08 20:41:46 -04:00
rocky
0893652943 Work around not having real flow-control analysis 2017-06-29 20:49:01 -04:00
rocky
6efd7afda3 continue non-detection in Python 2.7
fixes issue 122
2017-06-29 20:27:07 -04:00
rocky
af3d46b35c Use xdis' instruction offset calculation fns..
next_offset, op_size, has_argument
2017-06-24 06:43:04 -04:00
rocky
c7f8edd5ef Simplify access to L65536 ...
and fix use in scanner26.py. Thanks to AnythingTechPro
2017-06-09 18:22:02 -04:00
rocky
160ec0d9cc COME_FROM for 3.x POP_EXCEPT, DRY with op_name() ...
Start adding COME_FROMs for POP_EXCEPT in preparation for
getting tryelse blocks correct.

Simpler opname access functions:
  - self.op_name(op) is self.opc.opname[op]
  - self.op_name_from_offset(offset) is self.opc.opname[self.code[offset]]

verify.py: not all offsets are ints
2017-03-01 05:50:31 -05:00
rocky
1cb2cd7a82 More 2.6, 2.7 control flow
Todo more COME_FROMs but now need to check targets better. In some cases
we're relying on grammar ambiguity to work out right and in 2.7 it doesn't
2017-01-24 01:21:28 -05:00
rocky
9ec312ba5e More 2.6, 2.7 control-flow bugs
Wasn't limiting exception clause to try finally. Probably still has bugs
in try-finally nesting

Add another 2.6/2.7 COME_FROM to try to limit if/end scope better
2017-01-24 00:53:30 -05:00
rocky
597d51951e Improve Python 2.6 & 2.7 verification 2017-01-23 02:32:09 -05:00
rocky
8dc23e2cdc Python 2.1 doesn't have FOR_ITER or GET_ITER...
adjust locgic for this fact
2017-01-15 09:50:38 -05:00
rocky
ec0669367f Towards better 3.0 decompilation
Sync scanner2 and scanner3 better
2017-01-08 17:40:57 -05:00
rocky
6f097ff1ca dectect_structure() -> detect_control_flow() 2016-12-29 07:32:36 -05:00
rocky
8eb1a16f5b DRY code and emitted Python 3 source
* Python 3: break; continue -> break
* Use variable in detect_structure for pre[rtarget]
* Make Python 2 and Python 3 detect_structure more alie
2016-12-29 07:28:37 -05:00
rocky
6c546fe6e1 WIP : Add THEN to disambigute from "and" 2016-12-27 22:45:08 -05:00
rocky
e3f4beeb74 Lint 2016-12-24 07:45:02 -05:00
rocky
e6761e13bb Python flake8 crap
Was testing realgud's C-x!8 (goto flake8 warning/error)
2016-12-18 20:18:19 -05:00
rocky
eebec48308 show-asm on python2.5 is optional
make scanner2 look a little more like scanner3
2016-12-17 08:01:25 -05:00
rocky
d22931cb49 Get ready for release 2.9.7
Some of the many lint things. Linting is kind of stupid though.
2016-12-04 09:36:30 -05:00
rocky
1e324e0e8d Misc changes
scanner26.py: make scanner2.py and scanner26.py more alike
scanner2.py: check that return stmt is last in list. (May change)
main.py: show filename on verify error
test/*: add more
2016-11-26 21:41:45 -05:00
rocky
abecb21671 2.7 grammar bug workaround. Fix docstring bug 2016-11-24 21:57:39 -05:00
rocky
8941417a54 <2.7 "if" detection and dup Python 3 grammar rule 2016-11-24 05:33:08 -05:00
rocky
260ddedbfd More detailed COME_FROMs
For now we only add COME_FROM_FINALLY and COME_FROM_WITH
and even here only on 2.7
2016-11-22 19:42:26 -05:00
rocky
2bc316d6f0 more 2.7 control flow bug fixing 2016-11-20 06:55:08 -05:00
rocky
f2a3721d7d Start to improve detect_structure for 2.7 and 2.x
Add debug flag to find_jump_targets to show the structure we found.
When there are control-flow bugs, it's often reflected here.

scanner3.py: make code make more similar to 2.x code
2016-11-20 02:38:59 -05:00
rocky
76ae1592d0 verify scanner2 vs scanner3 small changes...
verify.py: allow LOAD_CONST None to make LOAD_NAME 'None'
scanner{2,3}.py: make them look more alike
2016-11-17 03:43:39 -05:00
rocky
9e3026bd78 WIP Grammar changes - reinstatng COME_FROMs around ignore_if's 2016-11-15 23:44:22 -05:00
rocky
766451cbb9 WIP remove COME_FROMs around ignore_if's 2016-11-14 09:27:56 -05:00
rocky
6073c77921 Show line numbers in 2.6 "after" asm ..
start to understand some of the Python 2.6 bytecode parse failures.
2016-11-14 00:30:23 -05:00
rocky
cd3cf5ec29 Use L. for line number prefix in asm and AST 2016-11-03 21:26:12 -04:00
rocky
d6f7ef4e17 DRY op_size
Move from scanner{2,3}.py to scanner.py
2016-10-21 07:40:35 -04:00
rocky
0826129112 DRY Python 2.x unmangle_classname
main.py: small typo: Disassembled -> Decompiled
2016-10-20 06:35:01 -04:00
rocky
5703ccd8b8 2.6 try statement (and below)
They may neeed arbitrary come_froms for each except clause
2016-10-13 19:37:38 -04:00
rocky
8b240a80e7 Get ready for release 2.9.0 ...
- Use xdis 3.0.0 protocol load_module. Needs bump in requirements.txt
  and _pkg_info_.py
- Start Python 1.5 decompiling - another round of work is needed to remove
  bugs
- small cleanups
2016-10-10 09:33:49 -04:00
rocky
4f83a87a00 Python 2 & 3 scanner code ever so slightly closer 2016-09-21 07:06:30 -04:00
rocky
c119965d96 Small changes 2016-09-18 17:23:49 -04:00
rocky
03e8995d18 ret_cond adjustment for < 2.7 and ...
"<= 2.6" -> "< 2.7" since python 2.6's version is 2.6000001
2016-09-09 15:55:15 -04:00
rocky
318311818e Python 2.6- try/except control flow detection 2016-09-05 04:02:37 -04:00
rocky
c7788e4545 disassemble -> ingest where appropriate
As part of tokenization for (de)parsing, we need to do something like a
disassembly, but is is really a little different.

Disassembly, strictly speaking, is done by the xdis module now.
What "ingestion" does is massage the instruction tokens to a form that is
more amenable for parsing.

In sum, ingestion is different than disassembly, although disassembly is
generally the first part of ingestion.
2016-09-04 11:43:02 -04:00
rocky
647248dfc8 A couple more 2.6 (and below) bugs fixed
* Detect "return None" inside if statement
* another case of triple ==, ==, ==
     scanner2.py: detect_structure: descriminate more on parent type
2016-09-04 04:10:08 -04:00
rocky
75c718bc5c Python 2.2..2.6 bug in a == b == c == d
Fix was to remove some come froms. Feels a little hacky though.
2016-09-03 10:05:55 -04:00