Commit Graph

154 Commits

Author SHA1 Message Date
rocky
6e2ca8f53d Add another guard on a test 2018-02-25 08:59:12 -05:00
rocky
e3d0b1b9ce move more toward instruction-based ingest...
rather than bytecode-based. This cleans up code
up a little bit better by using more general instructions
2018-02-25 08:51:50 -05:00
rocky
b2dfe0889a Mark more 3.6+ code that needs to be fixed 2018-01-20 08:09:18 -05:00
rocky
8194595ec9 Handle 3.6+ EXTENDED_ARGs for POP_JUMP_IF... instructions 2018-01-19 16:56:21 -05:00
rocky
f59174575e Fix 3.5+ bug in if's with pass bodies
Fixes #104 in a somewhat hacky way.
2018-01-08 10:21:53 -05:00
rocky
b0d18cae6a Fix bug in 2.5- try/else inside ifelsestmt 2018-01-06 22:19:44 -05:00
rocky
b139e21ca3 Isolation of <= 3.5 bogus COME_FROM workaround
Fixes #144
2017-12-12 12:12:29 -05:00
rocky
41bfa3fc01 Back off 3.5 control flow for now 2017-12-07 15:25:32 -05:00
rocky
b6413b6e6e Skirt around control-flow problems...
Tag some of the weaknesses if we can't address them now
2017-12-07 08:55:45 -05:00
rocky
dded92b85d Narrow unused classdefdeco2 rules 2017-11-28 12:49:37 -05:00
rocky
e9b60ddbf0 Better Python 3 ENDIF detection
If we have

COMPARE_OP exception-match
POP_JUMP_IF...
....
RETURN_VALUE

Then RETURN_VALUE can't be RETURN_END_IF
2017-11-08 23:05:01 -05:00
rocky
0bb793239b Add 3.6+ grammar for except's ending in RETURN...
Not totally out of the maze in 3.6 control flow...
There are still problems with erroneous RETURN_VALUEs becoming RETURN_END_IF,
2017-11-08 10:31:38 -05:00
rocky
4c77170ddf Small fixes and tweaks:
parser.py: handle errors when no tokens have been produced.
scanner3{,0}.py: DRY custom scanner 3.0 rem_or code.
scanner3.py misc other small tweaks
2017-11-07 12:48:03 -05:00
rocky
3e4889bcd7 Small tweaks to sync up better with scanner2.py 2017-11-06 13:30:49 -05:00
rocky
6b6755d599 Fix 3.{3,4} pytest. Remove dup find_jump_targets 2017-11-06 12:27:43 -05:00
rocky
4a904951f4 Move refactored find-jump-targets from 3.6 to 3.x 2017-11-06 11:54:01 -05:00
rocky
124267849c Move refactored ingest from 3.6 to 3.x...
We are getting away from working with bytecode in favor of
working with full-fledged structured instructions

Up next: find_jump_targets()
2017-11-06 09:43:49 -05:00
rocky
9379922c89 Iterate over instruction, not bytecode 2017-11-06 00:46:49 -05:00
rocky
b83d6c64ed Python 3.6 control flow bug...
Much more is needed, but it's a start
2017-10-29 23:52:58 -04:00
rocky
5df09540b5 Python 3.6-inspired instruction size cleanup
Revise and generalize for Python 3.6+ instructions vs < 3.6 instuctions.
Used more of the generalized methods in xdis and remove some (but not
all) of the magic numbers.

This is a lot of changes, but not all of the refactoring needed. Much
crap still remains. Also, there are still bugs in handling 3.6 bytecodes.
2017-10-29 11:46:28 -04:00
rocky
9cfd7d669e Get ready for release 2.13.0 2017-10-10 22:08:10 -04:00
rocky
92f5981661 Misc bugs 2017-10-10 16:08:24 -04:00
rocky
1392b18bd7 Adjust for spark-parser 2.7.0 incompatabilities 2017-10-10 14:14:26 -04:00
rocky
1d7a3c6444 Document hacky customize arg count better. 2017-09-30 18:02:35 -04:00
rocky
5bef5683e4 Match Python 3.4's terms a little names better 2017-09-10 00:48:54 -04:00
rocky
d138a01bf1 xdis's "exception match" is now "exception-match" 2017-07-17 22:42:57 -04:00
rocky
27ab6fe2f5 Use xdis 3.5.0's opcode sets 2017-07-08 20:41:46 -04:00
rocky
af3d46b35c Use xdis' instruction offset calculation fns..
next_offset, op_size, has_argument
2017-06-24 06:43:04 -04:00
rocky
d93b7a9eae Correct EXTENDED_ARG handling on Python 3.6...
where it can appear several times and xdis may handle it as well.
It possibly in other versions bug since EXTENDED_ARG is used so rarely
there because it has such a high value 1<<16, it's hard to test and
determine that.
2017-05-21 04:44:11 -04:00
rocky
2ab7aa2f48 Simplify EXTENDED_ARG on 3.x
We largely remove them and fold them itno the next op.
MAKE_FUNCTION though before 3.6 is an exception as that indicates an
annotated function
2017-05-19 22:06:18 -04:00
rocky
3084ac20e9 Another attempt at getting get_target() correct 2017-05-19 07:52:31 -04:00
rocky
b4efa62fad EXTENDED_ARG handling...
get_target() wasn't taking into account EXTENDED_ARG before opcode.

This is mostly relevant in Python 3.6 where the max size before needing
EXTENDED_ARG has been reduced to 256, but theoretically possible in earlier
versions.
2017-05-19 07:13:20 -04:00
rocky
6991a637a2 Small changes....
* __pkginfo__.py: Need spark parser 1.6.1 for corrected remove_rules() fn
* parser36.py: remove replaced Python3 rules
* scanner3.py: corrected comment. Thanks to moagstar here.
*
2017-05-17 23:31:56 -04:00
rocky
ea732acf49 In conjunction with MAKE_FUNCTION_FLAGS change...
Switched from tuple to string, but forgot to change the code that uses this.
2017-05-13 17:29:20 -04:00
rocky
da884487d5 MAKE_FUNCTION_FLAGS can be a simpler tuple 2017-05-13 11:47:27 -04:00
rocky
73857c831b Typo 2017-05-13 06:12:31 -04:00
rocky
d540146d5a WIP: start 3.6 MAKE_FUNCTION handling 2017-05-11 07:00:46 -04:00
rocky
f8a40c1949 Reduce spurious "continue" statements 2017-05-07 13:15:26 -04:00
rocky
605721c995 Python 3.x control-flow bug...
"pass" statement inside "while True"
2017-05-07 09:10:05 -04:00
rocky
26e8de8532 Fix improper COME_FROM_EXCEPT in Python 3.3+ 2017-05-07 03:19:53 -04:00
rocky
4e9555a7f6 Improve Python 3.2 decompilation ...
by removing a lot of the control-flow labels of 3.3+
2017-05-05 21:34:21 -04:00
rocky
47dbc57f3d Reduce adding RETURN_END_IF in 3.5+
The whole control flow determination has to be redone in a less
haphazard way using real flow-control analysis. Hopefully that's on the
way.

In the meantime we have this hack.
2017-04-14 06:57:25 -04:00
rocky
fdcb90f661 Python 3.0 doesn't have POP_JUMP_IF... 2017-03-12 10:32:05 -04:00
rocky
5856802902 Small cleanup - remove POP_JUMP_TF 2017-03-07 22:07:29 -05:00
rocky
4f2ae2f603 More accurate ranges of try blocks in 3.x 2017-03-05 00:05:52 -05:00
rocky
160ec0d9cc COME_FROM for 3.x POP_EXCEPT, DRY with op_name() ...
Start adding COME_FROMs for POP_EXCEPT in preparation for
getting tryelse blocks correct.

Simpler opname access functions:
  - self.op_name(op) is self.opc.opname[op]
  - self.op_name_from_offset(offset) is self.opc.opname[self.code[offset]]

verify.py: not all offsets are ints
2017-03-01 05:50:31 -05:00
rocky
493e4b14a1 Some Python 3.4 bugss fixed by using 3.5 rules 2017-02-20 08:17:17 -05:00
rocky
9491c67779 More COME_FROM's in Python 3...
Need this to find boundaries of simple if better
2017-02-20 04:17:46 -05:00
rocky
8333e4ae93 Handle BUILD_CONST_KEY_MAP as a varargs
custom rules with BUILD_CONST_KEY_MAP are pinned to the specific number
of args seen.
2017-01-20 20:41:10 -05:00
rocky
9c49b5d54b Handle 3.6 BUILD_CONST_KEYMAP 2017-01-15 11:10:13 -05:00