Compare commits

..

187 Commits

Author SHA1 Message Date
rocky
5566b9ba6c Get ready for release 2.9.11 2017-05-06 07:49:09 -04:00
rocky
e56ab2dcd5 Sync with master 2017-05-06 07:17:04 -04:00
rocky
d6c45979ba Merge branch 'master' into python-2.4 2017-05-06 07:16:39 -04:00
rocky
4a47822904 Fix more Python3.2 parser errors 2017-05-06 05:25:56 -04:00
rocky
4e9555a7f6 Improve Python 3.2 decompilation ...
by removing a lot of the control-flow labels of 3.3+
2017-05-05 21:34:21 -04:00
rocky
d1c0413b79 Try CI testing on Python 3.6 2017-05-05 17:55:01 -04:00
rocky
246495febd Bang more on BUIlD_MAP_UNPACK
there are still bugs. Note:

{**{'x': 1}, **{'y': 2}} and
{{'x': 1}, **{'y': 2}}

generate the same Python 3.5+ bytecode.
2017-05-02 21:55:41 -04:00
rocky
91b86ac156 BUILD_MAP_UNPACK'ing of dictionaries in 3.5 2017-05-02 05:51:48 -04:00
rocky
26cd91046e Remove extra unpack *. Issue #98 2017-05-01 05:26:55 -04:00
R. Bernstein
b42c66e091 Update HISTORY.md 2017-04-29 22:32:16 -04:00
rocky
364827a2f2 Handle BUILD_MAP_UNPACK in a build_list 2017-04-29 21:44:52 -04:00
rocky
819458564c A hacky way to get CALL_FUNCTION_EX_KW to work. 2017-04-27 21:38:30 -04:00
rocky
486f313532 remove debug code 2017-04-26 02:14:28 -04:00
rocky
84fd71b73b Python 3.6 CALL_FUNCTION_EX first attempt 2017-04-25 07:31:01 -04:00
rocky
50687e6317 Reduse scope of LOAD_ASSERT as expr to 3.4+ 2017-04-22 22:10:55 -04:00
rocky
b35546157f LOAD_ASSERT can also be an expr
This may have the undesirable property that assert statements might get
tagged with equivalant low-level Python code that uses "raise
AssertionError", but so be it.

Fixes #103
2017-04-22 20:03:21 -04:00
R. Bernstein
7755dddd94 Update HISTORY.md 2017-04-22 11:18:08 -04:00
R. Bernstein
ce1e841255 Update HISTORY.md 2017-04-22 11:15:45 -04:00
rocky
68f0f79030 History keeps gettting amended 2017-04-22 11:12:52 -04:00
rocky
bf195a234f Document Python 3.x status 2017-04-22 10:57:54 -04:00
rocky
87db833f62 Add await expr
Fixes #111
2017-04-22 10:39:20 -04:00
rocky
8081decf7c Update test 2017-04-22 04:29:18 -04:00
rocky
e5008693a1 3.3+ bug in handling single kwarg after *
Towards fixing issue #110
2017-04-22 04:19:04 -04:00
rocky
810649799c Add async for with pass statement
Fixes #109
2017-04-20 12:39:15 -04:00
rocky
d4be647bce 3.5 ifelsestmtl grammar bug.
Fixes #108
2017-04-19 05:08:48 -04:00
rocky
4a898ff4c1 Expand await stmt handling
Fixes #107
2017-04-18 01:51:23 -04:00
rocky
cb6925beec Add DELETE_DEREF grammar rule
Fixes Issue #106
2017-04-18 01:35:08 -04:00
rocky
2665f292c5 Rename test case to something more appropriate 2017-04-17 05:43:50 -04:00
rocky
33be34c6fb Fix botched test case
Thanks to Zm908 for pointing this out
2017-04-17 05:35:43 -04:00
rocky
3bbc94847d Comment on what's up with last change 2017-04-16 16:07:08 -04:00
rocky
3a8d4e1a12 Python 3.x ifelse in comprehension
Fixes Issue #91
2017-04-16 14:47:11 -04:00
rocky
87e005a7ba Add 2.7 complex test 2017-04-16 10:31:15 -04:00
rocky
5477ca294d Correct bug in 3.5+ build_list with UNPACK 2017-04-15 22:34:56 -04:00
R. Bernstein
31c28d0220 Update HOW-TO-REPORT-A-BUG.md 2017-04-15 18:56:56 -04:00
R. Bernstein
659e28d686 Update HOW-TO-REPORT-A-BUG.md 2017-04-15 18:42:01 -04:00
rocky
8a33a583cd 3.6 generates Wonky EXTENDED_ARG in expression
Fixes Issue #102
2017-04-15 18:31:39 -04:00
rocky
8a776176e2 Add how to report a bug
Add test case for ... if 1 else ...
2017-04-15 10:41:13 -04:00
rocky
03498963d4 Python 3.5+ BUILD_UNMAP_PACK rules
Towards addressing Issue #98
2017-04-14 23:39:56 -04:00
rocky
47dbc57f3d Reduce adding RETURN_END_IF in 3.5+
The whole control flow determination has to be redone in a less
haphazard way using real flow-control analysis. Hopefully that's on the
way.

In the meantime we have this hack.
2017-04-14 06:57:25 -04:00
rocky
a06e9bf32e Merge branch 'master' into python-2.4 2017-04-14 05:45:53 -04:00
rocky
7e8f7ba674 namedtuple25 -> namedtuple24 2017-04-14 05:42:44 -04:00
rocky
39b9810587 Better names for a test 2017-04-14 05:05:02 -04:00
rocky
8cdaac93ab Add if1else. Fixes #101 2017-04-13 21:27:22 -04:00
rocky
a9f7a3c6d0 In 3.x come_from should include COME_FROM_EXCEPT 2017-04-13 20:27:02 -04:00
rocky
495bdd7b64 Towards fixing issue #92 2017-04-13 01:48:17 -04:00
rocky
b4ded92822 Add Python 2.3 rule for "if 1: ..."
Fully fixes #97 for Python 2.3. Python 2.4 was fixed in a previous commit.
2017-04-13 01:14:49 -04:00
rocky
be9194c223 annotate args type need to be expr's not constants 2017-04-12 20:12:41 -04:00
rocky
45bd8e4058 Handle Python 2.4 "if 1...." 2017-04-12 04:50:22 -04:00
rocky
bb24df596d Bang on 3.x annotations 2017-04-11 17:09:10 -04:00
rocky
6acec471e3 Towards fixing annotated decorator functions...
and annotate functions
2017-04-11 05:56:20 -04:00
rocky
41343c27b7 Misc bugs
parse2.py: restore accidently-removed while1stmt rule
scanner27.py: grammar typo
check_ast: add while1else to list of looping constructs
pysource.py: CALL_FUNCTION_VAR_KW_ARGS with positional args rule is different?
2017-04-10 07:57:56 -04:00
rocky
9e34654b38 Add more while1else grammar rules
Towards addressing issue #93
2017-04-10 02:47:46 -04:00
rocky
09eb7f7f78 Merge branch 'master' into python-2.4 2017-04-10 00:48:04 -04:00
rocky
b9703cf6b4 One more FUNCTION_VAR test for 3.3 2017-04-09 06:58:41 -04:00
rocky
792df2a7a7 Another Python 3.5 FUNCTION_VAR bug
Fixes #94
2017-04-09 06:54:32 -04:00
rocky
b4a6c3c319 Merge branch 'master' of github.com:rocky/python-uncompyle6 2017-04-09 05:32:46 -04:00
rocky
4199bc7f61 Fix Python 3.5 CALL_FUNCTION_VAR_KW
Fixes Issue #95
2017-04-09 05:30:45 -04:00
rocky
91e1d2538f Merge branch 'master' of github.com:rocky/python-uncompyle6 2017-04-03 06:53:32 -04:00
rocky
6773a66b99 Tidy use of load_attrs 2017-04-03 06:53:12 -04:00
rocky
ed6cb9af79 Merge branch 'master' of github.com:rocky/python-uncompyle6 2017-03-27 07:10:16 -04:00
rocky
a91cd71667 Note we've run this on Python 3.0-3.1 bytecodes
__pkginfo__.py: use ore recent xdis
2017-03-27 07:08:59 -04:00
rocky
6f82ae3642 Use more-recent xdis 2017-03-19 14:01:59 -04:00
rocky
4e05c741e3 grammar typo and add another test 2017-03-15 03:59:07 -04:00
rocky
fdcb90f661 Python 3.0 doesn't have POP_JUMP_IF... 2017-03-12 10:32:05 -04:00
rocky
f416473562 Note problem in handling pathologically long lists 2017-03-12 10:16:10 -04:00
rocky
5856802902 Small cleanup - remove POP_JUMP_TF 2017-03-07 22:07:29 -05:00
rocky
4f2ae2f603 More accurate ranges of try blocks in 3.x 2017-03-05 00:05:52 -05:00
rocky
ea1651d8ca More accurate ranges of try blocks in 3.x 2017-03-05 00:03:01 -05:00
R. Bernstein
be769da401 Merge pull request #84 from moagstar/property_based_test_function_call
Property based test function call
2017-03-04 14:43:34 -05:00
Daniel Bradburn
cb3c5e7119 validation now uses xdis for python2 support 2017-03-04 20:23:39 +01:00
rocky
39e3582e72 README updates for 3.5 and 1.5 2017-03-04 11:54:02 -05:00
rocky
a0c090932e Bug found by hypothesis in creating function calls 2017-03-04 11:49:09 -05:00
Daniel Bradburn
d1e118afa3 marked all function call tests as failing until they pass across all python versions 2017-03-04 13:04:31 +01:00
Daniel Bradburn
f7da8fd8ab added minimal examples for various function call opcodes 2017-03-04 12:44:11 +01:00
Daniel Bradburn
3b1dd9d1c4 added property based test for verifying uncompylation of function calls. A number of minimal examples for the various function call opcodes have been generated with the majority marked as expected failure until python 3.6 opcode support is complete. I'm hoping this will make it easier to figure out what needs to be done to support the new opcodes and changed semntics for function calls 2017-03-04 12:43:12 +01:00
Daniel Bradburn
91fd1ce732 reduced errors when generating function call instances 2017-03-03 21:38:53 +01:00
Daniel Bradburn
a46e7cbfa4 added test file for function calls 2017-03-03 21:06:31 +01:00
Daniel Bradburn
d46873c44d added .idea to gitignore 2017-03-03 21:03:50 +01:00
Daniel Bradburn
54e50771ab added .venv to gitignore 2017-03-03 21:03:06 +01:00
rocky
f7a910ec66 Merge branch 'master' into python-2.4 2017-03-01 05:55:26 -05:00
rocky
160ec0d9cc COME_FROM for 3.x POP_EXCEPT, DRY with op_name() ...
Start adding COME_FROMs for POP_EXCEPT in preparation for
getting tryelse blocks correct.

Simpler opname access functions:
  - self.op_name(op) is self.opc.opname[op]
  - self.op_name_from_offset(offset) is self.opc.opname[self.code[offset]]

verify.py: not all offsets are ints
2017-03-01 05:50:31 -05:00
rocky
e1111e3f50 Python 2.6 a == b or c == d == 3 grammar bug 2017-02-28 09:18:36 -05:00
rocky
65913778a5 2.6 a == b or x == y == z bug 2017-02-28 03:12:48 -05:00
rocky
cf21fff38b Predidence of cmp_list: x == y == z
The x, y, z should not have parenthesis around pairs of them
(x == y) or (y == z)
2017-02-28 01:25:33 -05:00
rocky
29122340e6 Python 2.7 check jump targets of "and" 2017-02-28 00:15:39 -05:00
rocky
6d6a73eea7 Merge branch 'master' into python-2.4 2017-02-25 21:02:12 -05:00
rocky
1e3ea60055 Get ready for release 2.9.10 2017-02-25 20:35:00 -05:00
rocky
e4a7641927 Python <= 2.6 grammar fixes 2017-02-25 05:13:19 -05:00
rocky
b24b46d48c Merge branch 'master' into python-2.4 2017-02-25 04:48:06 -05:00
rocky
2fbbc728b1 Python 2.6 parsing bugs ..
and some parser list nonterminal cleanup
2017-02-25 04:45:10 -05:00
rocky
0a6c8ba909 Python 2.6 control flow bug with added COME_FROM 2017-02-24 21:29:28 -05:00
rocky
d3904527e6 Python 2.5 wasn't handling tryelse properly 2017-02-22 05:38:30 -05:00
rocky
a65d7dce5b Python 2.5 was missing try else stmt 2017-02-22 05:30:07 -05:00
rocky
718a0a5d34 Merge branch 'master' into python-2.4 2017-02-22 05:29:49 -05:00
rocky
b043f6bafc New test doesn't --verify correctly. Sigh. 2017-02-20 09:22:01 -05:00
rocky
aa207a3c77 Add test for last while1 bug fix 2017-02-20 09:15:39 -05:00
rocky
747212c62c Python 3.x needs more "while 1" grammar rules 2017-02-20 08:57:16 -05:00
rocky
493e4b14a1 Some Python 3.4 bugss fixed by using 3.5 rules 2017-02-20 08:17:17 -05:00
rocky
9491c67779 More COME_FROM's in Python 3...
Need this to find boundaries of simple if better
2017-02-20 04:17:46 -05:00
rocky
8ef5e5d12b Marginally better for Python 2.6 but...
control flow is still wrong.
2017-02-19 08:12:15 -05:00
rocky
222986640e Merge branch 'coverage'
Beef up coverage
2017-02-10 02:09:28 -05:00
rocky
f9d47abb2b Reduce withas and with semantic footprint
This appears in python 2.5+ only. 2.5 is via "from future"
2017-02-10 02:08:52 -05:00
rocky
31ed869a6f Beef up grammar coverage 2017-02-10 02:03:28 -05:00
rocky
ea9e3ab3f5 Group coverage Makefile targets 2017-02-10 01:00:26 -05:00
rocky
19d2569515 Changes based on grammar coverage info 2017-01-29 23:01:12 -05:00
rocky
770e988ff8 Changes based on coverage information 2017-01-29 22:54:30 -05:00
rocky
0fa0641974 Merge branch 'master' into python-2.4 2017-01-29 22:05:55 -05:00
R. Bernstein
9348411056 Merge pull request #83 from rocky/coverage
Coverage
2017-01-29 21:54:45 -05:00
rocky
e71dd010d7 Simplfy getting coverage
consts.py: notes on versions use which ops
2017-01-29 21:39:29 -05:00
rocky
dadd1c5c45 Add --coverage to test_pyenvlib and ...
improve grammar coverage on 2.7
2017-01-29 18:06:07 -05:00
rocky
99af1c9ffe Merge branch 'master' into coverage 2017-01-29 07:35:02 -05:00
rocky
3dc766d0a9 Update date 2017-01-29 07:34:49 -05:00
rocky
357005c814 Add --coverage option. WOOT! 2017-01-29 07:33:41 -05:00
rocky
41d63a0261 Bump min spark_parser version 2017-01-27 16:41:31 -05:00
rocky
1cb2cd7a82 More 2.6, 2.7 control flow
Todo more COME_FROMs but now need to check targets better. In some cases
we're relying on grammar ambiguity to work out right and in 2.7 it doesn't
2017-01-24 01:21:28 -05:00
rocky
9ec312ba5e More 2.6, 2.7 control-flow bugs
Wasn't limiting exception clause to try finally. Probably still has bugs
in try-finally nesting

Add another 2.6/2.7 COME_FROM to try to limit if/end scope better
2017-01-24 00:53:30 -05:00
rocky
597d51951e Improve Python 2.6 & 2.7 verification 2017-01-23 02:32:09 -05:00
rocky
cc2321f49e Fix up Python 3.0 handling 2017-01-22 03:45:40 -05:00
rocky
476a1c8ab5 Merge branch 'master' of github.com:rocky/python-uncompyle6 2017-01-21 06:25:54 -05:00
rocky
545a46dffa Correct spelling of Earley 2017-01-21 06:24:31 -05:00
rocky
8333e4ae93 Handle BUILD_CONST_KEY_MAP as a varargs
custom rules with BUILD_CONST_KEY_MAP are pinned to the specific number
of args seen.
2017-01-20 20:41:10 -05:00
R. Bernstein
e9057f378a Merge pull request #81 from moagstar/BUILD_CONST_KEY_MAP
fixed bug with BUILD_CONST_KEY_MAP
2017-01-19 20:43:10 -05:00
Daniel Bradburn
36b75abd90 fixed bug with BUILD_CONST_KEY_MAP 2017-01-19 21:58:56 +01:00
R. Bernstein
1528537ca4 Merge pull request #80 from moagstar/BUILD_CONST_KEY_MAP
Build const key map
2017-01-19 01:24:18 -05:00
Daniel Bradburn
6b8ae29267 added dev requirement six 2017-01-18 22:43:33 +01:00
Daniel Bradburn
33ec66a82f added generation of dict display from BUILD_CONST_KEY_MAP 2017-01-18 22:38:09 +01:00
Daniel Bradburn
b0493d1984 fixed typo 2017-01-18 22:34:12 +01:00
Daniel Bradburn
7f37c60c42 added some more test cases for BUILD_CONST_KEY_MAP 2017-01-18 22:33:44 +01:00
Daniel Bradburn
e2fd308928 simplified test cases for test_build_const_key_map 2017-01-17 23:07:27 +01:00
Daniel Bradburn
6d7cec002a added validation code for checking decompilation of an expression 2017-01-17 22:40:31 +01:00
rocky
9c49b5d54b Handle 3.6 BUILD_CONST_KEYMAP 2017-01-15 11:10:13 -05:00
rocky
8dc23e2cdc Python 2.1 doesn't have FOR_ITER or GET_ITER...
adjust locgic for this fact
2017-01-15 09:50:38 -05:00
rocky
a01b8be054 sys.recursionlimit is optional, not essential 2017-01-12 04:48:39 -05:00
rocky
c13e23cdae Get ready for release 2.9.9 2017-01-11 21:52:20 -05:00
rocky
114fe11e66 Merge branch 'master' of github.com:rocky/python-uncompyle6 2017-01-11 21:44:12 -05:00
rocky
b131c20e99 Get ready for release 2.9.9 2017-01-11 21:42:25 -05:00
rocky
5db1178b3e Get ready for release 2.10.9 2017-01-11 21:38:30 -05:00
rocky
fab4ebb768 Merge changes ...
* str() in Python 2.4 doesn't detect unicode.
* index() doesn't work on tuples
* ifelse change
2017-01-11 19:34:28 -05:00
rocky
89429339fa Merge branch 'master' into python-2.4 2017-01-11 19:25:44 -05:00
R. Bernstein
7ece296f76 Merge pull request #79 from rocky/revert-78-patch-1
Revert "fix bug : not generate all files when use "-ro""
2017-01-11 07:10:23 -05:00
R. Bernstein
5035d5433b Revert "fix bug : not generate all files when use "-ro"" 2017-01-11 07:09:18 -05:00
R. Bernstein
78a5b620a7 Merge pull request #78 from jlugjb/patch-1
fix bug : not generate all files when use "-ro"
2017-01-11 07:06:10 -05:00
jlugjb
e851c0d46a fix bug : not generate all files when use "-ro"
when use the args of "-ro  outdir inputdir", only the first file is generated, other files is covered.
2017-01-11 17:02:36 +08:00
rocky
a760188724 Improve BUILD_xxx_UNPACK slightly 2017-01-10 04:36:28 -05:00
rocky
ad345ef94a Add async_call_function for 3.5+ 2017-01-09 07:03:51 -05:00
rocky
d050dd3adb Reinstate test 2017-01-09 06:01:06 -05:00
rocky
9392103998 Works now 2017-01-08 22:28:33 -05:00
rocky
707770049f Python 3.0 decompile bugs 2017-01-08 22:19:15 -05:00
rocky
ec0669367f Towards better 3.0 decompilation
Sync scanner2 and scanner3 better
2017-01-08 17:40:57 -05:00
rocky
3f40c16587 Fix 3.5, 3.6 while true if/break bug 2017-01-08 15:54:49 -05:00
rocky
66518baed0 Misc cleanups
Favor "decompile" over "uncompyle" since "decompile" is in common use
Reduce size of pysource.py by splitting out constants
2017-01-08 09:26:19 -05:00
rocky
21023fea74 Add 3.5+ async with/for ..
scanner3.py: 3.6 bytecode vs wordcode fix
2017-01-08 08:54:03 -05:00
rocky
66741d16ba Start to add 3.5+ await and async 2017-01-07 21:36:37 -05:00
rocky
e02ebef45d More Python 3 annotation bugs 2017-01-07 10:27:42 -05:00
rocky
99fce6dfd7 Fix some errors in deparsing Python 3 annotations 2017-01-07 03:03:53 -05:00
rocky
7b8c5e091c Small Pyhton 3.x annotate bug 2017-01-07 00:21:59 -05:00
rocky
77caf515ea Note what's up with Python 3 decompile quality 2017-01-03 07:38:01 -05:00
rocky
e4c0d56947 3.5 continue check is needed on 3.6 2017-01-03 07:22:25 -05:00
rocky
4827b1e994 Towards better 3.6 support 2017-01-03 00:44:07 -05:00
rocky
2b46e71264 Python 3.5 continue detection bug 2017-01-02 10:06:52 -05:00
rocky
6ed129bd7a 2.4 verify hacks 2017-01-02 07:15:46 -05:00
rocky
c4fde6b53e Merge branch 'master' into python-2.4 2017-01-02 05:39:50 -05:00
rocky
a7d93e88b4 Merge branch 'master' into python-2.4 2017-01-02 05:39:13 -05:00
rocky
84c2932bc5 add come_from for setup_finally and setup_except 2017-01-01 21:11:35 -05:00
rocky
874b3c9d31 Towards fixing Python 3.5 return bugs 2017-01-01 04:56:15 -05:00
rocky
f6a997befc Note how to verify correctness ...
with --verify, --weak-verify and cross checking with pycdc
2017-01-01 02:13:13 -05:00
rocky
9891494142 We are version 2.9.9 2016-12-31 18:16:23 -05:00
rocky
f8544dfbbe 2.7->2.4 conversion 2016-12-31 10:56:43 -05:00
rocky
b00651d428 Merge master branche
Handle 2.2 list_if
2016-12-31 05:19:21 -05:00
rocky
da8dccbaca Merge branch 'master' into python-2.4 2016-12-29 02:08:12 -05:00
rocky
37272ae827 Merge commit '9b1dd0f' into python-2.4 2016-12-27 10:32:25 -05:00
rocky
7f2bee46b7 Bug in using python2 ast checking in python 2.5 2016-12-26 01:55:16 -05:00
rocky
c8a4dcf72b Removing NAME_MODULE, lint and bug fixes
scanner*.py: show_asm param is optional
verify.py: call correct scanners
main.py, verify.py: Use older Python print statements
2016-12-25 09:16:04 -05:00
rocky
012ff91cfb Merge branch 'master' into python-2.4 2016-12-25 07:57:17 -05:00
rocky
e690ddd50a Merge branch 'master' into python-2.4 2016-12-18 07:43:15 -05:00
rocky
45b7c1948c show-asm on python2.5 is optional
Make scanner2 a little more like scanner3.
2016-12-17 07:57:31 -05:00
rocky
e2fb7ca3d2 Python 2.6/2.7 tolerance in Python 2.4 branch 2016-12-17 06:51:47 -05:00
rocky
b3bda76582 Merge branch 'master' into python-2.4 2016-12-16 22:56:07 -05:00
rocky
ab6d322eca Get ready for release 2.9.7 2016-12-04 14:09:53 -05:00
rocky
1a8a0df107 Merge branch 'master' into python-2.4 2016-12-04 13:40:06 -05:00
rocky
0a37709b0a CircleCI build 2016-11-24 05:41:31 -05:00
rocky
98cd1417df Remove dup Python 3 grammar rule 2016-11-24 05:36:43 -05:00
rocky
460069ceaa Bug in 2.4 "if" dectection and...
Wrong language used in old-style exceptions: use "except Error,e" not
"except Error(e)""
2016-11-24 05:15:35 -05:00
rocky
316aa44f23 Python 2.6 grammary bug and..
__pkginfo.py__: Bump spark_parser version for parse_flags 'dups'
2016-11-24 04:09:32 -05:00
rocky
7133540c23 Make work on 2.4 2016-11-23 08:26:12 -05:00
rocky
590231741d Merge branch 'come-from-type' into python-2.4 2016-11-23 07:54:18 -05:00
rocky
a9349b8f3d Making it run on Python 2.4 and 2.5 2016-11-23 07:53:51 -05:00
144 changed files with 3484 additions and 1026 deletions

2
.gitignore vendored
View File

@@ -16,3 +16,5 @@
/unpyc
__pycache__
build
/.venv*
/.idea

View File

@@ -3,12 +3,7 @@ language: python
sudo: false
python:
- '3.5'
- '2.7.12'
- '2.6'
- '3.3'
- '3.4'
- '3.2'
- '2.7' # this is a cheat here because travis doesn't do 2.4-2.6
install:
- pip install -r requirements.txt

877
ChangeLog
View File

@@ -1,11 +1,782 @@
2017-05-06 rocky <rb@dustyfeet.com>
* uncompyle6/version.py: Get ready for release 2.9.11
2017-05-06 rocky <rb@dustyfeet.com>
* test/Makefile, uncompyle6/scanners/scanner2.py,
uncompyle6/scanners/scanner3.py, uncompyle6/semantics/fragments.py:
Sync with master
2017-05-06 rocky <rb@dustyfeet.com>
* : commit 4a4782290490187ac2fcaaecd3ca808f933722b2 Author: rocky
<rb@dustyfeet.com> Date: Sat May 6 05:25:56 2017 -0400
2017-05-05 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse32.py, uncompyle6/scanners/scanner3.py:
Improve Python 3.2 decompilation ... by removing a lot of the control-flow labels of 3.3+
2017-05-05 rocky <rb@dustyfeet.com>
* .travis.yml: Try CI testing on Python 3.6
2017-05-02 rocky <rb@dustyfeet.com>
* test/simple_source/bug35/01_map_unpack.py, uncompyle6/parser.py,
uncompyle6/parsers/parse3.py, uncompyle6/parsers/parse35.py,
uncompyle6/semantics/pysource.py: Bang more on BUIlD_MAP_UNPACK there are still bugs. Note: {**{'x': 1}, **{'y': 2}} and {{'x': 1}, **{'y': 2}} generate the same Python 3.5+ bytecode.
2017-05-02 rocky <rb@dustyfeet.com>
* test/simple_source/bug35/01_map_unpack.py, uncompyle6/parser.py,
uncompyle6/parsers/parse3.py, uncompyle6/parsers/parse35.py:
BUILD_MAP_UNPACK'ing of dictionaries in 3.5
2017-05-01 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/pysource.py: Remove extra unpack *. Issue #98
2017-04-29 R. Bernstein <rocky@users.noreply.github.com>
* HISTORY.md: Update HISTORY.md
2017-04-29 rocky <rb@dustyfeet.com>
* test/simple_source/bug35/01_map_unpack.py,
uncompyle6/parsers/parse35.py, uncompyle6/semantics/pysource.py:
Handle BUILD_MAP_UNPACK in a build_list
2017-04-27 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/pysource.py: A hacky way to get
CALL_FUNCTION_EX_KW to work.
2017-04-26 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/pysource.py: remove debug code
2017-04-25 rocky <rb@dustyfeet.com>
* test/simple_source/bug36/01_call_function.py,
uncompyle6/parsers/parse36.py, uncompyle6/scanners/scanner36.py,
uncompyle6/semantics/pysource.py: Python 3.6 CALL_FUNCTION_EX first
attempt
2017-04-22 rocky <rb@dustyfeet.com>
* uncompyle6/parser.py, uncompyle6/parsers/parse34.py: Reduse scope
of LOAD_ASSERT as expr to 3.4+
2017-04-22 rocky <rb@dustyfeet.com>
* uncompyle6/parser.py, uncompyle6/verify.py: LOAD_ASSERT can also
be an expr This may have the undesirable property that assert statements might
get tagged with equivalant low-level Python code that uses "raise
AssertionError", but so be it. Fixes #103
2017-04-22 R. Bernstein <rocky@users.noreply.github.com>
* HISTORY.md: Update HISTORY.md
2017-04-22 R. Bernstein <rocky@users.noreply.github.com>
* HISTORY.md: Update HISTORY.md
2017-04-22 rocky <rb@dustyfeet.com>
* HISTORY.md: History keeps gettting amended
2017-04-22 rocky <rb@dustyfeet.com>
* README.rst: Document Python 3.x status
2017-04-22 rocky <rb@dustyfeet.com>
* test/simple_source/bug35/03_async_await.py,
uncompyle6/parsers/parse35.py, uncompyle6/semantics/pysource.py: Add
await expr Fixes #111
2017-04-22 rocky <rb@dustyfeet.com>
* : Update test
2017-04-22 rocky <rb@dustyfeet.com>
* test/simple_source/bug33/02_pos_args.py,
uncompyle6/parsers/parse3.py, uncompyle6/semantics/make_function.py:
3.3+ bug in handling single kwarg after * Towards fixing issue #110
2017-04-20 rocky <rb@dustyfeet.com>
* test/simple_source/bug35/02_async_for.py,
uncompyle6/parsers/parse35.py: Add async for with pass statement Fixes #109
2017-04-19 rocky <rb@dustyfeet.com>
* test/simple_source/bug35/03_while-if-break.py,
uncompyle6/parsers/parse3.py: 3.5 ifelsestmtl grammar bug. Fixes #108
2017-04-18 rocky <rb@dustyfeet.com>
* test/simple_source/bug35/03_async_await.py,
uncompyle6/parsers/parse35.py: Expand await stmt handling Fixes #107
2017-04-18 rocky <rb@dustyfeet.com>
* test/simple_source/bug33/01_delete_deref.py,
uncompyle6/parsers/parse32.py, uncompyle6/semantics/pysource.py: Add
DELETE_DEREF grammar rule Fixes Issue #106
2017-04-17 rocky <rb@dustyfeet.com>
* test/simple_source/bug36/01_extended_arg.py,
test/simple_source/bug36/01_if_file.py: Rename test case to
something more appropriate
2017-04-17 rocky <rb@dustyfeet.com>
* test/simple_source/bug36/01_if_file.py: Fix botched test case Thanks to Zm908 for pointing this out
2017-04-16 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse3.py: Comment on what's up with last
change
2017-04-16 rocky <rb@dustyfeet.com>
* test/simple_source/bug22/03_if1.py,
test/simple_source/bug31/02_ifelse_comprehension.py,
uncompyle6/parsers/parse3.py: Python 3.x ifelse in comprehension Fixes Issue #91
2017-04-16 rocky <rb@dustyfeet.com>
* : Add 2.7 complex test
2017-04-15 rocky <rb@dustyfeet.com>
* test/simple_source/bug35/01_map_unpack.py,
uncompyle6/semantics/pysource.py: Correct bug in 3.5+ build_list
with UNPACK
2017-04-15 R. Bernstein <rocky@users.noreply.github.com>
* HOW-TO-REPORT-A-BUG.md: Update HOW-TO-REPORT-A-BUG.md
2017-04-15 R. Bernstein <rocky@users.noreply.github.com>
* HOW-TO-REPORT-A-BUG.md: Update HOW-TO-REPORT-A-BUG.md
2017-04-15 rocky <rb@dustyfeet.com>
* test/simple_source/bug36/01_if_file.py,
uncompyle6/parsers/parse36.py: 3.6 generates Wonky EXTENDED_ARG in
expression Fixes Issue #102
2017-04-15 rocky <rb@dustyfeet.com>
* HOW-TO-REPORT-A-BUG.md, MANIFEST.in: Add how to report a bug Add test case for ... if 1 else ...
2017-04-14 rocky <rb@dustyfeet.com>
* test/simple_source/bug35/01_map_unpack.py,
uncompyle6/parsers/parse3.py, uncompyle6/parsers/parse35.py,
uncompyle6/semantics/pysource.py: Python 3.5+ BUILD_UNMAP_PACK rules Towards addressing Issue #98
2017-04-14 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner3.py: Reduce adding RETURN_END_IF in
3.5+ The whole control flow determination has to be redone in a less
haphazard way using real flow-control analysis. Hopefully that's on
the way. In the meantime we have this hack.
2017-04-14 rocky <rb@dustyfeet.com>
* : commit 7e8f7ba67431725fceec08344934c929a517efc5 Author: rocky
<rb@dustyfeet.com> Date: Fri Apr 14 05:42:44 2017 -0400
2017-04-14 rocky <rb@dustyfeet.com>
* test/simple_source/bug27+/03_if_1_else.py,
test/simple_source/bug27+/03_if_true_else.py: Better names for a
test
2017-04-13 rocky <rb@dustyfeet.com>
* test/simple_source/bug27+/03_if_true_else.py,
uncompyle6/parser.py, uncompyle6/parsers/parse3.py,
uncompyle6/semantics/consts.py: Add if1else. Fixes #101
2017-04-13 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse3.py: In 3.x come_from should include
COME_FROM_EXCEPT
2017-04-13 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse35.py: Towards fixing issue #92
2017-04-13 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse23.py, uncompyle6/semantics/pysource.py:
Add Python 2.3 rule for "if 1: ..." Fully fixes #97 for Python 2.3. Python 2.4 was fixed in a previous
commit.
2017-04-12 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse3.py,
uncompyle6/semantics/make_function.py: annotate args type need to be
expr's not constants
2017-04-12 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse24.py: Handle Python 2.4 "if 1...."
2017-04-11 rocky <rb@dustyfeet.com>
* test/simple_source/bug31/04_def_annotate.py,
uncompyle6/semantics/fragments.py,
uncompyle6/semantics/make_function.py: Bang on 3.x annotations
2017-04-11 rocky <rb@dustyfeet.com>
* test/simple_source/bug31/04_def_annotate.py,
uncompyle6/parsers/parse3.py, uncompyle6/semantics/pysource.py:
Towards fixing annotated decorator functions... and annotate functions
2017-04-10 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse2.py, uncompyle6/scanners/scanner27.py,
uncompyle6/semantics/check_ast.py, uncompyle6/semantics/pysource.py:
Misc bugs parse2.py: restore accidently-removed while1stmt rule scanner27.py:
grammar typo check_ast: add while1else to list of looping constructs
pysource.py: CALL_FUNCTION_VAR_KW_ARGS with positional args rule is
different?
2017-04-10 rocky <rb@dustyfeet.com>
* test/simple_source/stmts/02_while1else.py,
uncompyle6/parsers/parse2.py, uncompyle6/parsers/parse3.py,
uncompyle6/parsers/parse35.py: Add more while1else grammar rules Towards addressing issue #93
2017-04-10 rocky <rb@dustyfeet.com>
* : commit b9703cf6b41138b717c282fc791c08d807692b07 Author: rocky
<rb@dustyfeet.com> Date: Sun Apr 9 06:58:41 2017 -0400
2017-04-09 rocky <rb@dustyfeet.com>
* test/simple_source/def/10_kw+pos_args-bug.py,
uncompyle6/parsers/parse3.py, uncompyle6/semantics/pysource.py:
Another Python 3.5 FUNCTION_VAR bug Fixes #94
2017-04-09 rocky <rb@dustyfeet.com>
* : commit 4199bc7f617e387fb03fc06939cd17366dc15c5e Author: rocky
<rb@dustyfeet.com> Date: Sun Apr 9 05:30:45 2017 -0400
2017-04-03 rocky <rb@dustyfeet.com>
* : commit 6773a66b99d07e48290a77dbbbe3c71cc39c31ba Author: rocky
<rb@dustyfeet.com> Date: Mon Apr 3 06:53:12 2017 -0400
2017-03-27 rocky <rb@dustyfeet.com>
* : commit a91cd716670be09d3cef34e1bb36a67f96f91712 Author: rocky
<rb@dustyfeet.com> Date: Mon Mar 27 07:08:59 2017 -0400
2017-03-19 rocky <rb@dustyfeet.com>
* __pkginfo__.py: Use more-recent xdis
2017-03-15 rocky <rb@dustyfeet.com>
* HISTORY.md, test/simple_source/bug33/01_if_try_except.py: grammar
typo and add another test
2017-03-12 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner3.py: Python 3.0 doesn't have
POP_JUMP_IF...
2017-03-12 rocky <rb@dustyfeet.com>
* README.rst: Note problem in handling pathologically long lists
2017-03-07 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner3.py: Small cleanup - remove
POP_JUMP_TF
2017-03-05 rocky <rb@dustyfeet.com>
* pytest/test_grammar.py, uncompyle6/parsers/parse3.py,
uncompyle6/parsers/parse33.py, uncompyle6/scanners/scanner3.py: More
accurate ranges of try blocks in 3.x
2017-03-05 rocky <rb@dustyfeet.com>
* test/simple_source/bug33/01_try_except.py: More accurate ranges of
try blocks in 3.x
2017-03-04 R. Bernstein <rocky@users.noreply.github.com>
* : Merge pull request #84 from
moagstar/property_based_test_function_call Property based test function call
2017-03-04 rocky <rb@dustyfeet.com>
* README.rst: README updates for 3.5 and 1.5
2017-03-04 rocky <rb@dustyfeet.com>
* test/simple_source/bug32/01_named_and_kwargs.py,
uncompyle6/parsers/parse3.py: Bug found by hypothesis in creating
function calls
2017-03-04 Daniel Bradburn <moagstar@gmail.com>
* pytest/test_function_call.py: marked all function call tests as
failing until they pass across all python versions
2017-03-04 Daniel Bradburn <moagstar@gmail.com>
* pytest/test_function_call.py: added minimal examples for various
function call opcodes
2017-03-04 Daniel Bradburn <moagstar@gmail.com>
* pytest/test_function_call.py: added property based test for
verifying uncompylation of function calls. A number of minimal
examples for the various function call opcodes have been generated
with the majority marked as expected failure until python 3.6 opcode
support is complete. I'm hoping this will make it easier to figure
out what needs to be done to support the new opcodes and changed
semntics for function calls
2017-03-03 Daniel Bradburn <moagstar@gmail.com>
* pytest/test_function_call.py: reduced errors when generating
function call instances
2017-03-03 Daniel Bradburn <moagstar@gmail.com>
* pytest/test_function_call.py: added test file for function calls
2017-03-03 Daniel Bradburn <moagstar@gmail.com>
* .gitignore: added .idea to gitignore
2017-03-03 Daniel Bradburn <moagstar@gmail.com>
* .gitignore: added .venv to gitignore
2017-03-01 rocky <rb@dustyfeet.com>
* : commit 160ec0d9cc5fe347f6e8bdb69515a28c76cfb368 Author: rocky
<rb@dustyfeet.com> Date: Wed Mar 1 05:50:31 2017 -0500
2017-02-28 rocky <rb@dustyfeet.com>
* README.rst, uncompyle6/parser.py, uncompyle6/parsers/parse26.py:
Python 2.6 a == b or c == d == 3 grammar bug
2017-02-28 rocky <rb@dustyfeet.com>
* : 2.6 a == b or x == y == z bug
2017-02-28 rocky <rb@dustyfeet.com>
* test/simple_source/bug26/03_double_equals.py,
uncompyle6/semantics/consts.py: Predidence of cmp_list: x == y == z The x, y, z should not have parenthesis around pairs of them (x ==
y) or (y == z)
2017-02-28 rocky <rb@dustyfeet.com>
* uncompyle6/parser.py, uncompyle6/parsers/parse27.py: Python 2.7
check jump targets of "and"
2017-02-25 rocky <rb@dustyfeet.com>
* : commit 1e3ea60055027dfd3f098661ac4f5979c5c48f7e Author: rocky
<rb@dustyfeet.com> Date: Sat Feb 25 20:18:03 2017 -0500
2017-02-25 rocky <rb@dustyfeet.com>
* uncompyle6/parser.py: Python <= 2.6 grammar fixes
2017-02-25 rocky <rb@dustyfeet.com>
* : commit 2fbbc728b10f0d3a754165708584bd80d33bc7f9 Author: rocky
<rb@dustyfeet.com> Date: Sat Feb 25 04:45:10 2017 -0500
2017-02-24 rocky <rb@dustyfeet.com>
* test/simple_source/bug25/03_if_for.py,
uncompyle6/parsers/parse26.py: Python 2.6 control flow bug with
added COME_FROM
2017-02-22 rocky <rb@dustyfeet.com>
* test/simple_source/bug25/02_try_else.py,
uncompyle6/parsers/parse25.py: Python 2.5 wasn't handling tryelse
properly
2017-02-22 rocky <rb@dustyfeet.com>
* test/Makefile, test/simple_source/bug25/02_try_else.py,
uncompyle6/parsers/parse25.py: Python 2.5 was missing try else stmt
2017-02-22 rocky <rb@dustyfeet.com>
* : commit b043f6bafc9b9ae26e64dc0f1441d7abae894c37 Author: rocky
<rb@dustyfeet.com> Date: Mon Feb 20 09:22:01 2017 -0500
2017-02-20 rocky <rb@dustyfeet.com>
* test/simple_source/bug33/02_while1.py: Add test for last while1
bug fix
2017-02-20 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse3.py, uncompyle6/parsers/parse35.py:
Python 3.x needs more "while 1" grammar rules
2017-02-20 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse3.py, uncompyle6/parsers/parse35.py,
uncompyle6/scanners/scanner3.py: Some Python 3.4 bugss fixed by
using 3.5 rules
2017-02-20 rocky <rb@dustyfeet.com>
* test/simple_source/exception/02_try_finally.py,
uncompyle6/parsers/parse3.py, uncompyle6/scanners/scanner3.py: More
COME_FROM's in Python 3... Need this to find boundaries of simple if better
2017-02-19 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse26.py: Marginally better for Python 2.6
but... control flow is still wrong.
2017-02-10 rocky <rb@dustyfeet.com>
* : commit f9d47abb2be7c3839df06c0ed69d3d513694af4e Author: rocky
<rb@dustyfeet.com> Date: Fri Feb 10 02:07:04 2017 -0500
2017-02-10 rocky <rb@dustyfeet.com>
* test/simple_source/bug22/01_ops.py, test/test_pythonlib.py: Beef
up grammar coverage
2017-02-10 rocky <rb@dustyfeet.com>
* test/Makefile: Group coverage Makefile targets
2017-01-29 rocky <rb@dustyfeet.com>
* test/Makefile, test/simple_source/bug22/01_ops.py,
uncompyle6/parsers/parse25.py, uncompyle6/semantics/consts.py,
uncompyle6/semantics/pysource.py: Changes based on grammar coverage
info
2017-01-29 rocky <rb@dustyfeet.com>
* test/Makefile, test/simple_source/bug22/01_ops.py,
uncompyle6/parsers/parse25.py, uncompyle6/semantics/consts.py,
uncompyle6/semantics/pysource.py: Changes based on coverage
information
2017-01-29 rocky <rb@dustyfeet.com>
* : commit 9348411056cbe809e07c4ef341effa17bca90e2f Merge: 3dc766d
e71dd01 Author: R. Bernstein <rocky@users.noreply.github.com> Date:
Sun Jan 29 21:54:45 2017 -0500
2017-01-29 rocky <rb@dustyfeet.com>
* test/Makefile, test/simple_source/bug22/01_ops.py,
test/test_pyenvlib.py, test/test_pythonlib.py,
uncompyle6/semantics/consts.py: Simplfy getting coverage consts.py: notes on versions use which ops
2017-01-29 rocky <rb@dustyfeet.com>
* test/simple_source/bug22/01_ops.py, test/test_pyenvlib.py: Add
--coverage to test_pyenvlib and ... improve grammar coverage on 2.7
2017-01-29 rocky <rb@dustyfeet.com>
* : commit 3dc766d0a9537842470c7b4f79e8ccb3d5a46843 Author: rocky
<rb@dustyfeet.com> Date: Sun Jan 29 07:34:49 2017 -0500
2017-01-29 rocky <rb@dustyfeet.com>
* test/test_pythonlib.py: Add --coverage option. WOOT!
2017-01-27 rocky <rb@dustyfeet.com>
* __pkginfo__.py: Bump min spark_parser version
2017-01-24 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner2.py, uncompyle6/semantics/consts.py,
uncompyle6/semantics/pysource.py: More 2.6, 2.7 control flow Todo more COME_FROMs but now need to check targets better. In some
cases we're relying on grammar ambiguity to work out right and in
2.7 it doesn't
2017-01-24 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse2.py, uncompyle6/parsers/parse27.py,
uncompyle6/scanners/scanner2.py, uncompyle6/semantics/consts.py,
uncompyle6/semantics/pysource.py: More 2.6, 2.7 control-flow bugs Wasn't limiting exception clause to try finally. Probably still has
bugs in try-finally nesting Add another 2.6/2.7 COME_FROM to try to limit if/end scope better
2017-01-23 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner2.py,
uncompyle6/scanners/scanner26.py, uncompyle6/verify.py: Improve
Python 2.6 & 2.7 verification
2017-01-22 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse30.py, uncompyle6/verify.py: Fix up Python
3.0 handling
2017-01-21 rocky <rb@dustyfeet.com>
* : commit 545a46dffaa0fe2246dd7cc1b560c58db525c2b4 Author: rocky
<rb@dustyfeet.com> Date: Sat Jan 21 06:24:31 2017 -0500
2017-01-20 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse3.py, uncompyle6/scanners/scanner3.py,
uncompyle6/semantics/pysource.py: Handle BUILD_CONST_KEY_MAP as a
varargs custom rules with BUILD_CONST_KEY_MAP are pinned to the specific
number of args seen.
2017-01-19 R. Bernstein <rocky@users.noreply.github.com>
* : Merge pull request #81 from moagstar/BUILD_CONST_KEY_MAP fixed bug with BUILD_CONST_KEY_MAP
2017-01-19 R. Bernstein <rocky@users.noreply.github.com>
* : Merge pull request #80 from moagstar/BUILD_CONST_KEY_MAP Build const key map
2017-01-18 Daniel Bradburn <moagstar@gmail.com>
* uncompyle6/semantics/pysource.py: added generation of dict display
from BUILD_CONST_KEY_MAP
2017-01-18 Daniel Bradburn <moagstar@gmail.com>
* pytest/test_build_const_key_map.py: fixed typo
2017-01-18 Daniel Bradburn <moagstar@gmail.com>
* pytest/test_build_const_key_map.py: added some more test cases for
BUILD_CONST_KEY_MAP
2017-01-17 Daniel Bradburn <moagstar@gmail.com>
* pytest/test_build_const_key_map.py: simplified test cases for
test_build_const_key_map
2017-01-17 Daniel Bradburn <moagstar@gmail.com>
* pytest/test_build_const_key_map.py, pytest/validate.py: added
validation code for checking decompilation of an expression
2017-01-15 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse3.py, uncompyle6/scanners/scanner3.py:
Handle 3.6 BUILD_CONST_KEYMAP
2017-01-15 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner2.py: Python 2.1 doesn't have FOR_ITER
or GET_ITER... adjust locgic for this fact
2017-01-12 rocky <rb@dustyfeet.com>
* uncompyle6/__init__.py: sys.recursionlimit is optional, not
essential
2017-01-11 rocky <rb@dustyfeet.com>
* NEWS, uncompyle6/version.py: Get ready for release 2.9.9
2017-01-11 rocky <rb@dustyfeet.com>
* : commit b131c20e99514d3a969a51e841d3a823017f1beb Author: rocky
<rb@dustyfeet.com> Date: Wed Jan 11 21:32:26 2017 -0500
2017-01-11 rocky <rb@dustyfeet.com>
* ChangeLog, NEWS: Get ready for release 2.10.9
2017-01-11 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse3.py, uncompyle6/scanner.py,
uncompyle6/semantics/make_function.py, uncompyle6/version.py: Merge
changes ... * str() in Python 2.4 doesn't detect unicode. * index() doesn't work on tuples * ifelse change
2017-01-11 rocky <rb@dustyfeet.com>
* : commit 7ece296f7638e71fad1117b940f7ffddbe095b1f Merge: 78a5b62
5035d54 Author: R. Bernstein <rocky@users.noreply.github.com> Date:
Wed Jan 11 07:10:23 2017 -0500
2017-01-11 R. Bernstein <rocky@users.noreply.github.com>
* uncompyle6/main.py: Revert "fix bug : not generate all files when
use "-ro""
2017-01-11 R. Bernstein <rocky@users.noreply.github.com>
* : Merge pull request #78 from jlugjb/patch-1 fix bug : not generate all files when use "-ro"
2017-01-10 rocky <rb@dustyfeet.com>
* test/simple_source/bug35/03_double_star_unpack.py,
uncompyle6/parsers/parse3.py, uncompyle6/semantics/pysource.py:
Improve BUILD_xxx_UNPACK slightly
2017-01-09 rocky <rb@dustyfeet.com>
* test/simple_source/bug35/03_async_await.py,
uncompyle6/parsers/parse3.py, uncompyle6/semantics/pysource.py: Add
async_call_function for 3.5+
2017-01-09 rocky <rb@dustyfeet.com>
* : Reinstate test
2017-01-08 rocky <rb@dustyfeet.com>
* : Works now
2017-01-08 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse30.py, uncompyle6/scanners/scanner3.py:
Python 3.0 decompile bugs
2017-01-08 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner2.py, uncompyle6/scanners/scanner3.py,
uncompyle6/scanners/scanner30.py: Towards better 3.0 decompilation Sync scanner2 and scanner3 better
2017-01-08 rocky <rb@dustyfeet.com>
* test/simple_source/bug35/03_while-if-break.py,
uncompyle6/parsers/parse35.py, uncompyle6/scanner.py,
uncompyle6/scanners/scanner3.py: Fix 3.5, 3.6 while true if/break
bug
2017-01-08 rocky <rb@dustyfeet.com>
* uncompyle6/__init__.py, uncompyle6/main.py,
uncompyle6/semantics/consts.py, uncompyle6/semantics/fragments.py,
uncompyle6/semantics/pysource.py: Misc cleanups Favor "decompile" over "uncompyle" since "decompile" is in common
use Reduce size of pysource.py by splitting out constants
2017-01-08 rocky <rb@dustyfeet.com>
* test/simple_source/bug35/03_async_await.py,
uncompyle6/parsers/parse35.py, uncompyle6/scanners/scanner3.py,
uncompyle6/semantics/pysource.py: Add 3.5+ async with/for .. scanner3.py: 3.6 bytecode vs wordcode fix
2017-01-07 rocky <rb@dustyfeet.com>
* test/simple_source/bug35/03_async_await.py,
uncompyle6/parsers/parse35.py, uncompyle6/semantics/pysource.py:
Start to add 3.5+ await and async
2017-01-07 rocky <rb@dustyfeet.com>
* test/simple_source/bug31/04_def_annotate.py,
uncompyle6/semantics/make_function.py: More Python 3 annotation bugs
2017-01-07 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse3.py, uncompyle6/parsers/parse31.py,
uncompyle6/parsers/parse32.py,
uncompyle6/semantics/make_function.py,
uncompyle6/semantics/pysource.py: Fix some errors in deparsing
Python 3 annotations
2017-01-07 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/make_function.py: Small Pyhton 3.x annotate
bug
2017-01-03 rocky <rb@dustyfeet.com>
* README.rst: Note what's up with Python 3 decompile quality
2017-01-03 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner3.py: 3.5 continue check is needed on
3.6
2017-01-03 rocky <rb@dustyfeet.com>
* test/test_pyenvlib.py, uncompyle6/parsers/parse36.py,
uncompyle6/scanners/scanner3.py: Towards better 3.6 support
2017-01-02 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse35.py, uncompyle6/scanners/scanner3.py:
Python 3.5 continue detection bug
2017-01-02 rocky <rb@dustyfeet.com>
* uncompyle6/verify.py: 2.4 verify hacks
2017-01-02 rocky <rb@dustyfeet.com>
* : commit a7d93e88b4e0dfd6876a7a31bd201a0e40f24bea Merge: 9891494
136f42a Author: rocky <rb@dustyfeet.com> Date: Mon Jan 2 05:39:13
2017 -0500
2017-01-01 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner3.py: add come_from for setup_finally
and setup_except
2017-01-01 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse35.py, uncompyle6/scanners/scanner3.py:
Towards fixing Python 3.5 return bugs
2017-01-01 rocky <rb@dustyfeet.com>
* README.rst: Note how to verify correctness ... with --verify, --weak-verify and cross checking with pycdc
2016-12-31 rocky <rb@dustyfeet.com>
* uncompyle6/version.py: Get ready for release 2.9.9
* uncompyle6/version.py: We are version 2.9.9
2016-12-31 rocky <rb@dustyfeet.com>
* uncompyle6/main.py: 2.7->2.4 conversion
2016-12-31 rocky <rb@dustyfeet.com>
* ChangeLog, NEWS, uncompyle6/version.py: Get ready for release
2.9.9
2016-12-31 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse26.py: 2.x list_if may have a THEN in it
2016-12-31 rocky <rb@dustyfeet.com>
* test/Makefile, uncompyle6/main.py, uncompyle6/parsers/parse26.py,
uncompyle6/scanners/scanner2.py, uncompyle6/scanners/scanner26.py,
uncompyle6/scanners/scanner3.py, uncompyle6/semantics/pysource.py,
uncompyle6/verify.py: Merge master branche Handle 2.2 list_if
2016-12-31 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner3.py: Towards fixing a Python 3.3
@@ -34,10 +805,9 @@
* : Merge pull request #73 from rocky/then-crap Add THEN token to improve Python 2.2-2.6 control flow detection
2016-12-28 rocky <rb@dustyfeet.com>
2016-12-29 R. Bernstein <rocky@users.noreply.github.com>
* uncompyle6/parsers/parse3.py, uncompyle6/scanners/tok.py: Misc
bugs
* : Merge pull request #72 from rocky/master THEN psuedo-ops for Python 2.x
2016-12-28 rocky <rb@dustyfeet.com>
@@ -64,8 +834,7 @@
2016-12-27 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner2.py: Make 2.6 and 2.7 ingest more
alike
* : Merge commit '9b1dd0f' into python-2.4
2016-12-26 rocky <rb@dustyfeet.com>
@@ -85,6 +854,11 @@
* uncompyle6/parsers/parse25.py: fix bug in using python2 AST rules
in python 2.5
2016-12-26 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse25.py: Bug in using python2 ast checking
in python 2.5
2016-12-26 rocky <rb@dustyfeet.com>
* : commit f1a947f106b231fb1480ba301b15e3ceaf78c94f Author: rocky
@@ -97,15 +871,19 @@
uncompyle6/verify.py: Scanner call fixes. NAME_MODULE removal for
<=2.4
2016-12-24 rocky <rb@dustyfeet.com>
2016-12-25 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/astnode.py, uncompyle6/parsers/parse2.py,
uncompyle6/parsers/parse26.py, uncompyle6/parsers/parse3.py,
uncompyle6/parsers/parse36.py, uncompyle6/scanners/scanner15.py,
uncompyle6/scanners/scanner2.py, uncompyle6/scanners/scanner21.py,
uncompyle6/scanners/scanner22.py,
uncompyle6/semantics/fragments.py, uncompyle6/semantics/pysource.py:
Lint
* uncompyle6/main.py, uncompyle6/scanners/scanner15.py,
uncompyle6/scanners/scanner21.py, uncompyle6/scanners/scanner22.py,
uncompyle6/scanners/scanner23.py, uncompyle6/scanners/scanner24.py,
uncompyle6/semantics/pysource.py, uncompyle6/verify.py: Removing
NAME_MODULE, lint and bug fixes scanner*.py: show_asm param is optional verify.py: call correct
scanners main.py, verify.py: Use older Python print statements
2016-12-25 rocky <rb@dustyfeet.com>
* : commit e3f4beeb74e33d5b404094765cc63040f62a0b41 Author: rocky
<rb@dustyfeet.com> Date: Sat Dec 24 07:45:02 2016 -0500
2016-12-24 rocky <rb@dustyfeet.com>
@@ -126,22 +904,34 @@
2016-12-18 rocky <rb@dustyfeet.com>
* pytest/.gitignore, test/simple_source/bug25/02_try_else.py,
uncompyle6/parsers/parse25.py, uncompyle6/parsers/parse26.py: Python
2.5 mistaken try/else
* : commit c7c0a989829a9a625333665516387c1177c611c2 Author: rocky
<rb@dustyfeet.com> Date: Sun Dec 18 00:56:07 2016 -0500
2016-12-17 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner2.py, uncompyle6/scanners/scanner25.py:
show-asm on python2.5 is optional make scanner2 look a little more like scanner3
2016-12-17 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner2.py, uncompyle6/scanners/scanner25.py:
show-asm on python2.5 is optional Make scanner2 a little more like scanner3.
2016-12-17 rocky <rb@dustyfeet.com>
* uncompyle6/parser.py, uncompyle6/scanner.py,
uncompyle6/scanners/scanner2.py, uncompyle6/scanners/scanner3.py,
uncompyle6/semantics/fragments.py: Python 2.6/2.7 tolerance in
Python 2.4 branch
2016-12-16 rocky <rb@dustyfeet.com>
* NEWS: Release 2.9.8 news
2016-12-16 rocky <rb@dustyfeet.com>
* __pkginfo__.py, uncompyle6/version.py: Get ready for release 2.9.8
* : commit 13d5cd1a588b7f4f2c233c436ce6b0b39db9950e Author: rocky
<rb@dustyfeet.com> Date: Fri Dec 16 22:42:46 2016 -0500
2016-12-16 rocky <rb@dustyfeet.com>
@@ -209,15 +999,12 @@
2016-12-04 rocky <rb@dustyfeet.com>
* ChangeLog, NEWS, uncompyle6/main.py, uncompyle6/parser.py,
uncompyle6/parsers/parse26.py, uncompyle6/parsers/parse3.py,
uncompyle6/parsers/parse34.py, uncompyle6/parsers/parse35.py,
uncompyle6/scanner.py, uncompyle6/scanners/scanner2.py,
uncompyle6/scanners/scanner23.py, uncompyle6/scanners/scanner24.py,
uncompyle6/scanners/scanner3.py, uncompyle6/scanners/tok.py,
uncompyle6/semantics/make_function.py,
uncompyle6/semantics/pysource.py, uncompyle6/verify.py,
uncompyle6/version.py: Get ready for release 2.9.7 Some of the many lint things. Linting is kind of stupid though.
* ChangeLog: Get ready for release 2.9.7
2016-12-04 rocky <rb@dustyfeet.com>
* : commit d22931cb49f0e28a0fbe48a7c1526b1f170a5b52 Author: rocky
<rb@dustyfeet.com> Date: Sun Dec 4 07:31:34 2016 -0500
2016-11-28 rocky <rb@dustyfeet.com>
@@ -268,11 +1055,34 @@
* uncompyle6/semantics/pysource.py: Better line number tracking Indent Python 2 list comprehensions, albeit badly. DRY code a
little via indent_if_source_nl
2016-11-24 rocky <rb@dustyfeet.com>
* circle.yml: CircleCI build
2016-11-24 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse3.py: Remove dup Python 3 grammar rule
2016-11-24 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse3.py, uncompyle6/scanners/scanner2.py:
<2.7 "if" detection and dup Python 3 grammar rule
2016-11-24 rocky <rb@dustyfeet.com>
* test/test_pyenvlib.py, uncompyle6/linenumbers.py,
uncompyle6/main.py, uncompyle6/scanners/scanner2.py,
uncompyle6/semantics/make_function.py,
uncompyle6/semantics/pysource.py, uncompyle6/verify.py: Bug in 2.4
"if" dectection and... Wrong language used in old-style exceptions: use "except Error,e"
not "except Error(e)""
2016-11-24 rocky <rb@dustyfeet.com>
* __pkginfo__.py, pytest/test_grammar.py, uncompyle6/parser.py,
uncompyle6/parsers/parse26.py: Python 2.6 grammary bug and.. __pkginfo.py__: Bump spark_parser
version for parse_flags 'dups'
2016-11-23 rocky <rb@dustyfeet.com>
* __pkginfo__.py, pytest/test_grammar.py, uncompyle6/parser.py,
@@ -284,8 +1094,17 @@
2016-11-23 rocky <rb@dustyfeet.com>
* : commit 6aa1531972de83ecab15b4c96b89c873ea5a7458 Author: rocky
<rb@dustyfeet.com> Date: Wed Nov 23 00:48:38 2016 -0500
* : commit 7133540c235e16f02d2db62cb903b70aa311de20 Author: rocky
<rb@dustyfeet.com> Date: Wed Nov 23 08:26:12 2016 -0500
2016-11-23 rocky <rb@dustyfeet.com>
* : commit a9349b8f3d12b2aa0cd88286617c1af9cccad018 Author: rocky
<rb@dustyfeet.com> Date: Tue Nov 22 17:49:47 2016 -0500
2016-11-23 rocky <rb@dustyfeet.com>
* circle.yml: Circle CI uses 2.7.10 and 2.7.12 is not available
2016-11-22 rocky <rb@dustyfeet.com>

View File

@@ -30,7 +30,7 @@ another clever idea: using table-driven semantics routines, using
format specifiers.
The last mention of a release of SPARK from John is around 2002. As
released, although the Early Algorithm parser was in good shape, this
released, although the Earley Algorithm parser was in good shape, this
code was woefully lacking as serious Python deparser.
In the fall of 2000, Hartmut Goebel
@@ -44,7 +44,8 @@ it appears that Hartmut did most of the work to get this code to
accept the full Python language. He added precedence to the table
specifiers, support for multiple versions of Python, the
pretty-printing of docstrings, lists, and hashes. He also wrote test and verification routines of
deparsed bytecode, and used this in an extensive set of tests that he also wrote. He could verify against the entire Python library.
deparsed bytecode, and used this in an extensive set of tests that he also wrote. He says he could verify against the
entire Python library. However I have subsequently found small and relatively obscure bugs in the decompilation code.
decompyle2.2 was packaged for Debian (sarge) by
[Ben Burton around 2002](https://packages.qa.debian.org/d/decompyle.html). As
@@ -65,10 +66,12 @@ code to handle first Python 2.3 and then 2.4 bytecodes. Because of
jump optimization introduced in the CPython bytecode compiler at that
time, various JUMP instructions were classifed as going backwards, and
COME FROM instructions were reintroduced. See
RELEASE-2.4-CHANGELOG.txt for more details here. There wasn't a public
[RELEASE-2.4-CHANGELOG.txt](https://github.com/rocky/python-uncompyle6/blob/master/DECOMPYLE-2.4-CHANGELOG.txt)
for more details here. There wasn't a public
release of RELEASE-2.4 and bytecodes other than Python 2.4 weren't
supported. Dan says the Python 2.3 version could verify the entire
python library.
Python library. But given subsequent bugs found like simply
recognizing complex-number constants in bytecode, decompilation wasn't perfect.
Next we get to ["uncompyle" and
PyPI](https://pypi.python.org/pypi/uncompyle/1.1) and the era of
@@ -95,17 +98,17 @@ so. Then hamled made a few commits earler on, while Eike Siewertsen
made a few commits later on. But mostly wibiti, and Guenther
Starnberger got the code to where uncompyle2 was around 2012.
In uncompyle2 decompilation of python bytecode 2.5 & 2.6 is done by
In `uncompyle`, decompilation of python bytecode 2.5 & 2.6 is done by
transforming the byte code into a a pseudo 2.7 python bytecode and is
based on code from Eloi Vanderbeken.
This project, uncompyle6, abandons that approach for various
This project, `uncompyle6`, abandons that approach for various
reasons. However the main reason is that we need offsets in fragment
deparsing to be exactly the same, and the transformation process can
remove instructions. Adding instructions with psuedo_offsets is
remove instructions. _Adding_ instructions with psuedo offsets is
however okay.
Uncompyle6, however owes its existence to the fork of uncompyle2 by
`Uncompyle6` however owes its existence to the fork of `uncompyle2` by
Myst herie (Mysterie) whose first commit picks up at
2012. I chose this since it seemed to have been at that time the most
actively, if briefly, worked on. Also starting around 2012 is Dark
@@ -115,9 +118,12 @@ I started working on this late 2015, mostly to add fragment support.
In that, I decided to make this runnable on Python 3.2+ and Python 2.6+
while, handling Python bytecodes from Python versions 2.5+ and
3.2+. In doing so, it has been expedient to separate this into three
projects: load loading and disassembly (xdis), parsing and tree
building (spark_parser), and grammar and semantic actions for
decompiling (uncompyle6).
projects:
* bytecode loading and disassembly ([xdis](https://pypi.python.org/pypi/xdis)),
* parsing and tree building ([spark_parser](https://pypi.python.org/pypi/spark_parser)),
* this project - grammar and semantic actions for decompiling
([uncompyle6](https://pypi.python.org/pypi/spark_parser)).
Over the many years, code styles and Python features have
@@ -135,23 +141,26 @@ Hartmut a decade an a half ago:
NB. This is not a masterpiece of software, but became more like a hack.
Probably a complete rewrite would be sensefull. hG/2000-12-27
This project deparses using an Early-algorithm parse with lots of
This project deparses using an Earley-algorithm parse with lots of
massaging of tokens and the grammar in the scanner
phase. Early-algorithm parsers are context free and tend to be linear
phase. Earley-algorithm parsers are context free and tend to be linear
if the grammar is LR or left recursive.
Another approach that doesn't use grammars is to do something like
simulate execution symbolically and build expression trees off of
stack results. The two important projects that work this way are
[unpyc3](https://code.google.com/p/unpyc3/) and most especially
[pycdc](https://github.com/zrax/pycdc) The latter project is largely
by Michael Hansen and Darryl Pogue. If they supported getting
source-code fragments and I could call it from Python, I'd probably
ditch this and use that. From what I've seen, the code runs blindingly
fast and spans all versions of Python.
stack results. Control flow in that apprproach still needs to be
handled somewhat ad hoc. The two important projects that work this
way are [unpyc3](https://code.google.com/p/unpyc3/) and most
especially [pycdc](https://github.com/zrax/pycdc) The latter project
is largely by Michael Hansen and Darryl Pogue. If they supported
getting source-code fragments, did a better job in supporting Python
more fully, and had a way I could call it from Python, I'd probably
would have ditched this and used that. The code runs blindingly fast
and spans all versions of Python, although more recently Python 3
support has been lagging.
Tests for the project have been, or are being, culled from all of the
projects mentioned.
NB. If you find mistakes, want corrections, or want your name added (or removed),
please contact me.
NB. If you find mistakes, want corrections, or want your name added
(or removed), please contact me.

63
HOW-TO-REPORT-A-BUG.md Normal file
View File

@@ -0,0 +1,63 @@
# How to report a Bug
## The difficulty of the problem
There is no Python decompiler yet, that I know about that will
decompyle everything. This one probably does the
best job of *any* Python decompiler. But it is a constant work in progress: Python keeps changing, and so does its code generation.
I have found bugs in *every* Python decompiler I have tried. Even
those where authors/maintainers claim that they have used it on
the entire Python standard library. And I don't mean that
the program doesn't come out with the same Python source instructions,
but that the program is *semantically* not equivalent.
So it is likely you'll find a mistranslation in decompiling.
## What to send (minimum requirements)
The basic requirement is pretty simple:
* Python bytecode
* Source text
## What to send (additional helpful information)
Some kind folks also give the invocation they used and the output
which usually includes an error message produced. This is helpful. I
can figure out what OS you are running this on and what version of
*uncomplye6* was used. Therefore, if you don't provide the input
command and the output from that, please give:
* _uncompile6_ version used
* OS that you used this on
* Python interpreter version used
### But I don't *have* the source code!
Sure, I get it. No problem. There is Python assembly code on parse
errors, so simply by hand decompile that. To get a full disassembly, use pydisasm from the [xdis](https://pypi.python.org/pypi/xdis) package. Opcodes are described in the documentation for the [dis](https://docs.python.org/3.6/library/dis.html) module.
### But I don't *have* the source code and am incapable of figuring how how to do a hand disassembly!
Well, you could learn. No one is born into this world knowing how to disassemble Python bytecode. And as Richard Feynman once said, "What one fool can learn, so can another."
## Narrowing the problem
I don't need the entire source code base for which one file or module
can't be decompiled. I just need that one file or module only. If
there are several files, file a bug report for each file.
Python modules can get quite large, and usually decompilation problems
occur in a single function or maybe the main-line code but not any of
the functions or classes. So please chop down the source code by
removing those parts that do to decompile properly.
By doing this, you'll probably have a better sense of what exactly is
the problem. Perhaps you can find the boundary of what decompiles, and
what doesn't. That is useful. Or maybe the same file will decompile
properly on a neighboring version of Python. That is helpful too.
In sum, the more you can isolate or narrow the problem, the more
likley the problem will be fixed and fixed sooner.

View File

@@ -1,6 +1,7 @@
include README.rst
include ChangeLog
include HISTORY.md
include HOW-TO-REPORT-A-BUG.md
include LICENSE
include Makefile
include requirements.txt

View File

@@ -37,7 +37,7 @@ check-3.0 check-3.1 check-3.2 check-3.5 check-3.6:
$(MAKE) -C test $@
#:Tests for Python 2.6 (doesn't have pytest)
check-2.6:
check-2.4 check-2.5 check-2.6:
$(MAKE) -C test $@
#:PyPy 2.6.1 or PyPy 5.0.1
@@ -59,7 +59,7 @@ clean: clean_pyc
#: Create source (tarball) and wheel distribution
dist:
$(PYTHON) ./setup.py sdist bdist_wheel
$(PYTHON) ./setup.py sdist bdist_egg
#: Remove .pyc files
clean_pyc:

24
NEWS
View File

@@ -1,5 +1,27 @@
uncompyle6 2.9.11 2016-04-06
- Better support for Python 3.5+ BUILD_MAP_UNPACK
- Start 3.6 CALL_FUNCTION_EX support
- Many decompilation bug fixes. (Many more remain). See ChangeLog
uncompyle6 2.9.10 2016-02-25
- Python grammar rule fixes
- Add ability to get grammar coverage on runs
- Handle Python 3.6 opcode BUILD_CONST_KEYMAP
uncompyle6 2.9.9 2016-12-16
- Remaining Python 3.5 ops handled
(this also means more Python 3.6 ops are handled)
- Python 3.5 and 3.6 async and await handled
- Python 3.0 decompilation improved
- Python 3 annotations fixed
- Better control-flow detection
- Code cleanups and misc bug fixes
uncompyle6 2.9.8 2016-12-16
- Better control-flow detection
- pseudo instruction THEN in 2.x
to disambiguate if from and
@@ -11,8 +33,6 @@ uncompyle6 2.9.9 2016-12-16
- verifycall fixes for Python <= 2.4
- more Python lint
uncompyle6 2.9.8 2016-12-16
uncompyle6 2.9.7 2016-12-16
- Start to handle 3.5/3.6 build_map_unpack_with_call

View File

@@ -11,8 +11,8 @@ Introduction
------------
*uncompyle6* translates Python bytecode back into equivalent Python
source code. It accepts bytecodes from Python version 2.1 to 3.6 or
so, including PyPy bytecode and Dropbox's Python 2.5 bytecode.
source code. It accepts bytecodes from Python version 1.5, and 2.1 to
3.6 or so, including PyPy bytecode and Dropbox's Python 2.5 bytecode.
Why this?
---------
@@ -21,7 +21,8 @@ There were a number of decompyle, uncompile, uncompyle2, uncompyle3
forks around. All of them came basically from the same code base, and
almost all of them no were no longer actively maintained. Only one
handled Python 3, and even there, only 3.2 or 3.3 depending on which
code is used. This code pulls these together and moves forward. It
code is used. This code pulls these together and moves forward. This
project has the most complete support for Python 3.3 and above. It
also addresses a number of open issues in the previous forks.
What makes this different from other CPython bytecode decompilers?: its
@@ -43,10 +44,10 @@ information.
Requirements
------------
This project requires Python 2.6 or later, PyPy 3-2.4, or PyPy-5.0.1.
Python versions 2.3-2.7 are supported in the python-2.4 branch.
This project requires Python 2.6 or later, PyPy 3-2.4, or PyPy-5.0.1.
Python versions 2.4-2.7 are supported in the python-2.4 branch.
The bytecode files it can read has been tested on Python bytecodes from
versions 2.1-2.7, and 3.2-3.6 and the above-mentioned PyPy versions.
versions 1.5, 2.1-2.7, and 3.0-3.6 and the above-mentioned PyPy versions.
Installation
------------
@@ -93,6 +94,16 @@ For usage help:
$ uncompyle6 -h
If you want strong verification of the correctness of the
decompilation process, add the `--verify` option. But there are
situations where this will indicate a failure, although the generated
program is semantically equivalent. Using option `--weak-verify` will
tell you if there is something definitely wrong. Generally, large
swaths of code are decompiled correctly, if not the entire program.
You can also cross compare the results with pycdc_ . Since they work
differently, bugs here often aren't in that, and vice versa.
Known Bugs/Restrictions
-----------------------
@@ -102,7 +113,7 @@ with handling control flow. All of the Python decompilers I have looked
at have the same problem. In some cases we can detect an erroneous
decompilation and report that.
About 90% of the decompilation of Python standard library packages in
Over 98% of the decompilation of Python standard library packages in
Python 2.7.12 verifies correctly. Over 99% of Python 2.7 and 3.3-3.5
"weakly" verify. Python 2.6 drops down to 96% weakly verifying.
Other versions drop off in quality too.
@@ -128,10 +139,12 @@ on the lower end Python versions which is more difficult for us to
handle since we don't have a Python interpreter for versions 1.5, 1.6,
and 2.0.
Python 3.0 support is weak; Python 3.5 largely works, but still has
some bugs in it. Python 3.6 changes things drastically by using word
codes rather than byte codes. That has been addressed, but then it also
changes function call opcodes and its semantics.
In the Python 3 series, Python support is is strongest around 3.4 or
3.3 and drops off as you move further away from those versions. Python
3.6 changes things drastically by using word codes rather than byte
codes. That has been addressed, but then it also changes function call
opcodes and its semantics and has more problems with control flow than
3.5 has.
Currently not all Python magic numbers are supported. Specifically in
some versions of Python, notably Python 3.6, the magic number has
@@ -145,13 +158,16 @@ We also don't handle PJOrion_ obfuscated code. For that try: PJOrion
Deobfuscator_ to unscramble the bytecode to get valid bytecode before
trying this tool.
Handling pathologically long lists of expressions or statements is
slow.
There is lots to do, so please dig in and help.
See Also
--------
* https://github.com/zrax/pycdc : supports all versions of Python and is written in C++
* https://github.com/zrax/pycdc : supports all versions of Python and is written in C++. Support for later Python 3 versions is a bit lacking though.
* https://code.google.com/archive/p/unpyc3/ : supports Python 3.2 only. The above projects use a different decompiling technique what is used here.
* https://github.com/figment/unpyc3/ : fork of above, but supports Python 3.3 only. Include some fixes like supporting function annotations
* The HISTORY_ file.

View File

@@ -9,7 +9,7 @@
# Things that change more often go here.
copyright = """
Copyright (C) 2015, 2016 Rocky Bernstein <rb@dustyfeet.com>.
Copyright (C) 2015-2017 Rocky Bernstein <rb@dustyfeet.com>.
"""
classifiers = ['Development Status :: 5 - Production/Stable',
@@ -39,8 +39,8 @@ entry_points={
'pydisassemble=uncompyle6.bin.pydisassemble:main',
]}
ftp_url = None
install_requires = ['spark-parser >= 1.5.1, < 1.6.0',
'xdis >= 3.2.4, < 3.3.0']
install_requires = ['spark-parser >= 1.6.0, < 1.7.0',
'xdis >= 3.3.0, < 3.4.0']
license = 'MIT'
mailing_list = 'python-debugger@googlegroups.com'
modname = 'uncompyle6'

View File

@@ -10,4 +10,4 @@ dependencies:
- pip install -r requirements-dev.txt
test:
override:
- python ./setup.py develop && make check-2.7
- python ./setup.py develop && make check-2.6

View File

@@ -0,0 +1,21 @@
import pytest
# uncompyle6
from uncompyle6 import PYTHON_VERSION
from validate import validate_uncompyle
@pytest.mark.skipif(PYTHON_VERSION < 3.6, reason='need at least python 3.6')
@pytest.mark.parametrize('text', (
"{0.: 'a', -1: 'b'}", # BUILD_MAP
"{'a':'b'}", # BUILD_MAP
"{0: 1}", # BUILD_MAP
"{b'0':1, b'2':3}", # BUILD_CONST_KEY_MAP
"{0: 1, 2: 3}", # BUILD_CONST_KEY_MAP
"{'a':'b','c':'d'}", # BUILD_CONST_KEY_MAP
"{0: 1, 2: 3}", # BUILD_CONST_KEY_MAP
"{'a': 1, 'b': 2}", # BUILD_CONST_KEY_MAP
"{'a':'b','c':'d'}", # BUILD_CONST_KEY_MAP
"{0.0:'b',0.1:'d'}", # BUILD_CONST_KEY_MAP
))
def test_build_const_key_map(text):
validate_uncompyle(text)

View File

@@ -1,150 +0,0 @@
# std
import os
# test
import pytest
import hypothesis
from hypothesis import strategies as st
# uncompyle6
from uncompyle6 import PYTHON_VERSION, deparse_code
@st.composite
def expressions(draw):
# todo : would be nice to generate expressions using hypothesis however
# this is pretty involved so for now just use a corpus of expressions
# from which to select.
return draw(st.sampled_from((
'abc',
'len(items)',
'x + 1',
'lineno',
'container',
'self.attribute',
'self.method()',
# These expressions are failing, I think these are control
# flow problems rather than problems with FORMAT_VALUE,
# however I need to confirm this...
#'sorted(items, key=lambda x: x.name)',
#'func(*args, **kwargs)',
#'text or default',
#'43 if life_the_universe and everything else None'
)))
@st.composite
def format_specifiers(draw):
"""
Generate a valid format specifier using the rules:
format_spec ::= [[fill]align][sign][#][0][width][,][.precision][type]
fill ::= <any character>
align ::= "<" | ">" | "=" | "^"
sign ::= "+" | "-" | " "
width ::= integer
precision ::= integer
type ::= "b" | "c" | "d" | "e" | "E" | "f" | "F" | "g" | "G" | "n" | "o" | "s" | "x" | "X" | "%"
See https://docs.python.org/2/library/string.html
:param draw: Let hypothesis draw from other strategies.
:return: An example format_specifier.
"""
alphabet_strategy = st.characters(min_codepoint=ord('a'), max_codepoint=ord('z'))
fill = draw(st.one_of(alphabet_strategy, st.none()))
align = draw(st.sampled_from(list('<>=^')))
fill_align = (fill + align or '') if fill else ''
type_ = draw(st.sampled_from('bcdeEfFgGnosxX%'))
can_have_sign = type_ in 'deEfFgGnoxX%'
can_have_comma = type_ in 'deEfFgG%'
can_have_precision = type_ in 'fFgG'
can_have_pound = type_ in 'boxX%'
can_have_zero = type_ in 'oxX'
sign = draw(st.sampled_from(list('+- ') + [''])) if can_have_sign else ''
pound = draw(st.sampled_from(('#', '',))) if can_have_pound else ''
zero = draw(st.sampled_from(('0', '',))) if can_have_zero else ''
int_strategy = st.integers(min_value=1, max_value=1000)
width = draw(st.one_of(int_strategy, st.none()))
width = str(width) if width is not None else ''
comma = draw(st.sampled_from((',', '',))) if can_have_comma else ''
if can_have_precision:
precision = draw(st.one_of(int_strategy, st.none()))
precision = '.' + str(precision) if precision else ''
else:
precision = ''
return ''.join((fill_align, sign, pound, zero, width, comma, precision, type_,))
@st.composite
def fstrings(draw):
"""
Generate a valid f-string.
See https://www.python.org/dev/peps/pep-0498/#specification
:param draw: Let hypothsis draw from other strategies.
:return: A valid f-string.
"""
character_strategy = st.characters(
blacklist_characters='\r\n\'\\s{}',
min_codepoint=1,
max_codepoint=1000,
)
is_raw = draw(st.booleans())
integer_strategy = st.integers(min_value=0, max_value=3)
expression_count = draw(integer_strategy)
content = []
for _ in range(expression_count):
expression = draw(expressions())
conversion = draw(st.sampled_from(('', '!s', '!r', '!a',)))
has_specifier = draw(st.booleans())
specifier = ':' + draw(format_specifiers()) if has_specifier else ''
content.append('{{{}{}}}'.format(expression, conversion, specifier))
content.append(draw(st.text(character_strategy)))
content = ''.join(content)
return "f{}'{}'".format('r' if is_raw else '', content)
@pytest.mark.skipif(PYTHON_VERSION < 3.6, reason='need at least python 3.6')
@hypothesis.given(format_specifiers())
def test_format_specifiers(format_specifier):
"""Verify that format_specifiers generates valid specifiers"""
try:
exec('"{:' + format_specifier + '}".format(0)')
except ValueError as e:
if 'Unknown format code' not in str(e):
raise
def run_test(text):
hypothesis.assume(len(text))
hypothesis.assume("f'{" in text)
expr = text + '\n'
code = compile(expr, '<string>', 'single')
deparsed = deparse_code(PYTHON_VERSION, code, compile_mode='single')
recompiled = compile(deparsed.text, '<string>', 'single')
if recompiled != code:
assert 'dis(' + deparsed.text.strip('\n') + ')' == 'dis(' + expr.strip('\n') + ')'
@pytest.mark.skipif(PYTHON_VERSION < 3.6, reason='need at least python 3.6')
@hypothesis.given(fstrings())
def test_uncompyle_fstring(fstring):
"""Verify uncompyling fstring bytecode"""
run_test(fstring)
@pytest.mark.skipif(PYTHON_VERSION < 3.6, reason='need at least python 3.6')
@pytest.mark.parametrize('fstring', [
"f'{abc}{abc!s}'",
"f'{abc}0'",
])
def test_uncompyle_direct(fstring):
"""useful for debugging"""
run_test(fstring)

View File

@@ -0,0 +1,128 @@
# std
import string
# 3rd party
from hypothesis import given, assume, strategies as st
import pytest
# uncompyle
from validate import validate_uncompyle
alpha = st.sampled_from(string.ascii_lowercase)
numbers = st.sampled_from(string.digits)
alphanum = st.sampled_from(string.ascii_lowercase + string.digits)
expressions = st.sampled_from([x for x in string.ascii_lowercase + string.digits] + ['x+1'])
@st.composite
def function_calls(draw):
"""
Strategy factory for generating function calls.
:param draw: Callable which draws examples from other strategies.
:return: The function call text.
"""
list1 = st.lists(alpha, min_size=0, max_size=1)
list3 = st.lists(alpha, min_size=0, max_size=3)
positional_args = draw(list3)
named_args = [x + '=0' for x in draw(list3)]
star_args = ['*' + x for x in draw(list1)]
double_star_args = ['**' + x for x in draw(list1)]
arguments = positional_args + named_args + star_args + double_star_args
draw(st.randoms()).shuffle(arguments)
arguments = ','.join(arguments)
function_call = 'fn({arguments})'.format(arguments=arguments)
try:
# TODO: Figure out the exact rules for ordering of positional, named,
# star args, double star args and in which versions the various
# types of arguments are supported so we don't need to check that the
# expression compiles like this.
compile(function_call, '<string>', 'single')
except:
assume(False)
return function_call
@pytest.mark.xfail()
def test_CALL_FUNCTION():
validate_uncompyle("fn(w,m,f)")
@pytest.mark.xfail()
def test_BUILD_CONST_KEY_MAP_BUILD_MAP_UNPACK_WITH_CALL_BUILD_TUPLE_CALL_FUNCTION_EX():
validate_uncompyle("fn(w=0,m=0,**v)")
@pytest.mark.xfail()
def test_BUILD_MAP_BUILD_MAP_UNPACK_WITH_CALL_BUILD_TUPLE_CALL_FUNCTION_EX():
validate_uncompyle("fn(a=0,**g)")
@pytest.mark.xfail()
def test_CALL_FUNCTION_KW():
validate_uncompyle("fn(j=0)")
@pytest.mark.xfail()
def test_CALL_FUNCTION_EX():
validate_uncompyle("fn(*g,**j)")
@pytest.mark.xfail()
def test_BUILD_MAP_CALL_FUNCTION_EX():
validate_uncompyle("fn(*z,u=0)")
@pytest.mark.xfail()
def test_BUILD_TUPLE_CALL_FUNCTION_EX():
validate_uncompyle("fn(**a)")
@pytest.mark.xfail()
def test_BUILD_MAP_BUILD_TUPLE_BUILD_TUPLE_UNPACK_WITH_CALL_CALL_FUNCTION_EX():
validate_uncompyle("fn(b,b,b=0,*a)")
@pytest.mark.xfail()
def test_BUILD_TUPLE_BUILD_TUPLE_UNPACK_WITH_CALL_CALL_FUNCTION_EX():
validate_uncompyle("fn(*c,v)")
@pytest.mark.xfail()
def test_BUILD_CONST_KEY_MAP_CALL_FUNCTION_EX():
validate_uncompyle("fn(i=0,y=0,*p)")
@pytest.mark.skip(reason='skipping property based test until all individual tests are passing')
@given(function_calls())
def test_function_call(function_call):
validate_uncompyle(function_call)
examples = set()
generate_examples = False
@pytest.mark.skipif(not generate_examples, reason='not generating examples')
@given(function_calls())
def test_generate_hypothesis(function_call):
examples.add(function_call)
@pytest.mark.skipif(not generate_examples, reason='not generating examples')
def test_generate_examples():
import dis
example_opcodes = {}
for example in examples:
opcodes = tuple(sorted(set(
instruction.opname
for instruction in dis.Bytecode(example)
if instruction.opname not in ('LOAD_CONST', 'LOAD_NAME', 'RETURN_VALUE')
)))
example_opcodes[opcodes] = example
for k, v in example_opcodes.items():
print('def test_' + '_'.join(k) + '():\n validate_uncompyle("' + v + '")\n\n')
return

View File

@@ -40,7 +40,9 @@ def test_grammar():
ignore_set = set(
"""
JUMP_BACK CONTINUE RETURN_END_IF
COME_FROM COME_FROM_EXCEPT COME_FROM_LOOP COME_FROM_WITH
COME_FROM COME_FROM_EXCEPT
COME_FROM_EXCEPT_CLAUSE
COME_FROM_LOOP COME_FROM_WITH
COME_FROM_FINALLY ELSE
LOAD_GENEXPR LOAD_ASSERT LOAD_SETCOMP LOAD_DICTCOMP
LAMBDA_MARKER RETURN_LAST

147
pytest/validate.py Normal file
View File

@@ -0,0 +1,147 @@
# future
from __future__ import print_function
# std
import os
import difflib
import subprocess
import tempfile
import functools
# compatability
import six
# uncompyle6 / xdis
from uncompyle6 import PYTHON_VERSION, IS_PYPY, deparse_code
# TODO : I think we can get xdis to support the dis api (python 3 version) by doing something like this there
from xdis.bytecode import Bytecode
from xdis.main import get_opcode
opc = get_opcode(PYTHON_VERSION, IS_PYPY)
Bytecode = functools.partial(Bytecode, opc=opc)
def _dis_to_text(co):
return Bytecode(co).dis()
def print_diff(original, uncompyled):
"""
Try and display a pretty html line difference between the original and
uncompyled code and bytecode if elinks and BeautifulSoup are installed
otherwise just show the diff.
:param original: Text describing the original code object.
:param uncompyled: Text describing the uncompyled code object.
"""
original_lines = original.split('\n')
uncompyled_lines = uncompyled.split('\n')
args = original_lines, uncompyled_lines, 'original', 'uncompyled'
try:
from bs4 import BeautifulSoup
diff = difflib.HtmlDiff().make_file(*args)
diff = BeautifulSoup(diff, "html.parser")
diff.select_one('table[summary="Legends"]').extract()
except ImportError:
print('\nTo display diff highlighting run:\n pip install BeautifulSoup4')
diff = difflib.HtmlDiff().make_table(*args)
with tempfile.NamedTemporaryFile(delete=False) as f:
f.write(str(diff).encode('utf-8'))
try:
print()
html = subprocess.check_output([
'elinks',
'-dump',
'-no-references',
'-dump-color-mode',
'1',
f.name,
]).decode('utf-8')
print(html)
except:
print('\nFor side by side diff install elinks')
diff = difflib.Differ().compare(original_lines, uncompyled_lines)
print('\n'.join(diff))
finally:
os.unlink(f.name)
def are_instructions_equal(i1, i2):
"""
Determine if two instructions are approximately equal,
ignoring certain fields which we allow to differ, namely:
* code objects are ignore (should probaby be checked) due to address
* line numbers
:param i1: left instruction to compare
:param i2: right instruction to compare
:return: True if the two instructions are approximately equal, otherwise False.
"""
result = (1==1
and i1.opname == i2.opname
and i1.opcode == i2.opcode
and i1.arg == i2.arg
# ignore differences due to code objects
# TODO : Better way of ignoring address
and (i1.argval == i2.argval or '<code object' in str(i1.argval))
# TODO : Should probably recurse to check code objects
and (i1.argrepr == i2.argrepr or '<code object' in i1.argrepr)
and i1.offset == i2.offset
# ignore differences in line numbers
#and i1.starts_line
and i1.is_jump_target == i2.is_jump_target
)
return result
def are_code_objects_equal(co1, co2):
"""
Determine if two code objects are approximately equal,
see are_instructions_equal for more information.
:param i1: left code object to compare
:param i2: right code object to compare
:return: True if the two code objects are approximately equal, otherwise False.
"""
instructions1 = Bytecode(co1)
instructions2 = Bytecode(co2)
for opcode1, opcode2 in zip(instructions1, instructions2):
if not are_instructions_equal(opcode1, opcode2):
return False
return True
def validate_uncompyle(text, mode='exec'):
"""
Validate decompilation of the given source code.
:param text: Source to validate decompilation of.
"""
original_code = compile(text, '<string>', mode)
original_dis = _dis_to_text(original_code)
original_text = text
deparsed = deparse_code(PYTHON_VERSION, original_code,
compile_mode=mode, out=six.StringIO())
uncompyled_text = deparsed.text
uncompyled_code = compile(uncompyled_text, '<string>', 'exec')
if not are_code_objects_equal(uncompyled_code, original_code):
uncompyled_dis = _dis_to_text(uncompyled_text)
def output(text, dis):
width = 60
return '\n\n'.join([
' SOURCE CODE '.center(width, '#'),
text.strip(),
' BYTECODE '.center(width, '#'),
dis
])
original = output(original_text, original_dis)
uncompyled = output(uncompyled_text, uncompyled_dis)
print_diff(original, uncompyled)
assert 'original' == 'uncompyled'

View File

@@ -1,3 +1,4 @@
pytest
flake8
hypothesis
hypothesis
six

View File

@@ -3,7 +3,7 @@ PHONY=check clean dist distclean test test-unit test-functional rmChangeLog clea
GIT2CL ?= git2cl
PYTHON ?= python
PYTHON_VERSION = $(shell $(PYTHON) -V | cut -d ' ' -f 2 | cut -d'.' -f1,2)
PYTHON_VERSION = $(shell $(PYTHON) -V 2>&1 | cut -d ' ' -f 2 | cut -d'.' -f1,2)
NATIVE_CHECK = check-$(PYTHON_VERSION)
# Set COMPILE='--compile' to force compilation before check
@@ -20,7 +20,7 @@ check:
$(MAKE) check-$$PYTHON_VERSION
#: Run working tests from Python 2.6 or 2.7
check-2.6 check-2.7: check-bytecode-2 check-bytecode-3 check-bytecode-1 check-2.7-ok
check-2.4 check-2.5 check-2.6 check-2.7: check-bytecode-2 check-bytecode-3 check-bytecode-1 check-native-short
#: Run working tests from Python 3.0
check-3.0: check-bytecode
@@ -68,7 +68,7 @@ check-bytecode-2:
check-bytecode-3:
$(PYTHON) test_pythonlib.py --bytecode-3.0 \
--bytecode-3.1 --bytecode-3.2 --bytecode-3.3 \
--bytecode-3.4 --bytecode-3.5 --bytecode-pypy3.2
--bytecode-3.4 --bytecode-3.5 --bytecode-3.6 --bytecode-pypy3.2
#: Check deparsing bytecode that works running Python 2 and Python 3
check-bytecode: check-bytecode-3
@@ -98,14 +98,6 @@ check-bytecode-2.4:
check-bytecode-2.5:
$(PYTHON) test_pythonlib.py --bytecode-2.5
#: Check deparsing Python 2.6
check-bytecode-2.6:
$(PYTHON) test_pythonlib.py --bytecode-2.6 --weak-verify
#: Check deparsing Python 2.7
check-bytecode-2.7:
$(PYTHON) test_pythonlib.py --bytecode-2.7 --verify
#: Check deparsing Python 3.0
check-bytecode-3.0:
$(PYTHON) test_pythonlib.py --bytecode-3.0
@@ -134,9 +126,33 @@ check-bytecode-3.5:
check-bytecode-3.6:
$(PYTHON) test_pythonlib.py --bytecode-3.6
#: Get grammar coverage for Python 2.4
grammar-coverage-2.4:
SPARK_PARSER_COVERAGE=/tmp/spark-grammar-24.cover $(PYTHON) test_pythonlib.py --bytecode-2.4
SPARK_PARSER_COVERAGE=/tmp/spark-grammar-24.cover $(PYTHON) test_pyenvlib.py --2.4.6
#: Get grammar coverage for Python 2.5
grammar-coverage-2.5:
SPARK_PARSER_COVERAGE=/tmp/spark-grammar-25.cover $(PYTHON) test_pythonlib.py --bytecode-2.5
SPARK_PARSER_COVERAGE=/tmp/spark-grammar-25.cover $(PYTHON) test_pyenvlib.py --2.5.6
#: Get grammar coverage for Python 2.6
grammar-coverage-2.6:
SPARK_PARSER_COVERAGE=/tmp/spark-grammar-26.cover $(PYTHON) test_pythonlib.py --bytecode-2.6
SPARK_PARSER_COVERAGE=/tmp/spark-grammar-26.cover $(PYTHON) test_pyenvlib.py --2.6.9
#: Get grammar coverage for Python 2.7
grammar-coverage-2.7:
SPARK_PARSER_COVERAGE=/tmp/spark-grammar-27.cover $(PYTHON) test_pythonlib.py --bytecode-2.7
SPARK_PARSER_COVERAGE=/tmp/spark-grammar-27.cover $(PYTHON) test_pyenvlib.py --2.7.13
#: short tests for bytecodes only for this version of Python
check-native-short:
$(PYTHON) test_pythonlib.py --bytecode-$(PYTHON_VERSION) --verify $(COMPILE)
$(PYTHON) test_pythonlib.py --bytecode-$(PYTHON_VERSION) --weak-verify $(COMPILE)
#: Run longer Python 2.6's lib files known to be okay
check-2.4-ok:
$(PYTHON) test_pythonlib.py --ok-2.4 --verify $(COMPILE)
#: Run longer Python 2.6's lib files known to be okay
check-2.6-ok:

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

BIN
test/bytecode_3.2/03_if.pyc Normal file

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

View File

@@ -0,0 +1,18 @@
# Statements to beef up grammar coverage rules
# Force "inplace" ops
y = +10 # UNARY_POSITIVE
y /= 1 # INPLACE_DIVIDE
y %= 4 # INPLACE_MODULO
y **= 1 # INPLACE POWER
y >>= 2 # INPLACE_RSHIFT
y <<= 2 # INPLACE_LSHIFT
y //= 1 # INPLACE_TRUE_DIVIDE
y &= 1 # INPLACE_AND
y ^= 1 # INPLACE_XOR
`y` # UNARY_CONVERT - No in Python 3.x
# Beef up augassign and STORE_SLICE+3
x = [1,2,3,4,5]
x[0:1] = 1
x[0:3] += 1, 2, 3

View File

@@ -0,0 +1,7 @@
# From https://github.com/ToontownInfinite /src/otp/avatar/LocalAvatar.py#L364
if 1:
def jumpLandAnimFix(self, jumpTime):
return 5
def jumpLand(self):
return 6

View File

@@ -1,7 +1,6 @@
# Python 2.5 bug
# Was turning into tryelse when there in fact is no else.
def options(self, section):
"""Return a list of option names for the given section name."""
try:
opts = self._sections[section].copy()
except KeyError:
@@ -10,3 +9,19 @@ def options(self, section):
if '__name__' in opts:
del opts['__name__']
return opts.keys()
# From python2.5/distutils/command/register.py
def post_to_server(self, urllib2):
try:
result = 5
except urllib2.HTTPError, e:
result = e.code, e.msg
except urllib2.URLError, e:
result = 500
else:
if self.show_response:
result = 10
result = 200
if self.show_response:
result = 8
return result

View File

@@ -0,0 +1,7 @@
# From Python 2.6. distutils/sysconfig.py
def get_config_vars(_config_vars, args):
if _config_vars:
if args == 1:
if args < 8:
for key in ('LDFLAGS', 'BASECFLAGS'):
_config_vars[key] = 4

View File

@@ -0,0 +1,6 @@
# From Python 2.7 parse_starttag HTMLParser.pyc
attrvalue = [1,2]
while attrvalue:
if attrvalue[:1] == 5 or \
attrvalue[:1] == 2 == attrvalue[-1:]:
attrvalue = 10

View File

@@ -0,0 +1 @@
1 if 1 else __file__

View File

@@ -0,0 +1,12 @@
# Python 2.7 sqlalchemy-1.013/sql/crud.py
def _extend_values_for_multiparams(compiler, stmt, c):
c(
[
(
(compiler() if compiler()
else compiler())
if c in stmt else compiler(),
)
]
for i in enumerate(stmt)
)

View File

@@ -8,3 +8,23 @@ def open(file, mode = "r", buffering = None,
encoding = None, errors = None,
newline = None, closefd = True) -> "IOBase":
return text
def foo(x: 'an argument that defaults to 5' = 5):
print(x)
def div(a: dict(type=float, help='the dividend'),
b: dict(type=float, help='the divisor (must be different than 0)')
) -> dict(type=float, help='the result of dividing a by b'):
"""Divide a by b"""
return a / b
class TestSignatureObject(unittest.TestCase):
def test_signature_on_wkwonly(self):
def test(*, a:float, b:str) -> int:
pass
class SupportsInt(_Protocol):
@abstractmethod
def __int__(self) -> int:
pass

View File

@@ -18,3 +18,12 @@ def __init__(self, defaults=None, dict_type=_default_dict,
default_section=DEFAULTSECT,
interpolation=_UNSET):
pass
# Bug found by hypothesis in creating function calls
# thanks to moagstar
def fn(a, b, d):
return (a, b, d)
b = {'b': 1,
'd': 2}
fn(a=0, **b)

View File

@@ -0,0 +1,9 @@
# From 3.2 _abcoll.py
def pop(self):
it = iter(self)
try:
value = next(it)
except StopIteration:
raise KeyError
self.discard(value)
return value

View File

@@ -0,0 +1,11 @@
# From 3.2 shlex.py
def _samefile(os, src, dst):
if hasattr(os.path, 'samefile'):
try:
return os.path.samefile(src, dst)
except OSError:
return False
# All other platforms: check for same pathname.
return (os.path.normcase(os.path.abspath(src)) ==
os.path.normcase(os.path.abspath(dst)))

View File

@@ -0,0 +1,4 @@
def a():
del y
def b():
return y

View File

@@ -0,0 +1,16 @@
# From 3.3.5 _osx_support.py
def _get_system_version():
if __file__ is None:
try:
m = 5
except IOError:
pass
else:
try:
m = 10
finally:
m = 15
if m is not None:
m = 20
return m

View File

@@ -0,0 +1,16 @@
# From 3.3.5 _osx_support.py
def _get_system_version():
if __file__ is None:
try:
m = 5
except IOError:
pass
else:
try:
m = 10
finally:
m = 15
if m is not None:
m = 20
return m

View File

@@ -1,3 +1,6 @@
# From Python 3.3.6 hmac.py
# Problem was getting wrong placement of positional args
digest_cons = lambda d=b'': 5
# Handle single kwarg
lambda *, d=0: None

View File

@@ -0,0 +1,14 @@
# From Python 3.4 mailcap
def readmailcapfile(caps):
while 1:
line = 'abc'
if line[0] == '#' or line == '':
continue
key, fields = (1,2)
if not (key and fields):
continue
if key in caps:
caps[key].append(fields)
else:
caps[key] = [fields]
return caps

View File

@@ -0,0 +1,9 @@
# Python 3.5+ PEP 448 - Additional Unpacking Generalizations for dictionaries
{**{}}
{**{'a': 1, 'b': 2}}
## {**{'x': 1}, **{'y': 2}}
# {'c': 1, {'d': 2}, **{'e': 3}}
[*[]]
{**{0:0 for a in b}}
## {**{}, **{}}
## {**{}, **{}, **{}}

View File

@@ -0,0 +1,3 @@
async def a(b, c):
async for b in c:
pass

View File

@@ -0,0 +1,32 @@
# Python 3.5+ async and await
async def await_test(asyncio):
reader, writer = await asyncio.open_connection(80)
await bar()
async def afor_test():
async for i in [1,2,3]:
x = i
async def afor_else_test():
async for i in [1,2,3]:
x = i
else:
z = 4
async def awith_test():
async with i:
print(i)
async def awith_as_test():
async with 1 as i:
print(i)
async def f(z):
await z
async def g(z):
return await z

View File

@@ -0,0 +1,9 @@
# Bug in Python 3.5 is getting the two star'd arguments right.
def sum(a,b,c,d):
return a + b + c + d
args=(1,2)
sum(*args, *args)
# FIXME: this is handled incorrectly
# (*c,) = (3,4)

View File

@@ -0,0 +1,14 @@
# Python 3.5 and 3.6 break inside a
# while True and if / break
def display_date(loop):
while True:
if loop.time():
break
x = 5
# Another loop to test 3.5 ifelsestmtl grammar rule
while loop:
if x:
True
else:
True

View File

@@ -0,0 +1,5 @@
# Python 3.6's changes for calling functions.
# See https://github.com/rocky/python-uncompyle6/issues/58
# CALL_FUNCTION_EX takes 2 to 3 arguments on the stack: the function, the tuple of positional arguments,
# and optionally the dict of keyword arguments if bit 0 of oparg is 1.
a(*[])

View File

@@ -0,0 +1,2 @@
if __file__:
0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0

View File

@@ -8,3 +8,20 @@ def __init__(self, defaults=None, dict_type=_default_dict,
default_section=DEFAULTSECT,
interpolation=_UNSET):
pass
# From 3.5 sqlalchemy/orm/__init__.py
# Python 3.5 changes the stack position of where * args are (furthest down the stack)
# Python 3.6+ replaces CALL_FUNCTION_VAR_KW with CALL_FUNCTION_EX
def deferred(*columns, **kw):
return ColumnProperty(deferred=True, *columns, **kw)
# From sqlalchemy/sql/selectable.py
class GenerativeSelect():
def __init__(self,
ClauseList,
util,
order_by=None):
self._order_by_clause = ClauseList(
*util.to_list(order_by),
_literal_as_text=5)

View File

@@ -0,0 +1,11 @@
# From 2.6.9 cmd.py
try:
if __file__:
x = 2
x = 3
finally:
if x and __file__:
try:
x = 1
except:
pass

View File

@@ -0,0 +1,8 @@
# From Python-3.5.2/Lib/multiprocessing/connection.py
def PipeClient(address):
while 1:
z = 2
else:
raise
return

View File

@@ -19,8 +19,6 @@ Step 2: Run the test:
test_pyenvlib --mylib --verify # decompile verify 'mylib'
"""
from __future__ import print_function
from uncompyle6 import main, PYTHON3
import os, time, shutil
from fnmatch import fnmatch
@@ -30,10 +28,10 @@ from fnmatch import fnmatch
TEST_VERSIONS=('2.3.7', '2.4.6', '2.5.6', '2.6.9',
'pypy-2.4.0', 'pypy-2.6.1',
'pypy-5.0.1', 'pypy-5.3.1',
'2.7.10', '2.7.11', '2.7.12',
'2.7.10', '2.7.11', '2.7.12', '2.7.13',
'3.0.1', '3.1.5', '3.2.6',
'3.3.5', '3.3.6',
'3.4.2', '3.5.1')
'3.4.2', '3.5.1', '3.6.0')
target_base = '/tmp/py-dis/'
lib_prefix = os.path.join(os.environ['HOME'], '.pyenv/versions')
@@ -106,28 +104,40 @@ def do_tests(src_dir, patterns, target_dir, start_with=None, do_verify=False):
if __name__ == '__main__':
import getopt, sys
do_verify = False
do_coverage = do_verify = False
test_dirs = []
start_with = None
test_options_keys = list(test_options.keys())
test_options_keys.sort()
opts, args = getopt.getopt(sys.argv[1:], '',
['start-with=', 'verify', 'weak-verify', 'all', ] \
['start-with=', 'verify', 'weak-verify',
'coverage', 'all', ] \
+ test_options_keys )
vers = ''
for opt, val in opts:
if opt == '--verify':
do_verify = True
if opt == '--weak-verify':
do_verify = 'weak'
if opt == '--coverage':
do_coverage = True
elif opt == '--start-with':
start_with = val
elif opt[2:] in test_options_keys:
test_dirs.append(test_options[opt[2:]])
triple = test_options[opt[2:]]
vers = triple[-1]
test_dirs.append(triple)
elif opt == '--all':
vers = 'all'
for val in test_options_keys:
test_dirs.append(test_options[val])
if do_coverage:
os.environ['SPARK_PARSER_COVERAGE'] = (
'/tmp/spark-grammar-%s.cover' % vers
)
for src_dir, pattern, target_dir in test_dirs:
if os.path.exists(src_dir):
target_dir = os.path.join(target_base, target_dir)

View File

@@ -27,8 +27,6 @@ Step 2: Run the test:
test_pythonlib.py --mylib --verify # decompile verify 'mylib'
"""
from __future__ import print_function
import getopt, os, py_compile, sys, shutil, tempfile, time
from uncompyle6 import PYTHON_VERSION
@@ -127,8 +125,10 @@ def do_tests(src_dir, obj_patterns, target_dir, opts):
if opts['do_compile']:
compiled_version = opts['compiled_version']
if compiled_version and PYTHON_VERSION != compiled_version:
print("Not compiling: desired Python version is %s but we are running %s" %
(compiled_version, PYTHON_VERSION), file=sys.stderr)
sys.stderr.write("Not compiling: "
"desired Python version is %s "
"but we are running %s" %
(compiled_version, PYTHON_VERSION))
else:
for root, dirs, basenames in os.walk(src_dir):
file_matches(files, root, basenames, PY)
@@ -146,8 +146,8 @@ def do_tests(src_dir, obj_patterns, target_dir, opts):
file_matches(files, dirname, basenames, obj_patterns)
if not files:
print("Didn't come up with any files to test! Try with --compile?",
file=sys.stderr)
sys.stderr.write("Didn't come up with any files to test! "
"Try with --compile?")
exit(1)
os.chdir(cwd)
@@ -161,9 +161,9 @@ def do_tests(src_dir, obj_patterns, target_dir, opts):
except ValueError:
pass
print(time.ctime())
print('Source directory: ', src_dir)
print('Output directory: ', target_dir)
print time.ctime()
print 'Source directory: ', src_dir
print 'Output directory: ', target_dir
try:
_, _, failed_files, failed_verify = \
main(src_dir, target_dir, files, [],
@@ -190,6 +190,7 @@ if __name__ == '__main__':
test_options_keys.sort()
opts, args = getopt.getopt(sys.argv[1:], '',
['start-with=', 'verify', 'weak-verify', 'all', 'compile',
'coverage',
'no-rm'] \
+ test_options_keys )
if not opts: help()
@@ -198,7 +199,8 @@ if __name__ == '__main__':
'do_compile': False,
'do_verify': False,
'start_with': None,
'rmtree' : True
'rmtree' : True,
'coverage' : False
}
for opt, val in opts:
@@ -217,24 +219,30 @@ if __name__ == '__main__':
elif opt == '--all':
for val in test_options_keys:
test_dirs.append(test_options[val])
elif opt == '--coverage':
test_opts['coverage'] = True
else:
help()
pass
pass
if test_opts['coverage']:
os.environ['SPARK_PARSER_COVERAGE'] = (
'/tmp/spark-grammar-python-lib%s.cover' % test_dirs[0][-1]
)
last_compile_version = None
for src_dir, pattern, target_dir, compiled_version in test_dirs:
if os.path.isdir(src_dir):
checked_dirs.append([src_dir, pattern, target_dir])
else:
print("Can't find directory %s. Skipping" % src_dir,
file=sys.stderr)
sys.stderr.write("Can't find directory %s. Skipping" % src_dir)
continue
last_compile_version = compiled_version
pass
if not checked_dirs:
print("No directories found to check", file=sys.stderr)
sys.stderr.write("No directories found to check\n")
sys.exit(1)
test_opts['compiled_version'] = last_compile_version

View File

@@ -41,13 +41,18 @@ PYTHON_VERSION_STR = "%s.%s" % (sys.version_info[0], sys.version_info[1])
IS_PYPY = '__pypy__' in sys.builtin_module_names
sys.setrecursionlimit(5000)
if hasattr(sys, 'setrecursionlimit'):
# pyston doesn't have setrecursionlimit
sys.setrecursionlimit(5000)
import uncompyle6.semantics.pysource
import uncompyle6.semantics.fragments
# Export some functions
from uncompyle6.main import uncompyle_file
from uncompyle6.main import decompile_file
# For compaitility
uncompyle_file = decompile_file
# Conventience functions so you can say:
# from uncompyle6 import deparse_code

Some files were not shown because too many files have changed in this diff Show More