Compare commits

...

182 Commits

Author SHA1 Message Date
rocky
2eaea447eb Get ready for release 2.9.4 2016-11-02 22:44:23 -04:00
rocky
287e98b4b1 Update unpyc3 info. 2016-11-02 20:42:31 -04:00
rocky
63e4c9343f Clean up annotation grammar a little 2016-11-01 15:50:19 -04:00
rocky
eab653afdd Full Python 3 annotations 2016-11-01 12:21:27 -04:00
rocky
7700446bb1 Note github unpyc3 and..
- Add source to bytecode_2.2/03_class_method.pyc
- more ignore
2016-10-30 21:16:33 -04:00
rocky
bfd2f77fbc More source-code line indention in make_function..
and remove Python 3 situations from make_function2()
2016-10-30 10:39:11 -04:00
rocky
1574bf4e1e More annotation processing in to make_function
Move return-value annotation determination from n_mkfunc_annotate to
make_function_annotate which is where other kinds of annotation handling
will also need to be done.
2016-10-29 16:03:02 -04:00
rocky
2328ca7a55 Break out make_function() into its own file.
It is already too complex and will get worse in Python 3.6.

Note: make_function in fragments.py is still inside and
probably needs fixup.
2016-10-29 07:22:58 -04:00
rocky
ccdd37611c More complete annotate handling
Still have a bit of work to do though.
2016-10-28 19:55:17 -04:00
rocky
2e355b6245 Expand annotate return to Python 3.4 2016-10-28 11:33:54 -04:00
rocky
9849f06ff6 Expand annotate handling to 3.3
(and possibly 3.2)

- DRY Python 3.1-3.3 grammar a little
2016-10-28 09:01:41 -04:00
rocky
0e7da031b2 Split out 3.1-3.3 parsers from parser3.py
This is anticipation of extending annotation to Python 3.2+
2016-10-28 07:07:18 -04:00
rocky
25dd67a135 Clean and fix Python 3 annotate arg return 2016-10-27 13:52:07 -04:00
rocky
1a38d3d9aa Dependencies stay within 2nd semantic level 2016-10-26 18:36:12 -04:00
rocky
de65a2c250 Get ready for release 2.9.3 2016-10-26 08:29:09 -04:00
rocky
7daec3352c Start to attack Python 3.1 def() -> xx construct
Start to localize make_function routines by Python version
2016-10-26 08:20:19 -04:00
rocky
8feb472d51 Split out Python 3.1 parser from rest.
__pkginfo__.py: use Python 3.1 bytecode fixes
2016-10-25 21:47:02 -04:00
rocky
7a10917857 Handle Python 3.1 "with ... as" statement 2016-10-25 02:05:10 -04:00
rocky
334f6935b6 Add python 3.1 bytecode testing 2016-10-24 20:49:05 -04:00
rocky
aff920d87b Python 3.1 "with" statement bug 2016-10-24 20:47:12 -04:00
rocky
6319d33fa0 Python 3.1 compile bug. DRY Python 3.x rules ...
via inheritance
2016-10-24 08:50:34 -04:00
rocky
abb61a4d7d Fix some Python 3.1 bugs 2016-10-24 02:16:23 -04:00
Daniel Bradburn
a4c943fe0d Merge pull request #60 from rocky/buildstring
Buildstring
2016-10-22 20:08:30 +02:00
rocky
0480455ae1 DRY Python 3.6 grammar rules 2016-10-22 12:29:36 -04:00
rocky
9b7d978944 Move fstring rules inside a 3.6+ check 2016-10-22 11:48:52 -04:00
rocky
a6befdee09 Merge branch 'buildstring' of github.com:moagstar/python-uncompyle6 into buildstring 2016-10-22 07:23:19 -04:00
rocky
d6f7ef4e17 DRY op_size
Move from scanner{2,3}.py to scanner.py
2016-10-21 07:40:35 -04:00
moagstar
cec80e696c further work on supporting single and multiple fstring decompilation 2016-10-20 20:44:27 +02:00
rocky
0826129112 DRY Python 2.x unmangle_classname
main.py: small typo: Disassembled -> Decompiled
2016-10-20 06:35:01 -04:00
moagstar
7beaa9f36c urther work on fstrings for python 3.6 - there is a new opcode build_string which is used to improve fstring performance, but broke the fstring support in uncompyle 2016-10-19 23:32:15 +02:00
rocky
78ef16e4d7 Change meta data info in uncompyle6:
* Show file size of source when possible,
  i.e. in Python 3.x
* Show full information about python interpreter
  used to decompile
2016-10-15 19:24:05 -04:00
rocky
59b597ea5d Get ready for release 2.9.2 2016-10-15 05:37:56 -04:00
rocky
cb8ffa51d7 Merge branch 'source-indent-respect' 2016-10-14 22:16:24 -04:00
rocky
a7ef513849 Python 2.6- _ifstmst_jump bug 2016-10-14 21:34:55 -04:00
rocky
8e62a48c96 Python 2.6- _ifstmst_jump bug 2016-10-14 21:32:35 -04:00
rocky
6d3b934bf2 Pretty-print constant tuples 2016-10-13 22:59:59 -04:00
rocky
27fb5758bd 2.6 and before: try_middle come_from per except 2016-10-13 20:49:51 -04:00
rocky
5703ccd8b8 2.6 try statement (and below)
They may neeed arbitrary come_froms for each except clause
2016-10-13 19:37:38 -04:00
rocky
20ba165e4a Bug in handling kv, kv2 2016-10-13 06:39:35 -04:00
rocky
7eb9210b0c Start to track line breaks on lists 2016-10-13 06:37:53 -04:00
rocky
fe072d8b57 DRY fragments.py preorder code
pysource.py: doc typo
2016-10-11 22:24:00 -04:00
rocky
f430b0dbe4 Need xdis 3.0.2 for Python 1.5 bug fixes 2016-10-11 02:41:44 -04:00
rocky
9ef670c872 Fix Python 1.5 bytecode deparse
Need xdis 3.0.2 though since the bug is really there.
2016-10-11 02:39:09 -04:00
rocky
c04fe00e50 Fix python 1.5 decompile bugs ...
add bytecode 1.5 tests from decompyle
2016-10-11 02:30:08 -04:00
rocky
52691c4e8a Map expression with source-directed linebreaks ...
for Python 3.4 and 3.5
2016-10-10 21:12:44 -04:00
rocky
f067148b6c Start using source linebreaks in formatting...
.. large literals like maps and dicts, and tuples
2016-10-10 18:28:02 -04:00
rocky
bb8d0a6389 Get ready for release 2.9.1 2016-10-10 12:29:57 -04:00
rocky
97ce330b00 Python 1.5 scanner and parser and ..
.. slightly improved Forgot to check in files before.

Handle Python pre 2.2. classes
2016-10-10 12:09:00 -04:00
rocky
8b240a80e7 Get ready for release 2.9.0 ...
- Use xdis 3.0.0 protocol load_module. Needs bump in requirements.txt
  and _pkg_info_.py
- Start Python 1.5 decompiling - another round of work is needed to remove
  bugs
- small cleanups
2016-10-10 09:33:49 -04:00
rocky
e888a87d15 Reinstate some tests 2016-10-09 06:44:08 -04:00
rocky
f41426e94b Simpify python 2.1 grammar Fix bug with -t ...
Wasn't showing source text when -t option was given
2016-10-08 21:52:10 -04:00
rocky
22dee55ff7 Python 2.1-2.6 bug in list comprehension 2016-10-08 19:51:57 -04:00
rocky
8916447adb Get ready for release 2.8.4 2016-10-08 06:30:44 -04:00
rocky
478602bfa5 Add decompyle's 2.1 tests
Update README.rst to note new pythons supported since last update
2016-10-08 05:54:35 -04:00
rocky
056f600da1 Start Python 2.1 bytecode decompile 2016-10-07 22:42:30 -04:00
rocky
2e1bd2dc13 Use .py extension in disassembled files more often 2016-10-06 04:05:38 -04:00
rocky
346d8678d2 Merge in COME_FROM disambiguation ...
from controlflowbranch. Should probably expand to Python2
parsers as well
2016-10-06 03:27:37 -04:00
rocky
4411ecceba Merge branch 'controlflow' 2016-10-06 03:26:44 -04:00
rocky
a4dd6e9805 Python 3: "or" doesn't have optional come_from 2016-10-05 22:56:48 -04:00
rocky
b52f341d46 Python 3: "or" doesn't have optional come_from 2016-10-05 22:54:50 -04:00
rocky
2ada29618f Python 3: "and" doesn't have optional come_from 2016-10-05 03:36:57 -04:00
rocky
649e4518fe Python 3: "and" doesn't have optional come_from 2016-10-05 03:34:29 -04:00
rocky
0df2c8b4bb Merge branch 'master' into controlflow 2016-10-05 00:01:04 -04:00
rocky
d2b477ae7d Add tuple comma when not in BUILD_LIST_n
Fixes issue #57

bin/uncompile.py: --verify now works on a single bytecode file. We will
set the output to be something created by tempfile.mktemps
2016-10-04 23:58:35 -04:00
rocky
a8f9f2170f Python 3 while1 grammar rules with COME_FROM_LOOP 2016-09-28 05:52:13 -04:00
rocky
fe46015b78 PY3 COME_FROM_LOOP bug
There are still more in sre...
2016-09-27 04:59:06 -04:00
rocky
051efb80f5 PY3: COME_FROM -> COME_FROM_FINALLY as appropriate
We now have all WITH_ op jumps associated in the grammar

Also, some COME_FROM_LOOP grammar fixes
2016-09-27 03:27:41 -04:00
rocky
1fc8ac4700 Interval order COME_FROMs in Python3
This bug had possibly caused lots of grammar pollution which may need
addressing.

We want to process COME_FROMs to the same offset to be in *descending*
order so we have the larger range or biggest instruction interval
last. (I think they are sorted in increasing order, but for safety
we sort them). That way, specific COME_FROM tags will match up
properly. For example, a "loop" with an "if" nested in it should have
the "loop" tag last so the grammar rule matches that properly

Adjust Python 3 grammar for more COME_FROM -> COME_FROM_LOOP. And
remove optional COME_FROM_LOOP where possible. Previously, the
optional-ness was a result of inner nestings gobbling up the
COME_FROM.

We'll probably want to go back and fix this up in Python2.
2016-09-26 09:26:51 -04:00
rocky
c87710dd4b Python 3 COME_FROM -> COME_FROM_WITH appropriately 2016-09-26 08:09:05 -04:00
rocky
ccd129b377 Try stronger verification
verify.py: add check in verification that magic is the same.
Otherwise we go for weak verification.
2016-09-25 17:48:33 -04:00
rocky
c03a8186b5 Address whileTrue if cond .. -> while cond .. bug
Reinstate Python standard library bisect test
2016-09-25 16:59:32 -04:00
rocky
8bc76f19b9 Tidy use of SETUP_ ops
Also possibly remove a bug in decompiling Python 3.0 .. 3.2 which doesn't
have SETUP_WITH
2016-09-23 19:22:09 -04:00
rocky
6908898b90 Tidy use of SETUP_ ops
Also possibly remove a bug in decompiling Python 3.0 .. 3.2 which doesn't
have SETUP_WITH
2016-09-23 19:18:54 -04:00
rocky
fa1a6347e9 Weaken testing for controlfow branch
We have more equivalent programs now
2016-09-23 04:58:29 -04:00
rocky
760532b218 Add COME_FROM_LOOP
Note: we have regressed in --verify and some tests,
but I believe that's because we are producing more equivalant
(if uglier) programs. That's a separate problem though.
2016-09-22 08:24:02 -04:00
rocky
88f2ad1f5a Merge branch 'master' into controlflow 2016-09-22 01:33:29 -04:00
rocky
d271e886d9 Fix pytest grammar test failure
dict and set comprehensions are only in 2.7+
2016-09-22 01:32:00 -04:00
rocky
c0b4a5e703 Fix some Test failures
Fix py.test pytest/test_grammar.py -

Comprehension sorts of things removed from parser that
don't exist in earlier pythons

scanners/tok.py Add back in "to" when needed
2016-09-22 01:27:58 -04:00
rocky
b6dee24289 First step towards managing control flow decoding 2016-09-21 21:30:57 -04:00
R. Bernstein
0e9ebca1a5 Merge pull request #56 from rocky/fstring
merge fstring changes from moagstar
2016-09-21 20:44:35 -04:00
moagstar
f3b9a5936e fixed generation of FSTRING_CONVERSION_MAP in < python 2.7 2016-09-21 22:13:25 +02:00
moagstar
bae3d2e361 merge fstring changes from moagstar 2016-09-21 22:04:46 +02:00
rocky
4f83a87a00 Python 2 & 3 scanner code ever so slightly closer 2016-09-21 07:06:30 -04:00
rocky
d8b7c3d813 Small NEWS doc typo 2016-09-21 03:26:44 -04:00
rocky
c119965d96 Small changes 2016-09-18 17:23:49 -04:00
rocky
b02754c954 Get ready for release 2.8.3 2016-09-11 19:22:07 -04:00
rocky
23770fca64 Tidy a bit 2016-09-11 03:39:19 -04:00
rocky
71591152ef 3.0 .. 3.4 bug in whileTrue 2016-09-09 22:27:13 -04:00
rocky
251de4338a 3.1 scanner small fixes 2016-09-09 18:23:43 -04:00
rocky
03e8995d18 ret_cond adjustment for < 2.7 and ...
"<= 2.6" -> "< 2.7" since python 2.6's version is 2.6000001
2016-09-09 15:55:15 -04:00
rocky
dd661bc94a Start accepting Python 3.1 bytecode 2016-09-09 15:32:46 -04:00
rocky
c4e6af6e4f Add 3.6 parser 2016-09-08 05:44:45 -04:00
rocky
57d1f3b9f9 More testing
- travis:
   * 2.7.12 - bump from 2.7.11
   * 3.3.6

test_pyenvlib: add --weak-verify to
Makefile: check-3.2.6 needs --weak-verify
2016-09-08 05:09:36 -04:00
rocky
2d11ffb669 Python 3.0-3.2 *args processing 2016-09-08 03:54:57 -04:00
rocky
b7f1f1b028 3.2 (and down to 3.0?) bug in fn name and kwargs 2016-09-08 02:35:06 -04:00
rocky
0fce4c6dc3 Another 3.5+ erroneous RETURN_END_IF misclassify 2016-09-07 04:03:21 -04:00
rocky
c44d4898cb But in Python 3.5+ erroneosly adding RETURN_END_IF 2016-09-07 03:37:48 -04:00
rocky
fecae9f902 Fix 3.3 named bug and ...
Parse 3.4 parameters correctly.
Allow test_pyenvlib to do 3.3.6
2016-09-06 02:15:47 -04:00
rocky
60b25f7596 Python 3.x bug in getting parameter of ** argument 2016-09-05 22:33:59 -04:00
rocky
318311818e Python 2.6- try/except control flow detection 2016-09-05 04:02:37 -04:00
rocky
c7788e4545 disassemble -> ingest where appropriate
As part of tokenization for (de)parsing, we need to do something like a
disassembly, but is is really a little different.

Disassembly, strictly speaking, is done by the xdis module now.
What "ingestion" does is massage the instruction tokens to a form that is
more amenable for parsing.

In sum, ingestion is different than disassembly, although disassembly is
generally the first part of ingestion.
2016-09-04 11:43:02 -04:00
rocky
979bca4fe0 Small cleanup/code optimization 2016-09-04 10:02:20 -04:00
rocky
47a56d3387 Python 2.3--2.6 bug in parsing try-middle
Note: it looks like COME_FROMs need to be classified better
with respect to whether they are from an except or not
we are getting if/else vs except nesting errors.
2016-09-04 09:28:47 -04:00
rocky
647248dfc8 A couple more 2.6 (and below) bugs fixed
* Detect "return None" inside if statement
* another case of triple ==, ==, ==
     scanner2.py: detect_structure: descriminate more on parent type
2016-09-04 04:10:08 -04:00
rocky
f4ac13ef0f Python 2.5-2.6 generator bug
Adjust for variable position of list iteration node in a genexpr node
2016-09-03 21:37:32 -04:00
rocky
8f95ec9882 Python 2.7 "return None" bug
Same as 2.3-2.6 "return None".
2016-09-03 10:27:21 -04:00
rocky
75c718bc5c Python 2.2..2.6 bug in a == b == c == d
Fix was to remove some come froms. Feels a little hacky though.
2016-09-03 10:05:55 -04:00
rocky
b6fd9088b8 Python 2.3..2.6 "return" bug
In Python 2.6 and possibly down to 2.3 we need to issue "return" not
"return None" inside a generator. So check for that "return None"
inside n_return and issue "return" for that.
2016-09-03 08:04:52 -04:00
rocky
4d046eb0bd Add hartmut Goebel's changes before 2.4 2016-09-03 00:50:18 -04:00
rocky
52a35e6c62 Fix Python 2.4-2.6 comp_for text generation...
Makefile: tolerate pypy 5.3.x
Rest: fix semantic action rule for comp_for and test this
2016-09-03 00:30:48 -04:00
rocky
f1bb40f485 Python 2.6- bug: RETURN_ENDIF, POP_TOP ..
POP_TOP should be excluded as a potentional statement beginning
2016-09-02 21:08:44 -04:00
rocky
136f935e26 Fix Python 3.x named param and kwargs bug 2016-09-02 06:30:39 -04:00
rocky
f5eeed6759 2.6- bug: while..and: stmt - on one line
If 2.6 or before POP_BLOCK after a JUMP_IF_FALSE does not
constitute a new statement. The POP_BLOCK is really part
of the JUMP_IF_FALSE. In Python 2.7+ it's a single op.
2016-09-01 20:44:17 -04:00
rocky
1d567d5d9a Handle Python 2.6 and below "except <cond>, <var>" 2016-09-01 02:20:07 -04:00
rocky
c9f364df9f Python 3.x bug in handling var number of args 2016-08-31 06:20:29 -04:00
rocky
6189ce3c04 Bug in 3.x detecting "if" structure and ...
scanner3.py: bug in 3.x detecting "if" structure
Make scanner2.py look more like scanner3.py
verify.py: add weak-verify which tests Pytyon syntax, but not code
2016-08-31 04:07:42 -04:00
rocky
6f2cdc164d Handle Python 2 vs 3 raise syntax change
raise_stmt ::=  "raise" expression "," expression
becomes:
   raise_stmt ::=  "raise" expression from expression

raise expr, expr -> raise
2016-08-30 00:42:24 -04:00
rocky
e4cc126b38 Get ready for release 2.8.2 2016-08-29 21:48:21 -04:00
rocky
da458bdce7 Correct PYPY bit logic in previous commit 2016-08-27 20:19:12 -04:00
rocky
f47aecae9f PYPY bugs and inspired changes ...
verify.py: Show co_flags when different.
pysource.py: PYPY also generates normal tryfinallystmt code
test_pyenvlib.py: allow pypy-5.3.1
2016-08-27 19:32:42 -04:00
rocky
ddc5460030 Start to handle Python 3.1 bytecode 2016-08-26 20:50:08 -04:00
rocky
835c4151c3 Bump min requirement versions
xdis we need increased so we don't catch old xdis bugs
2016-08-26 08:02:27 -04:00
rocky
1087613a27 Handle 3.6 Format String conversions !r, !s, !a 2016-08-25 07:26:01 -04:00
rocky
a67891c563 Get ready for release 2.8.1 2016-08-20 22:14:36 -04:00
rocky
31413be7a1 Python 2.2 doesn't have opcode LIST_APPEND 2016-08-16 12:45:43 -04:00
rocky
98a6f47ad6 Python 2.2 scanner bug: don't mung IMPORT_NAME op 2016-08-16 06:38:09 -04:00
rocky
2e3e6658ee Small pypy LOOKUP_METHOD cleanups 2016-08-16 06:09:10 -04:00
R. Bernstein
85c562cb36 Merge pull request #49 from moagstar/master
Fixed a bug with FORMAT_VALUE with sub expressions.
2016-08-16 05:33:09 -04:00
moagstar
5ab3e52c9c disable test_format_specifiers on python < 3.6 since this is only required for the fstring tests and was causing failures on 2.7 2016-08-16 08:38:38 +02:00
DanielBradburn
004ce5c491 Fixed bug with FORMAT_VALUE where a sub expression would not be correctly interpreted 2016-08-16 08:24:06 +02:00
rocky
599ceddd08 xdis 2.1.1 removes some bugs encountered here 2016-08-14 23:11:55 -04:00
rocky
6547d0230f Merge branch 'master' of github.com:rocky/python-uncompyle6 2016-08-14 23:00:16 -04:00
rocky
a65443ee02 Fix pypy semantic action bug in CALL_METHOD 2016-08-14 22:59:34 -04:00
DanielBradburn
2bd850f297 added examples for known failures 2016-08-14 20:44:23 +02:00
R. Bernstein
90477edf04 Merge pull request #45 from rocky/revert-43-patch-1
Revert "Cache pip installation in travis"
2016-08-14 07:54:46 -04:00
R. Bernstein
c912d16b50 Merge pull request #46 from rocky/revert-44-patch-2
Revert "Test with latest PyPy in Travis"
2016-08-14 07:53:59 -04:00
R. Bernstein
8dd405a5ee Revert "Test with latest PyPy in Travis" 2016-08-14 07:53:47 -04:00
R. Bernstein
116a22a425 Merge pull request #44 from thedrow/patch-2
Test with latest PyPy in Travis
2016-08-14 07:53:40 -04:00
R. Bernstein
7d771b9a8c Revert "Cache pip installation in travis" 2016-08-14 07:46:55 -04:00
R. Bernstein
a1972bbc08 Merge pull request #43 from thedrow/patch-1
Cache pip installation in travis
2016-08-14 07:41:46 -04:00
Omer Katz
8a91081535 Fix indentation. 2016-08-14 11:03:49 +03:00
rocky
0958dc889d Back off of 3.6 testing for now 2016-08-13 23:33:23 -04:00
rocky
33a0c75b69 Merge branch 'master' of github.com:rocky/python-uncompyle6 2016-08-13 20:26:52 -04:00
rocky
7ccbd419c6 Correct SET_LINENO handling in Python 2.2
Add more  2.2 tests
2016-08-13 20:25:19 -04:00
rocky
a45ee15cf2 I said - we test 2.2 now. 2016-08-13 09:14:33 -04:00
rocky
fb5ad76c4e Include Python 2.2 in testing 2016-08-13 09:07:15 -04:00
rocky
d8598f61e4 Start handling Python 2.2 bytecode and...
Fix some bugs in Python 2.3-2.5 bytecode handling
2016-08-13 07:16:50 -04:00
Omer Katz
5f52cce24d Test with latest PyPy. 2016-08-11 11:15:33 +03:00
Omer Katz
70463e036a Cache pip installation 2016-08-11 11:13:58 +03:00
DanielBradburn
7fba24198f small formatting change 2016-08-10 22:29:02 +02:00
DanielBradburn
e06a90ed27 added hypothesis to requirements-dev 2016-08-10 22:27:54 +02:00
DanielBradburn
d030a04c1a added hypothesis test (currently failing due to limited support) for testing fstring uncompyling 2016-08-10 22:20:43 +02:00
Daniel Bradburn
37d5a05241 Merge pull request #2 from rocky/master
Merging pypy and cpython 3.6 from rocky
2016-08-10 08:22:15 +02:00
DanielBradburn
5d27832d6f resolve merge conflicts from pull request 2016-08-10 08:20:51 +02:00
rocky
6b98432082 Merge branch 'master' of github.com:rocky/python-uncompyle6 2016-08-03 08:21:44 -04:00
rocky
109e813058 Get ready for release 2.8.0 2016-08-03 08:20:32 -04:00
rocky
4b8cb11d77 Need recent xdis fix for 3.6 wordcode 2016-08-01 11:02:29 -04:00
rocky
c77e9cdaf8 Add Python 3.6 formatted_str test 2016-08-01 09:47:45 -04:00
R. Bernstein
4c2f0df3dc Merge pull request #41 from rocky/3.6
Move forward on moagstar's Python 3.6 support
2016-08-01 09:16:45 -04:00
rocky
b49d30266f Tidy assembly output a little more 2016-08-01 08:42:52 -04:00
rocky
65a16327ce Moagstar's 3.6 wordcode + formattedValue rules 2016-08-01 03:16:26 -04:00
rocky
fff09db66e WIP try to keep line numbers the same 2016-07-30 14:25:43 -04:00
rocky
3ef0325cb8 Small changes 2016-07-29 19:54:16 -04:00
rocky
3a6f9d8f24 Fix 3.5 misclassifying RETURN_VALUE
We use location of SETUP_EXCEPT instructions to disambiguate.
2016-07-29 08:56:23 -04:00
Daniel Bradburn
d14865c1be Starting adding python 3.6 support to uncompyle 2016-07-28 09:07:12 +02:00
rocky
152935ab26 while1 bug applied to Python 2.3 and 2.4 2016-07-28 01:23:29 -04:00
rocky
5c9c0228ee PyPy 3.2 bug confusing RETURN_END_IF for except
Also fix a instruction formatting bug
2016-07-28 00:41:40 -04:00
rocky
ac121076e6 Split out 3.5 parser 2016-07-27 19:09:25 -04:00
rocky
04ae94ee9e Add python 3.4 grammar checking
DRY grammar testing
2016-07-27 18:32:15 -04:00
rocky
e8ed17967c Clean and check Python 2.6 grammar 2016-07-27 18:03:07 -04:00
rocky
3f7c4209d9 Start to segregate and clean up grammar 2016-07-27 17:35:21 -04:00
rocky
f33f425692 Add is_pypy parameter to places that need it 2016-07-27 16:08:40 -04:00
rocky
5ffd9b2be7 2.6 and 2.7 while1 grammar rule
Fixes issue #40
2016-07-27 13:19:42 -04:00
rocky
87dc5ad80c Start grammar checker 2016-07-27 11:02:01 -04:00
rocky
177a422b87 Show magic number in output
Fix bugs due to removal of token.format()
2016-07-27 09:44:05 -04:00
rocky
3a78332d59 tok.format -> tok.__str__; simplify pypy code 2016-07-27 09:26:39 -04:00
rocky
5e801b5d74 Python 2.7 set comprehension bug 2016-07-27 08:36:33 -04:00
rocky
2523b340cd separate semantic action version differences
Added customize_for_version which
uses is_pypy and version to adjust tables
2016-07-27 07:32:55 -04:00
rocky
c3f6fa32db Customize tables better for specific Python versions 2016-07-27 06:56:02 -04:00
rocky
6dc9d3ab2f Small code clean up 2016-07-27 06:01:49 -04:00
rocky
74f440bd0b Usuability fixes
* try using format for __str__
* Explicitly nuke self.attr and self.pattr when no arg
* Sync pysource and format wrt make_function
2016-07-26 20:49:41 -04:00
Daniel Bradburn
1be53ca729 Merge pull request #1 from rocky/master
Syncing with rocky
2016-07-24 16:56:36 +02:00
320 changed files with 71176 additions and 1122 deletions

3
.gitignore vendored
View File

@@ -1,7 +1,7 @@
*.pyo
*.pyc
*_dis
*~
*.pyc
/.cache
/.eggs
/.python-version
@@ -13,5 +13,6 @@
/nose-*.egg
/tmp
/uncompyle6.egg-info
/unpyc
__pycache__
build

View File

@@ -4,8 +4,9 @@ sudo: false
python:
- '3.5'
- '2.7.11'
- '2.7.12'
- '2.6'
- '3.3'
- '3.4'
install:

978
ChangeLog
View File

@@ -1,6 +1,978 @@
2016-11-02 rocky <rb@dustyfeet.com>
* __pkginfo__.py, uncompyle6/version.py: Get ready for release 2.9.4
2016-11-02 rocky <rb@dustyfeet.com>
* README.rst: Update unpyc3 info.
2016-11-01 rocky <rb@dustyfeet.com>
* pytest/test_grammar.py, uncompyle6/parsers/parse3.py,
uncompyle6/parsers/parse31.py, uncompyle6/parsers/parse32.py,
uncompyle6/semantics/make_function.py: Clean up annotation grammar a
little
2016-11-01 rocky <rb@dustyfeet.com>
* test/simple_source/bug31/04_def_annotate.py,
uncompyle6/semantics/make_function.py: Full Python 3 annotations
2016-10-30 rocky <rb@dustyfeet.com>
* .gitignore, README.rst, test/simple_source/def/03_class_method.py:
Note github unpyc3 and.. - Add source to bytecode_2.2/03_class_method.pyc - more ignore
2016-10-30 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/make_function.py: More source-code line
indention in make_function.. and remove Python 3 situations from make_function2()
2016-10-29 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/make_function.py,
uncompyle6/semantics/pysource.py: More annotation processing in to
make_function Move return-value annotation determination from n_mkfunc_annotate to
make_function_annotate which is where other kinds of annotation
handling will also need to be done.
2016-10-29 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/fragments.py,
uncompyle6/semantics/make_function.py,
uncompyle6/semantics/parser_error.py,
uncompyle6/semantics/pysource.py: Break out make_function() into its
own file. It is already too complex and will get worse in Python 3.6. Note: make_function in fragments.py is still inside and probably
needs fixup.
2016-10-28 rocky <rb@dustyfeet.com>
* pytest/test_grammar.py, uncompyle6/parsers/parse3.py,
uncompyle6/parsers/parse31.py, uncompyle6/parsers/parse32.py,
uncompyle6/parsers/parse35.py, uncompyle6/semantics/pysource.py:
More complete annotate handling Still have a bit of work to do though.
2016-10-28 rocky <rb@dustyfeet.com>
* pytest/test_grammar.py, uncompyle6/parsers/parse3.py,
uncompyle6/parsers/parse32.py, uncompyle6/parsers/parse33.py,
uncompyle6/parsers/parse34.py, uncompyle6/semantics/pysource.py:
Expand annotate return to Python 3.4
2016-10-28 rocky <rb@dustyfeet.com>
* pytest/test_grammar.py, uncompyle6/parsers/parse31.py,
uncompyle6/parsers/parse32.py, uncompyle6/semantics/pysource.py:
Expand annotate handling to 3.3 (and possibly 3.2) - DRY Python 3.1-3.3 grammar a little
2016-10-28 rocky <rb@dustyfeet.com>
* uncompyle6/parser.py, uncompyle6/parsers/parse3.py,
uncompyle6/parsers/parse31.py, uncompyle6/parsers/parse32.py,
uncompyle6/parsers/parse33.py, uncompyle6/parsers/parse35.py: Split
out 3.1-3.3 parsers from parser3.py This is anticipation of extending annotation to Python 3.2+
2016-10-27 rocky <rb@dustyfeet.com>
* test/simple_source/bug31/04_def_annotate.py,
test/simple_source/bug31/04_def_attr.py,
uncompyle6/parsers/parse31.py, uncompyle6/semantics/pysource.py:
Clean and fix Python 3 annotate arg return
2016-10-26 rocky <rb@dustyfeet.com>
* __pkginfo__.py: Dependencies stay within 2nd semantic level
2016-10-26 rocky <rb@dustyfeet.com>
* ChangeLog, NEWS, uncompyle6/version.py: Get ready for release
2.9.3
2016-10-26 rocky <rb@dustyfeet.com>
* test/simple_source/bug31/04_def_attr.py,
uncompyle6/parsers/parse31.py, uncompyle6/scanner.py,
uncompyle6/semantics/pysource.py: Start to attack Python 3.1 def()
-> xx construct Start to localize make_function routines by Python version
2016-10-25 rocky <rb@dustyfeet.com>
* __pkginfo__.py, uncompyle6/parser.py,
uncompyle6/parsers/parse3.py, uncompyle6/parsers/parse31.py: Split
out Python 3.1 parser from rest. __pkginfo__.py: use Python 3.1 bytecode fixes
2016-10-25 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse3.py: Handle Python 3.1 "with ... as"
statement
2016-10-24 rocky <rb@dustyfeet.com>
* test/Makefile: Add python 3.1 bytecode testing
2016-10-24 rocky <rb@dustyfeet.com>
* test/simple_source/stmts/07_withstmt_fn.py,
uncompyle6/parsers/parse3.py: Python 3.1 "with" statement bug
2016-10-24 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse3.py, uncompyle6/parsers/parse34.py,
uncompyle6/parsers/parse35.py: Python 3.1 compile bug. DRY Python
3.x rules ... via inheritance
2016-10-24 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse3.py, uncompyle6/scanners/scanner3.py: Fix
some Python 3.1 bugs
2016-10-22 Daniel Bradburn <moagstar@gmail.com>
* : Merge pull request #60 from rocky/buildstring Buildstring
2016-10-22 rocky <rb@dustyfeet.com>
* pytest/test_fstring.py, test/simple_source/bug36/01_fstring.py,
uncompyle6/semantics/pysource.py: Move fstring rules inside a 3.6+
check
2016-10-22 rocky <rb@dustyfeet.com>
* : commit d6f7ef4e178e04d9a612d3a6c0b77a008732357f Author: rocky
<rb@dustyfeet.com> Date: Fri Oct 21 07:40:35 2016 -0400
2016-10-20 moagstar <moagstar@gmail.com>
* pytest/test_fstring.py, uncompyle6/parsers/parse3.py,
uncompyle6/parsers/parse36.py, uncompyle6/semantics/pysource.py:
further work on supporting single and multiple fstring decompilation
2016-10-20 rocky <rb@dustyfeet.com>
* uncompyle6/main.py, uncompyle6/scanners/scanner2.py,
uncompyle6/scanners/scanner26.py: DRY Python 2.x unmangle_classname main.py: small typo: Disassembled -> Decompiled
2016-10-19 moagstar <moagstar@gmail.com>
* pytest/test_fstring.py, uncompyle6/parsers/parse3.py,
uncompyle6/parsers/parse36.py, uncompyle6/semantics/pysource.py:
urther work on fstrings for python 3.6 - there is a new opcode
build_string which is used to improve fstring performance, but broke
the fstring support in uncompyle
2016-10-15 rocky <rb@dustyfeet.com>
* uncompyle6/main.py: Change meta data info in uncompyle6: * Show file size of source when possible, i.e. in Python 3.x * Show full information about python interpreter used to decompile
2016-10-15 rocky <rb@dustyfeet.com>
* ChangeLog, NEWS, __pkginfo__.py, requirements.txt,
uncompyle6/version.py: Get ready for release 2.9.2
2016-10-14 rocky <rb@dustyfeet.com>
* : commit a7ef513849cc2564b35636d3865561242e46e63f Author: rocky
<rb@dustyfeet.com> Date: Fri Oct 14 21:34:55 2016 -0400
2016-10-14 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse26.py: Python 2.6- _ifstmst_jump bug
2016-10-13 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/pysource.py: Pretty-print constant tuples
2016-10-13 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse26.py, uncompyle6/scanners/scanner2.py,
uncompyle6/scanners/scanner26.py: 2.6 and before: try_middle
come_from per except
2016-10-13 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse26.py, uncompyle6/scanners/scanner2.py,
uncompyle6/scanners/scanner26.py: 2.6 try statement (and below) They may neeed arbitrary come_froms for each except clause
2016-10-13 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/pysource.py: Bug in handling kv, kv2
2016-10-13 rocky <rb@dustyfeet.com>
* test/simple_source/stmts/08_source_linebreaks.py,
uncompyle6/semantics/pysource.py: Start to track line breaks on
lists
2016-10-11 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/fragments.py,
uncompyle6/semantics/pysource.py: DRY fragments.py preorder code pysource.py: doc typo
2016-10-11 rocky <rb@dustyfeet.com>
* __pkginfo__.py, requirements.txt: Need xdis 3.0.2 for Python 1.5
bug fixes
2016-10-11 rocky <rb@dustyfeet.com>
* __pkginfo__.py, requirements.txt, test/Makefile,
test/test_pythonlib.py, uncompyle6/scanners/scanner26.py: Fix Python
1.5 bytecode deparse Need xdis 3.0.2 though since the bug is really there.
2016-10-11 rocky <rb@dustyfeet.com>
* test/Makefile, test/test_pythonlib.py,
uncompyle6/scanners/scanner26.py: Fix python 1.5 decompile bugs ... add bytecode 1.5 tests from decompyle
2016-10-10 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/fragments.py,
uncompyle6/semantics/pysource.py: Map expression with
source-directed linebreaks ... for Python 3.4 and 3.5
2016-10-10 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/fragments.py,
uncompyle6/semantics/pysource.py: Start using source linebreaks in
formatting... .. large literals like maps and dicts, and tuples
2016-10-10 rocky <rb@dustyfeet.com>
* ChangeLog, NEWS, README.rst, __pkginfo__.py, requirements.txt,
uncompyle6/version.py: Get ready for release 2.9.1
2016-10-10 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse15.py, uncompyle6/scanners/scanner15.py,
uncompyle6/semantics/pysource.py: Python 1.5 scanner and parser and
.. .. slightly improved Forgot to check in files before. Handle Python pre 2.2. classes
2016-10-10 rocky <rb@dustyfeet.com>
* ChangeLog, NEWS, __pkginfo__.py, requirements.txt,
uncompyle6/disas.py, uncompyle6/main.py, uncompyle6/parser.py,
uncompyle6/parsers/parse2.py, uncompyle6/scanner.py,
uncompyle6/scanners/scanner2.py, uncompyle6/scanners/scanner26.py,
uncompyle6/verify.py, uncompyle6/version.py: Get ready for release
2.9.0 ... - Use xdis 3.0.0 protocol load_module. Needs bump in
requirements.txt and _pkg_info_.py - Start Python 1.5 decompiling - another round of work is needed to
remove bugs - small cleanups
2016-10-09 rocky <rb@dustyfeet.com>
* : Reinstate some tests
2016-10-08 rocky <rb@dustyfeet.com>
* uncompyle6/bin/uncompile.py, uncompyle6/parsers/parse21.py,
uncompyle6/semantics/pysource.py: Simpify python 2.1 grammar Fix bug
with -t ... Wasn't showing source text when -t option was given
2016-10-08 rocky <rb@dustyfeet.com>
* test/simple_source/bug22/06_list_comprehension_x2_if.py,
uncompyle6/parsers/parse22.py, uncompyle6/parsers/parse26.py: Python
2.1-2.6 bug in list comprehension
2016-10-08 rocky <rb@dustyfeet.com>
* ChangeLog, NEWS, uncompyle6/version.py: Get ready for release
2.8.4
2016-10-08 rocky <rb@dustyfeet.com>
* README.rst, test/Makefile, test/decompyle/README.md,
test/decompyle/test_applyEquiv.py,
test/decompyle/test_augmentedAssign.py,
test/decompyle/test_class.py, test/decompyle/test_del.py,
test/decompyle/test_divide_future.py,
test/decompyle/test_divide_no_future.py,
test/decompyle/test_docstring.py, test/decompyle/test_empty.py,
test/decompyle/test_exceptions.py, test/decompyle/test_exec.py,
test/decompyle/test_expressions.py,
test/decompyle/test_extendedImport.py,
test/decompyle/test_extendedPrint.py,
test/decompyle/test_extendedarg.py,
test/decompyle/test_functions.py, test/decompyle/test_global.py,
test/decompyle/test_globals.py, test/decompyle/test_import.py,
test/decompyle/test_import_as.py, test/decompyle/test_integers.py,
test/decompyle/test_iterators.py, test/decompyle/test_lambda.py,
test/decompyle/test_listComprehensions.py,
test/decompyle/test_loops.py, test/decompyle/test_loops2.py,
test/decompyle/test_misc.py, test/decompyle/test_nested_elif.py,
test/decompyle/test_nested_scopes.py,
test/decompyle/test_prettyprint.py, test/decompyle/test_print.py,
test/decompyle/test_print_to.py,
test/decompyle/test_single_stmt.py, test/decompyle/test_slices.py,
test/decompyle/test_tuple_params.py, test/decompyle/test_tuples.py,
test/decompyle/test_yield.py, test/ok_lib2.7/compiler/syntax.py: Add
decompyle's 2.1 tests Update README.rst to note new pythons supported since last update
2016-10-07 rocky <rb@dustyfeet.com>
* __pkginfo__.py, test/test_pythonlib.py, uncompyle6/parser.py,
uncompyle6/parsers/parse2.py, uncompyle6/parsers/parse21.py,
uncompyle6/scanner.py, uncompyle6/scanners/scanner21.py,
uncompyle6/scanners/scanner22.py, uncompyle6/scanners/scanner27.py,
uncompyle6/semantics/pysource.py: Start Python 2.1 bytecode
decompile
2016-10-06 rocky <rb@dustyfeet.com>
* uncompyle6/main.py: Use .py extension in disassembled files more
often
2016-10-06 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/controlflow.py,
uncompyle6/scanners/scanner3.py: Merge in COME_FROM disambiguation
... from controlflowbranch. Should probably expand to Python2 parsers as
well
2016-10-06 rocky <rb@dustyfeet.com>
* : commit a4dd6e980566a83270b9b9d47fb7f3cb3e5e385a Author: rocky
<rb@dustyfeet.com> Date: Wed Oct 5 22:56:48 2016 -0400
2016-10-05 rocky <rb@dustyfeet.com>
* uncompyle6/parser.py, uncompyle6/parsers/parse2.py,
uncompyle6/parsers/parse3.py: Python 3: "or" doesn't have optional
come_from
2016-10-05 rocky <rb@dustyfeet.com>
* uncompyle6/parser.py, uncompyle6/parsers/parse2.py,
uncompyle6/parsers/parse3.py, uncompyle6/scanners/scanner3.py:
Python 3: "and" doesn't have optional come_from
2016-10-05 rocky <rb@dustyfeet.com>
* uncompyle6/parser.py, uncompyle6/parsers/parse2.py,
uncompyle6/parsers/parse3.py, uncompyle6/scanners/scanner3.py:
Python 3: "and" doesn't have optional come_from
2016-10-05 rocky <rb@dustyfeet.com>
* : commit d2b477ae7d5a4c081262cabdccdb314d90b39255 Author: rocky
<rb@dustyfeet.com> Date: Tue Oct 4 23:58:35 2016 -0400
2016-09-27 rocky <rb@dustyfeet.com>
* test/simple_source/looping/04_while1_while1.py,
uncompyle6/parsers/parse3.py: Python 3 while1 grammar rules with
COME_FROM_LOOP
2016-09-27 rocky <rb@dustyfeet.com>
* test/Makefile, test/simple_source/looping/04_while1_while1.py,
uncompyle6/parser.py, uncompyle6/parsers/parse2.py,
uncompyle6/parsers/parse3.py, uncompyle6/scanners/scanner3.py: PY3
COME_FROM_LOOP bug There are still more in sre...
2016-09-27 rocky <rb@dustyfeet.com>
* pytest/test_grammar.py, uncompyle6/parsers/parse3.py,
uncompyle6/scanners/scanner3.py: PY3: COME_FROM -> COME_FROM_FINALLY
as appropriate We now have all WITH_ op jumps associated in the grammar Also, some COME_FROM_LOOP grammar fixes
2016-09-26 rocky <rb@dustyfeet.com>
* HISTORY.md, uncompyle6/parser.py, uncompyle6/parsers/parse2.py,
uncompyle6/parsers/parse3.py, uncompyle6/scanners/scanner3.py:
Interval order COME_FROMs in Python3 This bug had possibly caused lots of grammar pollution which may
need addressing. We want to process COME_FROMs to the same offset to be in
*descending* order so we have the larger range or biggest
instruction interval last. (I think they are sorted in increasing
order, but for safety we sort them). That way, specific COME_FROM
tags will match up properly. For example, a "loop" with an "if"
nested in it should have the "loop" tag last so the grammar rule
matches that properly Adjust Python 3 grammar for more COME_FROM -> COME_FROM_LOOP. And
remove optional COME_FROM_LOOP where possible. Previously, the
optional-ness was a result of inner nestings gobbling up the
COME_FROM. We'll probably want to go back and fix this up in Python2.
2016-09-26 rocky <rb@dustyfeet.com>
* pytest/test_grammar.py, uncompyle6/parsers/parse3.py,
uncompyle6/parsers/parse35.py, uncompyle6/scanners/controlflow.py,
uncompyle6/scanners/scanner3.py: Python 3 COME_FROM ->
COME_FROM_WITH appropriately
2016-09-25 rocky <rb@dustyfeet.com>
* test/Makefile, uncompyle6/verify.py: Try stronger verification verify.py: add check in verification that magic is the same.
Otherwise we go for weak verification.
2016-09-25 rocky <rb@dustyfeet.com>
* test/Makefile, uncompyle6/parsers/parse3.py: Address whileTrue if
cond .. -> while cond .. bug Reinstate Python standard library bisect test
2016-09-23 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner3.py: Tidy use of SETUP_ ops Also possibly remove a bug in decompiling Python 3.0 .. 3.2 which
doesn't have SETUP_WITH
2016-09-23 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner3.py: Tidy use of SETUP_ ops Also possibly remove a bug in decompiling Python 3.0 .. 3.2 which
doesn't have SETUP_WITH
2016-09-23 rocky <rb@dustyfeet.com>
* test/Makefile, test/ok_lib3.4/bisect.py: Weaken testing for
controlfow branch We have more equivalent programs now
2016-09-22 rocky <rb@dustyfeet.com>
* pytest/test_grammar.py, pytest/test_single_compile.py,
test/Makefile, uncompyle6/parsers/parse3.py,
uncompyle6/scanners/scanner3.py: Add COME_FROM_LOOP Note: we have regressed in --verify and some tests, but I believe
that's because we are producing more equivalant (if uglier)
programs. That's a separate problem though.
2016-09-22 rocky <rb@dustyfeet.com>
* : commit d271e886d9e818a871f15d8876d6eb9d1ed452a4 Author: rocky
<rb@dustyfeet.com> Date: Thu Sep 22 01:32:00 2016 -0400
2016-09-22 rocky <rb@dustyfeet.com>
* uncompyle6/parser.py, uncompyle6/parsers/parse2.py,
uncompyle6/parsers/parse26.py, uncompyle6/parsers/parse27.py,
uncompyle6/parsers/parse3.py, uncompyle6/scanners/tok.py: Fix some
Test failures Fix py.test pytest/test_grammar.py - Comprehension sorts of things removed from parser that don't exist
in earlier pythons scanners/tok.py Add back in "to" when needed
2016-09-21 rocky <rb@dustyfeet.com>
* pytest/test_grammar.py, uncompyle6/parsers/parse3.py,
uncompyle6/scanners/controlflow.py,
uncompyle6/scanners/scanner3.py, uncompyle6/scanners/tok.py: First
step towards managing control flow decoding
2016-09-21 R. Bernstein <rocky@users.noreply.github.com>
* : Merge pull request #56 from rocky/fstring merge fstring changes from moagstar
2016-09-21 moagstar <moagstar@gmail.com>
* pytest/test_fstring.py, uncompyle6/parsers/parse3.py,
uncompyle6/semantics/pysource.py: merge fstring changes from
moagstar
2016-09-21 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner2.py, uncompyle6/scanners/scanner26.py:
Python 2 & 3 scanner code ever so slightly closer
2016-09-21 rocky <rb@dustyfeet.com>
* NEWS: Small NEWS doc typo
2016-09-18 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner2.py,
uncompyle6/scanners/scanner26.py, uncompyle6/semantics/pysource.py:
Small changes
2016-09-11 rocky <rb@dustyfeet.com>
* ChangeLog, NEWS, __pkginfo__.py, uncompyle6/version.py: Get ready
for release 2.8.3
2016-09-11 rocky <rb@dustyfeet.com>
* test/bytecode_3.6/fstring.py,
test/bytecode_3.6/fstring_single.py, uncompyle6/parsers/parse35.py:
Tidy a bit
2016-09-09 rocky <rb@dustyfeet.com>
* test/simple_source/looping/10_while.py,
uncompyle6/parsers/parse3.py: 3.0 .. 3.4 bug in whileTrue
2016-09-09 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner31.py: 3.1 scanner small fixes
2016-09-09 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner2.py, uncompyle6/semantics/pysource.py:
ret_cond adjustment for < 2.7 and ... "<= 2.6" -> "< 2.7" since python 2.6's version is 2.6000001
2016-09-09 rocky <rb@dustyfeet.com>
* test/Makefile, test/test_pyenvlib.py, test/test_pythonlib.py,
uncompyle6/scanners/scanner31.py, uncompyle6/verify.py: Start
accepting Python 3.1 bytecode
2016-09-08 rocky <rb@dustyfeet.com>
* uncompyle6/parser.py: Add 3.6 parser
2016-09-08 rocky <rb@dustyfeet.com>
* .travis.yml, test/Makefile,
test/simple_source/expression/06_frozenset.py,
test/test_pythonlib.py: More testing - travis: * 2.7.12 - bump from 2.7.11 * 3.3.6 test_pyenvlib: add --weak-verify to Makefile: check-3.2.6 needs
--weak-verify
2016-09-08 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse3.py, uncompyle6/semantics/pysource.py:
Python 3.0-3.2 *args processing
2016-09-08 rocky <rb@dustyfeet.com>
* test/simple_source/bug32/01_named_and_kwargs.py,
uncompyle6/semantics/pysource.py: 3.2 (and down to 3.0?) bug in fn
name and kwargs
2016-09-07 rocky <rb@dustyfeet.com>
* test/simple_source/bug35/05_return_in_else.py,
uncompyle6/scanners/scanner3.py: Another 3.5+ erroneous
RETURN_END_IF misclassify
2016-09-07 rocky <rb@dustyfeet.com>
* test/simple_source/bug35/05_return_in_else.py,
uncompyle6/scanners/scanner3.py: But in Python 3.5+ erroneosly
adding RETURN_END_IF
2016-09-05 rocky <rb@dustyfeet.com>
* test/simple_source/bug33/02_named_and_kwargs.py,
test/test_pyenvlib.py, uncompyle6/parsers/parse3.py,
uncompyle6/semantics/pysource.py: Fix 3.3 named bug and ... Parse 3.4 parameters correctly. Allow test_pyenvlib to do 3.3.6
2016-09-05 rocky <rb@dustyfeet.com>
* test/simple_source/bug33/02_named_and_kwargs.py: Python 3.x bug in
getting parameter of ** argument
2016-09-05 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse2.py, uncompyle6/parsers/parse26.py,
uncompyle6/parsers/parse27.py, uncompyle6/scanners/scanner2.py,
uncompyle6/scanners/scanner26.py, uncompyle6/scanners/scanner3.py:
Python 2.6- try/except control flow detection
2016-09-04 rocky <rb@dustyfeet.com>
* uncompyle6/disas.py, uncompyle6/main.py, uncompyle6/parser.py,
uncompyle6/scanner.py, uncompyle6/scanners/scanner2.py,
uncompyle6/scanners/scanner22.py, uncompyle6/scanners/scanner26.py,
uncompyle6/scanners/scanner27.py, uncompyle6/scanners/scanner3.py,
uncompyle6/scanners/scanner32.py, uncompyle6/scanners/scanner33.py,
uncompyle6/scanners/scanner34.py, uncompyle6/scanners/scanner35.py,
uncompyle6/scanners/scanner36.py, uncompyle6/semantics/aligner.py,
uncompyle6/semantics/fragments.py,
uncompyle6/semantics/pysource.py, uncompyle6/show.py,
uncompyle6/verify.py: disassemble -> ingest where appropriate As part of tokenization for (de)parsing, we need to do something
like a disassembly, but is is really a little different. Disassembly, strictly speaking, is done by the xdis module now.
What "ingestion" does is massage the instruction tokens to a form
that is more amenable for parsing. In sum, ingestion is different than disassembly, although
disassembly is generally the first part of ingestion.
2016-09-04 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/pysource.py: Small cleanup/code optimization
2016-09-04 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse26.py: Python 2.3--2.6 bug in parsing
try-middle Note: it looks like COME_FROMs need to be classified better with
respect to whether they are from an except or not we are getting
if/else vs except nesting errors.
2016-09-04 rocky <rb@dustyfeet.com>
* test/simple_source/bug26/07_generator_return.py,
uncompyle6/scanners/scanner2.py, uncompyle6/semantics/pysource.py: A
couple more 2.6 (and below) bugs fixed * Detect "return None" inside if statement * another case of triple ==, ==, == scanner2.py: detect_structure: descriminate more on parent type
2016-09-03 rocky <rb@dustyfeet.com>
* test/simple_source/bug26/05_generator.py,
uncompyle6/semantics/pysource.py: Python 2.5-2.6 generator bug Adjust for variable position of list iteration node in a genexpr
node
2016-09-03 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/pysource.py: Python 2.7 "return None" bug Same as 2.3-2.6 "return None".
2016-09-03 rocky <rb@dustyfeet.com>
* test/simple_source/bug26/08_triple_equals.py,
uncompyle6/scanners/scanner2.py: Python 2.2..2.6 bug in a == b == c
== d Fix was to remove some come froms. Feels a little hacky though.
2016-09-03 rocky <rb@dustyfeet.com>
* test/simple_source/bug26/07_generator_return.py,
uncompyle6/semantics/pysource.py: Python 2.3..2.6 "return" bug In Python 2.6 and possibly down to 2.3 we need to issue "return" not
"return None" inside a generator. So check for that "return None"
inside n_return and issue "return" for that.
2016-09-03 rocky <rb@dustyfeet.com>
* DECOMPYLE-2.4-CHANGELOG.txt: Add hartmut Goebel's changes before
2.4
2016-09-03 rocky <rb@dustyfeet.com>
* Makefile, test/simple_source/bug26/04_comp_for.py,
uncompyle6/semantics/pysource.py: Fix Python 2.4-2.6 comp_for text
generation... Makefile: tolerate pypy 5.3.x Rest: fix semantic action rule for
comp_for and test this
2016-09-02 rocky <rb@dustyfeet.com>
* test/simple_source/bug26/06_return_pop.py,
uncompyle6/parsers/parse26.py, uncompyle6/scanners/scanner2.py:
Python 2.6- bug: RETURN_ENDIF, POP_TOP .. POP_TOP should be excluded as a potentional statement beginning
2016-09-02 rocky <rb@dustyfeet.com>
* test/simple_source/bug33/02_named_and_kwargs.py,
uncompyle6/scanners/scanner2.py, uncompyle6/semantics/pysource.py:
Fix Python 3.x named param and kwargs bug
2016-09-01 rocky <rb@dustyfeet.com>
* test/simple_source/bug26/04_while_and_stmt_one_line.py,
uncompyle6/scanners/scanner2.py, uncompyle6/scanners/scanner26.py:
2.6- bug: while..and: stmt - on one line If 2.6 or before POP_BLOCK after a JUMP_IF_FALSE does not constitute
a new statement. The POP_BLOCK is really part of the JUMP_IF_FALSE.
In Python 2.7+ it's a single op.
2016-09-01 rocky <rb@dustyfeet.com>
* test/simple_source/bug26/02_except_as.py,
uncompyle6/parsers/parse26.py, uncompyle6/semantics/pysource.py:
Handle Python 2.6 and below "except <cond>, <var>"
2016-08-31 rocky <rb@dustyfeet.com>
* test/simple_source/bug35/02_fn_varargs.py,
uncompyle6/semantics/pysource.py: Python 3.x bug in handling var
number of args
2016-08-31 rocky <rb@dustyfeet.com>
* test/simple_source/bug33/04_aug_assign.py, test/test_pyenvlib.py,
uncompyle6/main.py, uncompyle6/scanners/scanner2.py,
uncompyle6/scanners/scanner3.py, uncompyle6/verify.py: Bug in 3.x
detecting "if" structure and ... scanner3.py: bug in 3.x detecting "if" structure Make scanner2.py
look more like scanner3.py verify.py: add weak-verify which tests
Pytyon syntax, but not code
2016-08-30 rocky <rb@dustyfeet.com>
* NEWS, test/simple_source/bug26/03_raise_from.py,
test/simple_source/bug33/03_raise_from.py,
uncompyle6/semantics/pysource.py: Handle Python 2 vs 3 raise syntax
change raise_stmt ::= "raise" expression "," expression becomes: raise_stmt ::= "raise" expression from expression raise expr, expr -> raise
2016-08-29 rocky <rb@dustyfeet.com>
* ChangeLog, __pkginfo__.py, uncompyle6/version.py: Get ready for
release 2.8.2
2016-08-27 rocky <rb@dustyfeet.com>
* test/test_pyenvlib.py, uncompyle6/verify.py: Correct PYPY bit
logic in previous commit
2016-08-27 rocky <rb@dustyfeet.com>
* test/test_pyenvlib.py, uncompyle6/semantics/pysource.py,
uncompyle6/verify.py: PYPY bugs and inspired changes ... verify.py: Show co_flags when different. pysource.py: PYPY also
generates normal tryfinallystmt code test_pyenvlib.py: allow
pypy-5.3.1
2016-08-26 rocky <rb@dustyfeet.com>
* Makefile, test/Makefile, uncompyle6/parser.py,
uncompyle6/parsers/parse3.py, uncompyle6/scanner.py,
uncompyle6/scanners/scanner3.py: Start to handle Python 3.1 bytecode
2016-08-26 rocky <rb@dustyfeet.com>
* __pkginfo__.py, requirements.txt: Bump min requirement versions xdis we need increased so we don't catch old xdis bugs
2016-08-25 rocky <rb@dustyfeet.com>
* test/simple_source/bug36/01_fstring.py,
uncompyle6/semantics/pysource.py: Handle 3.6 Format String
conversions !r, !s, !a
2016-08-20 rocky <rb@dustyfeet.com>
* ChangeLog, NEWS, uncompyle6/version.py: Get ready for release
2.8.1
2016-08-16 rocky <rb@dustyfeet.com>
* test/simple_source/bug22/05_test_yield.py,
uncompyle6/scanners/scanner2.py: Python 2.2 doesn't have opcode
LIST_APPEND
2016-08-16 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner26.py: Python 2.2 scanner bug: don't
mung IMPORT_NAME op
2016-08-16 rocky <rb@dustyfeet.com>
* test/simple_source/bug_pypy27/02_call_method.py,
uncompyle6/parsers/parse2.py, uncompyle6/parsers/parse3.py: Small
pypy LOOKUP_METHOD cleanups
2016-08-16 R. Bernstein <rocky@users.noreply.github.com>
* : Merge pull request #49 from moagstar/master Fixed a bug with FORMAT_VALUE with sub expressions.
2016-08-16 DanielBradburn <moagstar@gmail.com>
* pytest/test_fstring.py, uncompyle6/parsers/parse3.py: Fixed bug
with FORMAT_VALUE where a sub expression would not be correctly
interpreted
2016-08-14 rocky <rb@dustyfeet.com>
* __pkginfo__.py: xdis 2.1.1 removes some bugs encountered here
2016-08-14 rocky <rb@dustyfeet.com>
* : commit a65443ee0225933367be9c7640629298882532c9 Author: rocky
<rb@dustyfeet.com> Date: Sun Aug 14 22:59:34 2016 -0400
2016-08-14 DanielBradburn <moagstar@gmail.com>
* pytest/test_fstring.py: added examples for known failures
2016-08-14 R. Bernstein <rocky@users.noreply.github.com>
* : Merge pull request #45 from rocky/revert-43-patch-1 Revert "Cache pip installation in travis"
2016-08-14 R. Bernstein <rocky@users.noreply.github.com>
* .travis.yml: Revert "Test with latest PyPy in Travis"
2016-08-14 R. Bernstein <rocky@users.noreply.github.com>
* : Merge pull request #44 from thedrow/patch-2 Test with latest PyPy in Travis
2016-08-14 R. Bernstein <rocky@users.noreply.github.com>
* : Merge pull request #43 from thedrow/patch-1 Cache pip installation in travis
2016-08-13 rocky <rb@dustyfeet.com>
* test/Makefile: Back off of 3.6 testing for now
2016-08-13 rocky <rb@dustyfeet.com>
* : commit 7ccbd419c6b26e8ae9d0929f1bfddedebce6bbaf Author: rocky
<rb@dustyfeet.com> Date: Sat Aug 13 20:25:19 2016 -0400
2016-08-13 rocky <rb@dustyfeet.com>
* test/Makefile: I said - we test 2.2 now.
2016-08-13 rocky <rb@dustyfeet.com>
* test/Makefile, test/simple_source/stmts/00_import.py,
test/test_pythonlib.py: Include Python 2.2 in testing
2016-08-13 rocky <rb@dustyfeet.com>
* README.rst, uncompyle6/parser.py, uncompyle6/parsers/parse22.py,
uncompyle6/scanner.py, uncompyle6/scanners/scanner22.py,
uncompyle6/scanners/scanner23.py, uncompyle6/scanners/scanner24.py,
uncompyle6/scanners/scanner25.py, uncompyle6/semantics/pysource.py:
Start handling Python 2.2 bytecode and... Fix some bugs in Python 2.3-2.5 bytecode handling
2016-08-11 Omer Katz <omer.drow@gmail.com>
* .travis.yml: Test with latest PyPy.
2016-08-11 Omer Katz <omer.drow@gmail.com>
* .travis.yml: Cache pip installation
2016-08-10 DanielBradburn <moagstar@gmail.com>
* pytest/test_fstring.py: small formatting change
2016-08-10 DanielBradburn <moagstar@gmail.com>
* pytest/test_fstring.py, requirements-dev.txt: added hypothesis to
requirements-dev
2016-08-10 DanielBradburn <moagstar@gmail.com>
* pytest/test_fstring.py: added hypothesis test (currently failing
due to limited support) for testing fstring uncompyling
2016-08-10 Daniel Bradburn <moagstar@gmail.com>
* : Merge pull request #2 from rocky/master Merging pypy and cpython 3.6 from rocky
2016-08-03 rocky <rb@dustyfeet.com>
* : commit 109e813058380630bda82014eee94a9089cc4666 Author: rocky
<rb@dustyfeet.com> Date: Wed Aug 3 08:07:47 2016 -0400
2016-08-01 rocky <rb@dustyfeet.com>
* __pkginfo__.py, requirements.txt: Need recent xdis fix for 3.6
wordcode
2016-08-01 rocky <rb@dustyfeet.com>
* test/simple_source/bug36/01_fstring.py: Add Python 3.6
formatted_str test
2016-08-01 R. Bernstein <rocky@users.noreply.github.com>
* : Merge pull request #41 from rocky/3.6 Move forward on moagstar's Python 3.6 support
2016-08-01 rocky <rb@dustyfeet.com>
* Makefile, README.rst, test/Makefile, test/test_pythonlib.py,
uncompyle6/bin/uncompile.py, uncompyle6/parsers/parse3.py,
uncompyle6/parsers/parse36.py, uncompyle6/scanner.py,
uncompyle6/scanners/scanner36.py, uncompyle6/semantics/aligner.py,
uncompyle6/semantics/pysource.py, uncompyle6/verify.py: Moagstar's
3.6 wordcode + formattedValue rules
2016-07-30 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/aligner.py: WIP try to keep line numbers the
same
2016-07-29 rocky <rb@dustyfeet.com>
* test/Makefile, uncompyle6/semantics/fragments.py: Small changes
2016-07-29 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse35.py, uncompyle6/scanner.py,
uncompyle6/scanners/scanner3.py, uncompyle6/semantics/pysource.py:
Fix 3.5 misclassifying RETURN_VALUE We use location of SETUP_EXCEPT instructions to disambiguate.
2016-07-28 Daniel Bradburn <moagstar@gmail.com>
* README.rst, test/Makefile, test/bytecode_3.6/fstring.py,
test/bytecode_3.6/fstring_single.py, test/test_pythonlib.py,
uncompyle6/bin/uncompile.py, uncompyle6/parser.py,
uncompyle6/parsers/parse3.py, uncompyle6/scanner.py,
uncompyle6/scanners/scanner36.py, uncompyle6/semantics/pysource.py,
uncompyle6/verify.py: Starting adding python 3.6 support to
uncompyle
2016-07-28 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse2.py, uncompyle6/parsers/parse23.py,
uncompyle6/parsers/parse24.py: while1 bug applied to Python 2.3 and
2.4
2016-07-28 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner3.py, uncompyle6/scanners/tok.py: PyPy
3.2 bug confusing RETURN_END_IF for except Also fix a instruction formatting bug
2016-07-27 rocky <rb@dustyfeet.com>
* uncompyle6/parser.py, uncompyle6/parsers/parse3.py,
uncompyle6/parsers/parse35.py: Split out 3.5 parser
2016-07-27 rocky <rb@dustyfeet.com>
* pytest/test_grammar.py, uncompyle6/parser.py,
uncompyle6/parsers/parse2.py, uncompyle6/parsers/parse3.py,
uncompyle6/parsers/parse34.py: Add python 3.4 grammar checking DRY grammar testing
2016-07-27 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse25.py, uncompyle6/parsers/parse26.py,
uncompyle6/parsers/parse27.py, uncompyle6/parsers/parse3.py,
uncompyle6/parsers/parse34.py: Clean and check Python 2.6 grammar
2016-07-27 rocky <rb@dustyfeet.com>
* pytest/test_grammar.py, uncompyle6/parser.py,
uncompyle6/parsers/parse2.py, uncompyle6/parsers/parse26.py,
uncompyle6/parsers/parse27.py, uncompyle6/parsers/parse3.py: Start
to segregate and clean up grammar
2016-07-27 rocky <rb@dustyfeet.com>
* pytest/test_grammar.py, uncompyle6/disas.py,
uncompyle6/scanner.py, uncompyle6/semantics/fragments.py: Add
is_pypy parameter to places that need it
2016-07-27 rocky <rb@dustyfeet.com>
* test/simple_source/stmts/09_whiletrue_bug.py,
uncompyle6/parser.py, uncompyle6/parsers/parse2.py,
uncompyle6/parsers/parse26.py, uncompyle6/parsers/parse27.py,
uncompyle6/scanners/scanner2.py, uncompyle6/scanners/scanner3.py,
uncompyle6/semantics/pysource.py: 2.6 and 2.7 while1 grammar rule Fixes issue #40
2016-07-27 rocky <rb@dustyfeet.com>
* pytest/test_grammar.py, uncompyle6/parser.py,
uncompyle6/parsers/parse3.py: Start grammar checker
2016-07-27 rocky <rb@dustyfeet.com>
* uncompyle6/main.py, uncompyle6/show.py: Show magic number in
output Fix bugs due to removal of token.format()
2016-07-27 rocky <rb@dustyfeet.com>
* uncompyle6/disas.py, uncompyle6/parsers/parse2.py,
uncompyle6/parsers/parse3.py, uncompyle6/scanners/scanner2.py,
uncompyle6/scanners/scanner27.py, uncompyle6/scanners/scanner3.py,
uncompyle6/scanners/scanner35.py, uncompyle6/scanners/tok.py,
uncompyle6/show.py: tok.format -> tok.__str__; simplify pypy code
2016-07-27 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/pysource.py, uncompyle6/verify.py: Python 2.7
set comprehension bug
2016-07-27 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/pysource.py: separate semantic action version
differences Added customize_for_version which uses is_pypy and version to adjust
tables
2016-07-27 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/fragments.py,
uncompyle6/semantics/pysource.py: Customize tables better for
specific Python versions
2016-07-27 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse2.py, uncompyle6/semantics/pysource.py:
Small code clean up
2016-07-26 rocky <rb@dustyfeet.com>
* README.rst, __pkginfo__.py, requirements.txt,
* uncompyle6/scanners/tok.py, uncompyle6/semantics/fragments.py,
uncompyle6/verify.py: Usuability fixes * try using format for __str__ * Explicitly nuke self.attr and self.pattr when no arg * Sync pysource and format wrt make_function
2016-07-26 rocky <rb@dustyfeet.com>
* ChangeLog, NEWS, README.rst, __pkginfo__.py, requirements.txt,
test/test_pyenvlib.py, uncompyle6/version.py: Get ready for release
2.7.1
@@ -105,9 +1077,9 @@
uncompyle6/scanners/scanner3.py, uncompyle6/semantics/pysource.py:
PyPy support * Use proper PYPY 32 opcodes * handle opcodes LOOKUP_METHOD and CALL_METHOD * Administrative stuff for PyPy
2016-07-24 rocky <rb@dustyfeet.com>
2016-07-24 Daniel Bradburn <moagstar@gmail.com>
* test/add-test.py: add-test: Make sure PyPy bytecode is separated
* : Merge pull request #1 from rocky/master Syncing with rocky
2016-07-24 rocky <rb@dustyfeet.com>

View File

@@ -1,4 +1,5 @@
This is the changelog from *decompyle*'s release 2.4 passed on by Dan Pascu
This is the changelog from *decompyle*'s release 2.4 and before
passed on by Dan Pascu
release 2.4 (Dan Pascu)
@@ -74,3 +75,57 @@ release 2.2beta1 (hartmut Goebel)
- use a single Walker and a single Parser, thus saving time and memory
- use augmented assign and 'print >>' internally
- optimized 'Walker.engine', the main part of code generation
release 0.6.0: (hartmut Goebel)
- extended print (Python 2.0)
- extended import (Python 2.0) (may not cover all cases)
- augmented assign (Python 2.0) (may not cover all cases)
- list comprehensions (Python 2.0)
- equivalent for 'apply' (Python 1.6)
- if .. elif .. else are now nested as expected
- assert test, data
- unpack list corrected (was the same as unpack tuple)
- fixed unpack tuple (trailing semicolon was missing)
- major speed up :-)
- reduced memory usage (pre-alpha-0.5 has increased it a lot)
- still missing: EXTENDED_ARG
pre-alpha-0.5: (hartmut Goebel)
- *args, **kwargs
- global
- formal tuple parameters (eg. def a(self, (x,y,z)) )
- actual lambda parameters (eg. X(lambda z: z**2) )
- remove last 'return None' in procedures
- remove last 'return locals()' in class definitions
- docstrings
pre-alpha-0.4: (hartmut Goebel)
- assert
- try/except/finally
- parentheses in expressions
- nested expressions
- extracted dissassemble() from module dis and
removed ugly redirect of stdout, thus saved a lot of
ugly code and a lot of memory
pre-alpha-0.3: (hartmut Goebel)
- keyword arguments
- some boolean expressions
- and/or
- complex conditions in if/while
- read byte-code from .pyc without importing
- access to the body of classes and modules
- class and function definitions
- a = b = c = xxx
pre-alpha-0.1 -> pre-alpha-0.2:
- SET_LINENO filtered out in lexer now
- added support for subscripts (just for Christian Tismer :-)
- fixed bug with handling of BUILD_{LIST,TUPLE} & CALL_FUNCTION
- dict-building support
- comparison support
- exec support
- del support
- pass support
- slice support
- no more extraneous (albeit legal) commas
- finally, it excepts try [sic] but not all 42 variations of it

View File

@@ -29,7 +29,9 @@ augmented with pseudo instruction COME_FROM. This code introduced
another clever idea: using table-driven semantics routines, using
format specifiers.
The last mention of a release of SPARK from John is around 2002.
The last mention of a release of SPARK from John is around 2002. As
released, although the Early Algorithm parser was in good shape, this
code was woefully lacking as serious Python deparser.
In the fall of 2000, Hartmut Goebel
[took over maintaining the code](https://groups.google.com/forum/#!searchin/comp.lang.python/hartmut$20goebel/comp.lang.python/35s3mp4-nuY/UZALti6ujnQJ). The
@@ -112,12 +114,18 @@ Fenx's uncompyle3 which I used for inspiration for Python3 support.
I started working on this late 2015, mostly to add fragment support.
In that, I decided to make this runnable on Python 3.2+ and Python 2.6+
while, handling Python bytecodes from Python versions 2.5+ and
3.2+.
3.2+. In doing so, it has been expedient to separate this into three
projects: load loading and disassembly (xdis), parsing and tree
building (spark_parser), and grammar and semantic actions for
decompiling (uncompyle6).
Over the many years, code styles and Python features have
changed. However brilliant the code was and still is, it hasn't really
had a single public active maintainer. And there have been many forks
of the code.
of the code. I have spent a great deal of time trying to organize and
modularize the code so that it can handle more Python versions more
gracefully (with still only moderate success).
That it has been in need of an overhaul has been recognized by the
Hartmut a decade an a half ago:

View File

@@ -33,7 +33,7 @@ check-2.7 check-3.3 check-3.4: pytest
#: Tests for Python 3.2 and 3.5 - pytest doesn't work here
# Or rather 3.5 doesn't work not on Travis
check-3.2 check-3.5:
check-3.1 check-3.2 check-3.5 check-3.6:
$(MAKE) -C test $@
#:Tests for Python 2.6 (doesn't have pytest)
@@ -42,7 +42,7 @@ check-2.6:
#:PyPy 2.6.1 or PyPy 5.0.1
# Skip for now
2.6 5.0:
2.6 5.0 5.3:
#:PyPy pypy3-2.4.0 Python 3:
pypy-3.2 2.4:

106
NEWS
View File

@@ -1,3 +1,109 @@
uncompyle6 2.9.4 2016-11-02
- Handle Python 3.x function annotations
- track def keywoard-parameter line-splitting in source code better
- bump min xdis version to mask previous xdis bug
uncompyle6 2.9.3 2016-10-26
Release forced by incompatiblity change in xdis 3.2.0.
- Python 3.1 bugs:
* handle "with ... as"
* handle "with"
* Start handling def (...) -> yy (has bugs still)
- DRY Python 3.x via inheritance
- Python 3.6 work (from Daniel Bradburn)
* Handle 3.6 buildstring
* Handle 3.6 handle single and multiple fstring better
uncompyle6 2.9.2 2016-10-15
- use source-code line breaks to assist in where to break
in tuples and maps
- Fix Python 1.5 decompyle bugs
- Fix some Python 2.6 and below bugs
- DRY fragments.py code a little
uncompyle6 2.9.1 2016-10-09
- Improved Python 1.5 decompiling
- Handle old-style pre Python 2.2 classes
uncompyle6 2.9.0 2016-10-09
- Use xdis 3.0.0 protocol load_module.
this Forces change in requirements.txt and _pkg_info_.py
- Start Python 1.5 decompiling; another round of work is needed to
remove bugs
- Simpify python 2.1 grammar
- Fix bug with -t ... Wasn't showing source text when -t option was given
- Fix 2.1-2.6 bug in list comprehension
uncompyle6 2.8.4 2016-10-08
- Python 3 disassembly bug fixes
- Python 3.6 fstring bug fixes (from moagstar)
- Python 2.1 disassembly
- COME_FROM suffixes added in Python3
- use .py extension in verification disassembly
uncompyle6 2.8.3 2016-09-11 live from NYC!
NOTE: this is possibly the last release before a major reworking of
control-flow structure detection is done.
- Lots of bug fixes in decompilation:
* 3.0 .. 3.4 whileTrue bug
* 3.x function declaration deparsing:
. 3.0 .. 3.2 *args processing
. 3.0 .. 3.2 call name and kwargs bug
. 3.0 .. getting parameter of *
. 3.0 .. handling varible number of args
. 3.0 .. "if" structure bugs
* 3.5+ if/else bugs
* 2.2-2.6 bugs
. try/except control flow
. a == b == c -like detection
. generator detection
. "while .. and" statement bugs
. handle "except <cond>, <var>"
. use older raise format in 2.x
- scanner "disassemble" is now "ingest". True disassembly is done by xdis
- Start accepting Python 3.1 bytecode
- Add --weak-verify option on test_pyenvlib and test_pythonlib. This
catches more bugs more easily
- bump xdis requirement so we can deparse dropbox 2.5 code
- Added H. Goebel's changes before 2.4 in DECOMPYLE-2.4-CHANGELOG.txt
uncompyle6 2.8.2 2016-08-29
- Handle Python 3.6 format string conversions !r, !s, !a
- Start to handle 3.1 bytecode
- Fix some PyPy translation bugs
- We now only handle 3.6.0a3+ since that is incompatible with 3.6 before that
uncompyle6 2.8.1 2016-08-20
- Add Python 2.2 decompilation
- Fix bugs
* PyPy LOOKUP_METHOD bug
* Python 3.6 FORMAT_VALUE handles expressions now
uncompyle6 2.8.0 2016-08-03
- Start Python 3.6 support (moagstar)
more work on PEP 498 needed
- tidy bytecode/word output
- numerous decompiling bugs fixed
- grammar testing started
- show magic number in deparsed output
- better grammar and semantic action segregation based
on python bytecode version
uncompyle6 2.7.1 2016-07-26
- PyPy bytecodes for 2.7 and 3.2 added

View File

@@ -11,8 +11,8 @@ Introduction
------------
*uncompyle6* translates Python bytecode back into equivalent Python
source code. It accepts bytecodes from Python version 2.3 to 3.5 or
so, including PyPy bytecode.
source code. It accepts bytecodes from Python version 2.1 to 3.6 or
so, including PyPy bytecode and Dropbox's Python 2.5 bytecode.
Why this?
---------
@@ -20,9 +20,9 @@ Why this?
There were a number of decompyle, uncompile, uncompyle2, uncompyle3
forks around. All of them came basically from the same code base, and
almost all of them no were no longer actively maintained. Only one
handled Python 3, and even there, only 3.2. This code pulls these
together and moves forward. It also addresses a number of open issues
in the previous forks.
handled Python 3, and even there, only 3.2 or 3.3 depending on which
code is used. This code pulls these together and moves forward. It
also addresses a number of open issues in the previous forks.
What makes this different from other CPython bytecode decompilers?: its
ability to deparse just fragments and give source-code information
@@ -45,7 +45,7 @@ Requirements
This project requires Python 2.6 or later, PyPy 3-2.4, or PyPy-5.0.1.
The bytecode files it can read has been tested on Python bytecodes from
versions 2.3-2.7, and 3.2-3.5 and the above-mentioned PyPy versions.
versions 2.1-2.7, and 3.2-3.6 and the above-mentioned PyPy versions.
Installation
------------
@@ -96,22 +96,35 @@ For usage help:
Known Bugs/Restrictions
-----------------------
Python 2 deparsing decompiles and about 90% verifies from Python 2.3.7 to Python
About 90% of the decompilation verifies from Python 2.3.7 to Python
3.4.2 on the standard library packages I have on my system.
(Verification is the process of decompiling bytecode, compiling with a
Python for that byecode version, and then comparing the byetcode
produced by the decompiled/compiled program. Some allowance is made
for inessential differences.)
for inessential differences, but other semantically equivalent
differences are not caught.)
Later distributions average about 200 files. At this point, 2.7
decompilation is better than uncompyle2. A number of bugs have been
fixed.
decompilation is definitely better than uncompyle2. A number of bugs
have been fixed. We now handle more Python bytecodes than the old
decompyle program that handled Python bytecodes ranging from 1.5 to
2.4. There is some work do do on the lower end which is more
difficult for us since we don't have a Python interpreter for versions
1.5, 1.6 or 2.0.
Python 3.5 largely works, but still has some bugs in it.
Python 3.6 changes things drastically by using word codes rather than
byte codes, and that needs to be addressed.
Currently not all Python magic numbers are supported. Specifically in
some versions of Python, notably Python 3.6, the magic number has
changes several times within a version. We support only the released
magic. There are also customized Python interpreters, notably Dropbox,
which use their own magic and encrypt bytcode. With the exception of
the Dropbox's old Python 2.5 interpreter this kind of thing is not
handled.
There is lots to do, so please dig in and help.
See Also
@@ -119,6 +132,7 @@ See Also
* https://github.com/zrax/pycdc : supports all versions of Python and is written in C++
* https://code.google.com/archive/p/unpyc3/ : supports Python 3.2 only. The above projects use a different decompiling technique what is used here.
* https://github.com/figment/unpyc3/ : fork of above, but supports Python 3.3 only. Include some fixes like supporting function annotations
* The HISTORY_ file.
.. |downloads| image:: https://img.shields.io/pypi/dd/uncompyle6.svg

View File

@@ -23,6 +23,7 @@ classifiers = ['Development Status :: 4 - Beta',
'Programming Language :: Python :: 3.3',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Topic :: Software Development :: Debuggers',
'Topic :: Software Development :: Libraries :: Python Modules',
]
@@ -36,8 +37,8 @@ entry_points={
'pydisassemble=uncompyle6.bin.pydisassemble:main',
]}
ftp_url = None
install_requires = ['spark-parser >= 1.4.0',
'xdis >= 2.0.3']
install_requires = ['spark-parser >= 1.4.0, < 1.5.0',
'xdis >= 3.2.2, < 3.3.0']
license = 'MIT'
mailing_list = 'python-debugger@googlegroups.com'
modname = 'uncompyle6'

150
pytest/test_fstring.py Normal file
View File

@@ -0,0 +1,150 @@
# std
import os
# test
import pytest
import hypothesis
from hypothesis import strategies as st
# uncompyle6
from uncompyle6 import PYTHON_VERSION, deparse_code
@st.composite
def expressions(draw):
# todo : would be nice to generate expressions using hypothesis however
# this is pretty involved so for now just use a corpus of expressions
# from which to select.
return draw(st.sampled_from((
'abc',
'len(items)',
'x + 1',
'lineno',
'container',
'self.attribute',
'self.method()',
# These expressions are failing, I think these are control
# flow problems rather than problems with FORMAT_VALUE,
# however I need to confirm this...
#'sorted(items, key=lambda x: x.name)',
#'func(*args, **kwargs)',
#'text or default',
#'43 if life_the_universe and everything else None'
)))
@st.composite
def format_specifiers(draw):
"""
Generate a valid format specifier using the rules:
format_spec ::= [[fill]align][sign][#][0][width][,][.precision][type]
fill ::= <any character>
align ::= "<" | ">" | "=" | "^"
sign ::= "+" | "-" | " "
width ::= integer
precision ::= integer
type ::= "b" | "c" | "d" | "e" | "E" | "f" | "F" | "g" | "G" | "n" | "o" | "s" | "x" | "X" | "%"
See https://docs.python.org/2/library/string.html
:param draw: Let hypothesis draw from other strategies.
:return: An example format_specifier.
"""
alphabet_strategy = st.characters(min_codepoint=ord('a'), max_codepoint=ord('z'))
fill = draw(st.one_of(alphabet_strategy, st.none()))
align = draw(st.sampled_from(list('<>=^')))
fill_align = (fill + align or '') if fill else ''
type_ = draw(st.sampled_from('bcdeEfFgGnosxX%'))
can_have_sign = type_ in 'deEfFgGnoxX%'
can_have_comma = type_ in 'deEfFgG%'
can_have_precision = type_ in 'fFgG'
can_have_pound = type_ in 'boxX%'
can_have_zero = type_ in 'oxX'
sign = draw(st.sampled_from(list('+- ') + [''])) if can_have_sign else ''
pound = draw(st.sampled_from(('#', '',))) if can_have_pound else ''
zero = draw(st.sampled_from(('0', '',))) if can_have_zero else ''
int_strategy = st.integers(min_value=1, max_value=1000)
width = draw(st.one_of(int_strategy, st.none()))
width = str(width) if width is not None else ''
comma = draw(st.sampled_from((',', '',))) if can_have_comma else ''
if can_have_precision:
precision = draw(st.one_of(int_strategy, st.none()))
precision = '.' + str(precision) if precision else ''
else:
precision = ''
return ''.join((fill_align, sign, pound, zero, width, comma, precision, type_,))
@st.composite
def fstrings(draw):
"""
Generate a valid f-string.
See https://www.python.org/dev/peps/pep-0498/#specification
:param draw: Let hypothsis draw from other strategies.
:return: A valid f-string.
"""
character_strategy = st.characters(
blacklist_characters='\r\n\'\\s{}',
min_codepoint=1,
max_codepoint=1000,
)
is_raw = draw(st.booleans())
integer_strategy = st.integers(min_value=0, max_value=3)
expression_count = draw(integer_strategy)
content = []
for _ in range(expression_count):
expression = draw(expressions())
conversion = draw(st.sampled_from(('', '!s', '!r', '!a',)))
has_specifier = draw(st.booleans())
specifier = ':' + draw(format_specifiers()) if has_specifier else ''
content.append('{{{}{}}}'.format(expression, conversion, specifier))
content.append(draw(st.text(character_strategy)))
content = ''.join(content)
return "f{}'{}'".format('r' if is_raw else '', content)
@pytest.mark.skipif(PYTHON_VERSION < 3.6, reason='need at least python 3.6')
@hypothesis.given(format_specifiers())
def test_format_specifiers(format_specifier):
"""Verify that format_specifiers generates valid specifiers"""
try:
exec('"{:' + format_specifier + '}".format(0)')
except ValueError as e:
if 'Unknown format code' not in str(e):
raise
def run_test(text):
hypothesis.assume(len(text))
hypothesis.assume("f'{" in text)
expr = text + '\n'
code = compile(expr, '<string>', 'single')
deparsed = deparse_code(PYTHON_VERSION, code, compile_mode='single')
recompiled = compile(deparsed.text, '<string>', 'single')
if recompiled != code:
assert 'dis(' + deparsed.text.strip('\n') + ')' == 'dis(' + expr.strip('\n') + ')'
@pytest.mark.skipif(PYTHON_VERSION < 3.6, reason='need at least python 3.6')
@hypothesis.given(fstrings())
def test_uncompyle_fstring(fstring):
"""Verify uncompyling fstring bytecode"""
run_test(fstring)
@pytest.mark.skipif(PYTHON_VERSION < 3.6, reason='need at least python 3.6')
@pytest.mark.parametrize('fstring', [
"f'{abc}{abc!s}'",
"f'{abc}0'",
])
def test_uncompyle_direct(fstring):
"""useful for debugging"""
run_test(fstring)

55
pytest/test_grammar.py Normal file
View File

@@ -0,0 +1,55 @@
import re
from uncompyle6 import PYTHON_VERSION, PYTHON3, IS_PYPY # , PYTHON_VERSION
from uncompyle6.parser import get_python_parser
from uncompyle6.scanner import get_scanner
def test_grammar():
def check_tokens(tokens, opcode_set):
remain_tokens = set(tokens) - opcode_set
remain_tokens = set([re.sub('_\d+$','', t) for t in remain_tokens])
remain_tokens = set([re.sub('_CONT$','', t) for t in remain_tokens])
remain_tokens = set(remain_tokens) - opcode_set
assert remain_tokens == set([]), \
"Remaining tokens %s\n====\n%s" % (remain_tokens, p.dumpGrammar())
p = get_python_parser(PYTHON_VERSION, is_pypy=IS_PYPY)
lhs, rhs, tokens, right_recursive = p.checkSets()
expect_lhs = set(['expr1024', 'pos_arg'])
unused_rhs = set(['build_list', 'call_function', 'mkfunc',
'mklambda',
'unpack', 'unpack_list'])
expect_right_recursive = [['designList', ('designator', 'DUP_TOP', 'designList')]]
if PYTHON3:
expect_lhs.add('load_genexpr')
unused_rhs = unused_rhs.union(set("""
except_pop_except genexpr classdefdeco2 listcomp
""".split()))
if 3.0 <= PYTHON_VERSION:
expect_lhs.add("annotate_arg")
expect_lhs.add("annotate_tuple")
unused_rhs.add("mkfunc_annotate")
pass
else:
expect_lhs.add('kwarg')
assert expect_lhs == set(lhs)
assert unused_rhs == set(rhs)
assert expect_right_recursive == right_recursive
s = get_scanner(PYTHON_VERSION, IS_PYPY)
ignore_set = set(
"""
JUMP_BACK CONTINUE RETURN_END_IF
COME_FROM COME_FROM_EXCEPT COME_FROM_LOOP COME_FROM_WITH
COME_FROM_FINALLY
LOAD_GENEXPR LOAD_ASSERT LOAD_SETCOMP LOAD_DICTCOMP
LAMBDA_MARKER RETURN_LAST
""".split())
if 2.6 <= PYTHON_VERSION <= 2.7:
opcode_set = set(s.opc.opname).union(ignore_set)
check_tokens(tokens, opcode_set)
elif PYTHON_VERSION == 3.4:
ignore_set.add('LOAD_CLASSNAME')
ignore_set.add('STORE_LOCALS')
opcode_set = set(s.opc.opname).union(ignore_set)
check_tokens(tokens, opcode_set)

View File

@@ -9,8 +9,6 @@ def test_single_mode():
'i = j % 4',
'i = {}',
'i = []',
'while i < 1 or stop:\n i\n',
'while i < 1 or stop:\n print%s\n' % ('(i)' if PYTHON3 else ' i'),
'for i in range(10):\n i\n',
'for i in range(10):\n for j in range(10):\n i + j\n',
'try:\n i\nexcept Exception:\n j\nelse:\n k\n'

View File

@@ -1,2 +1,3 @@
pytest
flake8
hypothesis

View File

@@ -1,2 +1,2 @@
spark-parser >= 1.2.1
xdis >= 2.0.3
# Pick up stuff from setup.py
-e .

View File

@@ -20,42 +20,66 @@ check:
$(MAKE) check-$$PYTHON_VERSION
#: Run working tests from Python 2.6 or 2.7
check-2.6 check-2.7: check-bytecode-2 check-bytecode-3 check-2.7-ok
check-2.6 check-2.7: check-bytecode-2 check-bytecode-3 check-bytecode-1 check-2.7-ok
#: Run working tests from Python 3.1
check-3.1: check-bytecode
$(PYTHON) test_pythonlib.py --bytecode-3.1 --weak-verify $(COMPILE)
#: Run working tests from Python 3.2
check-3.2: check-bytecode
$(PYTHON) test_pythonlib.py --bytecode-3.2 --verify $(COMPILE)
$(PYTHON) test_pythonlib.py --bytecode-3.2 --weak-verify $(COMPILE)
#: Run working tests from Python 3.3
check-3.3: check-bytecode
$(PYTHON) test_pythonlib.py --bytecode-3.3 --verify $(COMPILE)
$(PYTHON) test_pythonlib.py --bytecode-3.3 --weak-verify $(COMPILE)
#: Run working tests from Python 3.4
check-3.4: check-bytecode check-3.4-ok check-2.7-ok
$(PYTHON) test_pythonlib.py --bytecode-3.4 --verify $(COMPILE)
$(PYTHON) test_pythonlib.py --bytecode-3.4 --weak-verify $(COMPILE)
#: Run working tests from Python 3.5
check-3.5: check-bytecode
$(PYTHON) test_pythonlib.py --bytecode-3.5 --verify $(COMPILE)
#: Run working tests from Python 3.6
check-3.6: check-bytecode
$(PYTHON) test_pythonlib.py --bytecode-3.6 --verify $(COMPILE)
#: Check deparsing only, but from a different Python version
check-disasm:
$(PYTHON) dis-compare.py
#: Check deparsing bytecode 1.x only
check-bytecode-1:
$(PYTHON) test_pythonlib.py --bytecode-1.5
#: Check deparsing bytecode 2.x only
check-bytecode-2:
$(PYTHON) test_pythonlib.py --bytecode-2.3 --bytecode-2.4 \
$(PYTHON) test_pythonlib.py \
--bytecode-2.1 --bytecode-2.2 --bytecode-2.3 --bytecode-2.4 \
--bytecode-2.5 --bytecode-2.6 --bytecode-2.7 --bytecode-pypy2.7
#: Check deparsing bytecode 3.x only
check-bytecode-3:
$(PYTHON) test_pythonlib.py --bytecode-3.2 --bytecode-3.3 \
$(PYTHON) test_pythonlib.py --bytecode-3.1 --bytecode-3.2 --bytecode-3.3 \
--bytecode-3.4 --bytecode-3.5 --bytecode-pypy3.2
#: Check deparsing bytecode that works running Python 2 and Python 3
check-bytecode: check-bytecode-3
$(PYTHON) test_pythonlib.py --bytecode-2.3 --bytecode-2.4 \
--bytecode-2.5 --bytecode-2.6 --bytecode-2.7 --bytecode-pypy2.7
$(PYTHON) test_pythonlib.py \
--bytecode-2.1 --bytecode-2.2 --bytecode-2.3 --bytecode-2.4 \
--bytecode-2.5 --bytecode-2.6 --bytecode-2.7 \
--bytecode-pypy2.7 --bytecode-1
#: Check deparsing Python 2.1
check-bytecode-2.1:
$(PYTHON) test_pythonlib.py --bytecode-2.1
#: Check deparsing Python 2.2
check-bytecode-2.2:
$(PYTHON) test_pythonlib.py --bytecode-2.2
#: Check deparsing Python 2.3
check-bytecode-2.3:
@@ -77,6 +101,10 @@ check-bytecode-2.6:
check-bytecode-2.7:
$(PYTHON) test_pythonlib.py --bytecode-2.7
#: Check deparsing Python 3.1
check-bytecode-3.1:
$(PYTHON) test_pythonlib.py --bytecode-3.1
#: Check deparsing Python 3.2
check-bytecode-3.2:
$(PYTHON) test_pythonlib.py --bytecode-3.2
@@ -93,6 +121,10 @@ check-bytecode-3.4:
check-bytecode-3.5:
$(PYTHON) test_pythonlib.py --bytecode-3.5
#: Check deparsing Python 3.6
check-bytecode-3.6:
$(PYTHON) test_pythonlib.py --bytecode-3.6
#: short tests for bytecodes only for this version of Python
check-native-short:
$(PYTHON) test_pythonlib.py --bytecode-$(PYTHON_VERSION) --verify $(COMPILE)
@@ -118,7 +150,7 @@ check-3.4-ok:
2.6:
#: PyPy 5.0.x with Python 2.7 ...
pypy-2.7 5.0:
pypy-2.7 5.0 5.3:
$(PYTHON) test_pythonlib.py --bytecode-pypy2.7 --verify
#: PyPy 2.4.x with Python 3.2 ...

Binary file not shown.

Binary file not shown.

BIN
test/bytecode_1.5/exec.pyc Normal file

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

BIN
test/bytecode_1.5/misc.pyc Normal file

Binary file not shown.

Binary file not shown.

BIN
test/bytecode_1.5/print.pyc Normal file

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

BIN
test/bytecode_2.1/exec.pyc Normal file

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

BIN
test/bytecode_2.1/misc.pyc Normal file

Binary file not shown.

Binary file not shown.

BIN
test/bytecode_2.1/print.pyc Normal file

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Some files were not shown because too many files have changed in this diff Show More