jinx/python-uncompyle6
1
Fork 0
mirror of https://github.com/rocky/python-uncompyle6.git synced 2025-08-04 01:09:52 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'python-2.4' of github.com:rocky/python-uncompyle6 into python-2.4

Browse Source
This commit is contained in:
rocky
2019-07-04 10:02:18 -04:00
parent 145c18fbeb e56088b566
commit 0639fdbbb7
31 changed files with 393 additions and 246 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
NEWS.md
View File

@@ -1,4 +1,18 @@
3.3.4 2019-05-19 Fleetwood at 65
3.3.5 2019-07-03 Pre Independence Day
=====================================
Again, most of the work in this is release is thanks to x0ret.
- Handle annotation args in Python 3.x
- Fix vararg and function signatures in 3.x
- Some 3.x < 3.6 while(1)/if fixes - others remain
- Start reinstating else if -> elif
- LOAD_CONST -> LOAD_CODE where appropriate
- option `weak-verify` is now `syntax-verify`
- code cleanups, start using "blacken" to reformat text
3.3.4 2019-06-19 Fleetwood at 65
================================
Most of the work in this is release is thanks to x0ret.

40
README.rst
View File

@@ -122,17 +122,6 @@ For usage help:
$ uncompyle6 -h
If you want strong verification of the correctness of the
decompilation process, add the `--verify` option. But there are
situations where this will indicate a failure, although the generated
program is semantically equivalent. Using option `--weak-verify` will
tell you if there is something definitely wrong. Generally, large
swaths of code are decompiled correctly, if not the entire program.
You can also cross compare the results with pycdc_ . Since they work
differently, bugs here often aren't in that, and vice versa.
Verification
------------
@@ -140,22 +129,25 @@ In older versions of Python it was possible to verify bytecode by
decompiling bytecode, and then compiling using the Python interpreter
for that bytecode version. Having done this the bytecode produced
could be compared with the original bytecode. However as Python's code
generation got better, this is no longer feasible.
generation got better, this no longer was feasible.
The verification that we use that doesn't check bytecode for
equivalence but does check to see if the resulting decompiled source
is a valid Python program by running the Python interpreter. Because
the Python language has changed so much, for best results you should
use the same Python version in checking as was used in creating the
bytecode.
If you want Python syntax verification of the correctness of the
decompilation process, add the `--syntax-verify` option. However since
Python syntax changes, you should use this option if the bytecode is
the right bytecode for the Python interpreter that will be checking
the syntax.
There are however an interesting class of these programs that is
readily available give stronger verification: those programs that
when run check some computation, or even better themselves.
You can also cross compare the results with another python decompiler
like pycdc_ . Since they work differently, bugs here often aren't in
that, and vice versa.
And already Python has a set of programs like this: the test suite
for the standard library that comes with Python. We have some
code in `test/stdlib` to facilitate this kind of checking.
There is an interesting class of these programs that is readily
available give stronger verification: those programs that when run
test themselves. Our test suite includes these.
And Python comes with another a set of programs like this: its test
suite for the standard library. We have some code in `test/stdlib` to
facilitate this kind of checking too.
Known Bugs/Restrictions
-----------------------

2
appveyor.yml
View File

@@ -61,7 +61,7 @@ build_script:
test_script:
# Run the project tests
- "%CMD_IN_ENV% python test/test_pyenvlib.py --native --weak-verify"
- "%CMD_IN_ENV% python test/test_pyenvlib.py --native --syntax-verify"
after_test:
# If tests are successful, create binary packages for the project.

50
test/Makefile
View File

@@ -34,47 +34,47 @@ check-2.4 check-2.5 check-2.6 check-2.7: check-bytecode-2 check-bytecode-3 check
#: Run working tests from Python 3.0
check-3.0: check-bytecode
$(PYTHON) test_pythonlib.py --bytecode-3.0-run --verify-run
$(PYTHON) test_pythonlib.py --bytecode-3.0 --weak-verify $(COMPILE)
$(PYTHON) test_pythonlib.py --bytecode-3.0 --syntax-verify $(COMPILE)
#: Run working tests from Python 3.1
check-3.1: check-bytecode
$(PYTHON) test_pythonlib.py --bytecode-3.1-run --verify-run
$(PYTHON) test_pythonlib.py --bytecode-3.1 --weak-verify $(COMPILE)
$(PYTHON) test_pythonlib.py --bytecode-3.1 --syntax-verify $(COMPILE)
#: Run working tests from Python 3.2
check-3.2: check-bytecode
$(PYTHON) test_pythonlib.py --bytecode-3.2-run --verify-run
$(PYTHON) test_pythonlib.py --bytecode-3.2 --weak-verify $(COMPILE)
$(PYTHON) test_pythonlib.py --bytecode-3.2 --syntax-verify $(COMPILE)
#: Run working tests from Python 3.3
check-3.3: check-bytecode
$(PYTHON) test_pythonlib.py --bytecode-3.3-run --verify-run
$(PYTHON) test_pythonlib.py --bytecode-3.3 --weak-verify $(COMPILE)
$(PYTHON) test_pythonlib.py --bytecode-3.3 --syntax-verify $(COMPILE)
#: Run working tests from Python 3.4
check-3.4: check-bytecode check-3.4-ok check-2.7-ok
$(PYTHON) test_pythonlib.py --bytecode-3.4-run --verify-run
$(PYTHON) test_pythonlib.py --bytecode-3.4 --weak-verify $(COMPILE)
$(PYTHON) test_pythonlib.py --bytecode-3.4 --syntax-verify $(COMPILE)
#: Run working tests from Python 3.5
check-3.5: check-bytecode
$(PYTHON) test_pythonlib.py --bytecode-3.5-run --verify-run
$(PYTHON) test_pythonlib.py --bytecode-3.5 --weak-verify $(COMPILE)
$(PYTHON) test_pythonlib.py --bytecode-3.5 --syntax-verify $(COMPILE)
#: Run working tests from Python 3.6
check-3.6: check-bytecode
$(PYTHON) test_pythonlib.py --bytecode-3.6-run --verify-run
$(PYTHON) test_pythonlib.py --bytecode-3.6 --weak-verify $(COMPILE)
$(PYTHON) test_pythonlib.py --bytecode-3.6 --syntax-verify $(COMPILE)
#: Run working tests from Python 3.7
check-3.7: check-bytecode
$(PYTHON) test_pythonlib.py --bytecode-3.7-run --verify-run
$(PYTHON) test_pythonlib.py --bytecode-3.7 --weak-verify $(COMPILE)
$(PYTHON) test_pythonlib.py --bytecode-3.7 --syntax-verify $(COMPILE)
# #: Run working tests from Python 3.8
# check-3.8: check-bytecode
# $(PYTHON) test_pythonlib.py --bytecode-3.8-run --verify-run
# $(PYTHON) test_pythonlib.py --bytecode-3.8 --weak-verify $(COMPILE)
# $(PYTHON) test_pythonlib.py --bytecode-3.8 --syntax-verify $(COMPILE)
# FIXME
#: this is called when running under pypy3.5-5.8.0 or pypy2-5.6.0
@@ -230,61 +230,61 @@ grammar-coverage-3.7:
#: Check deparsing Python 2.6
check-bytecode-2.6:
$(PYTHON) test_pythonlib.py --bytecode-2.6-run --verify-run
$(PYTHON) test_pythonlib.py --bytecode-2.6 --weak-verify
$(PYTHON) test_pythonlib.py --bytecode-2.6 --syntax-verify
#: Check deparsing Python 2.7
check-bytecode-2.7:
$(PYTHON) test_pythonlib.py --bytecode-2.7-run --verify-run
$(PYTHON) test_pythonlib.py --bytecode-2.7 --weak-verify
$(PYTHON) test_pythonlib.py --bytecode-2.7 --syntax-verify
#: Check deparsing Python 3.0
check-bytecode-3.0:
$(PYTHON) test_pythonlib.py --bytecode-3.0-run --verify-run
$(PYTHON) test_pythonlib.py --bytecode-3.0 --weak-verify
$(PYTHON) test_pythonlib.py --bytecode-3.0 --syntax-verify
#: Check deparsing Python 3.1
check-bytecode-3.1:
$(PYTHON) test_pythonlib.py --bytecode-3.1-run --verify-run
$(PYTHON) test_pythonlib.py --bytecode-3.1 --weak-verify
$(PYTHON) test_pythonlib.py --bytecode-3.1 --syntax-verify
#: Check deparsing Python 3.2
check-bytecode-3.2:
$(PYTHON) test_pythonlib.py --bytecode-3.2-run --verify-run
$(PYTHON) test_pythonlib.py --bytecode-3.2 --weak-verify
$(PYTHON) test_pythonlib.py --bytecode-3.2 --syntax-verify
#: Check deparsing Python 3.3
check-bytecode-3.3:
$(PYTHON) test_pythonlib.py --bytecode-3.3-run --verify-run
$(PYTHON) test_pythonlib.py --bytecode-3.3 --weak-verify
$(PYTHON) test_pythonlib.py --bytecode-3.3 --syntax-verify
#: Check deparsing Python 3.4
check-bytecode-3.4:
$(PYTHON) test_pythonlib.py --bytecode-3.4-run --verify-run
$(PYTHON) test_pythonlib.py --bytecode-3.4 --weak-verify
$(PYTHON) test_pythonlib.py --bytecode-3.4 --syntax-verify
#: Check deparsing Python 3.5
check-bytecode-3.5:
$(PYTHON) test_pythonlib.py --bytecode-3.5-run --verify-run
$(PYTHON) test_pythonlib.py --bytecode-3.5 --weak-verify
$(PYTHON) test_pythonlib.py --bytecode-3.5 --syntax-verify
#: Check deparsing Python 3.6
check-bytecode-3.6:
$(PYTHON) test_pythonlib.py --bytecode-3.6-run --verify-run
$(PYTHON) test_pythonlib.py --bytecode-3.6 --weak-verify
$(PYTHON) test_pythonlib.py --bytecode-3.6 --syntax-verify
#: Check deparsing Python 3.7
check-bytecode-3.7:
$(PYTHON) test_pythonlib.py --bytecode-3.7-run --verify-run
$(PYTHON) test_pythonlib.py --bytecode-3.7 --weak-verify
$(PYTHON) test_pythonlib.py --bytecode-3.7 --syntax-verify
# #: Check deparsing Python 3.8
# check-bytecode-3.8:
# $(PYTHON) test_pythonlib.py --bytecode-3.8-run --verify-run
# $(PYTHON) test_pythonlib.py --bytecode-3.8 --weak-verify
# $(PYTHON) test_pythonlib.py --bytecode-3.8 --syntax-verify
#: short tests for bytecodes only for this version of Python
check-native-short:
$(PYTHON) test_pythonlib.py --bytecode-$(PYTHON_VERSION) --weak-verify $(COMPILE)
$(PYTHON) test_pythonlib.py --bytecode-$(PYTHON_VERSION) --syntax-verify $(COMPILE)
$(PYTHON) test_pythonlib.py --bytecode-$(PYTHON_VERSION)-run --verify-run $(COMPILE)
#: Run longer Python 2.6's lib files known to be okay
@@ -293,19 +293,19 @@ check-2.4-ok:
#: Run longer Python 2.6's lib files known to be okay
check-2.6-ok:
$(PYTHON) test_pythonlib.py --ok-2.6 --weak-verify $(COMPILE)
$(PYTHON) test_pythonlib.py --ok-2.6 --syntax-verify $(COMPILE)
#: Run longer Python 2.7's lib files known to be okay
check-2.7-ok:
$(PYTHON) test_pythonlib.py --ok-2.7 --weak-verify $(COMPILE)
$(PYTHON) test_pythonlib.py --ok-2.7 --syntax-verify $(COMPILE)
#: Run longer Python 3.2's lib files known to be okay
check-3.2-ok:
$(PYTHON) test_pythonlib.py --ok-3.2 --weak-verify $(COMPILE)
$(PYTHON) test_pythonlib.py --ok-3.2 --syntax-verify $(COMPILE)
#: Run longer Python 3.4's lib files known to be okay
check-3.4-ok:
$(PYTHON) test_pythonlib.py --ok-3.4 --weak-verify $(COMPILE)
$(PYTHON) test_pythonlib.py --ok-3.4 --syntax-verify $(COMPILE)
#: PyPy of some sort. E.g. [PyPy 5.0.1 with GCC 4.8.4]
# Skip for now

BIN
test/bytecode_3.0_run/04_def_annotate.pyc Normal file
View File

Binary file not shown.

BIN
test/bytecode_3.1_run/04_def_annotate.pyc Normal file
View File

Binary file not shown.

BIN
test/bytecode_3.2_run/04_def_annotate.pyc Normal file
View File

Binary file not shown.

BIN
test/bytecode_3.3/09_yield_from.pyc Normal file
View File

Binary file not shown.

BIN
test/bytecode_3.3_run/04_def_annotate.pyc
View File

Binary file not shown.

BIN
test/bytecode_3.3_run/08_while_if.pyc Normal file
View File

Binary file not shown.

BIN
test/bytecode_3.4/09_yield_from.pyc Normal file
View File

Binary file not shown.

BIN
test/bytecode_3.4_run/04_def_annotate.pyc Normal file
View File

Binary file not shown.

BIN
test/bytecode_3.4_run/08_while_if.pyc Normal file
View File

Binary file not shown.

BIN
test/bytecode_3.5_run/04_def_annotate.pyc
View File

Binary file not shown.

BIN
test/bytecode_3.6_run/04_def_annotate.pyc
View File

Binary file not shown.

BIN
test/bytecode_3.7_run/04_def_annotate.pyc
View File

Binary file not shown.

2
test/run-and-email.sh
View File

@@ -42,7 +42,7 @@ for VERSION in $PYVERSION ; do
echo Python Version $(pyenv local) > $LOGFILE
echo "" >> $LOGFILE
typeset -i ALL_FILES_STARTTIME=$(date +%s)
python ./test_pyenvlib.py --max ${MAX_TESTS} --weak-verify --$VERSION >>$LOGFILE 2>&1
python ./test_pyenvlib.py --max ${MAX_TESTS} --syntax-verify --$VERSION >>$LOGFILE 2>&1
rc=$?
echo Python Version $(pyenv local) >> $LOGFILE

31
test/simple_source/bug31/04_def_annotate.py
View File

@@ -31,9 +31,28 @@ def test9(arg_1=55, *varargs: int, y=5, **kwargs):
def test10(args_1, b: 'annotating b', c: int) -> float:
return 5.4
class IOBase:
def test11(*, name):
return args, name
def test12(a, *args, name):
return a, args
pass
def test13(*args, name):
return args, name
def test14(*args, name: int=1, qname):
return args, name, qname
def test15(*args, name='S', fname, qname=4):
return args, name, fname, qname
# From 3.4 /asyncio/streams.py open_connection
_DEFAULT_LIMIT = 5
def test16(host=None, port=None, *,
loop=None, limit=_DEFAULT_LIMIT, **kwds):
return host, port, loop, limit, kwds
# Python 3.1 _pyio.py uses the -> "IOBase" annotation
def o(f, mode = "r", buffering = None) -> "IOBase":
return (f, mode, buffering)
@@ -109,6 +128,10 @@ def ann2(args_1, b: int = 5, **kwargs: float) -> float:
assert ann2.__annotations__['return'] == float
assert b == 5
class TestSignatureObject():
def test_signature_on_wkwonly(self):
def test(x:int=55, *args: (int, str), c='test', a:float, kwargs:str="S", **b: int) -> int:
pass
assert test1(1, 5) == (1, 5, 4, {})
assert test1(1, 5, 6, foo='bar') == (1, 5, 6, {'foo': 'bar'})
@@ -121,3 +144,9 @@ assert test6(2.3, 4, 5) == (2.3, 4, 5)
ann1(1, 'test', 5)
ann2(1)
### FIXME: fill in...
assert test12(1, 2, 3, name='hi') == (1, (2, 3)), "a, *args, name"
assert test13(1, 2, 3, name='hi') == ((1, 2, 3), 'hi'), "*args, name"
assert test16('localhost', loop=2, limit=3, a='b') == ('localhost', None, 2, 3, {'a': 'b'})

34
test/simple_source/bug34/08_while_if.py Normal file
View File

@@ -0,0 +1,34 @@
# Testing "while 1" versus "while" handling with if/elif/else's
def while_test(a, b, c):
while a != 2:
if b:
a += 1
elif c:
c = 0
else:
break
return a, b, c
def while1_test(a, b, c):
while 1:
if a != 2:
if b:
a = 3
b = 0
elif c:
c = 0
else:
a += b + c
break
return a, b, c
assert while_test(2, 0, 0) == (2, 0, 0), "no while loops"
assert while_test(0, 1, 0) == (2, 1, 0), "two while loops of b branch"
assert while_test(0, 0, 0) == (0, 0, 0), "0 while loops, else branch"
# FIXME: put this in a timer, and try with a=2
assert while1_test(4, 1, 1) == (3, 0, 0), "three while1 loops"
assert while1_test(4, 0, 0) == (4, 0, 0), " one while1 loop"

3
test/simple_source/expression/09_yield_from.py
View File

@@ -11,6 +11,9 @@
def _walk_dir(dir, dfile, ddir=None):
yield from _walk_dir(dir, ddir=dfile)
def ybug(g):
yield from g
# From 3.5.1 _wakrefset.py
#
# 3.5:

4
test/test_pyenvlib.py
View File

@@ -136,7 +136,7 @@ if __name__ == '__main__':
test_options_keys = list(test_options.keys())
test_options_keys.sort()
opts, args = getopt.getopt(sys.argv[1:], '',
['start-with=', 'verify', 'verify-run', 'weak-verify',
['start-with=', 'verify', 'verify-run', 'syntax-verify',
'max=', 'coverage', 'all', ] \
+ test_options_keys )
vers = ''
@@ -144,7 +144,7 @@ if __name__ == '__main__':
for opt, val in opts:
if opt == '--verify':
do_verify = 'strong'
elif opt == '--weak-verify':
elif opt == '--syntax-verify':
do_verify = 'weak'
elif opt == '--verify-run':
do_verify = 'verify-run'

4
test/test_pythonlib.py
View File

@@ -193,7 +193,7 @@ if __name__ == '__main__':
test_options_keys.sort()
opts, args = getopt.getopt(sys.argv[1:], '',
['start-with=', 'verify', 'verify-run',
'weak-verify', 'all',
'syntax-verify', 'all',
'compile', 'coverage',
'no-rm'] \
+ test_options_keys )
@@ -210,7 +210,7 @@ if __name__ == '__main__':
for opt, val in opts:
if opt == '--verify':
test_opts['do_verify'] = 'strong'
elif opt == '--weak-verify':
elif opt == '--syntax-verify':
test_opts['do_verify'] = 'weak'
elif opt == '--verify-run':
test_opts['do_verify'] = 'verify-run'

8
uncompyle6/bin/uncompile.py
View File

@@ -37,7 +37,7 @@ Options:
--fragments use fragments deparser
--verify compare generated source with input byte-code
--verify-run compile generated source, run it and check exit code
--weak-verify compile generated source
--syntax-verify compile generated source
--linemaps generated line number correspondencies between byte-code
and generated source output
--encoding <encoding>
@@ -86,9 +86,9 @@ def main_bin():
'help asm compile= grammar linemaps recurse '
'timestamp tree tree+ '
'fragments verify verify-run version '
'weak-verify '
'syntax-verify '
'showgrammar encoding='.split(' '))
except getopt.GetoptError(e):
except getopt.GetoptError, e:
sys.stderr.write('%s: %s\n' %
(os.path.basename(sys.argv[0]), e))
sys.exit(-1)
@@ -103,7 +103,7 @@ def main_bin():
sys.exit(0)
elif opt == '--verify':
options['do_verify'] = 'strong'
elif opt == '--weak-verify':
elif opt == '--syntax-verify':
options['do_verify'] = 'weak'
elif opt == '--fragments':
options['do_fragments'] = True

47
uncompyle6/parsers/parse3.py
View File

@@ -153,10 +153,7 @@ class Python3Parser(PythonParser):
_ifstmts_jump ::= c_stmts_opt COME_FROM
iflaststmt ::= testexpr c_stmts_opt JUMP_ABSOLUTE
iflaststmtl ::= testexpr c_stmts_opt JUMP_BACK
iflaststmtl ::= testexpr c_stmts_opt JUMP_BACK COME_FROM_LOOP
iflaststmtl ::= testexpr c_stmts_opt JUMP_BACK POP_BLOCK
# These are used to keep parse tree indices the same
jump_forward_else ::= JUMP_FORWARD ELSE
@@ -182,6 +179,8 @@ class Python3Parser(PythonParser):
ifelsestmtl ::= testexpr c_stmts_opt JUMP_BACK else_suitel
ifelsestmtl ::= testexpr c_stmts_opt cf_jump_back else_suitel
ifelsestmtl ::= testexpr c_stmts_opt continue else_suitel
cf_jump_back ::= COME_FROM JUMP_BACK
@@ -348,6 +347,8 @@ class Python3Parser(PythonParser):
def p_loop_stmt3(self, args):
"""
stmt ::= whileelsestmt2
for ::= SETUP_LOOP expr for_iter store for_block POP_BLOCK
COME_FROM_LOOP
@@ -363,6 +364,8 @@ class Python3Parser(PythonParser):
whilestmt ::= SETUP_LOOP testexpr l_stmts_opt COME_FROM JUMP_BACK POP_BLOCK
COME_FROM_LOOP
whilestmt ::= SETUP_LOOP testexpr l_stmts_opt JUMP_BACK POP_BLOCK JUMP_BACK
COME_FROM_LOOP
whilestmt ::= SETUP_LOOP testexpr l_stmts_opt JUMP_BACK POP_BLOCK
COME_FROM_LOOP
@@ -375,6 +378,9 @@ class Python3Parser(PythonParser):
whileelsestmt ::= SETUP_LOOP testexpr l_stmts_opt JUMP_BACK POP_BLOCK
else_suitel COME_FROM_LOOP
whileelsestmt2 ::= SETUP_LOOP testexpr l_stmts_opt JUMP_BACK POP_BLOCK
else_suitel JUMP_BACK COME_FROM_LOOP
whileTruestmt ::= SETUP_LOOP l_stmts_opt JUMP_BACK POP_BLOCK
COME_FROM_LOOP
@@ -1018,20 +1024,45 @@ class Python3Parser(PythonParser):
# Note order of kwargs and pos args changed between 3.3-3.4
if self.version <= 3.2:
if annotate_args > 0:
rule = "mkfunc_annotate ::= %s%s%sannotate_tuple load_closure LOAD_CODE %s" % (
kwargs_str,
"pos_arg " * args_pos,
"annotate_arg " * (annotate_args - 1),
opname,
)
else:
rule = "mkfunc ::= %s%sload_closure LOAD_CODE %s" % (
kwargs_str,
"expr " * args_pos,
"pos_arg " * args_pos,
opname,
)
elif self.version == 3.3:
rule = "mkfunc ::= %s%sload_closure LOAD_CODE LOAD_STR %s" % (
if annotate_args > 0:
rule = "mkfunc_annotate ::= %s%s%sannotate_tuple load_closure LOAD_CODE LOAD_STR %s" % (
kwargs_str,
"expr " * args_pos,
"pos_arg " * args_pos,
"annotate_arg " * (annotate_args - 1),
opname,
)
elif self.version >= 3.4:
else:
rule = "mkfunc ::= %s%sload_closure LOAD_CODE LOAD_STR %s" % (
"expr " * args_pos,
kwargs_str,
"pos_arg " * args_pos,
opname,
)
elif self.version >= 3.4:
if annotate_args > 0:
rule = "mkfunc_annotate ::= %s%s%sannotate_tuple load_closure LOAD_CODE LOAD_STR %s" % (
"pos_arg " * args_pos,
kwargs_str,
"annotate_arg " * (annotate_args - 1),
opname,
)
else:
rule = "mkfunc ::= %s%s load_closure LOAD_CODE LOAD_STR %s" % (
"pos_arg " * args_pos,
kwargs_str,
opname,
)

5
uncompyle6/semantics/consts.py
View File

@@ -377,6 +377,7 @@ TABLE_DIRECT = {
'while1stmt': ( '%|while 1:\n%+%c%-\n\n', 1 ),
'while1elsestmt': ( '%|while 1:\n%+%c%-%|else:\n%+%c%-\n\n', 1, -2 ),
'whileelsestmt': ( '%|while %c:\n%+%c%-%|else:\n%+%c%-\n\n', 1, 2, -2 ),
'whileelsestmt2': ( '%|while %c:\n%+%c%-%|else:\n%+%c%-\n\n', 1, 2, -3 ),
'whileelselaststmt': ( '%|while %c:\n%+%c%-%|else:\n%+%c%-', 1, 2, -2 ),
# Note: Python 3.8+ changes this
@@ -408,7 +409,9 @@ TABLE_DIRECT = {
'tf_tryelsestmt': ( '%c%-%c%|else:\n%+%c', 1, 3, 4 ),
'tryfinallystmt': ( '%|try:\n%+%c%-%|finally:\n%+%c%-\n\n', 1, 5 ),
'except': ( '%|except:\n%+%c%-', 3 ),
'except_cond1': ( '%|except %c:\n', 1 ),
'except_cond1': ( '%|except %c:\n', (1, 'expr') ),
'except_cond2': ( '%|except %c as %c:\n',
(1, 'expr'), (5, 'store') ),
'except_suite': ( '%+%c%-%C', 0, (1, maxint, '') ),
# In Python 3.6, this is more complicated in the presence of "returns"

4
uncompyle6/semantics/customize.py
View File

@@ -57,6 +57,10 @@ def customize_for_version(self, is_pypy, version):
from uncompyle6.semantics.customize3 import customize_for_version3
customize_for_version3(self, version)
else: # < 3.0
TABLE_DIRECT.update({
'except_cond3' : ( '%|except %c, %c:\n',
(1, 'expr'), (-2, 'store') )
})
if 2.4 <= version <= 2.6:
TABLE_DIRECT.update({
'comp_for': ( ' for %c in %c', 3, 1 ),

2
uncompyle6/semantics/customize25.py
View File

@@ -26,8 +26,6 @@ def customize_for_version25(self, version):
# Import style for 2.5+
########################
TABLE_DIRECT.update({
'except_cond3' : ( '%|except %c, %c:\n',
(1, 'expr'), (-2, 'store') ),
'importmultiple': ( '%|import %c%c\n', 2, 3 ),
'import_cont' : ( ', %c', 2 ),
# With/as is allowed as "from future" thing in 2.5

199
uncompyle6/semantics/customize3.py
View File

@@ -26,27 +26,31 @@ from uncompyle6.semantics.customize36 import customize_for_version36
from uncompyle6.semantics.customize37 import customize_for_version37
from uncompyle6.semantics.customize38 import customize_for_version38
def customize_for_version3(self, version):
TABLE_DIRECT.update({
'comp_for' : ( ' for %c in %c',
(2, 'store') , (0, 'expr') ),
'conditionalnot' : ( '%c if not %c else %c',
(2, 'expr') , (0, 'expr'), (4, 'expr') ),
'except_cond2' : ( '%|except %c as %c:\n', 1, 5 ),
'function_def_annotate': ( '\n\n%|def %c%c\n', -1, 0),
def customize_for_version3(self, version):
TABLE_DIRECT.update(
{
"comp_for": (" for %c in %c", (2, "store"), (0, "expr")),
"conditionalnot": (
"%c if not %c else %c",
(2, "expr"),
(0, "expr"),
(4, "expr"),
),
"except_cond2": ("%|except %c as %c:\n", 1, 5),
"function_def_annotate": ("\n\n%|def %c%c\n", -1, 0),
# When a generator is a single parameter of a function,
# it doesn't need the surrounding parenethesis.
'call_generator' : ('%c%P', 0, (1, -1, ', ', 100)),
'importmultiple' : ( '%|import %c%c\n', 2, 3 ),
'import_cont' : ( ', %c', 2 ),
'kwarg' : ( '%[0]{attr}=%c', 1),
'raise_stmt2' : ( '%|raise %c from %c\n', 0, 1),
'store_locals' : ( '%|# inspect.currentframe().f_locals = __locals__\n', ),
'withstmt' : ( '%|with %c:\n%+%c%-', 0, 3),
'withasstmt' : ( '%|with %c as (%c):\n%+%c%-', 0, 2, 3),
})
"call_generator": ("%c%P", 0, (1, -1, ", ", 100)),
"importmultiple": ("%|import %c%c\n", 2, 3),
"import_cont": (", %c", 2),
"kwarg": ("%[0]{attr}=%c", 1),
"raise_stmt2": ("%|raise %c from %c\n", 0, 1),
"store_locals": ("%|# inspect.currentframe().f_locals = __locals__\n",),
"withstmt": ("%|with %c:\n%+%c%-", 0, 3),
"withasstmt": ("%|with %c as (%c):\n%+%c%-", 0, 2, 3),
}
)
assert version >= 3.0
@@ -61,7 +65,7 @@ def customize_for_version3(self, version):
# ----------
# * subclass_code - the code for the subclass body
subclass_info = None
if node == 'classdefdeco2':
if node == "classdefdeco2":
if self.version >= 3.6:
class_name = node[1][1].attr
elif self.version <= 3.3:
@@ -72,17 +76,17 @@ def customize_for_version3(self, version):
else:
build_class = node[0]
if self.version >= 3.6:
if build_class == 'build_class_kw':
if build_class == "build_class_kw":
mkfunc = build_class[1]
assert mkfunc == 'mkfunc'
assert mkfunc == "mkfunc"
subclass_info = build_class
if hasattr(mkfunc[0], 'attr') and iscode(mkfunc[0].attr):
if hasattr(mkfunc[0], "attr") and iscode(mkfunc[0].attr):
subclass_code = mkfunc[0].attr
else:
assert mkfunc[0] == 'load_closure'
assert mkfunc[0] == "load_closure"
subclass_code = mkfunc[1].attr
assert iscode(subclass_code)
if build_class[1][0] == 'load_closure':
if build_class[1][0] == "load_closure":
code_node = build_class[1][1]
else:
code_node = build_class[1][0]
@@ -91,72 +95,72 @@ def customize_for_version3(self, version):
class_name = node[1][0].attr
build_class = node[0]
assert 'mkfunc' == build_class[1]
assert "mkfunc" == build_class[1]
mkfunc = build_class[1]
if mkfunc[0] in ('kwargs', 'no_kwargs'):
if mkfunc[0] in ("kwargs", "no_kwargs"):
if 3.0 <= self.version <= 3.2:
for n in mkfunc:
if hasattr(n, 'attr') and iscode(n.attr):
if hasattr(n, "attr") and iscode(n.attr):
subclass_code = n.attr
break
elif n == 'expr':
elif n == "expr":
subclass_code = n[0].attr
pass
pass
else:
for n in mkfunc:
if hasattr(n, 'attr') and iscode(n.attr):
if hasattr(n, "attr") and iscode(n.attr):
subclass_code = n.attr
break
pass
pass
if node == 'classdefdeco2':
if node == "classdefdeco2":
subclass_info = node
else:
subclass_info = node[0]
elif build_class[1][0] == 'load_closure':
elif build_class[1][0] == "load_closure":
# Python 3 with closures not functions
load_closure = build_class[1]
if hasattr(load_closure[-3], 'attr'):
if hasattr(load_closure[-3], "attr"):
# Python 3.3 classes with closures work like this.
# Note have to test before 3.2 case because
# index -2 also has an attr.
subclass_code = load_closure[-3].attr
elif hasattr(load_closure[-2], 'attr'):
elif hasattr(load_closure[-2], "attr"):
# Python 3.2 works like this
subclass_code = load_closure[-2].attr
else:
raise 'Internal Error n_classdef: cannot find class body'
if hasattr(build_class[3], '__len__'):
raise "Internal Error n_classdef: cannot find class body"
if hasattr(build_class[3], "__len__"):
if not subclass_info:
subclass_info = build_class[3]
elif hasattr(build_class[2], '__len__'):
elif hasattr(build_class[2], "__len__"):
subclass_info = build_class[2]
else:
raise 'Internal Error n_classdef: cannot superclass name'
elif self.version >= 3.6 and node == 'classdefdeco2':
raise "Internal Error n_classdef: cannot superclass name"
elif self.version >= 3.6 and node == "classdefdeco2":
subclass_info = node
subclass_code = build_class[1][0].attr
elif not subclass_info:
if mkfunc[0] in ('no_kwargs', 'kwargs'):
if mkfunc[0] in ("no_kwargs", "kwargs"):
subclass_code = mkfunc[1].attr
else:
subclass_code = mkfunc[0].attr
if node == 'classdefdeco2':
if node == "classdefdeco2":
subclass_info = node
else:
subclass_info = node[0]
if (node == 'classdefdeco2'):
self.write('\n')
if node == "classdefdeco2":
self.write("\n")
else:
self.write('\n\n')
self.write("\n\n")
self.currentclass = str(class_name)
self.write(self.indent, 'class ', self.currentclass)
self.write(self.indent, "class ", self.currentclass)
self.print_super_classes3(subclass_info)
self.println(':')
self.println(":")
# class body
self.indent_more()
@@ -165,11 +169,12 @@ def customize_for_version3(self, version):
self.currentclass = cclass
if len(self.param_stack) > 1:
self.write('\n\n')
self.write("\n\n")
else:
self.write('\n\n\n')
self.write("\n\n\n")
self.prune()
self.n_classdef3 = n_classdef3
if version == 3.0:
@@ -178,28 +183,28 @@ def customize_for_version3(self, version):
# since we pick up the iteration variable some other way and
# we definitely don't include in the source _[dd].
def n_comp_iter(node):
if node[0] == 'expr':
if node[0] == "expr":
n = node[0][0]
if (n == 'LOAD_FAST' and
n.pattr[0:2] == '_['):
if n == "LOAD_FAST" and n.pattr[0:2] == "_[":
self.prune()
pass
pass
# Not this special case, procede as normal...
# Not this special case, proceed as normal...
self.default(node)
self.n_comp_iter = n_comp_iter
if version >= 3.3:
elif version == 3.3:
# FIXME: perhaps this can be folded into the 3.4+ case?
def n_yield_from(node):
self.write('yield from')
self.write(' ')
if 3.3 <= self.version <= 3.4:
self.preorder(node[0][0][0][0])
elif self.version >= 3.5:
self.preorder(node[0])
else:
assert False, "dunno about this python version"
self.prune() # stop recursing
assert node[0] == "expr"
assert node[0][0] == "get_iter"
# Skip over yield_from.expr.get_iter which adds an
# extra iter(). Maybe we can do in tranformation phase instead?
template = ("yield from %c", (0, "expr"))
self.template_engine(template, node[0][0])
self.prune()
self.n_yield_from = n_yield_from
if 3.2 <= version <= 3.4:
@@ -211,11 +216,11 @@ def customize_for_version3(self, version):
for i in mapping[1:]:
key = key[i]
pass
if key.kind.startswith('CALL_FUNCTION_VAR_KW'):
if key.kind.startswith("CALL_FUNCTION_VAR_KW"):
# We may want to fill this in...
# But it is distinct from CALL_FUNCTION_VAR below
pass
elif key.kind.startswith('CALL_FUNCTION_VAR'):
elif key.kind.startswith("CALL_FUNCTION_VAR"):
# CALL_FUNCTION_VAR's top element of the stack contains
# the variable argument list, then comes
# annotation args, then keyword args.
@@ -229,12 +234,15 @@ def customize_for_version3(self, version):
# kwargs == 0 is handled by the table entry
# Should probably handle it here though.
if nargs == 0:
template = ('%c(*%c, %C)',
0, -2, (1, kwargs+1, ', '))
template = ("%c(*%c, %C)", 0, -2, (1, kwargs + 1, ", "))
else:
template = ('%c(%C, *%c, %C)',
0, (1, nargs+1, ', '),
-2, (-2-kwargs, -2, ', '))
template = (
"%c(%C, *%c, %C)",
0,
(1, nargs + 1, ", "),
-2,
(-2 - kwargs, -2, ", "),
)
self.template_engine(template, node)
self.prune()
else:
@@ -243,6 +251,7 @@ def customize_for_version3(self, version):
self.n_call = n_call
elif version < 3.2:
def n_call(node):
mapping = self._get_mapping(node)
key = node
@@ -251,17 +260,26 @@ def customize_for_version3(self, version):
pass
gen_function_parens_adjust(key, node)
self.default(node)
self.n_call = n_call
def n_mkfunc_annotate(node):
if self.version >= 3.3 or node[-2] == 'kwargs':
# Handling EXTENDED_ARG before MAKE_FUNCTION ...
if node[-2] == "EXTENDED_ARG":
i = -1
else:
i = 0
if self.version <= 3.2:
code = node[-2 + i]
elif self.version >= 3.3 or node[-2] == "kwargs":
# LOAD_CONST code object ..
# LOAD_CONST 'x0' if >= 3.3
# EXTENDED_ARG
# MAKE_FUNCTION ..
code = node[-4]
elif node[-3] == 'expr':
code = node[-3 + i]
elif node[-3] == "expr":
code = node[-3][0]
else:
# LOAD_CONST code object ..
@@ -270,41 +288,50 @@ def customize_for_version3(self, version):
self.indent_more()
for annotate_last in range(len(node) - 1, -1, -1):
if node[annotate_last] == 'annotate_tuple':
if node[annotate_last] == "annotate_tuple":
break
# FIXME: the real situation is that when derived from
# function_def_annotate we the name has been filled in.
# But when derived from funcdefdeco it hasn't Would like a better
# way to distinquish.
if self.f.getvalue()[-4:] == 'def ':
if self.f.getvalue()[-4:] == "def ":
self.write(code.attr.co_name)
# FIXME: handle and pass full annotate args
make_function3_annotate(self, node, is_lambda=False,
code_node=code, annotate_last=annotate_last)
make_function3_annotate(
self, node, is_lambda=False, code_node=code, annotate_last=annotate_last
)
if len(self.param_stack) > 1:
self.write('\n\n')
self.write("\n\n")
else:
self.write('\n\n\n')
self.write("\n\n\n")
self.indent_less()
self.prune() # stop recursing
self.n_mkfunc_annotate = n_mkfunc_annotate
TABLE_DIRECT.update({
'tryelsestmtl3': ( '%|try:\n%+%c%-%c%|else:\n%+%c%-',
(1, 'suite_stmts_opt'),
(3, 'except_handler'),
(5, 'else_suitel') ),
})
TABLE_DIRECT.update(
{
"tryelsestmtl3": (
"%|try:\n%+%c%-%c%|else:\n%+%c%-",
(1, "suite_stmts_opt"),
(3, "except_handler"),
(5, "else_suitel"),
)
}
)
if version >= 3.4:
#######################
# Python 3.4+ Changes #
#######################
TABLE_DIRECT.update({
'LOAD_CLASSDEREF': ( '%{pattr}', ),
})
TABLE_DIRECT.update(
{
"LOAD_CLASSDEREF": ("%{pattr}",),
"yield_from": ("yield from %c", (0, "expr")),
}
)
if version >= 3.5:
customize_for_version35(self, version)
if version >= 3.6:

79
uncompyle6/semantics/make_function.py
View File

@@ -1,4 +1,4 @@
# Copyright (c) 2015-2018 by Rocky Bernstein
# Copyright (c) 2015-2019 by Rocky Bernstein
# Copyright (c) 2000-2002 by hartmut Goebel <h.goebel@crazy-compilers.com>
#
# This program is free software: you can redistribute it and/or modify
@@ -67,7 +67,7 @@ def make_function3_annotate(self, node, is_lambda, nested=1,
i = -1
j = annotate_last-1
l = -len(node)
while j >= l and node[j].kind in ('annotate_arg' 'annotate_tuple'):
while j >= l and node[j].kind in ('annotate_arg', 'annotate_tuple'):
annotate_args[annotate_tup[i]] = node[j][0]
i -= 1
j -= 1
@@ -142,8 +142,6 @@ def make_function3_annotate(self, node, is_lambda, nested=1,
i = len(paramnames) - len(defparams)
suffix = ''
no_paramnames = len(paramnames[:i]) == 0
for param in paramnames[:i]:
self.write(suffix, param)
suffix = ', '
@@ -164,7 +162,6 @@ def make_function3_annotate(self, node, is_lambda, nested=1,
suffix = ''
for n in node:
if n == 'pos_arg':
no_paramnames = False
self.write(suffix)
param = paramnames[i]
self.write(param)
@@ -198,19 +195,16 @@ def make_function3_annotate(self, node, is_lambda, nested=1,
# self.println(indent, '#flags:\t', int(code.co_flags))
ends_in_comma = False
if kwonlyargcount > 0:
if no_paramnames:
if not code_has_star_arg(code):
if argc > 0:
self.write(", *, ")
else:
self.write("*, ")
pass
else:
self.write(", ")
ends_in_comma = True
else:
if argc > 0:
self.write(', ')
self.write(", ")
ends_in_comma = True
kw_args = [None] * kwonlyargcount
@@ -230,8 +224,8 @@ def make_function3_annotate(self, node, is_lambda, nested=1,
pass
# handling other args
ann_other_kw = [c == None for c in kw_args]
for i, flag in enumerate(ann_other_kw):
other_kw = [c == None for c in kw_args]
for i, flag in enumerate(other_kw):
if flag:
n = kwargs[i]
if n in annotate_dict:
@@ -239,7 +233,8 @@ def make_function3_annotate(self, node, is_lambda, nested=1,
else:
kw_args[i] = "%s" % n
self.write(', '.join(kw_args), ', ')
self.write(', '.join(kw_args))
ends_in_comma = False
else:
if argc == 0:
@@ -705,7 +700,6 @@ def make_function3(self, node, is_lambda, nested=1, code_node=None):
kw_pairs = 0
i = len(paramnames) - len(defparams)
no_paramnames = len(paramnames[:i]) == 0
# build parameters
params = []
@@ -768,52 +762,35 @@ def make_function3(self, node, is_lambda, nested=1, code_node=None):
# Unless careful, We might lose line breaks though.
ends_in_comma = False
if kwonlyargcount > 0:
if no_paramnames:
if not (4 & code.co_flags):
if argc > 0:
self.write(", *, ")
else:
self.write("*, ")
pass
else:
self.write(", ")
ends_in_comma = True
else:
if argc > 0:
self.write(', ')
self.write(", ")
ends_in_comma = True
# FIXME: this is not correct for 3.5. or 3.6 (which works different)
# and 3.7?
if 3.0 <= self.version <= 3.2:
kwargs = node[0]
last = len(kwargs)-1
i = 0
for n in node[0]:
if n == 'kwarg':
self.write('%s=' % n[0].attr)
self.preorder(n[1])
if i < last:
self.write(', ')
ends_in_comma = True
pass
else:
if 3.0 <= self.version <= 3.5:
kw_args = [None] * kwonlyargcount
kw_nodes = node[0]
if kw_nodes == "kwargs":
for n in kw_nodes:
name = eval(n[0].pattr)
default = self.traverse(n[1], indent='')
idx = kwargs.index(name)
kw_args[idx] = "%s=%s" % (name, default)
other_kw = [c == None for c in kw_args]
for i, flag in enumerate(other_kw):
if flag:
kw_args[i] = "%s" % kwargs[i]
self.write(', '.join(kw_args))
ends_in_comma = False
pass
i += 1
pass
pass
elif self.version <= 3.5:
# FIXME this is not qute right for 3.5
for n in node:
if n == 'pos_arg':
continue
elif self.version >= 3.4 and not (n.kind in ('kwargs', 'no_kwargs', 'kwarg')):
continue
else:
self.preorder(n)
ends_in_comma = False
break
elif self.version >= 3.6:
# argc = node[-1].attr
# co = node[-3].attr
@@ -862,16 +839,16 @@ def make_function3(self, node, is_lambda, nested=1, code_node=None):
pass
# handle others
if ann_dict:
ann_other_kw = [c == None for c in kw_args]
other_kw = [c == None for c in kw_args]
for i, flag in enumerate(ann_other_kw):
for i, flag in enumerate(other_kw):
if flag:
n = kwargs[i]
if n in annotate_dict:
if ann_dict and n in annotate_dict:
kw_args[i] = "%s: %s" %(n, annotate_dict[n])
else:
kw_args[i] = "%s" % n
self.write(', '.join(kw_args))
ends_in_comma = False

35
uncompyle6/semantics/pysource.py
View File

@@ -661,16 +661,33 @@ class SourceWalker(GenericASTTraversal, object):
if ..
elif ...
[else ...]
where appropriate
"""
else_suite = node[3]
n = else_suite[0]
old_stmts = None
if len(n) == 1 == len(n[0]) and n[0] == 'stmt':
n = n[0][0]
elif n[0].kind in ('lastc_stmt', 'lastl_stmt'):
n = n[0]
if n[0].kind in ('ifstmt', 'iflaststmt', 'iflaststmtl', 'ifelsestmtl', 'ifelsestmtc'):
# This seems needed for Python 2.5-2.7
n = n[0]
pass
pass
elif ( len(n) > 1 and 1 == len(n[0]) and n[0] == 'stmt'
and n[1].kind == "stmt" ):
else_suite_stmts = n[0]
if else_suite_stmts[0].kind not in ('ifstmt', 'iflaststmt', 'ifelsestmtl'):
if not preprocess:
self.default(node)
return
old_stmts = n
n = else_suite_stmts[0]
else:
if not preprocess:
self.default(node)
@@ -690,6 +707,18 @@ class SourceWalker(GenericASTTraversal, object):
elif n.kind in ('ifelsestmt', 'ifelsestmtc', 'ifelsestmtl'):
n.kind = 'elifelsestmt'
if not preprocess:
if old_stmts:
if n.kind == "elifstmt":
trailing_else = SyntaxTree("stmts", old_stmts[1:])
# We use elifelsestmtr because it has 3 nodes
elifelse_stmt = SyntaxTree(
'elifelsestmtr', [n[0], n[1], trailing_else])
node[3] = elifelse_stmt
pass
else:
# Other cases for n.kind may happen here
return
pass
self.default(node)
n_ifelsestmtc = n_ifelsestmtl = n_ifelsestmt
@@ -1798,9 +1827,15 @@ class SourceWalker(GenericASTTraversal, object):
self.write(', ')
self.prune()
return
for n in node[1:]:
if n[0].kind == 'unpack':
n[0].kind = 'unpack_w_parens'
# In Python 2.4, unpack is used in (a, b, c) of:
# except RuntimeError, (a, b, c):
if self.version < 2.7:
node.kind = 'unpack_w_parens'
self.default(node)
n_unpack_w_parens = n_unpack

2
uncompyle6/version.py
View File

@@ -12,4 +12,4 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# This file is suitable for sourcing inside bash as
# well as importing into Python
VERSION='3.3.4' # noqa
VERSION="3.3.5" # noqa