You've already forked python-uncompyle6
mirror of
https://github.com/rocky/python-uncompyle6.git
synced 2025-08-03 16:59:52 +08:00
Compare commits
45 Commits
release-py
...
release-py
Author | SHA1 | Date | |
---|---|---|---|
|
f7a8aabdee | ||
|
214f5f32a3 | ||
|
476db9ad1d | ||
|
9ba0b1bad2 | ||
|
b7942bc5f2 | ||
|
3ac4f1ee61 | ||
|
55d2df04f7 | ||
|
6d529d3cee | ||
|
53b471a3df | ||
|
201e5b18b1 | ||
|
ac2bbfc65a | ||
|
3b8e6635e2 | ||
|
7e172b63d1 | ||
|
c01eb554ed | ||
|
d32e67891b | ||
|
2b730628d5 | ||
|
78b8d1cd06 | ||
|
600e56b1d7 | ||
|
48006ab350 | ||
|
170504c518 | ||
|
b3642094b2 | ||
|
a574168ca8 | ||
|
e3d918df3d | ||
|
263b4b5653 | ||
|
e7b62a722f | ||
|
92d63ac598 | ||
|
79bed3419f | ||
|
0353b74a7a | ||
|
67910e7d8e | ||
|
61fa4fe391 | ||
|
19818ae632 | ||
|
a8cdcc4d85 | ||
|
477d73c71d | ||
|
f0176add7a | ||
|
7272ac4a60 | ||
|
45c8d62e68 | ||
|
096563cf91 | ||
|
7fd21aa227 | ||
|
82bc294995 | ||
|
9d3e4a6660 | ||
|
7dfade1195 | ||
|
1df5aa0ef9 | ||
|
c06ba45991 | ||
|
7fab91eb4e | ||
|
7659277c5c |
13
NEWS
13
NEWS
@@ -1,3 +1,16 @@
|
||||
uncompyle6 3.2.3 2018-06-04 Michael Cohen flips and Fleetwood Redux
|
||||
|
||||
- Python 1.3 support 3.0 bug and
|
||||
- fix botched parameter ordering of 3.x in last release
|
||||
|
||||
uncompyle6 3.2.2 2018-06-04 When I'm 64
|
||||
|
||||
- Python 3.0 support and bug fixes
|
||||
|
||||
uncompyle6 3.2.1 2018-06-04 MF
|
||||
|
||||
- Python 1.4 and 1.5 bug fixes
|
||||
|
||||
uncompyle6 3.2.0 2018-05-19 Rocket Scientist
|
||||
|
||||
- Add rudimentary 1.4 support (still a bit buggy)
|
||||
|
65
README.rst
65
README.rst
@@ -1,4 +1,4 @@
|
||||
|buildstatus|
|
||||
|buildstatus| |Latest Version| |Supported Python Versions|
|
||||
|
||||
uncompyle6
|
||||
==========
|
||||
@@ -11,8 +11,9 @@ Introduction
|
||||
------------
|
||||
|
||||
*uncompyle6* translates Python bytecode back into equivalent Python
|
||||
source code. It accepts bytecodes from Python version 1.5, and 2.1 to
|
||||
3.7 or so, including PyPy bytecode and Dropbox's Python 2.5 bytecode.
|
||||
source code. It accepts bytecodes from Python version 1.3 to version
|
||||
3.7, spanning over 22 years of Python releases. We include Dropbox's
|
||||
Python 2.5 bytecode and some PyPy bytecode.
|
||||
|
||||
Why this?
|
||||
---------
|
||||
@@ -29,11 +30,11 @@ CPython bytecode decompilers is the ability to deparse just
|
||||
*fragments* of source code and give source-code information around a
|
||||
given bytecode offset.
|
||||
|
||||
I use the tree fragments to deparse fragments of code inside my
|
||||
trepan_ debuggers_. For that, bytecode offsets are recorded and
|
||||
associated with fragments of the source code. This purpose, although
|
||||
compatible with the original intention, is yet a little bit different.
|
||||
See this_ for more information.
|
||||
I use the tree fragments to deparse fragments of code *at run time*
|
||||
inside my trepan_ debuggers_. For that, bytecode offsets are recorded
|
||||
and associated with fragments of the source code. This purpose,
|
||||
although compatible with the original intention, is yet a little bit
|
||||
different. See this_ for more information.
|
||||
|
||||
Python fragment deparsing given an instruction offset is useful in
|
||||
showing stack traces and can be encorporated into any program that
|
||||
@@ -58,7 +59,7 @@ provides decompilation for subset of Python versions, we generally do
|
||||
demonstrably better for those as well.
|
||||
|
||||
How can we tell? By taking Python bytecode that comes distributed with
|
||||
that version of Python and decompiling these. Among htose that
|
||||
that version of Python and decompiling these. Among those that
|
||||
successfully decompile, we can then make sure the resulting programs
|
||||
are syntactically correct by running the Python interpreter for that
|
||||
bytecode version. Finally, in cases where the program has a test for
|
||||
@@ -75,7 +76,7 @@ Requirements
|
||||
The code here can be run on Python versions 2.6 or later, PyPy 3-2.4,
|
||||
or PyPy-5.0.1. Python versions 2.4-2.7 are supported in the
|
||||
python-2.4 branch. The bytecode files it can read have been tested on
|
||||
Python bytecodes from versions 1.5, 2.1-2.7, and 3.0-3.6 and the
|
||||
Python bytecodes from versions 1.4, 2.1-2.7, and 3.0-3.6 and the
|
||||
above-mentioned PyPy versions.
|
||||
|
||||
Installation
|
||||
@@ -151,12 +152,12 @@ for that bytecode version. Having done this the bytecode produced
|
||||
could be compared with the original bytecode. However as Python's code
|
||||
generation got better, this is no longer feasible.
|
||||
|
||||
There is a kind of *weak verification* that we use that doesn't check
|
||||
bytecode for equivalence but does check to see if the resulting
|
||||
decompiled source is a valid Python program by running the Python
|
||||
interpreter. Because the Python language has changed so much, for best
|
||||
results you should use the same Python version in checking as was used
|
||||
in creating the bytecode.
|
||||
There verification that we use that doesn't check bytecode for
|
||||
equivalence but does check to see if the resulting decompiled source
|
||||
is a valid Python program by running the Python interpreter. Because
|
||||
the Python language has changed so much, for best results you should
|
||||
use the same Python version in checking as was used in creating the
|
||||
bytecode.
|
||||
|
||||
There are however an interesting class of these programs that is
|
||||
readily available give stronger verification: those programs that
|
||||
@@ -174,19 +175,21 @@ that era was minimal)
|
||||
|
||||
There is some work to do on the lower end Python versions which is
|
||||
more difficult for us to handle since we don't have a Python
|
||||
interpreter for versions 1.5, 1.6, and 2.0.
|
||||
interpreter for versions 1.6, and 2.0.
|
||||
|
||||
In the Python 3 series, Python support is is strongest around 3.4 or
|
||||
3.3 and drops off as you move further away from those versions. Python
|
||||
3.6 changes things drastically by using word codes rather than byte
|
||||
codes. As a result, the jump offset field in a jump instruction
|
||||
argument has been reduced. This makes the `EXTENDED_ARG` instructions
|
||||
are now more prevalent in jump instruction; previously they had been
|
||||
rare. Perhaps to compensate for the additional `EXTENDED_ARG`
|
||||
instructions, additional jump optimization has been added. So in sum
|
||||
handling control flow by ad hoc means as is currently done is worse.
|
||||
3.0 is weird in that it in some ways resembles 2.6 more than it does
|
||||
3.1 or 2.7. Python 3.6 changes things drastically by using word codes
|
||||
rather than byte codes. As a result, the jump offset field in a jump
|
||||
instruction argument has been reduced. This makes the `EXTENDED_ARG`
|
||||
instructions are now more prevalent in jump instruction; previously
|
||||
they had been rare. Perhaps to compensate for the additional
|
||||
`EXTENDED_ARG` instructions, additional jump optimization has been
|
||||
added. So in sum handling control flow by ad hoc means as is currently
|
||||
done is worse.
|
||||
|
||||
Also, between Python 3.5, 3.6 and 3.7 there have been major changes to the
|
||||
Between Python 3.5, 3.6 and 3.7 there have been major changes to the
|
||||
`MAKE_FUNCTION` and `CALL_FUNCTION` instructions.
|
||||
|
||||
Currently not all Python magic numbers are supported. Specifically in
|
||||
@@ -212,11 +215,12 @@ There is lots to do, so please dig in and help.
|
||||
See Also
|
||||
--------
|
||||
|
||||
* https://github.com/zrax/pycdc : supports all versions of Python and is written in C++. Support for Python 3 is a bit lacking though.
|
||||
* https://code.google.com/archive/p/unpyc3/ : supports Python 3.2 only. The above projects use a different decompiling technique than what is used here.
|
||||
* https://github.com/figment/unpyc3/ : fork of above, but supports Python 3.3 only. Includes some fixes like supporting function annotations
|
||||
* The HISTORY_ file.
|
||||
* https://github.com/zrax/pycdc : purports to support all versions of Python. It is written in C++ and is most accurate for Python versions around 2.7 and 3.3 when the code was more actively developed. Accuracy for more recent versions of Python 3 and early versions of Python are especially lacking. See its `issue tracker <https://github.com/zrax/pycdc/issues>`_ for details. Currently lightly maintained.
|
||||
* https://code.google.com/archive/p/unpyc3/ : supports Python 3.2 only. The above projects use a different decompiling technique than what is used here. Currently unmaintained.
|
||||
* https://github.com/figment/unpyc3/ : fork of above, but supports Python 3.3 only. Includes some fixes like supporting function annotations. Currently unmaintained.
|
||||
* https://github.com/wibiti/uncompyle2 : supports Python 2.7 only, but does that fairly well. There situtations where `uncompyle6` results are incorrect while `uncompyle2` results are not, but more often uncompyle6 is correct when uncompyle2 is not. Because `uncompyle6` adheres to accuracy over idiomatic Python, `uncompyle2` can produce more natural-looking code when it is correct. Currently `uncompyle2` is lightly maintained. See its issue `tracker <https://github.com/wibiti/uncompyle2/issues>`_ for more details
|
||||
* `How to report a bug <https://github.com/rocky/python-uncompyle6/blob/master/HOW-TO-REPORT-A-BUG.md>`_
|
||||
* The HISTORY_ file.
|
||||
* https://github.com/rocky/python-xdis : Cross Python version disassembler
|
||||
* https://github.com/rocky/python-xasm : Cross Python version assembler
|
||||
* https://github.com/rocky/python-uncompyle6/wiki : Wiki Documents which describe the code and aspects of it in more detail
|
||||
@@ -234,3 +238,6 @@ See Also
|
||||
.. _PJOrion: http://www.koreanrandom.com/forum/topic/15280-pjorion-%D1%80%D0%B5%D0%B4%D0%B0%D0%BA%D1%82%D0%B8%D1%80%D0%BE%D0%B2%D0%B0%D0%BD%D0%B8%D0%B5-%D0%BA%D0%BE%D0%BC%D0%BF%D0%B8%D0%BB%D1%8F%D1%86%D0%B8%D1%8F-%D0%B4%D0%B5%D0%BA%D0%BE%D0%BC%D0%BF%D0%B8%D0%BB%D1%8F%D1%86%D0%B8%D1%8F-%D0%BE%D0%B1%D1%84
|
||||
.. _Deobfuscator: https://github.com/extremecoders-re/PjOrion-Deobfuscator
|
||||
.. _Py2EXE: https://en.wikipedia.org/wiki/Py2exe
|
||||
.. |Supported Python Versions| image:: https://img.shields.io/pypi/pyversions/uncompyle6.svg
|
||||
.. |Latest Version| image:: https://badge.fury.io/py/uncompyle6.svg
|
||||
:target: https://badge.fury.io/py/uncompyle6
|
||||
|
@@ -35,6 +35,7 @@ classifiers = ['Development Status :: 5 - Production/Stable',
|
||||
'Programming Language :: Python :: 2.5',
|
||||
'Programming Language :: Python :: 2.6',
|
||||
'Programming Language :: Python :: 2.7',
|
||||
'Programming Language :: Python :: 3.0',
|
||||
'Programming Language :: Python :: 3.1',
|
||||
'Programming Language :: Python :: 3.2',
|
||||
'Programming Language :: Python :: 3.3',
|
||||
@@ -56,7 +57,8 @@ entry_points = {
|
||||
]}
|
||||
ftp_url = None
|
||||
install_requires = ['spark-parser >= 1.8.5, < 1.9.0',
|
||||
'xdis >= 3.8.2, < 3.9.0']
|
||||
'xdis >= 3.8.4, < 3.9.0']
|
||||
|
||||
license = 'GPL3'
|
||||
mailing_list = 'python-debugger@googlegroups.com'
|
||||
modname = 'uncompyle6'
|
||||
|
@@ -16,6 +16,7 @@ if ! source ./setup-master.sh ; then
|
||||
fi
|
||||
cd ..
|
||||
for version in $PYVERSIONS; do
|
||||
echo --- $version ---
|
||||
if ! pyenv local $version ; then
|
||||
exit $?
|
||||
fi
|
||||
@@ -23,4 +24,5 @@ for version in $PYVERSIONS; do
|
||||
if ! make check; then
|
||||
exit $?
|
||||
fi
|
||||
echo === $version ===
|
||||
done
|
||||
|
@@ -15,6 +15,7 @@ fi
|
||||
|
||||
cd ..
|
||||
for version in $PYVERSIONS; do
|
||||
echo --- $version ---
|
||||
if ! pyenv local $version ; then
|
||||
exit $?
|
||||
fi
|
||||
@@ -22,4 +23,5 @@ for version in $PYVERSIONS; do
|
||||
if ! make check ; then
|
||||
exit $?
|
||||
fi
|
||||
echo === $version ===
|
||||
done
|
||||
|
@@ -47,7 +47,7 @@ install:
|
||||
|
||||
# Upgrade to the latest version of pip to avoid it displaying warnings
|
||||
# about it being out of date.
|
||||
- "pip install --disable-pip-version-check --user --upgrade pip"
|
||||
- "%PYTHON%\\python.exe -m pip install --disable-pip-version-check --user --upgrade pip"
|
||||
|
||||
# Install the build dependencies of the project. If some dependencies contain
|
||||
# compiled extensions and are not provided as pre-built wheel packages,
|
||||
|
@@ -1,6 +1,6 @@
|
||||
from uncompyle6 import PYTHON_VERSION, deparse_code
|
||||
|
||||
if PYTHON_VERSION >= 2.6:
|
||||
if PYTHON_VERSION > 2.6:
|
||||
def test_single_mode():
|
||||
single_expressions = (
|
||||
'i = 1',
|
||||
|
@@ -3,8 +3,8 @@ import os
|
||||
import difflib
|
||||
import subprocess
|
||||
import tempfile
|
||||
|
||||
from StringIO import StringIO
|
||||
|
||||
# uncompyle6 / xdis
|
||||
from uncompyle6 import PYTHON_VERSION, IS_PYPY, deparse_code
|
||||
# TODO : I think we can get xdis to support the dis api (python 3 version) by doing something like this there
|
||||
@@ -20,8 +20,6 @@ try:
|
||||
except:
|
||||
pass
|
||||
|
||||
|
||||
|
||||
def print_diff(original, uncompyled):
|
||||
"""
|
||||
Try and display a pretty html line difference between the original and
|
||||
|
@@ -1,2 +1,4 @@
|
||||
# Pick up stuff from setup.py
|
||||
hypothesis==2.0.0
|
||||
pytest
|
||||
-e .
|
||||
|
@@ -22,7 +22,7 @@ COVER_DIR=../tmp/grammar-cover
|
||||
# Run short tests
|
||||
check-short:
|
||||
@$(PYTHON) -V && PYTHON_VERSION=`$(PYTHON) -V 2>&1 | cut -d ' ' -f 2 | cut -d'.' -f1,2`; \
|
||||
$(MAKE) check-bytecode
|
||||
$(MAKE) check-bytecode-short
|
||||
|
||||
# Run all tests
|
||||
check:
|
||||
@@ -91,16 +91,31 @@ check-bytecode-3:
|
||||
--bytecode-3.1 --bytecode-3.2 --bytecode-3.3 \
|
||||
--bytecode-3.4 --bytecode-3.5 --bytecode-3.6 --bytecode-pypy3.2
|
||||
|
||||
#: Check deparsing bytecode that works running Python 2 and Python 3
|
||||
#: Check deparsing on selected bytecode 3.x
|
||||
check-bytecode-3-short:
|
||||
$(PYTHON) test_pythonlib.py \
|
||||
--bytecode-3.4 --bytecode-3.5 --bytecode-3.6
|
||||
|
||||
#: Check deparsing bytecode on all Python 2 and Python 3 versions
|
||||
check-bytecode: check-bytecode-3
|
||||
$(PYTHON) test_pythonlib.py \
|
||||
--bytecode-1.4 --bytecode-1.5 \
|
||||
--bytecode-1.3 --bytecode-1.4 --bytecode-1.5 \
|
||||
--bytecode-2.2 --bytecode-2.3 --bytecode-2.4 \
|
||||
--bytecode-2.1 --bytecode-2.2 --bytecode-2.3 --bytecode-2.4 \
|
||||
--bytecode-2.5 --bytecode-2.6 --bytecode-2.7 \
|
||||
--bytecode-pypy2.7
|
||||
|
||||
|
||||
#: Check deparsing bytecode on selected Python 2 and Python 3 versions
|
||||
check-bytecode-short: check-bytecode-3-short
|
||||
$(PYTHON) test_pythonlib.py \
|
||||
--bytecode-2.6 --bytecode-2.7 --bytecode-pypy2.7
|
||||
|
||||
|
||||
#: Check deparsing bytecode 1.3 only
|
||||
check-bytecode-1.3:
|
||||
$(PYTHON) test_pythonlib.py --bytecode-1.3
|
||||
|
||||
#: Check deparsing bytecode 1.4 only
|
||||
check-bytecode-1.4:
|
||||
$(PYTHON) test_pythonlib.py --bytecode-1.4
|
||||
@@ -210,6 +225,11 @@ check-bytecode-2.7:
|
||||
check-bytecode-3.0:
|
||||
$(PYTHON) test_pythonlib.py --bytecode-3.0 --weak-verify
|
||||
|
||||
#: Check deparsing Python 3.0
|
||||
check-bytecode-3.0:
|
||||
$(PYTHON) test_pythonlib.py --bytecode-3.0 --weak-verify
|
||||
# $(PYTHON) test_pythonlib.py --bytecode-3.0-run --verify-run
|
||||
|
||||
#: Check deparsing Python 3.1
|
||||
check-bytecode-3.1:
|
||||
$(PYTHON) test_pythonlib.py --bytecode-3.1 --weak-verify
|
||||
@@ -274,7 +294,7 @@ check-3.4-ok:
|
||||
2.6:
|
||||
|
||||
#: PyPy 5.0.x with Python 2.7 ...
|
||||
pypy-2.7 5.0 5.3:
|
||||
pypy-2.7 5.0 5.3 6.0:
|
||||
$(PYTHON) test_pythonlib.py --bytecode-pypy2.7 --verify
|
||||
|
||||
#: PyPy 2.4.x with Python 3.2 ...
|
||||
|
BIN
test/bytecode_1.3/test_builtin.pyc
Normal file
BIN
test/bytecode_1.3/test_builtin.pyc
Normal file
Binary file not shown.
BIN
test/bytecode_1.3/test_exceptions.pyc
Normal file
BIN
test/bytecode_1.3/test_exceptions.pyc
Normal file
Binary file not shown.
BIN
test/bytecode_1.3/test_grammar.pyc
Normal file
BIN
test/bytecode_1.3/test_grammar.pyc
Normal file
Binary file not shown.
BIN
test/bytecode_1.3/test_operations.pyc
Normal file
BIN
test/bytecode_1.3/test_operations.pyc
Normal file
Binary file not shown.
BIN
test/bytecode_1.3/testall.pyc
Normal file
BIN
test/bytecode_1.3/testall.pyc
Normal file
Binary file not shown.
BIN
test/bytecode_1.4/01_print-1.4.pyc
Normal file
BIN
test/bytecode_1.4/01_print-1.4.pyc
Normal file
Binary file not shown.
BIN
test/bytecode_1.4/02_continue.pyc
Normal file
BIN
test/bytecode_1.4/02_continue.pyc
Normal file
Binary file not shown.
BIN
test/bytecode_1.4/addpack.pyc
Normal file
BIN
test/bytecode_1.4/addpack.pyc
Normal file
Binary file not shown.
BIN
test/bytecode_1.4/anydbm.pyc
Normal file
BIN
test/bytecode_1.4/anydbm.pyc
Normal file
Binary file not shown.
BIN
test/bytecode_1.4/bisect.pyc
Normal file
BIN
test/bytecode_1.4/bisect.pyc
Normal file
Binary file not shown.
BIN
test/bytecode_1.4/cmp.pyc
Normal file
BIN
test/bytecode_1.4/cmp.pyc
Normal file
Binary file not shown.
BIN
test/bytecode_1.4/cmpcache.pyc
Normal file
BIN
test/bytecode_1.4/cmpcache.pyc
Normal file
Binary file not shown.
BIN
test/bytecode_1.4/dbhash.pyc
Normal file
BIN
test/bytecode_1.4/dbhash.pyc
Normal file
Binary file not shown.
BIN
test/bytecode_1.4/glob.pyc
Normal file
BIN
test/bytecode_1.4/glob.pyc
Normal file
Binary file not shown.
Binary file not shown.
BIN
test/bytecode_1.4/test_empty.pyc
Normal file
BIN
test/bytecode_1.4/test_empty.pyc
Normal file
Binary file not shown.
BIN
test/bytecode_1.4/test_exec.pyc
Normal file
BIN
test/bytecode_1.4/test_exec.pyc
Normal file
Binary file not shown.
BIN
test/bytecode_1.5/00_unpack_list.pyc
Normal file
BIN
test/bytecode_1.5/00_unpack_list.pyc
Normal file
Binary file not shown.
Binary file not shown.
BIN
test/bytecode_3.0/01_comprehension.pyc
Normal file
BIN
test/bytecode_3.0/01_comprehension.pyc
Normal file
Binary file not shown.
BIN
test/bytecode_3.0/02_ifelse_lambda.pyc
Normal file
BIN
test/bytecode_3.0/02_ifelse_lambda.pyc
Normal file
Binary file not shown.
BIN
test/bytecode_3.0/03_pop_top.pyc
Normal file
BIN
test/bytecode_3.0/03_pop_top.pyc
Normal file
Binary file not shown.
3
test/simple_source/bug14/00_unpack_list.py
Normal file
3
test/simple_source/bug14/00_unpack_list.py
Normal file
@@ -0,0 +1,3 @@
|
||||
# Python 1.4 tzparse.py, but also appears in 1.5
|
||||
|
||||
[tzname, delta] = __file__
|
6
test/simple_source/bug14/01_print.py
Normal file
6
test/simple_source/bug14/01_print.py
Normal file
@@ -0,0 +1,6 @@
|
||||
# Python 1.4 aifc.py
|
||||
# Something weird about the final "print" and PRINT_NL_CONT followed by PRINT_NL
|
||||
def _readmark(self, markers):
|
||||
if self._markers: print 'marker',
|
||||
else: print 'markers',
|
||||
print 'instead of', markers
|
11
test/simple_source/bug14/02_continue.py
Normal file
11
test/simple_source/bug14/02_continue.py
Normal file
@@ -0,0 +1,11 @@
|
||||
# Python 1.4 cgi.py
|
||||
# Bug was in "continue" detection.
|
||||
# 1.4 doesn't have lnotab and our CONTINUE detection is off.
|
||||
def parse_multipart(params, pdict):
|
||||
while params:
|
||||
if params.has_key('name'):
|
||||
params = None
|
||||
else:
|
||||
continue
|
||||
|
||||
return None
|
20
test/simple_source/bug30/03_pop_top.py
Normal file
20
test/simple_source/bug30/03_pop_top.py
Normal file
@@ -0,0 +1,20 @@
|
||||
# From 3.0.1 __dummy_thread.py
|
||||
# bug was handling else:
|
||||
def interrupt_main():
|
||||
"""Set _interrupt flag to True to have start_new_thread raise
|
||||
KeyboardInterrupt upon exiting."""
|
||||
if _main:
|
||||
raise KeyboardInterrupt
|
||||
else:
|
||||
global _interrupt
|
||||
_interrupt = True
|
||||
|
||||
# From 3.0.1 ast.py bug was mangling prototype
|
||||
# def parse(expr, filename='<unknown>', mode='exec'):
|
||||
|
||||
# From 3.0.1 bisect
|
||||
def bisect_left(a, x, lo=0, hi=None):
|
||||
while lo:
|
||||
if a[mid] < x: lo = mid+1
|
||||
else: hi = mid
|
||||
return lo
|
22
test/stdlib/compile-file-1x.py
Executable file
22
test/stdlib/compile-file-1x.py
Executable file
@@ -0,0 +1,22 @@
|
||||
#!/usr/bin/env python
|
||||
"""byte compiles a Python 1.x program"""
|
||||
import sys
|
||||
if len(sys.argv) != 2:
|
||||
print("Usage: compile-file.py *python-file*")
|
||||
sys.exit(1)
|
||||
source = sys.argv[1]
|
||||
|
||||
# assert source.endswith('.py')
|
||||
basename = source[:-3]
|
||||
|
||||
# We do this crazy way to support Python 1.4 which
|
||||
# doesn't support version_info.
|
||||
PY_VERSION = sys.version[:3]
|
||||
|
||||
bytecode = "%s-%s.pyc" % (basename, PY_VERSION)
|
||||
|
||||
import py_compile
|
||||
print("# compiling %s to %s" % (source, bytecode))
|
||||
py_compile.compile(source, bytecode)
|
||||
# import os
|
||||
# os.system("../bin/uncompyle6 %s" % bytecode)
|
@@ -1,4 +1,5 @@
|
||||
#!/usr/bin/env python
|
||||
"""byte compiles a Python program after version 2.2 or so. Also see compile_file_1x.py"""
|
||||
import sys
|
||||
if len(sys.argv) != 2:
|
||||
print("Usage: compile-file.py *python-file*")
|
||||
|
@@ -105,6 +105,7 @@ case $PYVERSION in
|
||||
[test_curses.py]=1 # Possibly fails on its own but not detected
|
||||
[test_dis.py]=1 # We change line numbers - duh!
|
||||
[test_doctest.py]=1 # Fails on its own
|
||||
[test_format.py]=1 # control flow. uncompyle2 does not have problems here
|
||||
[test_generators.py]=1 # control flow. uncompyle2 has problem here too
|
||||
[test_grammar.py]=1 # Too many stmts. Handle large stmts
|
||||
[test_io.py]=1 # Test takes too long to run
|
||||
|
@@ -76,7 +76,7 @@ for vers in (2.7, 3.4, 3.5, 3.6):
|
||||
test_options[key] = (os.path.join(src_dir, pythonlib), PYOC, key, vers)
|
||||
pass
|
||||
|
||||
for vers in (1.4, 1.5,
|
||||
for vers in (1.3, 1.4, 1.5,
|
||||
2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7,
|
||||
3.0, 3.1, 3.2, 3.3,
|
||||
3.4, 3.5, 3.6, 3.7, 'pypy3.2', 'pypy2.7'):
|
||||
|
@@ -68,11 +68,9 @@ def usage():
|
||||
|
||||
|
||||
def main_bin():
|
||||
if not (sys.version_info[0:2] in ((2, 4), (2, 5), (2, 6), (2, 7),
|
||||
(3, 2), (3, 3),
|
||||
(3, 4), (3, 5), (3, 6), (3, 7))):
|
||||
sys.stderr.write('Error: %s requires Python 2.4 2.5 2.6, 2.7, '
|
||||
'3.2, 3.3, 3.4, 3.5, or 3.6' % program)
|
||||
if not (sys.version_info[0:2] in ((2, 4), (2, 5), (2, 6), (2, 7))):
|
||||
sys.stderr.write('Error: this branch of %s requires Python 2.4, 2.5, 2.6 or 2.7'
|
||||
% program)
|
||||
sys.exit(-1)
|
||||
|
||||
do_verify = recurse_dirs = False
|
||||
|
@@ -73,6 +73,19 @@ def disco_loop(disasm, queue, real_out):
|
||||
pass
|
||||
pass
|
||||
|
||||
def disassemble_fp(fp, outstream=None):
|
||||
"""
|
||||
disassemble Python byte-code from an open file
|
||||
"""
|
||||
(version, timestamp, magic_int, co, is_pypy,
|
||||
source_size) = load_from_fp(fp)
|
||||
if type(co) == list:
|
||||
for con in co:
|
||||
disco(version, con, outstream)
|
||||
else:
|
||||
disco(version, co, outstream, is_pypy=is_pypy)
|
||||
co = None
|
||||
|
||||
def disassemble_file(filename, outstream=None):
|
||||
"""
|
||||
disassemble Python byte-code file (.pyc)
|
||||
|
@@ -618,18 +618,31 @@ def get_python_parser(
|
||||
# a lazy way of doing the import?
|
||||
|
||||
if version < 3.0:
|
||||
if version == 1.5:
|
||||
import uncompyle6.parsers.parse15 as parse15
|
||||
if compile_mode == 'exec':
|
||||
p = parse15.Python15Parser(debug_parser)
|
||||
else:
|
||||
p = parse15.Python15ParserSingle(debug_parser)
|
||||
elif version == 2.1:
|
||||
import uncompyle6.parsers.parse21 as parse21
|
||||
if compile_mode == 'exec':
|
||||
p = parse21.Python21Parser(debug_parser)
|
||||
else:
|
||||
p = parse21.Python21ParserSingle(debug_parser)
|
||||
if version < 2.2:
|
||||
if version == 1.3:
|
||||
import uncompyle6.parsers.parse13 as parse13
|
||||
if compile_mode == 'exec':
|
||||
p = parse13.Python14Parser(debug_parser)
|
||||
else:
|
||||
p = parse13.Python14ParserSingle(debug_parser)
|
||||
elif version == 1.4:
|
||||
import uncompyle6.parsers.parse14 as parse14
|
||||
if compile_mode == 'exec':
|
||||
p = parse14.Python14Parser(debug_parser)
|
||||
else:
|
||||
p = parse14.Python14ParserSingle(debug_parser)
|
||||
elif version == 1.5:
|
||||
import uncompyle6.parsers.parse15 as parse15
|
||||
if compile_mode == 'exec':
|
||||
p = parse15.Python15Parser(debug_parser)
|
||||
else:
|
||||
p = parse15.Python15ParserSingle(debug_parser)
|
||||
elif version == 2.1:
|
||||
import uncompyle6.parsers.parse21 as parse21
|
||||
if compile_mode == 'exec':
|
||||
p = parse21.Python21Parser(debug_parser)
|
||||
else:
|
||||
p = parse21.Python21ParserSingle(debug_parser)
|
||||
elif version == 2.2:
|
||||
import uncompyle6.parsers.parse22 as parse22
|
||||
if compile_mode == 'exec':
|
||||
|
49
uncompyle6/parsers/parse13.py
Normal file
49
uncompyle6/parsers/parse13.py
Normal file
@@ -0,0 +1,49 @@
|
||||
# Copyright (c) 2018 Rocky Bernstein
|
||||
|
||||
from spark_parser import DEFAULT_DEBUG as PARSER_DEFAULT_DEBUG
|
||||
from uncompyle6.parser import PythonParserSingle
|
||||
from uncompyle6.parsers.parse14 import Python14Parser
|
||||
|
||||
class Python13Parser(Python14Parser):
|
||||
|
||||
def p_misc13(self, args):
|
||||
"""
|
||||
# Nothing here yet, but will need to add LOAD_GLOBALS
|
||||
"""
|
||||
|
||||
def __init__(self, debug_parser=PARSER_DEFAULT_DEBUG):
|
||||
super(Python13Parser, self).__init__(debug_parser)
|
||||
self.customized = {}
|
||||
|
||||
# def customize_grammar_rules(self, tokens, customize):
|
||||
# super(Python13Parser, self).customize_grammar_rules(tokens, customize)
|
||||
# self.remove_rules("""
|
||||
# whileelsestmt ::= SETUP_LOOP testexpr l_stmts_opt
|
||||
# jb_pop
|
||||
# POP_BLOCK else_suitel COME_FROM
|
||||
# """)
|
||||
# self.check_reduce['doc_junk'] = 'tokens'
|
||||
|
||||
|
||||
# def reduce_is_invalid(self, rule, ast, tokens, first, last):
|
||||
# invalid = super(Python14Parser,
|
||||
# self).reduce_is_invalid(rule, ast,
|
||||
# tokens, first, last)
|
||||
# if invalid or tokens is None:
|
||||
# return invalid
|
||||
# if rule[0] == 'doc_junk':
|
||||
# return not isinstance(tokens[first].pattr, str)
|
||||
|
||||
|
||||
|
||||
class Python13ParserSingle(Python13Parser, PythonParserSingle):
|
||||
pass
|
||||
|
||||
if __name__ == '__main__':
|
||||
# Check grammar
|
||||
p = Python13Parser()
|
||||
p.check_grammar()
|
||||
p.dump_grammar()
|
||||
|
||||
# local variables:
|
||||
# tab-width: 4
|
@@ -8,15 +8,53 @@ class Python14Parser(Python15Parser):
|
||||
|
||||
def p_misc14(self, args):
|
||||
"""
|
||||
# Nothing here yet, but will need to add UNARY_CALL, BINARY_CALL,
|
||||
# Not much here yet, but will probably need to add UNARY_CALL, BINARY_CALL,
|
||||
# RAISE_EXCEPTION, BUILD_FUNCTION, UNPACK_ARG, UNPACK_VARARG, LOAD_LOCAL,
|
||||
# SET_FUNC_ARGS, and RESERVE_FAST
|
||||
|
||||
# Not strictly needed, but tidies up output
|
||||
stmt ::= doc_junk
|
||||
doc_junk ::= LOAD_CONST POP_TOP
|
||||
|
||||
# Not sure why later Python's omit the COME_FROM
|
||||
jb_pop14 ::= JUMP_BACK COME_FROM POP_TOP
|
||||
|
||||
whileelsestmt ::= SETUP_LOOP testexpr l_stmts_opt
|
||||
jb_pop14
|
||||
POP_BLOCK else_suitel COME_FROM
|
||||
|
||||
print_items_nl_stmt ::= expr PRINT_ITEM_CONT print_items_opt PRINT_NEWLINE_CONT
|
||||
|
||||
|
||||
# 1.4 doesn't have linenotab, and although this shouldn't
|
||||
# be a show stopper, our CONTINUE detection is off here.
|
||||
continue ::= JUMP_BACK
|
||||
"""
|
||||
|
||||
def __init__(self, debug_parser=PARSER_DEFAULT_DEBUG):
|
||||
super(Python14Parser, self).__init__(debug_parser)
|
||||
self.customized = {}
|
||||
|
||||
def customize_grammar_rules(self, tokens, customize):
|
||||
super(Python14Parser, self).customize_grammar_rules(tokens, customize)
|
||||
self.remove_rules("""
|
||||
whileelsestmt ::= SETUP_LOOP testexpr l_stmts_opt
|
||||
jb_pop
|
||||
POP_BLOCK else_suitel COME_FROM
|
||||
""")
|
||||
self.check_reduce['doc_junk'] = 'tokens'
|
||||
|
||||
|
||||
def reduce_is_invalid(self, rule, ast, tokens, first, last):
|
||||
invalid = super(Python14Parser,
|
||||
self).reduce_is_invalid(rule, ast,
|
||||
tokens, first, last)
|
||||
if invalid or tokens is None:
|
||||
return invalid
|
||||
if rule[0] == 'doc_junk':
|
||||
return not isinstance(tokens[first].pattr, str)
|
||||
|
||||
|
||||
|
||||
class Python14ParserSingle(Python14Parser, PythonParserSingle):
|
||||
pass
|
||||
|
@@ -2,7 +2,7 @@
|
||||
# Copyright (c) 2000-2002 by hartmut Goebel <hartmut@goebel.noris.de>
|
||||
|
||||
from spark_parser import DEFAULT_DEBUG as PARSER_DEFAULT_DEBUG
|
||||
from uncompyle6.parser import PythonParserSingle
|
||||
from uncompyle6.parser import PythonParserSingle, nop_func
|
||||
from uncompyle6.parsers.parse21 import Python21Parser
|
||||
|
||||
class Python15Parser(Python21Parser):
|
||||
@@ -23,6 +23,17 @@ class Python15Parser(Python21Parser):
|
||||
importlist ::= IMPORT_FROM
|
||||
"""
|
||||
|
||||
def customize_grammar_rules(self, tokens, customize):
|
||||
super(Python15Parser, self).customize_grammar_rules(tokens, customize)
|
||||
for i, token in enumerate(tokens):
|
||||
opname = token.kind
|
||||
opname_base = opname[:opname.rfind('_')]
|
||||
|
||||
if opname_base == 'UNPACK_LIST':
|
||||
self.addRule("store ::= unpack_list", nop_func)
|
||||
|
||||
|
||||
|
||||
class Python15ParserSingle(Python15Parser, PythonParserSingle):
|
||||
pass
|
||||
|
||||
|
@@ -292,7 +292,6 @@ class Python2Parser(PythonParser):
|
||||
|
||||
# The order of opname listed is roughly sorted below
|
||||
if opname_base in ('BUILD_LIST', 'BUILD_SET', 'BUILD_TUPLE'):
|
||||
v = token.attr
|
||||
collection = opname_base[opname_base.find('_')+1:].lower()
|
||||
rule = '%s ::= %s%s' % (collection, (token.attr * 'expr '), opname)
|
||||
self.add_unique_rules([
|
||||
@@ -393,7 +392,6 @@ class Python2Parser(PythonParser):
|
||||
""", nop_func)
|
||||
continue
|
||||
elif opname == 'JUMP_IF_NOT_DEBUG':
|
||||
v = token.attr
|
||||
self.addRule("""
|
||||
jmp_true_false ::= POP_JUMP_IF_TRUE
|
||||
jmp_true_false ::= POP_JUMP_IF_FALSE
|
||||
|
@@ -349,7 +349,7 @@ class Python26Parser(Python2Parser):
|
||||
# For now, we won't let the 2nd 'expr' be a "conditional_not"
|
||||
# However in < 2.6 where we don't have if/else expression it *can*
|
||||
# be.
|
||||
if ast[2][0] == 'conditional_not':
|
||||
if self.version >= 2.6 and ast[2][0] == 'conditional_not':
|
||||
return True
|
||||
|
||||
test_index = last
|
||||
|
@@ -52,6 +52,8 @@ class Python27Parser(Python2Parser):
|
||||
def p_try27(self, args):
|
||||
"""
|
||||
# If the last except is a "raise" we might not have a final COME_FROM
|
||||
# FIXME: need a check on this rule since this accepts try_except when
|
||||
# we shouldn't
|
||||
try_except ::= SETUP_EXCEPT suite_stmts_opt POP_BLOCK
|
||||
except_handler
|
||||
|
||||
|
@@ -11,6 +11,12 @@ class Python30Parser(Python31Parser):
|
||||
def p_30(self, args):
|
||||
"""
|
||||
|
||||
assert ::= assert_expr jmp_true LOAD_ASSERT RAISE_VARARGS_1 POP_TOP
|
||||
return_if_lambda ::= RETURN_END_IF_LAMBDA POP_TOP
|
||||
compare_chained1 ::= expr DUP_TOP ROT_THREE COMPARE_OP
|
||||
JUMP_IF_FALSE POP_TOP compare_chained2
|
||||
compare_chained2 ::= expr COMPARE_OP RETURN_END_IF_LAMBDA
|
||||
|
||||
# FIXME: combine with parse3.2
|
||||
whileTruestmt ::= SETUP_LOOP l_stmts_opt JUMP_BACK
|
||||
COME_FROM_LOOP
|
||||
@@ -19,10 +25,10 @@ class Python30Parser(Python31Parser):
|
||||
|
||||
# In many ways Python 3.0 code generation is more like Python 2.6 than
|
||||
# it is 2.7 or 3.1. So we have a number of 2.6ish (and before) rules below
|
||||
# Specifically POP_TOP is more prevelant since there is no POP_JUMP_IF_...
|
||||
# instructions
|
||||
|
||||
_ifstmts_jump ::= c_stmts_opt JUMP_FORWARD _come_froms POP_TOP COME_FROM
|
||||
jmp_true ::= JUMP_IF_TRUE POP_TOP
|
||||
jmp_false ::= JUMP_IF_FALSE POP_TOP
|
||||
|
||||
# Used to keep index order the same in semantic actions
|
||||
jb_pop_top ::= JUMP_BACK POP_TOP
|
||||
@@ -32,16 +38,67 @@ class Python30Parser(Python31Parser):
|
||||
else_suitel ::= l_stmts COME_FROM_LOOP JUMP_BACK
|
||||
|
||||
ifelsestmtl ::= testexpr c_stmts_opt jb_pop_top else_suitel
|
||||
iflaststmtl ::= testexpr c_stmts_opt jb_pop_top
|
||||
iflaststmt ::= testexpr c_stmts_opt JUMP_ABSOLUTE POP_TOP
|
||||
|
||||
withasstmt ::= expr setupwithas store suite_stmts_opt
|
||||
POP_BLOCK LOAD_CONST COME_FROM_FINALLY
|
||||
LOAD_FAST DELETE_FAST WITH_CLEANUP END_FINALLY
|
||||
setupwithas ::= DUP_TOP LOAD_ATTR STORE_FAST LOAD_ATTR CALL_FUNCTION_0 setup_finally
|
||||
setup_finally ::= STORE_FAST SETUP_FINALLY LOAD_FAST DELETE_FAST
|
||||
|
||||
# Need to keep LOAD_FAST as index 1
|
||||
set_comp_func_header ::= BUILD_SET_0 DUP_TOP STORE_FAST
|
||||
set_comp_func ::= set_comp_func_header
|
||||
LOAD_FAST FOR_ITER store comp_iter
|
||||
JUMP_BACK POP_TOP JUMP_BACK RETURN_VALUE RETURN_LAST
|
||||
|
||||
list_comp_header ::= BUILD_LIST_0 DUP_TOP STORE_FAST
|
||||
list_comp ::= list_comp_header
|
||||
LOAD_FAST FOR_ITER store comp_iter
|
||||
JUMP_BACK
|
||||
|
||||
|
||||
comp_if ::= expr jmp_false comp_iter
|
||||
comp_iter ::= expr expr SET_ADD
|
||||
comp_iter ::= expr expr LIST_APPEND
|
||||
|
||||
jump_forward_else ::= JUMP_FORWARD POP_TOP
|
||||
jump_absolute_else ::= JUMP_ABSOLUTE POP_TOP
|
||||
|
||||
# In many ways 3.0 is like 2.6. The below rules in fact are the same or similar.
|
||||
|
||||
jmp_true ::= JUMP_IF_TRUE POP_TOP
|
||||
jmp_false ::= JUMP_IF_FALSE POP_TOP
|
||||
for_block ::= l_stmts_opt _come_froms POP_TOP JUMP_BACK
|
||||
except_handler ::= JUMP_FORWARD COME_FROM_EXCEPT except_stmts
|
||||
POP_TOP END_FINALLY come_froms
|
||||
except_handler ::= jmp_abs COME_FROM_EXCEPT except_stmts
|
||||
POP_TOP END_FINALLY
|
||||
return_if_stmt ::= ret_expr RETURN_END_IF POP_TOP
|
||||
and ::= expr JUMP_IF_FALSE POP_TOP expr COME_FROM
|
||||
whilestmt ::= SETUP_LOOP testexpr l_stmts_opt
|
||||
JUMP_BACK POP_TOP POP_BLOCK COME_FROM_LOOP
|
||||
"""
|
||||
|
||||
def customize_grammar_rules(self, tokens, customize):
|
||||
super(Python30Parser, self).customize_grammar_rules(tokens, customize)
|
||||
self.remove_rules("""
|
||||
iflaststmtl ::= testexpr c_stmts_opt JUMP_BACK COME_FROM_LOOP
|
||||
ifelsestmtl ::= testexpr c_stmts_opt JUMP_BACK else_suitel
|
||||
iflaststmt ::= testexpr c_stmts_opt JUMP_ABSOLUTE
|
||||
_ifstmts_jump ::= c_stmts_opt JUMP_FORWARD _come_froms
|
||||
jump_forward_else ::= JUMP_FORWARD ELSE
|
||||
jump_absolute_else ::= JUMP_ABSOLUTE ELSE
|
||||
whilestmt ::= SETUP_LOOP testexpr l_stmts_opt COME_FROM JUMP_BACK POP_BLOCK
|
||||
COME_FROM_LOOP
|
||||
assert ::= assert_expr jmp_true LOAD_ASSERT RAISE_VARARGS_1
|
||||
return_if_lambda ::= RETURN_END_IF_LAMBDA
|
||||
compare_chained1 ::= expr DUP_TOP ROT_THREE COMPARE_OP JUMP_IF_FALSE_OR_POP compare_chained2 COME_FROM
|
||||
except_handler ::= jmp_abs COME_FROM_EXCEPT except_stmts END_FINALLY
|
||||
|
||||
""")
|
||||
|
||||
return
|
||||
pass
|
||||
|
||||
|
@@ -39,7 +39,7 @@ else:
|
||||
|
||||
# The byte code versions we support.
|
||||
# Note: these all have to be floats
|
||||
PYTHON_VERSIONS = frozenset((1.4, 1.5,
|
||||
PYTHON_VERSIONS = frozenset((1.3, 1.4, 1.5,
|
||||
2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7,
|
||||
3.0, 3.1, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7))
|
||||
|
||||
@@ -505,22 +505,26 @@ def get_scanner(version, is_pypy=False, show_asm=None):
|
||||
# Pick up appropriate scanner
|
||||
if version in PYTHON_VERSIONS:
|
||||
v_str = "%s" % (int(version * 10))
|
||||
if PYTHON3:
|
||||
try:
|
||||
import importlib
|
||||
if is_pypy:
|
||||
scan = importlib.import_module("uncompyle6.scanners.pypy%s" % v_str)
|
||||
else:
|
||||
scan = importlib.import_module("uncompyle6.scanners.scanner%s" % v_str)
|
||||
if False: print(scan) # Avoid unused scan
|
||||
else:
|
||||
except ImportError:
|
||||
if is_pypy:
|
||||
exec("import uncompyle6.scanners.pypy%s as scan" % v_str)
|
||||
exec("import uncompyle6.scanners.pypy%s as scan" % v_str,
|
||||
locals(), globals())
|
||||
else:
|
||||
exec("import uncompyle6.scanners.scanner%s as scan" % v_str)
|
||||
exec("import uncompyle6.scanners.scanner%s as scan" % v_str,
|
||||
locals(), globals())
|
||||
if is_pypy:
|
||||
scanner = eval("scan.ScannerPyPy%s(show_asm=show_asm)" % v_str)
|
||||
scanner = eval("scan.ScannerPyPy%s(show_asm=show_asm)" % v_str,
|
||||
locals(), globals())
|
||||
else:
|
||||
scanner = eval("scan.Scanner%s(show_asm=show_asm)" % v_str)
|
||||
scanner = eval("scan.Scanner%s(show_asm=show_asm)" % v_str,
|
||||
locals(), globals())
|
||||
else:
|
||||
raise RuntimeError("Unsupported Python version %s" % version)
|
||||
return scanner
|
||||
|
35
uncompyle6/scanners/scanner13.py
Normal file
35
uncompyle6/scanners/scanner13.py
Normal file
@@ -0,0 +1,35 @@
|
||||
# Copyright (c) 2018 by Rocky Bernstein
|
||||
"""
|
||||
Python 1.3 bytecode decompiler massaging.
|
||||
|
||||
This massages tokenized 1.3 bytecode to make it more amenable for
|
||||
grammar parsing.
|
||||
"""
|
||||
|
||||
import uncompyle6.scanners.scanner14 as scan
|
||||
# from uncompyle6.scanners.scanner26 import ingest as ingest26
|
||||
|
||||
# bytecode verification, verify(), uses JUMP_OPs from here
|
||||
from xdis.opcodes import opcode_13
|
||||
JUMP_OPS = opcode_13.JUMP_OPS
|
||||
|
||||
# We base this off of 1.4 instead of the other way around
|
||||
# because we cleaned things up this way.
|
||||
# The history is that 2.7 support is the cleanest,
|
||||
# then from that we got 2.6 and so on.
|
||||
class Scanner13(scan.Scanner14):
|
||||
def __init__(self, show_asm=False):
|
||||
scan.Scanner14.__init__(self, show_asm)
|
||||
self.opc = opcode_13
|
||||
self.opname = opcode_13.opname
|
||||
self.version = 1.3
|
||||
return
|
||||
|
||||
# def ingest22(self, co, classname=None, code_objects={}, show_asm=None):
|
||||
# tokens, customize = self.parent_ingest(co, classname, code_objects, show_asm)
|
||||
# tokens = [t for t in tokens if t.kind != 'SET_LINENO']
|
||||
|
||||
# # for t in tokens:
|
||||
# # print(t)
|
||||
#
|
||||
# return tokens, customize
|
@@ -32,5 +32,5 @@ class Scanner14(scan.Scanner15):
|
||||
|
||||
# # for t in tokens:
|
||||
# # print(t)
|
||||
|
||||
return tokens, customize
|
||||
#
|
||||
# return tokens, customize
|
||||
|
@@ -25,3 +25,17 @@ class Scanner15(scan.Scanner21):
|
||||
self.version = 1.5
|
||||
self.genexpr_name = '<generator expression>'
|
||||
return
|
||||
|
||||
def ingest(self, co, classname=None, code_objects={}, show_asm=None):
|
||||
"""
|
||||
Pick out tokens from an uncompyle6 code object, and transform them,
|
||||
returning a list of uncompyle6 Token's.
|
||||
|
||||
The transformations are made to assist the deparsing grammar.
|
||||
"""
|
||||
tokens, customize = scan.Scanner21.ingest(self, co, classname, code_objects, show_asm)
|
||||
for t in tokens:
|
||||
if t.op == self.opc.UNPACK_LIST:
|
||||
t.kind = 'UNPACK_LIST_%d' % t.attr
|
||||
pass
|
||||
return tokens, customize
|
||||
|
@@ -1081,7 +1081,7 @@ class Scanner2(Scanner):
|
||||
# FIXME FIXME FIXME
|
||||
# All the conditions are horrible, and I am not sure I
|
||||
# undestand fully what's going l
|
||||
# WeR REALLY REALLY need a better way to handle control flow
|
||||
# We REALLY REALLY need a better way to handle control flow
|
||||
# Expecially for < 2.7
|
||||
if label is not None and label != -1:
|
||||
if self.version == 2.7:
|
||||
|
@@ -78,8 +78,10 @@ class Scanner3(Scanner):
|
||||
|
||||
if self.version == 3.0:
|
||||
self.pop_jump_tf = frozenset([self.opc.JUMP_IF_FALSE, self.opc.JUMP_IF_TRUE])
|
||||
self.not_continue_follow = ('END_FINALLY', 'POP_BLOCK', 'POP_TOP')
|
||||
else:
|
||||
self.pop_jump_tf = frozenset([self.opc.PJIF, self.opc.PJIT])
|
||||
self.not_continue_follow = ('END_FINALLY', 'POP_BLOCK')
|
||||
|
||||
self.setup_ops_no_loop = frozenset(setup_ops) - frozenset([self.opc.SETUP_LOOP])
|
||||
|
||||
@@ -217,7 +219,12 @@ class Scanner3(Scanner):
|
||||
# If we have a JUMP_FORWARD after the
|
||||
# RAISE_VARARGS then we have a "raise" statement
|
||||
# else we have an "assert" statement.
|
||||
if inst.opname == 'POP_JUMP_IF_TRUE' and i+1 < n:
|
||||
if self.version == 3.0:
|
||||
# There is a an implied JUMP_IF_TRUE that we are not testing for (yet?) here
|
||||
assert_can_follow = inst.opname == 'POP_TOP' and i+1 < n
|
||||
else:
|
||||
assert_can_follow = inst.opname == 'POP_JUMP_IF_TRUE' and i+1 < n
|
||||
if assert_can_follow:
|
||||
next_inst = self.insts[i+1]
|
||||
if (next_inst.opname == 'LOAD_GLOBAL' and
|
||||
next_inst.argval == 'AssertionError'):
|
||||
@@ -395,13 +402,9 @@ class Scanner3(Scanner):
|
||||
.opname == 'FOR_ITER'
|
||||
and self.insts[i+1].opname == 'JUMP_FORWARD')
|
||||
|
||||
|
||||
if (is_continue or
|
||||
(inst.offset in self.stmts and
|
||||
(self.version != 3.0 or (hasattr(inst, 'linestart'))) and
|
||||
(next_opname not in ('END_FINALLY', 'POP_BLOCK',
|
||||
# Python 3.0 only uses POP_TOP
|
||||
'POP_TOP')))):
|
||||
(inst.offset in self.stmts and (inst.starts_line and
|
||||
next_opname not in self.not_continue_follow))):
|
||||
opname = 'CONTINUE'
|
||||
else:
|
||||
opname = 'JUMP_BACK'
|
||||
|
@@ -520,7 +520,7 @@ def make_function3(self, node, is_lambda, nested=1, code_node=None):
|
||||
lc_index = -3
|
||||
pass
|
||||
|
||||
if (3.1 <= self.version <= 3.3 and len(node) > 2 and
|
||||
if (3.0 <= self.version <= 3.3 and len(node) > 2 and
|
||||
node[lambda_index] != 'LOAD_LAMBDA' and
|
||||
(have_kwargs or node[lc_index].kind != 'load_closure')):
|
||||
|
||||
@@ -595,8 +595,8 @@ def make_function3(self, node, is_lambda, nested=1, code_node=None):
|
||||
paramnames = list(scanner_code.co_varnames[:argc])
|
||||
|
||||
# defaults are for last n parameters, thus reverse
|
||||
if not 3.0 == self.version or self.version >= 3.6:
|
||||
paramnames.reverse(); defparams.reverse()
|
||||
paramnames.reverse();
|
||||
defparams.reverse()
|
||||
|
||||
try:
|
||||
ast = self.build_ast(scanner_code._tokens,
|
||||
@@ -625,72 +625,37 @@ def make_function3(self, node, is_lambda, nested=1, code_node=None):
|
||||
else:
|
||||
params = paramnames
|
||||
|
||||
if not 3.0 == self.version or self.version >= 3.6:
|
||||
params.reverse() # back to correct order
|
||||
params.reverse() # back to correct order
|
||||
|
||||
if code_has_star_arg(code):
|
||||
if self.version > 3.0:
|
||||
params.append('*%s' % code.co_varnames[argc + kw_pairs])
|
||||
else:
|
||||
params.append('*%s' % code.co_varnames[argc])
|
||||
argc += 1
|
||||
|
||||
# dump parameter list (with default values)
|
||||
if is_lambda:
|
||||
self.write("lambda ", ", ".join(params))
|
||||
if code_has_star_arg(code):
|
||||
if self.version > 3.0:
|
||||
params.append('*%s' % code.co_varnames[argc + kw_pairs])
|
||||
else:
|
||||
self.write("(", ", ".join(params))
|
||||
# self.println(indent, '#flags:\t', int(code.co_flags))
|
||||
params.append('*%s' % code.co_varnames[argc])
|
||||
argc += 1
|
||||
|
||||
# dump parameter list (with default values)
|
||||
if is_lambda:
|
||||
self.write("lambda ", ", ".join(params))
|
||||
# If the last statement is None (which is the
|
||||
# same thing as "return None" in a lambda) and the
|
||||
# next to last statement is a "yield". Then we want to
|
||||
# drop the (return) None since that was just put there
|
||||
# to have something to after the yield finishes.
|
||||
# FIXME: this is a bit hoaky and not general
|
||||
if (len(ast) > 1 and
|
||||
self.traverse(ast[-1]) == 'None' and
|
||||
self.traverse(ast[-2]).strip().startswith('yield')):
|
||||
del ast[-1]
|
||||
# Now pick out the expr part of the last statement
|
||||
ast_expr = ast[-1]
|
||||
while ast_expr.kind != 'expr':
|
||||
ast_expr = ast_expr[0]
|
||||
ast[-1] = ast_expr
|
||||
pass
|
||||
else:
|
||||
if is_lambda:
|
||||
self.write("lambda ")
|
||||
# If the last statement is None (which is the
|
||||
# same thing as "return None" in a lambda) and the
|
||||
# next to last statement is a "yield". Then we want to
|
||||
# drop the (return) None since that was just put there
|
||||
# to have something to after the yield finishes.
|
||||
# FIXME: this is a bit hoaky and not general
|
||||
if (len(ast) > 1 and
|
||||
self.traverse(ast[-1]) == 'None' and
|
||||
self.traverse(ast[-2]).strip().startswith('yield')):
|
||||
del ast[-1]
|
||||
# Now pick out the expr part of the last statement
|
||||
ast_expr = ast[-1]
|
||||
while ast_expr.kind != 'expr':
|
||||
ast_expr = ast_expr[0]
|
||||
ast[-1] = ast_expr
|
||||
pass
|
||||
else:
|
||||
self.write("(")
|
||||
pass
|
||||
|
||||
last_line = self.f.getvalue().split("\n")[-1]
|
||||
l = len(last_line)
|
||||
indent = ' ' * l
|
||||
line_number = self.line_number
|
||||
|
||||
if code_has_star_arg(code):
|
||||
self.write('*%s' % code.co_varnames[argc + kw_pairs])
|
||||
argc += 1
|
||||
|
||||
i = len(paramnames) - len(defparams)
|
||||
self.write(", ".join(paramnames[:i]))
|
||||
if i > 0:
|
||||
suffix = ', '
|
||||
else:
|
||||
suffix = ''
|
||||
for n in node:
|
||||
if n == 'pos_arg':
|
||||
self.write(suffix)
|
||||
self.write(paramnames[i] + '=')
|
||||
i += 1
|
||||
self.preorder(n)
|
||||
if (line_number != self.line_number):
|
||||
suffix = ",\n" + indent
|
||||
line_number = self.line_number
|
||||
else:
|
||||
suffix = ', '
|
||||
self.write("(", ", ".join(params))
|
||||
# self.println(indent, '#flags:\t', int(code.co_flags))
|
||||
|
||||
ends_in_comma = False
|
||||
if kw_args > 0:
|
||||
|
@@ -940,7 +940,7 @@ class SourceWalker(GenericASTTraversal, object):
|
||||
self.prec = 27
|
||||
|
||||
# FIXME: clean this up
|
||||
if self.version > 3.0 and node == 'dict_comp':
|
||||
if self.version >= 3.0 and node == 'dict_comp':
|
||||
cn = node[1]
|
||||
elif self.version < 2.7 and node == 'generator_exp':
|
||||
if node[0] == 'LOAD_GENEXPR':
|
||||
@@ -948,7 +948,7 @@ class SourceWalker(GenericASTTraversal, object):
|
||||
elif node[0] == 'load_closure':
|
||||
cn = node[1]
|
||||
|
||||
elif self.version > 3.0 and node == 'generator_exp':
|
||||
elif self.version >= 3.0 and node == 'generator_exp':
|
||||
if node[0] == 'load_genexpr':
|
||||
load_genexpr = node[0]
|
||||
elif node[1] == 'load_genexpr':
|
||||
@@ -1052,7 +1052,9 @@ class SourceWalker(GenericASTTraversal, object):
|
||||
ast = ast[0]
|
||||
|
||||
store = None
|
||||
if ast in ['set_comp_func', 'dict_comp_func']:
|
||||
n = ast[iter_index]
|
||||
if ast in ['set_comp_func', 'dict_comp_func',
|
||||
'list_comp', 'set_comp_func_header']:
|
||||
for k in ast:
|
||||
if k == 'comp_iter':
|
||||
n = k
|
||||
@@ -1062,7 +1064,6 @@ class SourceWalker(GenericASTTraversal, object):
|
||||
pass
|
||||
pass
|
||||
else:
|
||||
n = ast[iter_index]
|
||||
assert n == 'list_iter', n
|
||||
|
||||
# FIXME: I'm not totally sure this is right.
|
||||
@@ -1070,15 +1071,19 @@ class SourceWalker(GenericASTTraversal, object):
|
||||
# Find the list comprehension body. It is the inner-most
|
||||
# node that is not list_.. .
|
||||
if_node = None
|
||||
comp_for = None
|
||||
comp_store = None
|
||||
if n == 'comp_iter':
|
||||
comp_for = n
|
||||
comp_store = ast[3]
|
||||
|
||||
have_not = False
|
||||
while n in ('list_iter', 'comp_iter'):
|
||||
n = n[0] # iterate one nesting deeper
|
||||
# iterate one nesting deeper
|
||||
if self.version == 3.0 and len(n) == 3:
|
||||
assert n[0] == 'expr' and n[1] == 'expr'
|
||||
n = n[1]
|
||||
else:
|
||||
n = n[0]
|
||||
|
||||
if n in ('list_for', 'comp_for'):
|
||||
if n[2] == 'store':
|
||||
store = n[2]
|
||||
@@ -1094,7 +1099,9 @@ class SourceWalker(GenericASTTraversal, object):
|
||||
|
||||
# Python 2.7+ starts including set_comp_body
|
||||
# Python 3.5+ starts including set_comp_func
|
||||
assert n.kind in ('lc_body', 'comp_body', 'set_comp_func', 'set_comp_body'), ast
|
||||
# Python 3.0 is yet another snowflake
|
||||
if self.version != 3.0:
|
||||
assert n.kind in ('lc_body', 'comp_body', 'set_comp_func', 'set_comp_body'), ast
|
||||
assert store, "Couldn't find store in list/set comprehension"
|
||||
|
||||
# A problem created with later Python code generation is that there
|
||||
@@ -1114,11 +1121,10 @@ class SourceWalker(GenericASTTraversal, object):
|
||||
self.preorder(store)
|
||||
|
||||
# FIXME this is all merely approximate
|
||||
# from trepan.api import debug; debug()
|
||||
self.write(' in ')
|
||||
self.preorder(node[-3])
|
||||
|
||||
if ast == 'list_comp':
|
||||
if ast == 'list_comp' and self.version != 3.0:
|
||||
list_iter = ast[1]
|
||||
assert list_iter == 'list_iter'
|
||||
if list_iter == 'list_for':
|
||||
@@ -1127,9 +1133,7 @@ class SourceWalker(GenericASTTraversal, object):
|
||||
return
|
||||
pass
|
||||
|
||||
if comp_store:
|
||||
self.preorder(comp_for)
|
||||
elif if_node:
|
||||
if if_node:
|
||||
self.write(' if ')
|
||||
if have_not:
|
||||
self.write('not ')
|
||||
|
@@ -12,4 +12,4 @@
|
||||
# along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
# This file is suitable for sourcing inside bash as
|
||||
# well as importing into Python
|
||||
VERSION='3.2.0'
|
||||
VERSION='3.2.3'
|
||||
|
Reference in New Issue
Block a user