Merge branch 'master' into python-2.4

This commit is contained in:
rocky
2017-08-09 21:45:50 -04:00
25 changed files with 246 additions and 83 deletions

View File

@@ -1,3 +1,85 @@
2017-08-09 rocky <rb@dustyfeet.com>
* README.rst, __pkginfo__.py, uncompyle6/version.py: Get ready for
release 2.11.3
2017-08-02 rocky <rb@dustyfeet.com>
* __pkginfo__.py: Revert commit to wrong branch
2017-08-02 rocky <rb@dustyfeet.com>
* __pkginfo__.py: Remove six from Python-2.4/2.5 package
2017-07-17 rocky <rb@dustyfeet.com>
* __pkginfo__.py, uncompyle6/scanners/scanner2.py,
uncompyle6/scanners/scanner3.py, uncompyle6/scanners/scanner30.py:
xdis's "exception match" is now "exception-match"
2017-07-15 rocky <rb@dustyfeet.com>
* __pkginfo__.py: xdis 3.5.1 is botched?
2017-07-14 rocky <rb@dustyfeet.com>
* __pkginfo__.py: Use newer xdis
2017-07-14 R. Bernstein <rocky@users.noreply.github.com>
* README.rst: Fixes issue #124
2017-07-14 rocky <rb@dustyfeet.com>
* HISTORY.md: History updates
2017-07-09 rocky <rb@dustyfeet.com>
* README.rst: RsT doc formatting
2017-07-09 rocky <rb@dustyfeet.com>
* ChangeLog, HOW-TO-REPORT-A-BUG.md, NEWS, uncompyle6/version.py:
Get ready for release 2.11.2
2017-07-08 rocky <rb@dustyfeet.com>
* __pkginfo__.py, uncompyle6/scanner.py,
uncompyle6/scanners/scanner2.py, uncompyle6/scanners/scanner26.py,
uncompyle6/scanners/scanner3.py, uncompyle6/scanners/scanner30.py,
uncompyle6/scanners/tok.py: Use xdis 3.5.0's opcode sets
2017-07-08 rocky <rb@dustyfeet.com>
* test/test_pyenvlib.py, uncompyle6/scanners/pypy32.py,
uncompyle6/scanners/pypy35.py, uncompyle6/scanners/scanner15.py,
uncompyle6/scanners/scanner32.py, uncompyle6/scanners/scanner34.py,
uncompyle6/scanners/scanner35.py, uncompyle6/scanners/scanner36.py:
Start supporting Pypy 3.5 (5.7.1-beta)
2017-07-05 rocky <rb@dustyfeet.com>
* test/simple_source/bug26/03_loop_if_cf.py,
uncompyle6/parsers/parse26.py: Loops in Python 2.4-2.6 loop
come_from Looks like Python 2.4-2.6 may have a COME_FROM(_LOOP) before the
jump_back. Fixes Issue #123
2017-06-29 rocky <rb@dustyfeet.com>
* : Work around not having real flow-control analysis
2017-06-28 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/make_function.py: A guard against badly
formated bytecode
2017-06-25 rocky <rb@dustyfeet.com>
* ChangeLog, NEWS, test/simple_source/bug31/04_def_annotate.py,
uncompyle6/semantics/make_function.py,
uncompyle6/semantics/pysource.py: 3.x funciton and annotation bug
fixes
2017-06-25 rocky <rb@dustyfeet.com> 2017-06-25 rocky <rb@dustyfeet.com>
* uncompyle6/version.py: Get ready for release 2.11.1 * uncompyle6/version.py: Get ready for release 2.11.1

View File

@@ -44,8 +44,8 @@ it appears that Hartmut did most of the work to get this code to
accept the full Python language. He added precedence to the table accept the full Python language. He added precedence to the table
specifiers, support for multiple versions of Python, the specifiers, support for multiple versions of Python, the
pretty-printing of docstrings, lists, and hashes. He also wrote test and verification routines of pretty-printing of docstrings, lists, and hashes. He also wrote test and verification routines of
deparsed bytecode, and used this in an extensive set of tests that he also wrote. He says he could verify against the deparsed bytecode, and used this in an extensive set of tests that he also wrote. He says he could verify against the
entire Python library. However I have subsequently found small and relatively obscure bugs in the decompilation code. entire Python library. However I have subsequently found small and relatively obscure bugs in the decompilation code.
decompyle2.2 was packaged for Debian (sarge) by decompyle2.2 was packaged for Debian (sarge) by
[Ben Burton around 2002](https://packages.qa.debian.org/d/decompyle.html). As [Ben Burton around 2002](https://packages.qa.debian.org/d/decompyle.html). As
@@ -66,7 +66,7 @@ code to handle first Python 2.3 and then 2.4 bytecodes. Because of
jump optimization introduced in the CPython bytecode compiler at that jump optimization introduced in the CPython bytecode compiler at that
time, various JUMP instructions were classifed as going backwards, and time, various JUMP instructions were classifed as going backwards, and
COME FROM instructions were reintroduced. See COME FROM instructions were reintroduced. See
[RELEASE-2.4-CHANGELOG.txt](https://github.com/rocky/python-uncompyle6/blob/master/DECOMPYLE-2.4-CHANGELOG.txt) [RELEASE-2.4-CHANGELOG.txt](https://github.com/rocky/python-uncompyle6/blob/master/DECOMPYLE-2.4-CHANGELOG.txt)
for more details here. There wasn't a public for more details here. There wasn't a public
release of RELEASE-2.4 and bytecodes other than Python 2.4 weren't release of RELEASE-2.4 and bytecodes other than Python 2.4 weren't
supported. Dan says the Python 2.3 version could verify the entire supported. Dan says the Python 2.3 version could verify the entire
@@ -99,7 +99,7 @@ made a few commits later on. But mostly wibiti, and Guenther
Starnberger got the code to where uncompyle2 was around 2012. Starnberger got the code to where uncompyle2 was around 2012.
In `uncompyle`, decompilation of python bytecode 2.5 & 2.6 is done by In `uncompyle`, decompilation of python bytecode 2.5 & 2.6 is done by
transforming the byte code into a a pseudo 2.7 python bytecode and is transforming the byte code into a pseudo-2.7 Python bytecode and is
based on code from Eloi Vanderbeken. based on code from Eloi Vanderbeken.
This project, `uncompyle6`, abandons that approach for various This project, `uncompyle6`, abandons that approach for various
@@ -120,10 +120,10 @@ while, handling Python bytecodes from Python versions 2.5+ and
3.2+. In doing so, it has been expedient to separate this into three 3.2+. In doing so, it has been expedient to separate this into three
projects: projects:
* bytecode loading and disassembly ([xdis](https://pypi.python.org/pypi/xdis)), * marshaling/unmarshaling, bytecode loading and disassembly ([xdis](https://pypi.python.org/pypi/xdis)),
* parsing and tree building ([spark_parser](https://pypi.python.org/pypi/spark_parser)), * parsing and tree building ([spark_parser](https://pypi.python.org/pypi/spark_parser)),
* this project - grammar and semantic actions for decompiling * this project - grammar and semantic actions for decompiling
([uncompyle6](https://pypi.python.org/pypi/spark_parser)). ([uncompyle6](https://pypi.python.org/pypi/uncompyle6)).
Over the many years, code styles and Python features have Over the many years, code styles and Python features have
@@ -162,5 +162,8 @@ support has been lagging.
Tests for the project have been, or are being, culled from all of the Tests for the project have been, or are being, culled from all of the
projects mentioned. projects mentioned.
For a little bit of the history of changes to the Early-algorithm parser,
see the file [NEW-FEATURES.rst](https://github.com/rocky/python-spark/blob/master/NEW-FEATURES.rst) in the [python-spark github repository](https://github.com/rocky/python-spark).
NB. If you find mistakes, want corrections, or want your name added NB. If you find mistakes, want corrections, or want your name added
(or removed), please contact me. (or removed), please contact me.

View File

@@ -19,7 +19,7 @@ So it is likely you'll find a mistranslation in decompiling.
The basic requirement is pretty simple: The basic requirement is pretty simple:
* Python bytecode * Python bytecode
* Source text * Python source text
## What to send (additional helpful information) ## What to send (additional helpful information)
@@ -50,7 +50,7 @@ one fool can learn, so can another."
## Narrowing the problem ## Narrowing the problem
I don't need the entire source code base for which one file or module I don't need or want the entire source code base for which one file or module
can't be decompiled. I just need that one file or module only. If can't be decompiled. I just need that one file or module only. If
there are several files, file a bug report for each file. there are several files, file a bug report for each file.

56
NEWS
View File

@@ -1,10 +1,26 @@
uncompyle6 2.11.1 2016-06-22 uncompyle6 2.11.2 2017-08-09
- Python 3.x annotatoin and function signature fixes Very minor changes
- RsT doc fixes and updates
- use newer xdis, but not too new; 3.5.2 breaks uncompyle6
- use xdis opcode sets
- xdis "exception match" is now "exception-match"
uncompyle6 2.11.2 2017-07-09
- Start supporting Pypy 3.5 (5.7.1-beta)
- use xdis 3.5.0's opcode sets and require xdis 3.5.0
- Correct some Python 2.4-2.6 loop detection
- guard against badly formatted bytecode
uncompyle6 2.11.1 2017-06-25
- Python 3.x annotation and function signature fixes
- Bump xdis version - Bump xdis version
- Small pysource bug fixes - Small pysource bug fixes
uncompyle6 2.11.0 2016-06-18 Fleetwood uncompyle6 2.11.0 2017-06-18 Fleetwood
- Major improvements in fragment tracking - Major improvements in fragment tracking
* Add nonterminal node in extractInfo * Add nonterminal node in extractInfo
* tag more offsets in expressions * tag more offsets in expressions
@@ -14,17 +30,17 @@ uncompyle6 2.11.0 2016-06-18 Fleetwood
- Fixes yet again for make_function node handling; document what's up here - Fixes yet again for make_function node handling; document what's up here
- Fix bug in snowflake Python 3.5 *args kwargs - Fix bug in snowflake Python 3.5 *args kwargs
uncompyle6 2.10.1 2016-06-3 Marylin Frankel uncompyle6 2.10.1 2017-06-3 Marylin Frankel
- fix some fragments parsing bugs - fix some fragments parsing bugs
- was returning the wrong type sometimes in deparse_code_around_offset() - was returning the wrong type sometimes in deparse_code_around_offset()
- capture function name in offsets - capture function name in offsets
- track changes to ifelstrmtr node from pysource into fragments - track changes to ifelstrmtr node from pysource into fragments
uncompyle6 2.10.0 2016-05-30 Elaine Gordon uncompyle6 2.10.0 2017-05-30 Elaine Gordon
- Add fuzzy offset deparse lookup - Add fuzzy offset deparse look up
- 3.6 bugfixes - 3.6 bug fixes
- fix EXTENDED_ARGS handling (and in 2.6 and others) - fix EXTENDED_ARGS handling (and in 2.6 and others)
- semantic routine make_function fragments.py - semantic routine make_function fragments.py
- MAKE_FUNCTION handling - MAKE_FUNCTION handling
@@ -35,19 +51,19 @@ uncompyle6 2.10.0 2016-05-30 Elaine Gordon
- 3.5 FUNCTION_VAR bug - 3.5 FUNCTION_VAR bug
- 3.x pass statement insdie while True - 3.x pass statement insdie while True
- Improve 3.2 decompilation - Improve 3.2 decompilation
- Fixed -o argument processing (Gregrory) - Fixed -o argument processing (grkov90)
- Reduce scope of LOAD_ASSERT as expr to 3.4+ - Reduce scope of LOAD_ASSERT as expr to 3.4+
- "await" statement fixes - "await" statement fixes
- 2.3, 2.4 "if 1 .." fixes - 2.3, 2.4 "if 1 .." fixes
- 3.x annotation fixes - 3.x annotation fixes
uncompyle6 2.9.11 2016-04-06 uncompyle6 2.9.11 2017-04-06
- Better support for Python 3.5+ BUILD_MAP_UNPACK - Better support for Python 3.5+ BUILD_MAP_UNPACK
- Start 3.6 CALL_FUNCTION_EX support - Start 3.6 CALL_FUNCTION_EX support
- Many decompilation bug fixes. (Many more remain). See ChangeLog - Many decompilation bug fixes. (Many more remain). See ChangeLog
uncompyle6 2.9.10 2016-02-25 uncompyle6 2.9.10 2017-02-25
- Python grammar rule fixes - Python grammar rule fixes
- Add ability to get grammar coverage on runs - Add ability to get grammar coverage on runs
@@ -114,7 +130,7 @@ uncompyle6 2.9.6 2016-11-20
uncompyle6 2.9.5 2016-11-13 uncompyle6 2.9.5 2016-11-13
- Fix Python 3 bugs: - Fix Python 3 bugs:
* improprer while 1 else * improper while 1 else
* docstring indent * docstring indent
* 3.3 default values in lambda expressions * 3.3 default values in lambda expressions
* start 3.0 decompilation (needs newer xdis) * start 3.0 decompilation (needs newer xdis)
@@ -124,12 +140,12 @@ uncompyle6 2.9.5 2016-11-13
uncompyle6 2.9.4 2016-11-02 uncompyle6 2.9.4 2016-11-02
- Handle Python 3.x function annotations - Handle Python 3.x function annotations
- track def keywoard-parameter line-splitting in source code better - track def keyword-parameter line-splitting in source code better
- bump min xdis version to mask previous xdis bug - bump min xdis version to mask previous xdis bug
uncompyle6 2.9.3 2016-10-26 uncompyle6 2.9.3 2016-10-26
Release forced by incompatiblity change in xdis 3.2.0. Release forced by incompatibility change in xdis 3.2.0.
- Python 3.1 bugs: - Python 3.1 bugs:
* handle "with ... as" * handle "with ... as"
@@ -161,7 +177,7 @@ uncompyle6 2.9.0 2016-10-09
this Forces change in requirements.txt and _pkg_info_.py this Forces change in requirements.txt and _pkg_info_.py
- Start Python 1.5 decompiling; another round of work is needed to - Start Python 1.5 decompiling; another round of work is needed to
remove bugs remove bugs
- Simpify python 2.1 grammar - Simplify python 2.1 grammar
- Fix bug with -t ... Wasn't showing source text when -t option was given - Fix bug with -t ... Wasn't showing source text when -t option was given
- Fix 2.1-2.6 bug in list comprehension - Fix 2.1-2.6 bug in list comprehension
@@ -184,7 +200,7 @@ control-flow structure detection is done.
. 3.0 .. 3.2 *args processing . 3.0 .. 3.2 *args processing
. 3.0 .. 3.2 call name and kwargs bug . 3.0 .. 3.2 call name and kwargs bug
. 3.0 .. getting parameter of * . 3.0 .. getting parameter of *
. 3.0 .. handling varible number of args . 3.0 .. handling variable number of args
. 3.0 .. "if" structure bugs . 3.0 .. "if" structure bugs
* 3.5+ if/else bugs * 3.5+ if/else bugs
* 2.2-2.6 bugs * 2.2-2.6 bugs
@@ -235,7 +251,7 @@ uncompyle6 2.7.1 2016-07-26
uncompyle6 2.7.0 2016-07-15 uncompyle6 2.7.0 2016-07-15
- Many Syntax and verifification bugs removed - Many Syntax and verification bugs removed
tested on standard libraries from 2.3.7 to 3.5.1 tested on standard libraries from 2.3.7 to 3.5.1
and they all decompile and verify fine. and they all decompile and verify fine.
I'm sure there are more bugs though. I'm sure there are more bugs though.
@@ -262,9 +278,9 @@ uncompyle6 2.6.0 2016-07-07
- Better <2.6 vs. 2.7 grammar separation - Better <2.6 vs. 2.7 grammar separation
- Fix some 2.7 deparsing bugs - Fix some 2.7 deparsing bugs
- Fix bug in installing uncompyle6 script - Fix bug in installing uncompyle6 script
- Doc improvments - Doc improvements
uncompyle6 2.5.0 2016-06-22 Summer Solstace uncompyle6 2.5.0 2016-06-22 Summer Solstice
- Much better Python 3.2-3.5 coverage. - Much better Python 3.2-3.5 coverage.
3.4.6 is probably the best;3.2 and 3.5 are weaker 3.4.6 is probably the best;3.2 and 3.5 are weaker
@@ -276,7 +292,7 @@ uncompyle6 2.5.0 2016-06-22 Summer Solstace
uncompyle6 2.4.0 2016-05-18 (in memory of Lewis Bernstein) uncompyle6 2.4.0 2016-05-18 (in memory of Lewis Bernstein)
- Many Python 3 bugs fixed: - Many Python 3 bugs fixed:
* Python 3.2 to 3.5 libaries largely * Python 3.2 to 3.5 libraries largely
uncompyle and most verify uncompyle and most verify
- pydisassembler: - pydisassembler:
* disassembles all code objects in a file * disassembles all code objects in a file
@@ -334,7 +350,7 @@ uncompyle6 2.2.0 2016-04-30
uncompyle6 2.2.0 2016-04-02 uncompyle6 2.2.0 2016-04-02
- Support single-mode (in addtion to exec-mode) compilation - Support single-mode (in addition to exec-mode) compilation
- Start to DRY Python 2 and Python 3 grammars - Start to DRY Python 2 and Python 3 grammars
- Fix bug in if else ternary construct - Fix bug in if else ternary construct
- Fix bug in uncomplye6 -d and -r options (via lelicopter) - Fix bug in uncomplye6 -d and -r options (via lelicopter)

View File

@@ -171,8 +171,11 @@ See Also
* https://code.google.com/archive/p/unpyc3/ : supports Python 3.2 only. The above projects use a different decompiling technique than what is used here. * https://code.google.com/archive/p/unpyc3/ : supports Python 3.2 only. The above projects use a different decompiling technique than what is used here.
* https://github.com/figment/unpyc3/ : fork of above, but supports Python 3.3 only. Include some fixes like supporting function annotations * https://github.com/figment/unpyc3/ : fork of above, but supports Python 3.3 only. Include some fixes like supporting function annotations
* The HISTORY_ file. * The HISTORY_ file.
* `How to report a bug <https://github.com/rocky/python-uncompyle6/blob/master/HISTORY.md>`_ * `How to report a bug <https://github.com/rocky/python-uncompyle6/blob/master/HOW-TO-REPORT-A-BUG.md>`_
.. |downloads| image:: https://img.shields.io/pypi/dd/uncompyle6.svg * https://github.com/rocky/python-xdis : Cross Python version disassembler
* https://github.com/rocky/python-xasm : Cross Python version assembler
.. _trepan: https://pypi.python.org/pypi/trepan .. _trepan: https://pypi.python.org/pypi/trepan
.. _HISTORY: https://github.com/rocky/python-uncompyle6/blob/master/HISTORY.md .. _HISTORY: https://github.com/rocky/python-uncompyle6/blob/master/HISTORY.md
.. _debuggers: https://pypi.python.org/pypi/trepan3k .. _debuggers: https://pypi.python.org/pypi/trepan3k

View File

@@ -40,7 +40,7 @@ entry_points = {
]} ]}
ftp_url = None ftp_url = None
install_requires = ['spark-parser >= 1.6.1, < 1.7.0', install_requires = ['spark-parser >= 1.6.1, < 1.7.0',
'xdis >= 3.5.1, < 3.6.0'] 'xdis == 3.5.1, < 3.6.0']
license = 'MIT' license = 'MIT'
mailing_list = 'python-debugger@googlegroups.com' mailing_list = 'python-debugger@googlegroups.com'
modname = 'uncompyle6' modname = 'uncompyle6'

Binary file not shown.

View File

@@ -0,0 +1,19 @@
# Bug in < 2.6 is having a COME_FROM_LOOP (but we
# don't tag that so it is just COME_FROM *before*
# a jump back to the loop.
def pickup(self, open_players, open_buf, wrap_buf):
for aplayer in self._game.active_players:
if aplayer in open_players:
aplayer.send(open_players)
if self == aplayer:
for awatcher in self._watchers:
if awatcher._can_see_detail:
awatcher.send(open_buf)
else:
awatcher.send(wrap_buf)
else:
self._game.send(aplayer.side)
else:
self._game.send(aplayer.side, wrap_buf)

View File

@@ -27,7 +27,7 @@ from fnmatch import fnmatch
TEST_VERSIONS=('2.3.7', '2.4.6', '2.5.6', '2.6.9', TEST_VERSIONS=('2.3.7', '2.4.6', '2.5.6', '2.6.9',
'pypy-2.4.0', 'pypy-2.6.1', 'pypy-2.4.0', 'pypy-2.6.1',
'pypy-5.0.1', 'pypy-5.3.1', 'pypy-5.0.1', 'pypy-5.3.1', 'pypy3.5-5.7.1-beta',
'2.7.10', '2.7.11', '2.7.12', '2.7.13', '2.7.10', '2.7.11', '2.7.12', '2.7.13',
'3.0.1', '3.1.5', '3.2.6', '3.0.1', '3.1.5', '3.2.6',
'3.3.5', '3.3.6', '3.3.5', '3.3.6',

View File

@@ -84,6 +84,12 @@ class Python26Parser(Python2Parser):
ja_cf_pop ::= JUMP_ABSOLUTE come_froms POP_TOP ja_cf_pop ::= JUMP_ABSOLUTE come_froms POP_TOP
jf_cf_pop ::= JUMP_FORWARD come_froms POP_TOP jf_cf_pop ::= JUMP_FORWARD come_froms POP_TOP
# The first optional COME_FROM when it appears is really
# COME_FROM_LOOP, but in <= 2.6 we don't distinguish
# this
cf_jb_cf_pop ::= _come_from JUMP_BACK come_froms POP_TOP
bp_come_from ::= POP_BLOCK COME_FROM bp_come_from ::= POP_BLOCK COME_FROM
jb_bp_come_from ::= JUMP_BACK bp_come_from jb_bp_come_from ::= JUMP_BACK bp_come_from
@@ -111,7 +117,8 @@ class Python26Parser(Python2Parser):
break_stmt ::= BREAK_LOOP JUMP_BACK break_stmt ::= BREAK_LOOP JUMP_BACK
# Semantic actions want else_suitel to be at index 3 # Semantic actions want else_suitel to be at index 3
ifelsestmtl ::= testexpr c_stmts_opt jb_cf_pop else_suitel ifelsestmtl ::= testexpr c_stmts_opt cf_jb_cf_pop else_suitel
ifelsestmtc ::= testexpr c_stmts_opt ja_cf_pop else_suitec ifelsestmtc ::= testexpr c_stmts_opt ja_cf_pop else_suitec
# Semantic actions want suite_stmts_opt to be at index 3 # Semantic actions want suite_stmts_opt to be at index 3

View File

@@ -87,7 +87,7 @@ class Scanner(object):
if op is None: if op is None:
op = self.code[pos] op = self.code[pos]
target = self.get_argument(pos) target = self.get_argument(pos)
if op in self.opc.hasjrel: if op in self.opc.JREL_OPS:
target += pos + 3 target += pos + 3
return target return target
@@ -98,7 +98,7 @@ class Scanner(object):
def print_bytecode(self): def print_bytecode(self):
for i in self.op_range(0, len(self.code)): for i in self.op_range(0, len(self.code)):
op = self.code[i] op = self.code[i]
if op in self.opc.hasjabs+self.opc.hasjrel: if op in self.JUMP_OPs:
dest = self.get_target(i, op) dest = self.get_target(i, op)
print('%i\t%s\t%i' % (i, self.opname[op], dest)) print('%i\t%s\t%i' % (i, self.opname[op], dest))
else: else:

View File

@@ -1,22 +1,18 @@
# Copyright (c) 2016 by Rocky Bernstein # Copyright (c) 2017 by Rocky Bernstein
""" """
Python PyPy 3.2 bytecode scanner/deparser Python PyPy 3.2 decompiler scanner.
This overlaps Python's 3.2's dis module, but it can be run from Does some additional massaging of xdis-disassembled instructions to
Python 3 and other versions of Python. Also, we save token make things easier for decompilation.
information for later use in deparsing.
""" """
import uncompyle6.scanners.scanner32 as scan import uncompyle6.scanners.scanner32 as scan
# bytecode verification, verify(), uses JUMP_OPs from here # bytecode verification, verify(), uses JUMP_OPs from here
from xdis.opcodes import opcode_32 as opc # is this rgith? from xdis.opcodes import opcode_32 as opc # is this right?
JUMP_OPs = map(lambda op: opc.opname[op], opc.hasjrel + opc.hasjabs) JUMP_OPs = map(lambda op: opc.opname[op], opc.hasjrel + opc.hasjabs)
# We base this off of 2.6 instead of the other way around # We base this off of 3.2
# because we cleaned things up this way.
# The history is that 2.7 support is the cleanest,
# then from that we got 2.6 and so on.
class ScannerPyPy32(scan.Scanner32): class ScannerPyPy32(scan.Scanner32):
def __init__(self, show_asm): def __init__(self, show_asm):
# There are no differences in initialization between # There are no differences in initialization between

View File

@@ -0,0 +1,22 @@
# Copyright (c) 2017 by Rocky Bernstein
"""
Python PyPy 3.2 decompiler scanner.
Does some additional massaging of xdis-disassembled instructions to
make things easier for decompilation.
"""
import uncompyle6.scanners.scanner35 as scan
# bytecode verification, verify(), uses JUMP_OPs from here
from xdis.opcodes import opcode_35 as opc # is this right?
JUMP_OPs = map(lambda op: opc.opname[op], opc.hasjrel + opc.hasjabs)
# We base this off of 3.5
class ScannerPyPy35(scan.Scanner35):
def __init__(self, show_asm):
# There are no differences in initialization between
# pypy 3.5 and 3.5
scan.Scanner35.__init__(self, show_asm, is_pypy=True)
self.version = 3.5
return

View File

@@ -1,6 +1,6 @@
# Copyright (c) 2016 by Rocky Bernstein # Copyright (c) 2016-2017 by Rocky Bernstein
""" """
Python 1.5 bytecode scanner/deparser Python 1.5 bytecode decompiler scanner.
This massages tokenized 1.5 bytecode to make it more amenable for This massages tokenized 1.5 bytecode to make it more amenable for
grammar parsing. grammar parsing.

View File

@@ -195,7 +195,7 @@ class Scanner2(Scanner):
if op == self.opc.EXTENDED_ARG: if op == self.opc.EXTENDED_ARG:
extended_arg = oparg * L65536 extended_arg = oparg * L65536
continue continue
if op in self.opc.hasconst: if op in self.opc.CONST_OPS:
const = co.co_consts[oparg] const = co.co_consts[oparg]
if iscode(const): if iscode(const):
oparg = const oparg = const
@@ -216,23 +216,23 @@ class Scanner2(Scanner):
pattr = '<code_object ' + const.co_name + '>' pattr = '<code_object ' + const.co_name + '>'
else: else:
pattr = const pattr = const
elif op in self.opc.hasname: elif op in self.opc.NAME_OPS:
pattr = names[oparg] pattr = names[oparg]
elif op in self.opc.hasjrel: elif op in self.opc.JREL_OPS:
# use instead: hasattr(self, 'patch_continue'): ? # use instead: hasattr(self, 'patch_continue'): ?
if self.version == 2.7: if self.version == 2.7:
self.patch_continue(tokens, offset, op) self.patch_continue(tokens, offset, op)
pattr = repr(offset + 3 + oparg) pattr = repr(offset + 3 + oparg)
elif op in self.opc.hasjabs: elif op in self.opc.JABS_OPS:
# use instead: hasattr(self, 'patch_continue'): ? # use instead: hasattr(self, 'patch_continue'): ?
if self.version == 2.7: if self.version == 2.7:
self.patch_continue(tokens, offset, op) self.patch_continue(tokens, offset, op)
pattr = repr(oparg) pattr = repr(oparg)
elif op in self.opc.haslocal: elif op in self.opc.LOCAL_OPS:
pattr = varnames[oparg] pattr = varnames[oparg]
elif op in self.opc.hascompare: elif op in self.opc.COMPARE_OPS:
pattr = self.opc.cmp_op[oparg] pattr = self.opc.cmp_op[oparg]
elif op in self.opc.hasfree: elif op in self.opc.FREE_OPS:
pattr = free[oparg] pattr = free[oparg]
if op in self.varargs_ops: if op in self.varargs_ops:
@@ -458,7 +458,7 @@ class Scanner2(Scanner):
self.not_continue.add(jmp) self.not_continue.add(jmp)
jmp = self.get_target(jmp) jmp = self.get_target(jmp)
prev_offset = self.prev[except_match] prev_offset = self.prev[except_match]
# COMPARE_OP argument should be "exception match" or 10 # COMPARE_OP argument should be "exception-match" or 10
if (self.code[prev_offset] == self.opc.COMPARE_OP and if (self.code[prev_offset] == self.opc.COMPARE_OP and
self.code[prev_offset+1] != 10): self.code[prev_offset+1] != 10):
return None return None
@@ -609,7 +609,7 @@ class Scanner2(Scanner):
if test == offset: if test == offset:
loop_type = 'while 1' loop_type = 'while 1'
elif self.code[test] in self.opc.hasjabs + self.opc.hasjrel: elif self.code[test] in self.opc.JUMP_OPs:
self.ignore_if.add(test) self.ignore_if.add(test)
test_target = self.get_target(test) test_target = self.get_target(test)
if test_target > (jump_back+3): if test_target > (jump_back+3):
@@ -911,7 +911,9 @@ class Scanner2(Scanner):
'start': start-3, 'start': start-3,
'end': pre_rtarget}) 'end': pre_rtarget})
self.not_continue.add(pre_rtarget) # FIXME: this is yet another case were we need dominators.
if pre_rtarget not in self.linestartoffsets or self.version < 2.7:
self.not_continue.add(pre_rtarget)
if rtarget < end: if rtarget < end:
# We have an "else" block of some kind. # We have an "else" block of some kind.
@@ -998,11 +1000,11 @@ class Scanner2(Scanner):
oparg = self.get_argument(offset) oparg = self.get_argument(offset)
if label is None: if label is None:
if op in self.opc.hasjrel and self.op_name(op) != 'FOR_ITER': if op in self.opc.JREL_OPS and self.op_name(op) != 'FOR_ITER':
# if (op in self.opc.hasjrel and # if (op in self.opc.JREL_OPS and
# (self.version < 2.0 or op != self.opc.FOR_ITER)): # (self.version < 2.0 or op != self.opc.FOR_ITER)):
label = offset + 3 + oparg label = offset + 3 + oparg
elif self.version == 2.7 and op in self.opc.hasjabs: elif self.version == 2.7 and op in self.opc.JABS_OPS:
if op in (self.opc.JUMP_IF_FALSE_OR_POP, if op in (self.opc.JUMP_IF_FALSE_OR_POP,
self.opc.JUMP_IF_TRUE_OR_POP): self.opc.JUMP_IF_TRUE_OR_POP):
if (oparg > offset): if (oparg > offset):

View File

@@ -183,7 +183,7 @@ class Scanner26(scan.Scanner2):
if op == self.opc.EXTENDED_ARG: if op == self.opc.EXTENDED_ARG:
extended_arg = oparg * L65536 extended_arg = oparg * L65536
continue continue
if op in self.opc.hasconst: if op in self.opc.CONST_OPS:
const = co.co_consts[oparg] const = co.co_consts[oparg]
# We can't use inspect.iscode() because we may be # We can't use inspect.iscode() because we may be
# using a different version of Python than the # using a different version of Python than the
@@ -208,9 +208,9 @@ class Scanner26(scan.Scanner2):
pattr = '<code_object ' + const.co_name + '>' pattr = '<code_object ' + const.co_name + '>'
else: else:
pattr = const pattr = const
elif op in self.opc.hasname: elif op in self.opc.NAME_OPS:
pattr = names[oparg] pattr = names[oparg]
elif op in self.opc.hasjrel: elif op in self.opc.JREL_OPS:
pattr = repr(offset + 3 + oparg) pattr = repr(offset + 3 + oparg)
if op == self.opc.JUMP_FORWARD: if op == self.opc.JUMP_FORWARD:
target = self.get_target(offset) target = self.get_target(offset)
@@ -220,13 +220,13 @@ class Scanner26(scan.Scanner2):
if len(tokens) and tokens[-1].type == 'JUMP_BACK': if len(tokens) and tokens[-1].type == 'JUMP_BACK':
tokens[-1].type = intern('CONTINUE') tokens[-1].type = intern('CONTINUE')
elif op in self.opc.hasjabs: elif op in self.opc.JABS_OPS:
pattr = repr(oparg) pattr = repr(oparg)
elif op in self.opc.haslocal: elif op in self.opc.LOCAL_OPS:
pattr = varnames[oparg] pattr = varnames[oparg]
elif op in self.opc.hascompare: elif op in self.opc.COMPARE_OPS:
pattr = self.opc.cmp_op[oparg] pattr = self.opc.cmp_op[oparg]
elif op in self.opc.hasfree: elif op in self.opc.FREE_OPS:
pattr = free[oparg] pattr = free[oparg]
if op in self.varargs_ops: if op in self.varargs_ops:
# CE - Hack for >= 2.5 # CE - Hack for >= 2.5

View File

@@ -639,11 +639,11 @@ class Scanner3(Scanner):
rel_offset = 0 rel_offset = 0
if self.version >= 3.6: if self.version >= 3.6:
target = self.code[offset+1] target = self.code[offset+1]
if op in self.opc.hasjrel: if op in self.opc.JREL_OPS:
rel_offset = offset + 2 rel_offset = offset + 2
else: else:
target = self.code[offset+1] + self.code[offset+2] * 256 target = self.code[offset+1] + self.code[offset+2] * 256
if op in self.opc.hasjrel: if op in self.opc.JREL_OPS:
rel_offset = offset + 3 rel_offset = offset + 3
pass pass
pass pass

View File

@@ -118,7 +118,7 @@ class Scanner30(Scanner3):
if test == offset: if test == offset:
loop_type = 'while 1' loop_type = 'while 1'
elif self.code[test] in opc.hasjabs+opc.hasjrel: elif self.code[test] in opc.JUMP_OPs:
self.ignore_if.add(test) self.ignore_if.add(test)
test_target = self.get_target(test) test_target = self.get_target(test)
if test_target > (jump_back+3): if test_target > (jump_back+3):

View File

@@ -1,6 +1,9 @@
# Copyright (c) 2015-2016 by Rocky Bernstein # Copyright (c) 2015-2017 by Rocky Bernstein
""" """
Python 3.2 bytecode scanner/deparser Python 3.2 bytecode decompiler scanner.
Does some additional massaging of xdis-disassembled instructions to
make things easier for decompilation.
This sets up opcodes Python's 3.2 and calls a generalized This sets up opcodes Python's 3.2 and calls a generalized
scanner routine for Python 3. scanner routine for Python 3.

View File

@@ -1,6 +1,9 @@
# Copyright (c) 2015-2016 by Rocky Bernstein # Copyright (c) 2015-2017 by Rocky Bernstein
""" """
Python 3.4 bytecode scanner/deparser Python 3.4 bytecode decompiler scanner
Does some additional massaging of xdis-disassembled instructions to
make things easier for decompilation.
This sets up opcodes Python's 3.4 and calls a generalized This sets up opcodes Python's 3.4 and calls a generalized
scanner routine for Python 3. scanner routine for Python 3.

View File

@@ -1,6 +1,9 @@
# Copyright (c) 2016 by Rocky Bernstein # Copyright (c) 2017 by Rocky Bernstein
""" """
Python 3.5 bytecode scanner/deparser Python 3.5 bytecode decompiler scanner
Does some additional massaging of xdis-disassembled instructions to
make things easier for decompilation.
This sets up opcodes Python's 3.5 and calls a generalized This sets up opcodes Python's 3.5 and calls a generalized
scanner routine for Python 3. scanner routine for Python 3.
@@ -14,8 +17,8 @@ JUMP_OPs = map(lambda op: opc.opname[op], opc.hasjrel + opc.hasjabs)
class Scanner35(Scanner3): class Scanner35(Scanner3):
def __init__(self, show_asm=None): def __init__(self, show_asm=None, is_pypy=False):
Scanner3.__init__(self, 3.5, show_asm) Scanner3.__init__(self, 3.5, show_asm, is_pypy)
return return
pass pass

View File

@@ -1,6 +1,9 @@
# Copyright (c) 2016-2017 by Rocky Bernstein # Copyright (c) 2016-2017 by Rocky Bernstein
""" """
Python 3.6 bytecode scanner/deparser Python 3.6 bytecode decompiler scanner
Does some additional massaging of xdis-disassembled instructions to
make things easier for decompilation.
This sets up opcodes Python's 3.6 and calls a generalized This sets up opcodes Python's 3.6 and calls a generalized
scanner routine for Python 3. scanner routine for Python 3.

View File

@@ -71,10 +71,10 @@ class Token:
if self.pattr: if self.pattr:
pattr = self.pattr pattr = self.pattr
if self.opc: if self.opc:
if self.op in self.opc.hasjrel: if self.op in self.opc.JREL_OPS:
if not self.pattr.startswith('to '): if not self.pattr.startswith('to '):
pattr = "to " + self.pattr pattr = "to " + self.pattr
elif self.op in self.opc.hasjabs: elif self.op in self.opc.JABS_OPS:
self.pattr= str(self.pattr) self.pattr= str(self.pattr)
if not self.pattr.startswith('to '): if not self.pattr.startswith('to '):
pattr = "to " + str(self.pattr) pattr = "to " + str(self.pattr)

View File

@@ -413,7 +413,8 @@ def make_function2(self, node, isLambda, nested=1, codeNode=None):
if code_has_star_star_arg(code): if code_has_star_star_arg(code):
if argc > 0: if argc > 0:
self.write(', ') self.write(', ')
self.write('**%s' % code.co_varnames[argc + kw_pairs]) if argc + kw_pairs > 0:
self.write('**%s' % code.co_varnames[argc + kw_pairs])
if isLambda: if isLambda:
self.write(": ") self.write(": ")

View File

@@ -1,3 +1,3 @@
# This file is suitable for sourcing inside bash as # This file is suitable for sourcing inside bash as
# well as importing into Python # well as importing into Python
VERSION='2.11.1' VERSION='2.11.3'