Realign make_function3 with master

Reduced checking via "make check-short"

Makefile

@@ -38,6 +38,7 @@ check-3.0 check-3.1 check-3.2 check-3.6:
 	$(MAKE) -C test $@
 
 check-3.7: pytest
+	$(MAKE) -C test check
 
 #:Tests for Python 2.4-2.5 (don't have pytest)
 check-2.4 check-2.5:

NEWS

@@ -1,3 +1,23 @@
+uncompyle6 3.2.3 2018-06-04 Michael Cohen flips and Fleetwood Redux
+
+- Python 1.3 support 3.0 bug and
+- fix botched parameter ordering of 3.x in last release
+
+uncompyle6 3.2.2 2018-06-04 When I'm 64
+
+- Python 3.0 support and bug fixes
+
+uncompyle6 3.2.1 2018-06-04 MF
+
+- Python 1.4 and 1.5 bug fixes
+
+uncompyle6 3.2.0 2018-05-19 Rocket Scientist
+
+- Add rudimentary 1.4 support (still a bit buggy)
+- add --tree+ option to show formatting rule, when it is constant
+- Python 2.7.15candidate1 support (via xdis)
+- bug fixes, especially for 3.7 (but 2.7 and 3.6 and others as well)
+
 uncompyle6 3.1.3 2018-04-16
 
 - Add some Python 3.7 rules, such as for handling LOAD_METHOD (not complete)

README.rst

@@ -1,4 +1,4 @@
-|buildstatus|
+|buildstatus| |Latest Version| |Supported Python Versions|
 
 uncompyle6
 ==========
@@ -11,8 +11,9 @@ Introduction
 ------------
 
 *uncompyle6* translates Python bytecode back into equivalent Python
-source code. It accepts bytecodes from Python version 1.5, and 2.1 to
-3.7 or so, including PyPy bytecode and Dropbox's Python 2.5 bytecode.
+source code. It accepts bytecodes from Python version 1.3 to version
+3.7, spanning over 22 years of Python releases. We include Dropbox's
+Python 2.5 bytecode and some PyPy bytecode.
 
 Why this?
 ---------
@@ -29,11 +30,11 @@ CPython bytecode decompilers is the ability to deparse just
 *fragments* of source code and give source-code information around a
 given bytecode offset.
 
-I use the tree fragments to deparse fragments of code inside my
-trepan_ debuggers_. For that, bytecode offsets are recorded and
-associated with fragments of the source code. This purpose, although
-compatible with the original intention, is yet a little bit different.
-See this_ for more information.
+I use the tree fragments to deparse fragments of code *at run time*
+inside my trepan_ debuggers_. For that, bytecode offsets are recorded
+and associated with fragments of the source code. This purpose,
+although compatible with the original intention, is yet a little bit
+different.  See this_ for more information.
 
 Python fragment deparsing given an instruction offset is useful in
 showing stack traces and can be encorporated into any program that
@@ -58,7 +59,7 @@ provides decompilation for subset of Python versions, we generally do
 demonstrably better for those as well.
 
 How can we tell? By taking Python bytecode that comes distributed with
-that version of Python and decompiling these.  Among htose that
+that version of Python and decompiling these.  Among those that
 successfully decompile, we can then make sure the resulting programs
 are syntactically correct by running the Python interpreter for that
 bytecode version.  Finally, in cases where the program has a test for
@@ -75,7 +76,7 @@ Requirements
 The code here can be run on Python versions 2.6 or later, PyPy 3-2.4,
 or PyPy-5.0.1.  Python versions 2.4-2.7 are supported in the
 python-2.4 branch.  The bytecode files it can read have been tested on
-Python bytecodes from versions 1.5, 2.1-2.7, and 3.0-3.6 and the
+Python bytecodes from versions 1.4, 2.1-2.7, and 3.0-3.6 and the
 above-mentioned PyPy versions.
 
 Installation
@@ -151,12 +152,12 @@ for that bytecode version. Having done this the bytecode produced
 could be compared with the original bytecode. However as Python's code
 generation got better, this is no longer feasible.
 
-There is a kind of *weak verification* that we use that doesn't check
-bytecode for equivalence but does check to see if the resulting
-decompiled source is a valid Python program by running the Python
-interpreter. Because the Python language has changed so much, for best
-results you should use the same Python version in checking as was used
-in creating the bytecode.
+There verification that we use that doesn't check bytecode for
+equivalence but does check to see if the resulting decompiled source
+is a valid Python program by running the Python interpreter. Because
+the Python language has changed so much, for best results you should
+use the same Python version in checking as was used in creating the
+bytecode.
 
 There are however an interesting class of these programs that is
 readily available give stronger verification: those programs that
@@ -174,19 +175,21 @@ that era was minimal)
 
 There is some work to do on the lower end Python versions which is
 more difficult for us to handle since we don't have a Python
-interpreter for versions 1.5, 1.6, and 2.0.
+interpreter for versions 1.6, and 2.0.
 
 In the Python 3 series, Python support is is strongest around 3.4 or
 3.3 and drops off as you move further away from those versions. Python
-3.6 changes things drastically by using word codes rather than byte
-codes. As a result, the jump offset field in a jump instruction
-argument has been reduced. This makes the `EXTENDED_ARG` instructions
-are now more prevalent in jump instruction; previously they had been
-rare.  Perhaps to compensate for the additional `EXTENDED_ARG`
-instructions, additional jump optimization has been added. So in sum
-handling control flow by ad hoc means as is currently done is worse.
+3.0 is weird in that it in some ways resembles 2.6 more than it does
+3.1 or 2.7. Python 3.6 changes things drastically by using word codes
+rather than byte codes. As a result, the jump offset field in a jump
+instruction argument has been reduced. This makes the `EXTENDED_ARG`
+instructions are now more prevalent in jump instruction; previously
+they had been rare.  Perhaps to compensate for the additional
+`EXTENDED_ARG` instructions, additional jump optimization has been
+added. So in sum handling control flow by ad hoc means as is currently
+done is worse.
 
-Also, between Python 3.5, 3.6 and 3.7 there have been major changes to the
+Between Python 3.5, 3.6 and 3.7 there have been major changes to the
 `MAKE_FUNCTION` and `CALL_FUNCTION` instructions.
 
 Currently not all Python magic numbers are supported. Specifically in
@@ -212,11 +215,12 @@ There is lots to do, so please dig in and help.
 See Also
 --------
 
-* https://github.com/zrax/pycdc : supports all versions of Python and is written in C++. Support for Python 3 is a bit lacking though.
-* https://code.google.com/archive/p/unpyc3/ : supports Python 3.2 only. The above projects use a different decompiling technique than what is used here.
-* https://github.com/figment/unpyc3/ : fork of above, but supports Python 3.3 only. Includes some fixes like supporting function annotations
-* The HISTORY_ file.
+* https://github.com/zrax/pycdc : purports to support all versions of Python. It is written in C++ and is most accurate for Python versions around 2.7 and 3.3 when the code was more actively developed. Accuracy for more recent versions of Python 3 and early versions of Python are especially lacking. See its `issue tracker <https://github.com/zrax/pycdc/issues>`_ for details. Currently lightly maintained.
+* https://code.google.com/archive/p/unpyc3/ : supports Python 3.2 only. The above projects use a different decompiling technique than what is used here. Currently unmaintained.
+* https://github.com/figment/unpyc3/ : fork of above, but supports Python 3.3 only. Includes some fixes like supporting function annotations. Currently unmaintained.
+* https://github.com/wibiti/uncompyle2 : supports Python 2.7 only, but does that fairly well. There situtations where `uncompyle6` results are incorrect while `uncompyle2` results are not, but more often uncompyle6 is correct when uncompyle2 is not. Because `uncompyle6` adheres to accuracy over idiomatic Python, `uncompyle2` can produce more natural-looking code when it is correct. Currently `uncompyle2` is lightly maintained. See its issue `tracker <https://github.com/wibiti/uncompyle2/issues>`_ for more details
 * `How to report a bug <https://github.com/rocky/python-uncompyle6/blob/master/HOW-TO-REPORT-A-BUG.md>`_
+* The HISTORY_ file.
 * https://github.com/rocky/python-xdis : Cross Python version disassembler
 * https://github.com/rocky/python-xasm : Cross Python version assembler
 * https://github.com/rocky/python-uncompyle6/wiki : Wiki Documents which describe the code and aspects of it in more detail
@@ -234,3 +238,6 @@ See Also
 .. _PJOrion: http://www.koreanrandom.com/forum/topic/15280-pjorion-%D1%80%D0%B5%D0%B4%D0%B0%D0%BA%D1%82%D0%B8%D1%80%D0%BE%D0%B2%D0%B0%D0%BD%D0%B8%D0%B5-%D0%BA%D0%BE%D0%BC%D0%BF%D0%B8%D0%BB%D1%8F%D1%86%D0%B8%D1%8F-%D0%B4%D0%B5%D0%BA%D0%BE%D0%BC%D0%BF%D0%B8%D0%BB%D1%8F%D1%86%D0%B8%D1%8F-%D0%BE%D0%B1%D1%84
 .. _Deobfuscator: https://github.com/extremecoders-re/PjOrion-Deobfuscator
 .. _Py2EXE: https://en.wikipedia.org/wiki/Py2exe
+.. |Supported Python Versions| image:: https://img.shields.io/pypi/pyversions/uncompyle6.svg
+.. |Latest Version| image:: https://badge.fury.io/py/uncompyle6.svg
+		 :target: https://badge.fury.io/py/uncompyle6

__pkginfo__.py

@@ -35,6 +35,7 @@ classifiers =  ['Development Status :: 5 - Production/Stable',
                 'Programming Language :: Python :: 2.5',
                 'Programming Language :: Python :: 2.6',
                 'Programming Language :: Python :: 2.7',
+                'Programming Language :: Python :: 3.0',
                 'Programming Language :: Python :: 3.1',
                 'Programming Language :: Python :: 3.2',
                 'Programming Language :: Python :: 3.3',
@@ -56,7 +57,8 @@ entry_points       = {
     ]}
 ftp_url            = None
 install_requires   = ['spark-parser >= 1.8.5, < 1.9.0',
-                      'xdis >= 3.7.0, < 3.8.0']
+                      'xdis >= 3.8.4, < 3.9.0']
+
 license            = 'GPL3'
 mailing_list       = 'python-debugger@googlegroups.com'
 modname            = 'uncompyle6'

admin-tools/check-newer-versions.sh

@@ -16,6 +16,7 @@ if ! source ./setup-master.sh ; then
 fi
 cd ..
 for version in $PYVERSIONS; do
+    echo --- $version ---
     if ! pyenv local $version ; then
 	exit $?
     fi
@@ -23,4 +24,5 @@ for version in $PYVERSIONS; do
     if ! make check; then
 	exit $?
     fi
+    echo === $version ===
 done

admin-tools/check-older-versions.sh

@@ -15,6 +15,7 @@ fi
 
 cd ..
 for version in $PYVERSIONS; do
+    echo --- $version ---
     if ! pyenv local $version ; then
 	exit $?
     fi
@@ -22,4 +23,5 @@ for version in $PYVERSIONS; do
     if ! make check ; then
 	exit $?
     fi
+    echo === $version ===
 done

admin-tools/setup-master.sh

@@ -1,5 +1,5 @@
 #!/bin/bash
-PYTHON_VERSION=3.6.4
+PYTHON_VERSION=3.6.5
 
 # FIXME put some of the below in a common routine
 function finish {

appveyor.yml

@@ -47,7 +47,7 @@ install:
 
   # Upgrade to the latest version of pip to avoid it displaying warnings
   # about it being out of date.
-  - "pip install --disable-pip-version-check --user --upgrade pip"
+  - "%PYTHON%\\python.exe -m pip install --disable-pip-version-check --user --upgrade pip"
 
   # Install the build dependencies of the project. If some dependencies contain
   # compiled extensions and are not provided as pre-built wheel packages,

circle.yml

@@ -6,6 +6,7 @@ machine:
 
 dependencies:
     override:
+        - pip install --upgrade setuptools
         - pip install -e .
         - pip install pytest==3.2.5 hypothesis
 test:

pytest/test_single_compile.py

@@ -1,6 +1,6 @@
 from uncompyle6 import PYTHON_VERSION, deparse_code
 
-if PYTHON_VERSION >= 2.6:
+if PYTHON_VERSION > 2.6:
     def test_single_mode():
         single_expressions = (
             'i = 1',

pytest/validate.py

@@ -3,8 +3,8 @@ import os
 import difflib
 import subprocess
 import tempfile
-
 from StringIO import StringIO
+
 # uncompyle6 / xdis
 from uncompyle6 import PYTHON_VERSION, IS_PYPY, deparse_code
 # TODO : I think we can get xdis to support the dis api (python 3 version) by doing something like this there
@@ -20,8 +20,6 @@ try:
 except:
     pass
 
-
-
 def print_diff(original, uncompyled):
     """
     Try and display a pretty html line difference between the original and

requirements.txt

@@ -1,2 +1,4 @@
 # Pick up stuff from setup.py
+hypothesis==2.0.0
+pytest
 -e .

test/Makefile

@@ -1,5 +1,6 @@
 PHONY=check clean dist distclean test test-unit test-functional rmChangeLog clean_pyc nosetests \
-      check-bytecode-1.5 check-bytecode-1 check-bytecode-2 check-bytecode-3 \
+      check-bytecode-1  check-bytecode-1.4 check-bytecode-1.5 \
+      check-bytecode-2 check-bytecode-3 \
       check-bytecode-2.2 check-byteocde-2.3 check-bytecode-2.4 \
       check-short check-2.6 check-2.7 check-3.0 check-3.1 check-3.2 check-3.3 \
       check-3.4 check-3.5 check-3.6 check-3.7 check-5.6 5.6 5.8 \
@@ -21,7 +22,7 @@ COVER_DIR=../tmp/grammar-cover
 # Run short tests
 check-short:
 	@$(PYTHON) -V && PYTHON_VERSION=`$(PYTHON) -V 2>&1 | cut -d ' ' -f 2 | cut -d'.' -f1,2`; \
-	$(MAKE) check-bytecode
+	$(MAKE) check-bytecode-short
 
 # Run all tests
 check:
@@ -76,7 +77,7 @@ check-disasm:
 	$(PYTHON) dis-compare.py
 
 #: Check deparsing bytecode 1.x only
-check-bytecode-1: check-bytecode-1.5
+check-bytecode-1: check-bytecode-1.4 check-bytecode-1.5
 
 #: Check deparsing bytecode 2.x only
 check-bytecode-2:
@@ -90,14 +91,35 @@ check-bytecode-3:
        --bytecode-3.1 --bytecode-3.2 --bytecode-3.3 \
        --bytecode-3.4 --bytecode-3.5 --bytecode-3.6 --bytecode-pypy3.2
 
-#: Check deparsing bytecode that works running Python 2 and Python 3
+#: Check deparsing on selected bytecode 3.x
+check-bytecode-3-short:
+	$(PYTHON) test_pythonlib.py \
+       --bytecode-3.4 --bytecode-3.5 --bytecode-3.6
+
+#: Check deparsing bytecode on all Python 2 and Python 3 versions
 check-bytecode: check-bytecode-3
 	$(PYTHON) test_pythonlib.py \
+	--bytecode-1.3 --bytecode-1.4 --bytecode-1.5 \
+        --bytecode-2.2 --bytecode-2.3 --bytecode-2.4 \
 	--bytecode-2.1 --bytecode-2.2 --bytecode-2.3 --bytecode-2.4 \
         --bytecode-2.5 --bytecode-2.6 --bytecode-2.7 \
         --bytecode-pypy2.7
 
 
+#: Check deparsing bytecode on selected Python 2 and Python 3 versions
+check-bytecode-short: check-bytecode-3-short
+	$(PYTHON) test_pythonlib.py \
+        --bytecode-2.6 --bytecode-2.7 --bytecode-pypy2.7
+
+
+#: Check deparsing bytecode 1.3 only
+check-bytecode-1.3:
+	$(PYTHON) test_pythonlib.py --bytecode-1.3
+
+#: Check deparsing bytecode 1.4 only
+check-bytecode-1.4:
+	$(PYTHON) test_pythonlib.py --bytecode-1.4
+
 #: Check deparsing bytecode 1.5 only
 check-bytecode-1.5:
 	$(PYTHON) test_pythonlib.py --bytecode-1.5
@@ -203,6 +225,11 @@ check-bytecode-2.7:
 check-bytecode-3.0:
 	$(PYTHON) test_pythonlib.py --bytecode-3.0 --weak-verify
 
+#: Check deparsing Python 3.0
+check-bytecode-3.0:
+	$(PYTHON) test_pythonlib.py --bytecode-3.0 --weak-verify
+        # $(PYTHON) test_pythonlib.py --bytecode-3.0-run --verify-run
+
 #: Check deparsing Python 3.1
 check-bytecode-3.1:
 	$(PYTHON) test_pythonlib.py --bytecode-3.1 --weak-verify
@@ -267,7 +294,7 @@ check-3.4-ok:
 2.6:
 
 #: PyPy 5.0.x with Python 2.7 ...
-pypy-2.7 5.0 5.3:
+pypy-2.7 5.0 5.3 6.0:
 	$(PYTHON) test_pythonlib.py --bytecode-pypy2.7 --verify
 
 #: PyPy 2.4.x with Python 3.2 ...

test/simple_source/branching/02_ifelse_lambda.py

@@ -2,7 +2,8 @@
 # lambda's have to be more or less on a line
 
 f = lambda x: 1 if x<2 else 3
-f(5)
+assert f(3) == 3
+assert f(1) == 1
 
 # If that wasn't enough ...
 # Python will create dead code
@@ -10,10 +11,18 @@ f(5)
 # not to include the else expression
 
 g = lambda: 1 if True else 3
-g()
+assert g() == 1
 
 h = lambda: 1 if False else 3
-h()
+assert h() == 3
 
 # From 2.7 test_builtin
-lambda c: 'a' <= c <= 'z', 'Hello World'
+i = lambda c: 'a' <= c <= 'z', 'Hello World'
+assert i[0]('a') == True
+assert i[0]('A') == False
+
+# Issue #170. Bug is needing an "conditional_not_lambda" grammar rule
+# in addition the the "conditional_lambda" rule
+j = lambda a: False if not a else True
+assert j(True) == True
+assert j(False) == False

test/simple_source/bug14/00_unpack_list.py

@@ -0,0 +1,3 @@
+# Python 1.4 tzparse.py, but also appears in 1.5
+
+[tzname, delta] = __file__

test/simple_source/bug14/01_print.py

@@ -0,0 +1,6 @@
+# Python 1.4 aifc.py
+# Something weird about the final "print" and PRINT_NL_CONT followed by PRINT_NL
+def _readmark(self, markers):
+    if self._markers: print 'marker',
+    else: print 'markers',
+    print 'instead of', markers

test/simple_source/bug14/02_continue.py

@@ -0,0 +1,11 @@
+# Python 1.4 cgi.py
+# Bug was in "continue" detection.
+# 1.4 doesn't have lnotab and our CONTINUE detection is off.
+def parse_multipart(params, pdict):
+    while params:
+	if params.has_key('name'):
+	   params = None
+	else:
+	    continue
+
+    return None

test/simple_source/bug26/01_ifelse_listcomp.py

@@ -0,0 +1,4 @@
+# Bug from issue #171:  parsing "if x if a else y" inside a list comprehension on 2.7
+# This is RUNNABLE!
+assert [False, True, True, True, True] == [False if not a else True for a in range(5)]
+assert [True, False, False, False, False] == [False if a else True for a in range(5)]

test/simple_source/bug26/03_complex_and.py

@@ -0,0 +1,17 @@
+# From 2.6 test_datetime.py
+# Bug is in parsing (x is 0 or x is 1) and (y is 5 or y is 2)
+# correctly.
+
+# This code is RUNNABLE!
+result = []
+for y in (1, 2, 10):
+    x = cmp(1, y)
+    if (x is 0 or x is 1) and (y is 5 or y is 2):
+        expected = 10
+    elif y is 2:
+        expected = 2
+    else:
+        expected = 3
+    result.append(expected)
+
+assert result == [10, 2, 3]

test/simple_source/bug27+/03_for_try_raise.py

@@ -0,0 +1,12 @@
+# Code in 2.7 needing rule:
+#   try_except ::= SETUP_EXCEPT suite_stmts_opt POP_BLOCK except_handler
+# Generally we need a COME_FROM. But not in the situation below.
+
+for package in [1,2]:
+    try:
+        pass
+    except IndexError:
+        with __file__ as f:
+            pass
+    except:
+        raise

test/simple_source/bug27+/05_try_else.py

@@ -0,0 +1,11 @@
+# From Python 2.7 test_ziplib.py
+# Bug is distinguishing try from try/else.
+def testAFakeZlib(self):
+    try:
+        self.doTest()
+    except ImportError:
+        if self.compression != 3:
+            self.fail("expected test to not raise ImportError")
+    else:
+        if self.compression != 4:
+            self.fail("expected test to raise ImportError")

test/simple_source/bug275/01_while1.py

@@ -0,0 +1,9 @@
+# Issue #173. Bug is that 2.7.5 omits POP_BLOCK in
+# in later 2.7 grammar.
+#  while1stmt        ::= SETUP_LOOP l_stmts_opt JUMP_BACK COME_FROM
+#  while1stmt        ::= SETUP_LOOP l_stmts_opt CONTINUE COME_FROM
+# which is included in later code generation
+ms=0
+if ms==1:
+    while 1:
+        pass

test/simple_source/bug30/03_pop_top.py

@@ -0,0 +1,20 @@
+# From 3.0.1 __dummy_thread.py
+# bug was handling else:
+def interrupt_main():
+    """Set _interrupt flag to True to have start_new_thread raise
+    KeyboardInterrupt upon exiting."""
+    if _main:
+        raise KeyboardInterrupt
+    else:
+        global _interrupt
+        _interrupt = True
+
+# From 3.0.1 ast.py bug was mangling prototype
+# def parse(expr, filename='<unknown>', mode='exec'):
+
+# From 3.0.1 bisect
+def bisect_left(a, x, lo=0, hi=None):
+    while lo:
+        if a[mid] < x: lo = mid+1
+        else: hi = mid
+    return lo

test/simple_source/comprehension/05_dict_comp.py

@@ -1,4 +1,7 @@
+# Bug was in dictionary comprehension involving "if not"
 # Issue #162
+#
+# This code is RUNNABLE!
 def x(s):
     return {k: v
             for (k, v) in s

test/simple_source/looping/04_while1_while1.py

@@ -19,10 +19,15 @@ while 1:
 while 1:
     if __name__:
         while 1:
-            if y:
+            if __name__:
                 break
             raise RuntimeError
     elif __file__:
         x = 2
     else:
         raise RuntimeError
+
+# Degenerate case. Note: we can't run becase this causes an infinite loop.
+# Suggested in issue #172
+while 1:
+    pass

test/stdlib/compile-file-1x.py

@@ -0,0 +1,22 @@
+#!/usr/bin/env python
+"""byte compiles a Python 1.x program"""
+import sys
+if len(sys.argv) != 2:
+    print("Usage: compile-file.py *python-file*")
+    sys.exit(1)
+source = sys.argv[1]
+
+# assert source.endswith('.py')
+basename = source[:-3]
+
+# We do this crazy way to support Python 1.4 which
+# doesn't support version_info.
+PY_VERSION = sys.version[:3]
+
+bytecode = "%s-%s.pyc" % (basename, PY_VERSION)
+
+import py_compile
+print("# compiling %s to %s" % (source, bytecode))
+py_compile.compile(source, bytecode)
+# import os
+# os.system("../bin/uncompyle6 %s" % bytecode)

test/stdlib/compile-file.py

@@ -1,4 +1,5 @@
 #!/usr/bin/env python
+"""byte compiles a Python program after version 2.2 or so. Also see compile_file_1x.py"""
 import sys
 if len(sys.argv) != 2:
     print("Usage: compile-file.py *python-file*")

test/stdlib/runtests.sh

@@ -56,6 +56,8 @@ case $PYVERSION in
     2.6)
 	SKIP_TESTS=(
 	    [test_compile.py]=1  # Intermittent - sometimes works and sometimes doesn't
+	    [test_grammar.py]=1  # Need real flow control. "and" in side "or"
+	                         # "and" inside ifelse need to simulatenously work
 	    [test_grp.py]=1      # Long test - might work Control flow?
 	    [test_opcodes.py]=1
 	    [test_pwd.py]=1 # Long test - might work? Control flow?
@@ -73,12 +75,24 @@ case $PYVERSION in
 	    # .pyenv/versions/2.6.9/lib/python2.6/sre_parse.pyc
 	    # .pyenv/versions/2.6.9/lib/python2.6/tabnanny.pyc
 	    # .pyenv/versions/2.6.9/lib/python2.6/tarfile.pyc
+
+	    # Not getting set by bach below?
+	    [test_pprint.py]=1
+
 	)
 	if (( batch )) ; then
 	    # Fails in crontab environment?
 	    # Figure out what's up here
 	    SKIP_TESTS[test_aifc.py]=1
 	    SKIP_TESTS[test_array.py]=1
+
+	    # SyntaxError: Non-ASCII character '\xdd' in file test_base64.py on line 153, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details
+	    SKIP_TESTS[test_base64.py]=1
+
+	    # output indicates expected == output, but this fails anyway.
+	    # Maybe the underlying encoding is subtlely different so it
+	    # looks the same?
+	    SKIP_TESTS[test_pprint.py]=1
 	fi
 	;;
     2.7)
@@ -91,6 +105,7 @@ case $PYVERSION in
 	    [test_curses.py]=1  # Possibly fails on its own but not detected
 	    [test_dis.py]=1   # We change line numbers - duh!
 	    [test_doctest.py]=1 # Fails on its own
+	    [test_format.py]=1  # control flow. uncompyle2 does not have problems here
 	    [test_generators.py]=1  # control flow. uncompyle2 has problem here too
 	    [test_grammar.py]=1     # Too many stmts. Handle large stmts
 	    [test_io.py]=1 # Test takes too long to run
@@ -109,12 +124,17 @@ case $PYVERSION in
 	    [test_unicode.py]=1  # Too long to run 11 seconds
 	    [test_xpickle.py]=1 # Runs ok but takes 72 seconds
 	    [test_zipfile64.py]=1  # Runs ok but takes 204 seconds
+	    [test_zipimport.py]=1  # We can't distinguish try from try/else yet
         )
 	if (( batch )) ; then
 	    # Fails in crontab environment?
 	    # Figure out what's up here
 	    SKIP_TESTS[test_array.py]=1
 	    SKIP_TESTS[test_ast.py]=1
+	    SKIP_TESTS[test_audioop.py]=1
+
+	    # SyntaxError: Non-ASCII character '\xdd' in file test_base64.py on line 153, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details
+	    SKIP_TESTS[test_base64.py]=1
 	fi
 	;;
     3.5)
@@ -171,7 +191,7 @@ if [[ -n $1 ]] ; then
 	SKIP_TESTS=()
     fi
 else
-    files=test_a*.py
+    files=test_*.py
 fi
 
 typeset -i ALL_FILES_STARTTIME=$(date +%s)

test/test_pythonlib.py

@@ -76,7 +76,7 @@ for  vers in (2.7, 3.4, 3.5, 3.6):
     test_options[key] = (os.path.join(src_dir, pythonlib), PYOC, key, vers)
     pass
 
-for  vers in (1.5,
+for  vers in (1.3, 1.4, 1.5,
               2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7,
               3.0, 3.1, 3.2, 3.3,
               3.4, 3.5, 3.6, 3.7, 'pypy3.2', 'pypy2.7'):

uncompyle6/bin/uncompile.py

@@ -45,6 +45,7 @@ Debugging Options:
   --asm     -a  include byte-code         (disables --verify)
   --grammar -g  show matching grammar
   --tree    -t  include syntax tree       (disables --verify)
+  --tree++      add template rules to --tree when possible
 
 Extensions of generated files:
   '.pyc_dis' '.pyo_dis'   successfully decompiled (and verified if --verify)
@@ -60,18 +61,16 @@ from uncompyle6.version import VERSION
 
 def usage():
     print("""usage:
-   %s [--verify | --weak-verify ] [--asm] [--tree] [--grammar] [-o <path>] FILE|DIR...
+    %s [--verify | --weak-verify ] [--asm] [--tree[+]] [--grammar] [-o <path>] FILE|DIR...
    %s [--help | -h | --version | -V]
 """  % (program, program))
     sys.exit(1)
 
 
 def main_bin():
-    if not (sys.version_info[0:2] in ((2, 4), (2, 5), (2, 6), (2, 7),
-                                      (3, 2), (3, 3),
-                                      (3, 4), (3, 5), (3, 6), (3, 7))):
-        sys.stderr.write('Error: %s requires Python 2.4 2.5 2.6, 2.7, '
-                         '3.2, 3.3, 3.4, 3.5, or 3.6' % program)
+    if not (sys.version_info[0:2] in ((2, 4), (2, 5), (2, 6), (2, 7))):
+        sys.stderr.write('Error: this branch of %s requires Python 2.4, 2.5, 2.6 or 2.7'
+                         % program)
         sys.exit(-1)
 
     do_verify = recurse_dirs = False
@@ -84,8 +83,10 @@ def main_bin():
 
     try:
         opts, files = getopt.getopt(sys.argv[1:], 'hagtdrVo:c:p:',
-                                    'help asm grammar linemaps recurse timestamp tree '
-                                    'fragments verify verify-run version weak-verify '
+                                    'help asm grammar linemaps recurse '
+                                    'timestamp tree tree+ '
+                                    'fragments verify verify-run version '
+                                    'weak-verify '
                                     'showgrammar'.split(' '))
     except getopt.GetoptError(e):
         sys.stderr.write('%s: %s\n' % (os.path.basename(sys.argv[0]), e))
@@ -115,6 +116,9 @@ def main_bin():
         elif opt in ('--tree', '-t'):
             options['showast'] = True
             options['do_verify'] = None
+        elif opt in ('--tree+',):
+            options['showast'] = 'Full'
+            options['do_verify'] = None
         elif opt in ('--grammar', '-g'):
             options['showgrammar'] = True
         elif opt == '-o':

uncompyle6/disas.py

@@ -73,6 +73,19 @@ def disco_loop(disasm, queue, real_out):
             pass
         pass
 
+def disassemble_fp(fp, outstream=None):
+    """
+    disassemble Python byte-code from an open file
+    """
+    (version, timestamp, magic_int, co, is_pypy,
+     source_size) = load_from_fp(fp)
+    if type(co) == list:
+        for con in co:
+            disco(version, con, outstream)
+    else:
+        disco(version, co, outstream, is_pypy=is_pypy)
+    co = None
+
 def disassemble_file(filename, outstream=None):
     """
     disassemble Python byte-code file (.pyc)

uncompyle6/parser.py

@@ -618,18 +618,31 @@ def get_python_parser(
     # a lazy way of doing the import?
 
     if version < 3.0:
-        if version == 1.5:
-            import uncompyle6.parsers.parse15 as parse15
-            if compile_mode == 'exec':
-                p = parse15.Python15Parser(debug_parser)
-            else:
-                p = parse15.Python15ParserSingle(debug_parser)
-        elif version == 2.1:
-            import uncompyle6.parsers.parse21 as parse21
-            if compile_mode == 'exec':
-                p = parse21.Python21Parser(debug_parser)
-            else:
-                p = parse21.Python21ParserSingle(debug_parser)
+        if version < 2.2:
+            if version == 1.3:
+                import uncompyle6.parsers.parse13 as parse13
+                if compile_mode == 'exec':
+                    p = parse13.Python14Parser(debug_parser)
+                else:
+                    p = parse13.Python14ParserSingle(debug_parser)
+            elif version == 1.4:
+                import uncompyle6.parsers.parse14 as parse14
+                if compile_mode == 'exec':
+                    p = parse14.Python14Parser(debug_parser)
+                else:
+                    p = parse14.Python14ParserSingle(debug_parser)
+            elif version == 1.5:
+                import uncompyle6.parsers.parse15 as parse15
+                if compile_mode == 'exec':
+                    p = parse15.Python15Parser(debug_parser)
+                else:
+                    p = parse15.Python15ParserSingle(debug_parser)
+            elif version == 2.1:
+                import uncompyle6.parsers.parse21 as parse21
+                if compile_mode == 'exec':
+                    p = parse21.Python21Parser(debug_parser)
+                else:
+                    p = parse21.Python21ParserSingle(debug_parser)
         elif version == 2.2:
             import uncompyle6.parsers.parse22 as parse22
             if compile_mode == 'exec':

uncompyle6/parsers/parse13.py

@@ -0,0 +1,49 @@
+#  Copyright (c) 2018 Rocky Bernstein
+
+from spark_parser import DEFAULT_DEBUG as PARSER_DEFAULT_DEBUG
+from uncompyle6.parser import PythonParserSingle
+from uncompyle6.parsers.parse14 import Python14Parser
+
+class Python13Parser(Python14Parser):
+
+    def p_misc13(self, args):
+        """
+        # Nothing here yet, but will need to add LOAD_GLOBALS
+        """
+
+    def __init__(self, debug_parser=PARSER_DEFAULT_DEBUG):
+        super(Python13Parser, self).__init__(debug_parser)
+        self.customized = {}
+
+    # def customize_grammar_rules(self, tokens, customize):
+    #     super(Python13Parser, self).customize_grammar_rules(tokens, customize)
+    #     self.remove_rules("""
+    #     whileelsestmt ::= SETUP_LOOP testexpr l_stmts_opt
+    #                       jb_pop
+    #                       POP_BLOCK else_suitel COME_FROM
+    #     """)
+    #     self.check_reduce['doc_junk'] = 'tokens'
+
+
+    # def reduce_is_invalid(self, rule, ast, tokens, first, last):
+    #     invalid = super(Python14Parser,
+    #                     self).reduce_is_invalid(rule, ast,
+    #                                             tokens, first, last)
+    #     if invalid or tokens is None:
+    #         return invalid
+    #     if rule[0] == 'doc_junk':
+    #         return not isinstance(tokens[first].pattr, str)
+
+
+
+class Python13ParserSingle(Python13Parser, PythonParserSingle):
+    pass
+
+if __name__ == '__main__':
+    # Check grammar
+    p = Python13Parser()
+    p.check_grammar()
+    p.dump_grammar()
+
+# local variables:
+# tab-width: 4

uncompyle6/parsers/parse14.py

@@ -0,0 +1,69 @@
+#  Copyright (c) 2018 Rocky Bernstein
+
+from spark_parser import DEFAULT_DEBUG as PARSER_DEFAULT_DEBUG
+from uncompyle6.parser import PythonParserSingle
+from uncompyle6.parsers.parse15 import Python15Parser
+
+class Python14Parser(Python15Parser):
+
+    def p_misc14(self, args):
+        """
+        # Not much here yet, but will probably need to add UNARY_CALL, BINARY_CALL,
+        # RAISE_EXCEPTION, BUILD_FUNCTION, UNPACK_ARG, UNPACK_VARARG, LOAD_LOCAL,
+        # SET_FUNC_ARGS, and RESERVE_FAST
+
+        # Not strictly needed, but tidies up output
+        stmt     ::= doc_junk
+        doc_junk ::= LOAD_CONST POP_TOP
+
+        # Not sure why later Python's omit the COME_FROM
+        jb_pop14  ::= JUMP_BACK COME_FROM POP_TOP
+
+        whileelsestmt ::= SETUP_LOOP testexpr l_stmts_opt
+                          jb_pop14
+                          POP_BLOCK else_suitel COME_FROM
+
+        print_items_nl_stmt ::= expr PRINT_ITEM_CONT print_items_opt PRINT_NEWLINE_CONT
+
+
+        # 1.4 doesn't have linenotab, and although this shouldn't
+        # be a show stopper, our CONTINUE detection is off here.
+        continue ::= JUMP_BACK
+        """
+
+    def __init__(self, debug_parser=PARSER_DEFAULT_DEBUG):
+        super(Python14Parser, self).__init__(debug_parser)
+        self.customized = {}
+
+    def customize_grammar_rules(self, tokens, customize):
+        super(Python14Parser, self).customize_grammar_rules(tokens, customize)
+        self.remove_rules("""
+        whileelsestmt ::= SETUP_LOOP testexpr l_stmts_opt
+                          jb_pop
+                          POP_BLOCK else_suitel COME_FROM
+        """)
+        self.check_reduce['doc_junk'] = 'tokens'
+
+
+    def reduce_is_invalid(self, rule, ast, tokens, first, last):
+        invalid = super(Python14Parser,
+                        self).reduce_is_invalid(rule, ast,
+                                                tokens, first, last)
+        if invalid or tokens is None:
+            return invalid
+        if rule[0] == 'doc_junk':
+            return not isinstance(tokens[first].pattr, str)
+
+
+
+class Python14ParserSingle(Python14Parser, PythonParserSingle):
+    pass
+
+if __name__ == '__main__':
+    # Check grammar
+    p = Python14Parser()
+    p.check_grammar()
+    p.dump_grammar()
+
+# local variables:
+# tab-width: 4

uncompyle6/parsers/parse15.py

@@ -2,7 +2,7 @@
 #  Copyright (c) 2000-2002 by hartmut Goebel <hartmut@goebel.noris.de>
 
 from spark_parser import DEFAULT_DEBUG as PARSER_DEFAULT_DEBUG
-from uncompyle6.parser import PythonParserSingle
+from uncompyle6.parser import PythonParserSingle, nop_func
 from uncompyle6.parsers.parse21 import Python21Parser
 
 class Python15Parser(Python21Parser):
@@ -23,7 +23,18 @@ class Python15Parser(Python21Parser):
         importlist  ::= IMPORT_FROM
         """
 
-class Python15ParserSingle(Python21Parser, PythonParserSingle):
+    def customize_grammar_rules(self, tokens, customize):
+        super(Python15Parser, self).customize_grammar_rules(tokens, customize)
+        for i, token in enumerate(tokens):
+            opname = token.kind
+            opname_base = opname[:opname.rfind('_')]
+
+            if opname_base == 'UNPACK_LIST':
+                self.addRule("store ::= unpack_list", nop_func)
+
+
+
+class Python15ParserSingle(Python15Parser, PythonParserSingle):
     pass
 
 if __name__ == '__main__':