Get ready for release 2.3.3

parse{2,3,r}.py: DRY Python expressions between Python 2 and 3
pysource.py, fragment.py, parser.py: handle long lists by grouping in chunks of 32
and 256
bin/uncompyle6: count %s properly

ChangeLog

@@ -1,6 +1,71 @@
+2016-05-03  rocky <rocky@gnu.org>
+
+	* uncompyle6/version.py: Get ready for release 2.3.3
+
+2016-05-02  rocky <rocky@gnu.org>
+
+	* README.rst: Be more explicit that we need Python 2.6 or later
+
+2016-05-02  rocky <rocky@gnu.org>
+
+	* : commit feec241da88107b97bbdfbabeb3ae7131a7aa923 Author: rocky
+	<rocky@gnu.org> Date:   Mon May 2 21:20:17 2016 -0400
+
+2016-05-02  rocky <rb@dustyfeet.com>
+
+	* README.rst: Note relation to other uncompyle forks Add some other minor corrections and additions as well.
+
+2016-05-02  rocky <rb@dustyfeet.com>
+
+	* uncompyle6/__init__.py: Trivial spacing change
+
+2016-05-02  rocky <rocky@gnu.org>
+
+	* ChangeLog, NEWS, __pkginfo__.py, bin/pydisassemble,
+	bin/uncompyle6, setup.py, uncompyle6/__init__.py,
+	uncompyle6/version.py: Add -V | --version and simplfy changing it
+
+2016-05-01  rocky <rocky@gnu.org>
+
+	* uncompyle6/__init__.py: Expose uncompyle_file
+
+2016-05-01  rocky <rocky@gnu.org>
+
+	* test/Makefile, uncompyle6/semantics/pysource.py: Bug
+
+2016-05-01  rocky <rocky@gnu.org>
+
+	* test/Makefile, test/simple_source/expression/05_const_map.py: Add
+	test for last fix.  Drop 2.5 test until we figure out what's wrong
+
+2016-05-01  rocky <rocky@gnu.org>
+
+	* uncompyle6/parsers/parse3.py, uncompyle6/scanners/scanner35.py,
+	uncompyle6/semantics/pysource.py: Bug in 3.5 constant map parsing
+
+2016-05-01  rocky <rocky@gnu.org>
+
+	* uncompyle6/__init__.py: Export module load and fns load_file,
+	load_module
+
+2016-05-01  rocky <rocky@gnu.org>
+
+	* __pkginfo__.py, setup.py, uncompyle6/marsh.py: License is MIT marsh.py: remove unused import
+
+2016-05-01  rocky <rocky@gnu.org>
+
+	* uncompyle6/parsers/parse3.py: Forgot to define Python3ParserSingle
+
+2016-05-01  rocky <rocky@gnu.org>
+
+	* uncompyle6/parser.py, uncompyle6/parsers/parse2.py,
+	uncompyle6/parsers/parse3.py: Start to DRY Python2 and Python3
+	grammars Separate out 3.2, and 3.5+ specific grammar code
+
 2016-04-30  rocky <rocky@gnu.org>
 
-	* README.rst, __pkginfo__.py: Get ready for release 2.3.0
+	* ChangeLog, NEWS, README.rst, __pkginfo__.py: Get ready for release
+	2.3.1
 
 2016-04-30  rocky <rocky@gnu.org>
 

NEWS

@@ -1,4 +1,19 @@
-uncompyle6 2.2.1 2016-04-30
+uncompyle6 2.3.3 2016-05-3
+
+- Fix bug in running uncompyle6 script on Python 3
+- Speed up performance on deparsing long lists by grouping in chunks of 32 and 256 items
+- DRY Python expressions between Python 2 and 3
+
+uncompyle6 2.3.2 2016-05-1
+
+- Add --version option standalone scripts
+- Correct License information in package
+- expose fns uncompyle_file, load_file, and load_module
+- Start to DRY Python2 and Python3 grammars Separate out 3.2, and 3.5+
+  specific grammar code
+- Fix bug in 3.5+ constant map parsing
+
+uncompyle6 2.3.0, 2.3.1 2016-04-30
 
 - Require spark_parser  >= 1.1.0
 

README.rst

@@ -4,7 +4,7 @@ uncompyle6
 ==========
 
 A native Python bytecode Disassembler, Decompiler, Fragment Decompiler
-and bytecode library
+and bytecode library. Follows in the tradition of decompyle, uncompyle, and uncompyle2.
 
 
 Introduction
@@ -12,12 +12,17 @@ Introduction
 
 *uncompyle6* translates Python bytecode back into equivalent Python
 source code. It accepts bytecodes from Python version 2.5 to 3.4 or
-so and has been tested on Python running versions 2.6, 2.7, 3.3,
-3.4 and 3.5.
+so. The code requires Python 2.6 or later and has been tested on Python
+running versions 2.6, 2.7, 3.3, 3.4 and 3.5.
 
 Why this?
 ---------
 
+There were a number of decompyle, uncompile, uncompyle2, uncompyle3
+forks around. All of them come basically from the same code base, and
+almost all of them not maintained very well. This code pulls these together
+and addresses a number of open issues in those.
+
 What makes this different from other CPython bytecode decompilers?  Its
 ability to deparse just fragments and give source-code information
 around a given bytecode offset.
@@ -81,20 +86,22 @@ Run
      ./bin/uncompyle6 -h
      ./bin/pydisassemble -h
 
-for usage help
+for usage help.
 
 
 Known Bugs/Restrictions
 -----------------------
 
 Python 2 deparsing is probably as solid as the various versions of
-uncompyle2.  Python 3 deparsing is okay but not as solid.
+uncompyle2.  Python 3 deparsing is okay but not as solid. Python 3.5 is missing some of new opcodes added, but still often works.
 
 See Also
 --------
 
-* https://github.com/zrax/pycdc
-* https://code.google.com/p/unpyc3/
+* https://github.com/zrax/pycdc : supports all versions of Python and is written in C++
+* https://code.google.com/archive/p/unpyc3/ : supports Python 3.2 only
+
+The above projects use a different decompiling technique what is used here.
 
 The HISTORY file.
 

__pkginfo__.py

@@ -34,7 +34,7 @@ ftp_url            = None
 install_requires   = ['python-spark >= 1.1.0']
 license            = 'GPL'
 
-# license            = 'BSDish'
+license            = 'MIT'
 mailing_list       = 'python-debugger@googlegroups.com'
 modname            = 'uncompyle6'
 packages           = ['uncompyle6', 'uncompyle6.opcodes', 'uncompyle6.semantics', 'uncompyle6.scanners', 'uncompyle6.parsers']
@@ -50,7 +50,6 @@ def get_srcdir():
     return os.path.realpath(filename)
 
 ns = {}
-version            = '2.3.1'
 web                = 'https://github.com/rocky/python-uncompyle6/'
 
 # tracebacks in zip files are funky and not debuggable

bin/pydisassemble

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 # Mode: -*- python -*-
 #
-# Copyright (c) 2015 by Rocky Bernstein <rb@dustyfeet.com>
+# Copyright (c) 2015-2016 by Rocky Bernstein <rb@dustyfeet.com>
 #
 from __future__ import print_function
 import sys, os, getopt
@@ -9,11 +9,13 @@ import sys, os, getopt
 program =  os.path.basename(__file__)
 
 __doc__ = """
-Usage: %s [OPTIONS]... FILE
+Usage:
+  %s [OPTIONS]... FILE
+  %s [--help | -h | -V | --version]
 
 Examples:
-  %s      foo.pyc
-  %s      foo.py
+  %s foo.pyc
+  %s foo.py
   %s -o foo.pydis foo.pyc
   %s -o /tmp foo.pyc
 
@@ -24,7 +26,7 @@ Options:
                 <path>
   --help        show this message
 
-""" % ((program,) * 5)
+""" % ((program,) * 6)
 
 
 Usage_short = \
@@ -32,6 +34,7 @@ Usage_short = \
 
 from uncompyle6 import check_python_version
 from uncompyle6.disas import disassemble_files
+from uncompyle6.version import VERSION
 
 check_python_version(program)
 
@@ -40,7 +43,7 @@ out_base = None
 
 
 try:
-    opts, files = getopt.getopt(sys.argv[1:], 'ho:', ['help'])
+    opts, files = getopt.getopt(sys.argv[1:], 'hVo:', ['help', 'version'])
 except getopt.GetoptError as e:
     print('%s: %s' % (os.path.basename(sys.argv[0]), e),  file=sys.stderr)
     sys.exit(-1)
@@ -48,6 +51,9 @@ except getopt.GetoptError as e:
 for opt, val in opts:
     if opt in ('-h', '--help'):
         print(__doc__)
+        sys.exit(1)
+    elif opt in ('-V', '--version'):
+        print("%s %s" % (program, VERSION))
         sys.exit(0)
     elif opt == '-o':
         outfile = val

bin/uncompyle6

@@ -1,16 +1,23 @@
 #!/usr/bin/env python
 # Mode: -*- python -*-
 #
+# Copyright (c) 2015-2016 by Rocky Bernstein
 # Copyright (c) 2000-2002 by hartmut Goebel <h.goebel@crazy-compilers.com>
-# Copyright (c) 2015 by Rocky Bernstein
+#
+from __future__ import print_function
+import sys, os, getopt, time
 
-"""
-Usage: uncompyle6 [OPTIONS]... [ FILE | DIR]...
+program =  os.path.basename(__file__)
+
+__doc__ = """
+Usage:
+  %s [OPTIONS]... [ FILE | DIR]...
+  %s [--help | -h | --V | --version]
 
 Examples:
-  uncompyle6      foo.pyc bar.pyc       # decompile foo.pyc, bar.pyc to stdout
-  uncompyle6 -o . foo.pyc bar.pyc       # decompile to ./foo.pyc_dis and ./bar.pyc_dis
-  uncompyle6 -o /tmp /usr/lib/python1.5 # decompile whole library
+  %s      foo.pyc bar.pyc       # decompile foo.pyc, bar.pyc to stdout
+  %s -o . foo.pyc bar.pyc       # decompile to ./foo.pyc_dis and ./bar.pyc_dis
+  %s -o /tmp /usr/lib/python1.5 # decompile whole library
 
 Options:
   -o <path>     output decompiled files to this path:
@@ -34,26 +41,25 @@ Options:
 Debugging Options:
   --asm     -a  include byte-code         (disables --verify)
   --grammar -g  show matching grammar
-  --treee   -t  include syntax tree       (disables --verify)
+  --tree    -t  include syntax tree       (disables --verify)
 
 Extensions of generated files:
   '.pyc_dis' '.pyo_dis'   successfully decompiled (and verified if --verify)
     + '_unverified'       successfully decompile but --verify failed
     + '_failed'           decompile failed (contact author for enhancement)
-"""
-
-from __future__ import print_function
-import sys, os, getopt, time
+""" % ((program,) * 5)
 
 program =  os.path.basename(__file__)
 
 from uncompyle6 import verify, check_python_version
 from uncompyle6.main import main, status_msg
+from uncompyle6.version import VERSION
 
 def usage():
     print("""usage:
-   %s [--help] [--verify] [--asm] [--tree] [--grammar] [-o <path>] FILE|DIR...
-"""  % program)
+   %s [--verify] [--asm] [--tree] [--grammar] [-o <path>] FILE|DIR...
+   %s [--help | -h | --version | -V]
+"""  % (program, program))
     sys.exit(1)
 
 
@@ -68,8 +74,8 @@ timestamp = False
 timestampfmt = "# %Y.%m.%d %H:%M:%S %Z"
 
 try:
-    opts, files = getopt.getopt(sys.argv[1:], 'hagtdro:c:p:',
-                                'help asm grammar recurse timestamp tree verify '
+    opts, files = getopt.getopt(sys.argv[1:], 'hagtdrVo:c:p:',
+                                'help asm grammar recurse timestamp tree verify version '
                                 'showgrammar'.split(' '))
 except getopt.GetoptError as e:
     print('%s: %s' % (os.path.basename(sys.argv[0]), e),  file=sys.stderr)
@@ -80,6 +86,9 @@ for opt, val in opts:
     if opt in ('-h', '--help'):
         print(__doc__)
         sys.exit(0)
+    elif opt in ('-V', '--version'):
+        print("%s %s" % (program, VERSION))
+        sys.exit(0)
     elif opt == '--verify':
         options['do_verify'] = True
     elif opt in ('--asm', '-a'):

setup.py

@@ -2,29 +2,24 @@
 
 """Setup script for the 'uncompyle6' distribution."""
 
-# Get the package information used in setup().
-# from __pkginfo__ import \
-#     author,           author_email,       classifiers,                    \
-#     install_requires, license,            long_description,               \
-#     modname,          packages,           py_modules,   \
-#     short_desc,       version,            web,              zip_safe
-
 from __pkginfo__ import \
     author,           author_email,                                  \
-    long_description,                                                \
+    license,          long_description,                              \
     modname,          packages,           py_modules,       scripts, \
-    short_desc,       version,            web,              zip_safe
+    short_desc,       web,                zip_safe
 
 __import__('pkg_resources')
 from setuptools import setup
 
+exec(open('uncompyle6/version.py').read())
+
 setup(
        author             = author,
        author_email       = author_email,
        # classifiers        = classifiers,
        description        = short_desc,
        # install_requires   = install_requires,
-       # license            = license,
+       license            = license,
        long_description   = long_description,
        py_modules         = py_modules,
        name               = modname,
@@ -33,5 +28,5 @@ setup(
        url                = web,
        setup_requires     = ['nose>=1.0'],
        scripts            = scripts,
-       version            = version,
+       version            = VERSION,
        zip_safe           = zip_safe)

test/simple_source/expression/05_const_map.py

@@ -0,0 +1,7 @@
+# Addresses a bug in the way Python 3.5+ handles
+# creation of map constants
+opts = {'highlight': True,
+            'start_line': -1,
+            'end_line': None
+            }
+print(opts)

test/simple_source/expression/05_long_list.py

@@ -0,0 +1,3 @@
+# Long lists pose a slowdown in uncompiling.
+x = [1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1]
+print(x)

uncompyle6/__init__.py

@@ -30,13 +30,15 @@ from __future__ import print_function
 
 import sys
 
+__docformat__ = 'restructuredtext'
+
 PYTHON3 = (sys.version_info >= (3, 0))
 
 # We do this crazy way to support Python 2.6 which
 # doesn't support version_major, and has a bug in
 # floating point so we can't divide 26 by 10 and get
 # 2.6
-PYTHON_VERSION = sys.version_info[0]+ (sys.version_info[1] / 10.0)
+PYTHON_VERSION = sys.version_info[0] + (sys.version_info[1] / 10.0)
 PYTHON_VERSION_STR  = "%s.%s" % (sys.version_info[0], sys.version_info[1])
 
 sys.setrecursionlimit(5000)
@@ -50,6 +52,11 @@ def check_python_version(program):
 
 import uncompyle6.semantics.pysource
 import uncompyle6.semantics.fragments
+import uncompyle6.load
+
+# Export some functions
+from uncompyle6.load import load_module, load_file
+from uncompyle6.main import uncompyle_file
 
 # Conventience functions so you can say:
 # from uncompyle6 import deparse_code

uncompyle6/main.py

@@ -118,14 +118,14 @@ def main(in_base, out_base, files, codes, outfile=None,
                 os.remove(outfile)
             sys.stderr.write("\nLast file: %s   " % (infile))
             raise
-        except:
-            failed_files += 1
-            if outfile:
-                outstream.close()
-                os.rename(outfile, outfile + '_failed')
-            else:
-                sys.stderr.write("\n# %s" % sys.exc_info()[1])
-                sys.stderr.write("\n# Can't uncompile %s\n" % infile)
+        # except:
+        #     failed_files += 1
+        #     if outfile:
+        #         outstream.close()
+        #         os.rename(outfile, outfile + '_failed')
+        #     else:
+        #         sys.stderr.write("\n# %s" % sys.exc_info()[1])
+        #         sys.stderr.write("\n# Can't uncompile %s\n" % infile)
         else: # uncompile successful
             if outfile:
                 outstream.close()

uncompyle6/marsh.py

@@ -18,7 +18,6 @@ from __future__ import print_function
 import sys, types
 from struct import unpack
 
-import uncompyle6.scanners.scanner3 as scan3
 from uncompyle6.magics import PYTHON_MAGIC_INT
 from uncompyle6.code import Code3
 

uncompyle6/parser.py

@@ -192,6 +192,199 @@ class PythonParser(GenericASTBuilder):
         load_attrs ::= load_attrs LOAD_ATTR
         '''
 
+    def p_list_comprehension(self, args):
+        """
+        expr ::= list_compr
+        list_compr ::= BUILD_LIST_0 list_iter
+
+        list_iter ::= list_for
+        list_iter ::= list_if
+        list_iter ::= list_if_not
+        list_iter ::= lc_body
+
+        _come_from ::= COME_FROM
+        _come_from ::=
+
+        list_if ::= expr jmp_false list_iter
+        list_if_not ::= expr jmp_true list_iter
+
+        lc_body ::= expr LIST_APPEND
+        """
+
+    def p_setcomp(self, args):
+        """
+        expr ::= setcomp
+
+        setcomp ::= LOAD_SETCOMP MAKE_FUNCTION_0 expr GET_ITER CALL_FUNCTION_1
+
+        stmt ::= setcomp_func
+
+        setcomp_func ::= BUILD_SET_0 LOAD_FAST FOR_ITER designator comp_iter
+                JUMP_BACK RETURN_VALUE RETURN_LAST
+
+        comp_iter ::= comp_if
+        comp_iter ::= comp_ifnot
+        comp_iter ::= comp_for
+        comp_iter ::= comp_body
+        comp_body ::= set_comp_body
+        comp_body ::= gen_comp_body
+        comp_body ::= dict_comp_body
+        set_comp_body ::= expr SET_ADD
+        gen_comp_body ::= expr YIELD_VALUE POP_TOP
+        dict_comp_body ::= expr expr MAP_ADD
+
+        comp_if ::= expr jmp_false comp_iter
+        comp_ifnot ::= expr jmp_true comp_iter
+        """
+
+    def p_expr(self, args):
+        '''
+        expr ::= _mklambda
+        expr ::= SET_LINENO
+        expr ::= LOAD_FAST
+        expr ::= LOAD_NAME
+        expr ::= LOAD_CONST
+        expr ::= LOAD_GLOBAL
+        expr ::= LOAD_DEREF
+        expr ::= load_attr
+        expr ::= binary_expr
+        expr ::= binary_expr_na
+        expr ::= build_list
+        expr ::= cmp
+        expr ::= mapexpr
+        expr ::= and
+        expr ::= and2
+        expr ::= or
+        expr ::= unary_expr
+        expr ::= call_function
+        expr ::= unary_not
+        expr ::= unary_convert
+        expr ::= binary_subscr
+        expr ::= binary_subscr2
+        expr ::= load_attr
+        expr ::= get_iter
+        expr ::= slice0
+        expr ::= slice1
+        expr ::= slice2
+        expr ::= slice3
+        expr ::= buildslice2
+        expr ::= buildslice3
+        expr ::= yield
+
+        binary_expr ::= expr expr binary_op
+        binary_op ::= BINARY_ADD
+        binary_op ::= BINARY_MULTIPLY
+        binary_op ::= BINARY_AND
+        binary_op ::= BINARY_OR
+        binary_op ::= BINARY_XOR
+        binary_op ::= BINARY_SUBTRACT
+        binary_op ::= BINARY_DIVIDE
+        binary_op ::= BINARY_TRUE_DIVIDE
+        binary_op ::= BINARY_FLOOR_DIVIDE
+        binary_op ::= BINARY_MODULO
+        binary_op ::= BINARY_LSHIFT
+        binary_op ::= BINARY_RSHIFT
+        binary_op ::= BINARY_POWER
+
+        unary_expr ::= expr unary_op
+        unary_op ::= UNARY_POSITIVE
+        unary_op ::= UNARY_NEGATIVE
+        unary_op ::= UNARY_INVERT
+
+        unary_not ::= expr UNARY_NOT
+        unary_convert ::= expr UNARY_CONVERT
+
+        binary_subscr ::= expr expr BINARY_SUBSCR
+        binary_subscr2 ::= expr expr DUP_TOPX_2 BINARY_SUBSCR
+
+        load_attr ::= expr LOAD_ATTR
+        get_iter ::= expr GET_ITER
+        slice0 ::= expr SLICE+0
+        slice0 ::= expr DUP_TOP SLICE+0
+        slice1 ::= expr expr SLICE+1
+        slice1 ::= expr expr DUP_TOPX_2 SLICE+1
+        slice2 ::= expr expr SLICE+2
+        slice2 ::= expr expr DUP_TOPX_2 SLICE+2
+        slice3 ::= expr expr expr SLICE+3
+        slice3 ::= expr expr expr DUP_TOPX_3 SLICE+3
+        buildslice3 ::= expr expr expr BUILD_SLICE_3
+        buildslice2 ::= expr expr BUILD_SLICE_2
+
+        yield ::= expr YIELD_VALUE
+
+        _mklambda ::= load_closure mklambda
+        _mklambda ::= mklambda
+
+        or   ::= expr JUMP_IF_TRUE_OR_POP expr COME_FROM
+        or   ::= expr jmp_true expr _come_from
+        and  ::= expr jmp_false expr _come_from
+        and  ::= expr JUMP_IF_FALSE_OR_POP expr COME_FROM
+        and2 ::= _jump jmp_false COME_FROM expr COME_FROM
+
+        expr ::= conditional
+        conditional ::= expr jmp_false expr JUMP_FORWARD expr COME_FROM
+        conditional ::= expr jmp_false expr JUMP_ABSOLUTE expr
+        expr ::= conditionalnot
+        conditionalnot ::= expr jmp_true expr _jump expr COME_FROM
+
+        ret_expr ::= expr
+        ret_expr ::= ret_and
+        ret_expr ::= ret_or
+
+        ret_expr_or_cond ::= ret_expr
+        ret_expr_or_cond ::= ret_cond
+        ret_expr_or_cond ::= ret_cond_not
+
+        ret_and  ::= expr JUMP_IF_FALSE_OR_POP ret_expr_or_cond COME_FROM
+        ret_or   ::= expr JUMP_IF_TRUE_OR_POP ret_expr_or_cond COME_FROM
+        ret_cond ::= expr POP_JUMP_IF_FALSE expr RETURN_END_IF ret_expr_or_cond
+        ret_cond_not ::= expr POP_JUMP_IF_TRUE expr RETURN_END_IF ret_expr_or_cond
+
+        stmt ::= return_lambda
+        stmt ::= conditional_lambda
+
+        return_lambda ::= ret_expr RETURN_VALUE LAMBDA_MARKER
+        conditional_lambda ::= expr jmp_false return_if_stmt return_stmt LAMBDA_MARKER
+
+        cmp ::= cmp_list
+        cmp ::= compare
+        compare ::= expr expr COMPARE_OP
+        cmp_list ::= expr cmp_list1 ROT_TWO POP_TOP
+                _come_from
+        cmp_list1 ::= expr DUP_TOP ROT_THREE
+                COMPARE_OP JUMP_IF_FALSE_OR_POP
+                cmp_list1 COME_FROM
+        cmp_list1 ::= expr DUP_TOP ROT_THREE
+                COMPARE_OP jmp_false
+                cmp_list1 _come_from
+        cmp_list1 ::= expr DUP_TOP ROT_THREE
+                COMPARE_OP JUMP_IF_FALSE_OR_POP
+                cmp_list2 COME_FROM
+        cmp_list1 ::= expr DUP_TOP ROT_THREE
+                COMPARE_OP jmp_false
+                cmp_list2 _come_from
+        cmp_list2 ::= expr COMPARE_OP JUMP_FORWARD
+        cmp_list2 ::= expr COMPARE_OP RETURN_VALUE
+        mapexpr ::= BUILD_MAP kvlist
+
+        kvlist ::= kvlist kv
+        kvlist ::= kvlist kv2
+        kvlist ::= kvlist kv3
+        kvlist ::=
+
+        kv ::= DUP_TOP expr ROT_TWO expr STORE_SUBSCR
+        kv2 ::= DUP_TOP expr expr ROT_THREE STORE_SUBSCR
+        kv3 ::= expr expr STORE_MAP
+
+        exprlist ::= exprlist expr
+        exprlist ::= expr
+
+        nullexprlist ::=
+
+        expr32 ::= expr expr expr expr expr expr expr expr expr expr expr expr expr expr expr expr expr expr expr expr expr expr expr expr expr expr expr expr expr expr expr expr
+        expr1024 ::= expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32 expr32
+        '''
+
 
 def parse(p, tokens, customize):
     p.add_custom_rules(tokens, customize)
@@ -201,12 +394,11 @@ def parse(p, tokens, customize):
 
 
 def get_python_parser(version, debug_parser, compile_mode='exec'):
-    """
-    Returns parser object for Python version 2 or 3
-    depending on the parameter passed.  *compile_mode*
-    is either 'exec', 'eval', or 'single'. See
-    https://docs.python.org/3.6/library/functions.html#compile for an explanation
-    of the different modes.
+    """Returns parser object for Python version 2 or 3, 3.2, 3.5on,
+    etc., depending on the parameters passed.  *compile_mode* is either
+    'exec', 'eval', or 'single'. See
+    https://docs.python.org/3.6/library/functions.html#compile for an
+    explanation of the different modes.
     """
 
     if version < 3.0:
@@ -217,13 +409,34 @@ def get_python_parser(version, debug_parser, compile_mode='exec'):
             p = parse2.Python2ParserSingle(debug_parser)
     else:
         import uncompyle6.parsers.parse3 as parse3
-        if compile_mode == 'exec':
-            p = parse3.Python3Parser(debug_parser)
+        if version == 3.2:
+            if compile_mode == 'exec':
+                p = parse3.Python32Parser(debug_parser)
+            else:
+                p = parse3.Python32ParserSingle(debug_parser)
+        elif version >= 3.5:
+            if compile_mode == 'exec':
+                p = parse3.Python35onParser(debug_parser)
+            else:
+                p = parse3.Python35onParserSingle(debug_parser)
         else:
-            p = parse3.Python3ParserSingle(debug_parser)
+            if compile_mode == 'exec':
+                p = parse3.Python3Parser(debug_parser)
+            else:
+                p = parse3.Python3ParserSingle(debug_parser)
     p.version = version
     return p
 
+class PythonParserSingle(PythonParser):
+    def p_call_stmt(self, args):
+        '''
+        # single-mode compilation. Eval-mode interactive compilation
+        # drops the last rule.
+
+        call_stmt ::= expr POP_TOP
+        call_stmt ::= expr PRINT_EXPR
+        '''
+
 def python_parser(version, co, out=sys.stdout, showasm=False,
                   parser_debug=PARSER_DEFAULT_DEBUG):
     assert iscode(co)

uncompyle6/parsers/parse2.py

@@ -17,9 +17,9 @@ that a later phase can tern into a sequence of ASCII text.
 
 from __future__ import print_function
 
-from uncompyle6.parser import PythonParser, nop_func
+from uncompyle6.parser import PythonParser, PythonParserSingle, nop_func
 from uncompyle6.parsers.astnode import AST
-from spark_parser import GenericASTBuilder, DEFAULT_DEBUG as PARSER_DEFAULT_DEBUG
+from spark_parser import DEFAULT_DEBUG as PARSER_DEFAULT_DEBUG
 from uncompyle6 import PYTHON3
 
 class Python2Parser(PythonParser):
@@ -31,52 +31,14 @@ class Python2Parser(PythonParser):
             super(Python2Parser, self).__init__(AST, 'stmts', debug=debug_parser)
         self.customized = {}
 
-    def p_list_comprehension(self, args):
-        '''
-        expr ::= list_compr
-        list_compr ::= BUILD_LIST_0 list_iter
-
-        list_iter ::= list_for
-        list_iter ::= list_if
-        list_iter ::= list_if_not
-        list_iter ::= lc_body
-
-        _come_from ::= COME_FROM
-        _come_from ::=
-
+    def p_list_comprehension2(self, args):
+        """
         list_for ::= expr _for designator list_iter JUMP_BACK
-        list_if ::= expr jmp_false list_iter
-        list_if_not ::= expr jmp_true list_iter
+        """
 
-        lc_body ::= expr LIST_APPEND
+    def p_setcomp2(self, args):
         '''
-
-    def p_setcomp(self, args):
-        '''
-        expr ::= setcomp
-
-        setcomp ::= LOAD_SETCOMP MAKE_FUNCTION_0 expr GET_ITER CALL_FUNCTION_1
-
-        stmt ::= setcomp_func
-
-        setcomp_func ::= BUILD_SET_0 LOAD_FAST FOR_ITER designator comp_iter
-                JUMP_BACK RETURN_VALUE RETURN_LAST
-
-        comp_iter ::= comp_if
-        comp_iter ::= comp_ifnot
-        comp_iter ::= comp_for
-        comp_iter ::= comp_body
-        comp_body ::= set_comp_body
-        comp_body ::= gen_comp_body
-        comp_body ::= dict_comp_body
-        set_comp_body ::= expr SET_ADD
-        gen_comp_body ::= expr YIELD_VALUE POP_TOP
-        dict_comp_body ::= expr expr MAP_ADD
-
-        comp_if ::= expr jmp_false comp_iter
-        comp_ifnot ::= expr jmp_true comp_iter
-
-        # This is different in python3 - shout it be?
+        # This is different in python3 - should it be?
         comp_for ::= expr _for designator comp_iter JUMP_BACK
         '''
 
@@ -400,69 +362,10 @@ class Python2Parser(PythonParser):
 
         '''
 
-    def p_expr(self, args):
+    def p_expr2(self, args):
         '''
-        expr ::= _mklambda
-        expr ::= SET_LINENO
-        expr ::= LOAD_FAST
-        expr ::= LOAD_NAME
-        expr ::= LOAD_CONST
-        expr ::= LOAD_GLOBAL
-        expr ::= LOAD_DEREF
         expr ::= LOAD_LOCALS
-        expr ::= load_attr
-        expr ::= binary_expr
-        expr ::= binary_expr_na
-        expr ::= build_list
-        expr ::= cmp
-        expr ::= mapexpr
-        expr ::= and
-        expr ::= and2
-        expr ::= or
-        expr ::= unary_expr
-        expr ::= call_function
-        expr ::= unary_not
-        expr ::= unary_convert
-        expr ::= binary_subscr
-        expr ::= binary_subscr2
-        expr ::= load_attr
-        expr ::= get_iter
-        expr ::= slice0
-        expr ::= slice1
-        expr ::= slice2
-        expr ::= slice3
-        expr ::= buildslice2
-        expr ::= buildslice3
-        expr ::= yield
 
-        binary_expr ::= expr expr binary_op
-        binary_op ::= BINARY_ADD
-        binary_op ::= BINARY_MULTIPLY
-        binary_op ::= BINARY_AND
-        binary_op ::= BINARY_OR
-        binary_op ::= BINARY_XOR
-        binary_op ::= BINARY_SUBTRACT
-        binary_op ::= BINARY_DIVIDE
-        binary_op ::= BINARY_TRUE_DIVIDE
-        binary_op ::= BINARY_FLOOR_DIVIDE
-        binary_op ::= BINARY_MODULO
-        binary_op ::= BINARY_LSHIFT
-        binary_op ::= BINARY_RSHIFT
-        binary_op ::= BINARY_POWER
-
-        unary_expr ::= expr unary_op
-        unary_op ::= UNARY_POSITIVE
-        unary_op ::= UNARY_NEGATIVE
-        unary_op ::= UNARY_INVERT
-
-        unary_not ::= expr UNARY_NOT
-        unary_convert ::= expr UNARY_CONVERT
-
-        binary_subscr ::= expr expr BINARY_SUBSCR
-        binary_subscr2 ::= expr expr DUP_TOPX_2 BINARY_SUBSCR
-
-        load_attr ::= expr LOAD_ATTR
-        get_iter ::= expr GET_ITER
         slice0 ::= expr SLICE+0
         slice0 ::= expr DUP_TOP SLICE+0
         slice1 ::= expr expr SLICE+1
@@ -471,79 +374,6 @@ class Python2Parser(PythonParser):
         slice2 ::= expr expr DUP_TOPX_2 SLICE+2
         slice3 ::= expr expr expr SLICE+3
         slice3 ::= expr expr expr DUP_TOPX_3 SLICE+3
-        buildslice3 ::= expr expr expr BUILD_SLICE_3
-        buildslice2 ::= expr expr BUILD_SLICE_2
-
-        yield ::= expr YIELD_VALUE
-
-        _mklambda ::= load_closure mklambda
-        _mklambda ::= mklambda
-
-        or   ::= expr JUMP_IF_TRUE_OR_POP expr COME_FROM
-        or   ::= expr jmp_true expr _come_from
-        and  ::= expr jmp_false expr _come_from
-        and  ::= expr JUMP_IF_FALSE_OR_POP expr COME_FROM
-        and2 ::= _jump jmp_false COME_FROM expr COME_FROM
-
-        expr ::= conditional
-        conditional ::= expr jmp_false expr JUMP_FORWARD expr COME_FROM
-        conditional ::= expr jmp_false expr JUMP_ABSOLUTE expr
-        expr ::= conditionalnot
-        conditionalnot ::= expr jmp_true expr _jump expr COME_FROM
-
-        ret_expr ::= expr
-        ret_expr ::= ret_and
-        ret_expr ::= ret_or
-
-        ret_expr_or_cond ::= ret_expr
-        ret_expr_or_cond ::= ret_cond
-        ret_expr_or_cond ::= ret_cond_not
-
-        ret_and  ::= expr JUMP_IF_FALSE_OR_POP ret_expr_or_cond COME_FROM
-        ret_or   ::= expr JUMP_IF_TRUE_OR_POP ret_expr_or_cond COME_FROM
-        ret_cond ::= expr POP_JUMP_IF_FALSE expr RETURN_END_IF ret_expr_or_cond
-        ret_cond_not ::= expr POP_JUMP_IF_TRUE expr RETURN_END_IF ret_expr_or_cond
-
-        stmt ::= return_lambda
-        stmt ::= conditional_lambda
-
-        return_lambda ::= ret_expr RETURN_VALUE LAMBDA_MARKER
-        conditional_lambda ::= expr jmp_false return_if_stmt return_stmt LAMBDA_MARKER
-
-        cmp ::= cmp_list
-        cmp ::= compare
-        compare ::= expr expr COMPARE_OP
-        cmp_list ::= expr cmp_list1 ROT_TWO POP_TOP
-                _come_from
-        cmp_list1 ::= expr DUP_TOP ROT_THREE
-                COMPARE_OP JUMP_IF_FALSE_OR_POP
-                cmp_list1 COME_FROM
-        cmp_list1 ::= expr DUP_TOP ROT_THREE
-                COMPARE_OP jmp_false
-                cmp_list1 _come_from
-        cmp_list1 ::= expr DUP_TOP ROT_THREE
-                COMPARE_OP JUMP_IF_FALSE_OR_POP
-                cmp_list2 COME_FROM
-        cmp_list1 ::= expr DUP_TOP ROT_THREE
-                COMPARE_OP jmp_false
-                cmp_list2 _come_from
-        cmp_list2 ::= expr COMPARE_OP JUMP_FORWARD
-        cmp_list2 ::= expr COMPARE_OP RETURN_VALUE
-        mapexpr ::= BUILD_MAP kvlist
-
-        kvlist ::= kvlist kv
-        kvlist ::= kvlist kv2
-        kvlist ::= kvlist kv3
-        kvlist ::=
-
-        kv ::= DUP_TOP expr ROT_TWO expr STORE_SUBSCR
-        kv2 ::= DUP_TOP expr expr ROT_THREE STORE_SUBSCR
-        kv3 ::= expr expr STORE_MAP
-
-        exprlist ::= exprlist expr
-        exprlist ::= expr
-
-        nullexprlist ::=
         '''
 
     def add_custom_rules(self, tokens, customize):
@@ -573,7 +403,8 @@ class Python2Parser(PythonParser):
 
             op = k[:k.rfind('_')]
             if op in ('BUILD_LIST', 'BUILD_TUPLE', 'BUILD_SET'):
-                rule = 'build_list ::= ' + 'expr '*v + k
+                rule = ('build_list ::= ' + 'expr1024 '*(v//1024) +
+                                        'expr32 '*((v//32)%32) + 'expr '*(v%32) + k)
             elif op in ('UNPACK_TUPLE', 'UNPACK_SEQUENCE'):
                 rule = 'unpack ::= ' + k + ' designator'*v
             elif op == 'UNPACK_LIST':
@@ -609,12 +440,5 @@ class Python2Parser(PythonParser):
                 raise Exception('unknown customize token %s' % k)
             self.addRule(rule, nop_func)
 
-class Python2ParserSingle(Python2Parser):
-    def p_call_stmt(self, args):
-        '''
-        # single-mode compilation. eval-mode interactive compilation
-        # drops the last rule.
-
-        call_stmt ::= expr POP_TOP
-        call_stmt ::= expr PRINT_EXPR
-        '''
+class Python2ParserSingle(Python2Parser, PythonParserSingle):
+    pass

uncompyle6/parsers/parse3.py

@@ -17,7 +17,7 @@ that a later phase can tern into a sequence of ASCII text.
 
 from __future__ import print_function
 
-from uncompyle6.parser import PythonParser, nop_func
+from uncompyle6.parser import PythonParser, PythonParserSingle, nop_func
 from uncompyle6.parsers.astnode import AST
 from spark_parser import DEFAULT_DEBUG as PARSER_DEFAULT_DEBUG
 from uncompyle6 import PYTHON3
@@ -42,60 +42,26 @@ class Python3Parser(PythonParser):
             pass
         return
 
-    def p_list_comprehension(self, args):
-        '''
+    def p_list_comprehension3(self, args):
+        """
         # Python3 scanner adds LOAD_LISTCOMP. Python3 does list comprehension like
         # other comprehensions (set, dictionary).
 
-        # listcomp is a custom rule
+        # listcomp is a custom Python3 rule
         expr ::= listcomp
 
-        expr ::= list_compr
-        list_compr ::= BUILD_LIST_0 list_iter
-
-        list_iter ::= list_for
-        list_iter ::= list_if
-        list_iter ::= list_if_not
-        list_iter ::= lc_body
-
-        _come_from ::= COME_FROM
-        _come_from ::=
-
         list_for ::= expr FOR_ITER designator list_iter JUMP_BACK
-        list_if ::= expr jmp_false list_iter
-        list_if_not ::= expr jmp_true list_iter
 
-        lc_body ::= expr LIST_APPEND
-        '''
+        # See also common Python p_list_comprehension
+        """
 
-    def p_setcomp(self, args):
-        '''
-        expr ::= setcomp
-
-        setcomp ::= LOAD_SETCOMP MAKE_FUNCTION_0 expr GET_ITER CALL_FUNCTION_1
-
-        stmt ::= setcomp_func
-
-        setcomp_func ::= BUILD_SET_0 LOAD_FAST FOR_ITER designator comp_iter
-                JUMP_BACK RETURN_VALUE RETURN_LAST
-
-        comp_iter ::= comp_if
-        comp_iter ::= comp_ifnot
-        comp_iter ::= comp_for
-        comp_iter ::= comp_body
-        comp_body ::= set_comp_body
-        comp_body ::= gen_comp_body
-        comp_body ::= dict_comp_body
-        set_comp_body ::= expr SET_ADD
-        gen_comp_body ::= expr YIELD_VALUE POP_TOP
-        dict_comp_body ::= expr expr MAP_ADD
-
-        comp_if ::= expr jmp_false comp_iter
-        comp_ifnot ::= expr jmp_true comp_iter
-
-        # This is different in python2 - should it be?
+    def p_setcomp3(self, args):
+        """
+        # This is different in Python 2 - should it be?
         comp_for ::= expr _for designator comp_iter JUMP_ABSOLUTE
-        '''
+
+        # See also common Python p_setcomp
+        """
 
     def p_grammar(self, args):
         '''
@@ -154,9 +120,6 @@ class Python3Parser(PythonParser):
         designList ::= designator designator
         designList ::= designator DUP_TOP designList
 
-        # FIXME: Store local is only used in Python 3.2
-        designator ::= STORE_LOCALS
-
         designator ::= STORE_FAST
         designator ::= STORE_NAME
         designator ::= STORE_GLOBAL
@@ -278,9 +241,6 @@ class Python3Parser(PythonParser):
         _ifstmts_jump ::= return_if_stmts
         _ifstmts_jump ::= c_stmts_opt JUMP_FORWARD _come_from _come_from
 
-        # FIXME: this optimization is only used in Python 3.5 and beyond
-        _ifstmts_jump ::= c_stmts_opt
-
         iflaststmt ::= testexpr c_stmts_opt JUMP_ABSOLUTE
 
         iflaststmtl ::= testexpr c_stmts_opt JUMP_BACK
@@ -411,145 +371,10 @@ class Python3Parser(PythonParser):
 
         '''
 
-    def p_expr(self, args):
+    def p_expr3(self, args):
         '''
-        expr ::= _mklambda
-        expr ::= SET_LINENO
-        expr ::= LOAD_FAST
-        expr ::= LOAD_NAME
-        expr ::= LOAD_CONST
-        expr ::= LOAD_GLOBAL
-        expr ::= LOAD_DEREF
-        expr ::= LOAD_LOCALS
         expr ::= LOAD_CLASSNAME
-        expr ::= load_attr
-        expr ::= binary_expr
-        expr ::= binary_expr_na
-        expr ::= build_list
-        expr ::= cmp
-        expr ::= mapexpr
-        expr ::= and
-        expr ::= and2
-        expr ::= or
-        expr ::= unary_expr
-        expr ::= call_function
-        expr ::= unary_not
-        expr ::= unary_convert
-        expr ::= binary_subscr
-        expr ::= binary_subscr2
-        expr ::= load_attr
-        expr ::= get_iter
-        expr ::= slice0
-        expr ::= slice1
-        expr ::= slice2
-        expr ::= slice3
-        expr ::= buildslice2
-        expr ::= buildslice3
-        expr ::= yield
-
-        binary_expr ::= expr expr binary_op
-        binary_op ::= BINARY_ADD
-        binary_op ::= BINARY_MULTIPLY
-        binary_op ::= BINARY_AND
-        binary_op ::= BINARY_OR
-        binary_op ::= BINARY_XOR
-        binary_op ::= BINARY_SUBTRACT
-        binary_op ::= BINARY_DIVIDE
-        binary_op ::= BINARY_TRUE_DIVIDE
-        binary_op ::= BINARY_FLOOR_DIVIDE
-        binary_op ::= BINARY_MODULO
-        binary_op ::= BINARY_LSHIFT
-        binary_op ::= BINARY_RSHIFT
-        binary_op ::= BINARY_POWER
-
-        unary_expr ::= expr unary_op
-        unary_op ::= UNARY_POSITIVE
-        unary_op ::= UNARY_NEGATIVE
-        unary_op ::= UNARY_INVERT
-
-        unary_not ::= expr UNARY_NOT
-        unary_convert ::= expr UNARY_CONVERT
-
-        binary_subscr ::= expr expr BINARY_SUBSCR
-        binary_subscr2 ::= expr expr DUP_TOPX_2 BINARY_SUBSCR
-
-        load_attr ::= expr LOAD_ATTR
-        get_iter ::= expr GET_ITER
-
         # Python3 drops slice0..slice3
-        buildslice3 ::= expr expr expr BUILD_SLICE_3
-        buildslice2 ::= expr expr BUILD_SLICE_2
-
-        yield ::= expr YIELD_VALUE
-
-        _mklambda ::= load_closure mklambda
-        _mklambda ::= mklambda
-
-        or   ::= expr jmp_true expr _come_from
-        or   ::= expr JUMP_IF_TRUE_OR_POP expr COME_FROM
-        and  ::= expr jmp_false expr _come_from
-        and  ::= expr JUMP_IF_FALSE_OR_POP expr COME_FROM
-        and2 ::= _jump jmp_false COME_FROM expr COME_FROM
-
-        expr ::= conditional
-        conditional ::= expr jmp_false expr JUMP_FORWARD expr COME_FROM
-        conditional ::= expr jmp_false expr JUMP_ABSOLUTE expr
-        expr ::= conditionalnot
-        conditionalnot ::= expr jmp_true expr _jump expr COME_FROM
-
-        ret_expr ::= expr
-        ret_expr ::= ret_and
-        ret_expr ::= ret_or
-
-        ret_expr_or_cond ::= ret_expr
-        ret_expr_or_cond ::= ret_cond
-        ret_expr_or_cond ::= ret_cond_not
-
-        ret_and  ::= expr JUMP_IF_FALSE_OR_POP ret_expr_or_cond COME_FROM
-        ret_or   ::= expr JUMP_IF_TRUE_OR_POP ret_expr_or_cond COME_FROM
-        ret_cond ::= expr POP_JUMP_IF_FALSE expr RETURN_END_IF ret_expr_or_cond
-        ret_cond_not ::= expr POP_JUMP_IF_TRUE expr RETURN_END_IF ret_expr_or_cond
-
-        stmt ::= return_lambda
-        stmt ::= conditional_lambda
-
-        return_lambda ::= ret_expr RETURN_VALUE LAMBDA_MARKER
-        conditional_lambda ::= expr jmp_false return_if_stmt return_stmt LAMBDA_MARKER
-
-        cmp ::= cmp_list
-        cmp ::= compare
-        compare ::= expr expr COMPARE_OP
-        cmp_list ::= expr cmp_list1 ROT_TWO POP_TOP
-                _come_from
-        cmp_list1 ::= expr DUP_TOP ROT_THREE
-                COMPARE_OP JUMP_IF_FALSE_OR_POP
-                cmp_list1 COME_FROM
-        cmp_list1 ::= expr DUP_TOP ROT_THREE
-                COMPARE_OP jmp_false
-                cmp_list1 _come_from
-        cmp_list1 ::= expr DUP_TOP ROT_THREE
-                COMPARE_OP JUMP_IF_FALSE_OR_POP
-                cmp_list2 COME_FROM
-        cmp_list1 ::= expr DUP_TOP ROT_THREE
-                COMPARE_OP jmp_false
-                cmp_list2 _come_from
-        cmp_list2 ::= expr COMPARE_OP JUMP_FORWARD
-        cmp_list2 ::= expr COMPARE_OP RETURN_VALUE
-        mapexpr ::= BUILD_MAP kvlist
-
-        kvlist ::= kvlist kv
-        kvlist ::= kvlist kv2
-        kvlist ::= kvlist kv3
-        kvlist ::=
-
-        kv ::= DUP_TOP expr ROT_TWO expr STORE_SUBSCR
-        kv2 ::= DUP_TOP expr expr ROT_THREE STORE_SUBSCR
-        kv3 ::= expr expr STORE_MAP
-
-        exprlist ::= exprlist expr
-        exprlist ::= expr
-
-        nullexprlist ::=
         '''
 
     @staticmethod
@@ -653,7 +478,15 @@ class Python3Parser(PythonParser):
             elif opname == 'LOAD_BUILD_CLASS':
                 self.custom_build_class_rule(opname, i, token, tokens, customize)
             elif opname_base in ('BUILD_LIST', 'BUILD_TUPLE', 'BUILD_SET'):
-                rule = 'build_list ::= ' + 'expr ' * token.attr + opname
+                v = token.attr
+                rule = ('build_list ::= ' + 'expr1024 ' * int(v//1024) +
+                                        'expr32 ' * int((v//32)%32) + 'expr '*(v%32) + opname)
+                self.add_unique_rule(rule, opname, token.attr, customize)
+            elif self.version >= 3.5 and opname_base == 'BUILD_MAP':
+                kvlist_n = "kvlist_%s" % token.attr
+                rule = kvlist_n + ' ::= ' + 'expr ' * (token.attr*2)
+                self.add_unique_rule(rule, opname, token.attr, customize)
+                rule = "mapexpr ::=  %s %s" % (kvlist_n, opname)
                 self.add_unique_rule(rule, opname, token.attr, customize)
             elif opname_base in ('UNPACK_TUPLE', 'UNPACK_SEQUENCE'):
                 rule = 'unpack ::= ' + opname + ' designator' * token.attr
@@ -693,12 +526,25 @@ class Python3Parser(PythonParser):
                 self.add_unique_rule(rule, opname, token.attr, customize)
         return
 
-class Python3ParserSingle(Python3Parser):
-    def p_call_stmt(self, args):
-        '''
-        # single-mode compilation. Eval-mode interactive compilation
-        # drops the last rule.
+class Python32Parser(Python3Parser):
+    def p_32(self, args):
+        """
+        # Store locals is only used in Python 3.2
+        designator ::= STORE_LOCALS
+        """
 
-        call_stmt ::= expr POP_TOP
-        call_stmt ::= expr PRINT_EXPR
-        '''
+class Python3ParserSingle(Python3Parser, PythonParserSingle):
+    pass
+
+class Python32ParserSingle(Python32Parser, PythonParserSingle):
+    pass
+
+class Python35onParser(Python3Parser):
+    def p_35on(self, args):
+        """
+        # this optimization is only used in Python 3.5 and beyond
+        _ifstmts_jump ::= c_stmts_opt
+        """
+
+class Python35onParserSingle(Python35onParser, PythonParserSingle):
+    pass

uncompyle6/scanners/scanner35.py

@@ -14,7 +14,6 @@ import dis, inspect
 from array import array
 import uncompyle6.scanners.scanner3 as scan3
 
-from uncompyle6 import PYTHON_VERSION
 from uncompyle6.code import iscode
 from uncompyle6.scanner import Token
 
@@ -29,13 +28,10 @@ from uncompyle6.opcodes.opcode_35 import *
 
 class Scanner35(scan3.Scanner3):
 
-    def disassemble(self, co, classname=None, code_objects={}):
-        fn = self.disassemble_built_in if PYTHON_VERSION == 3.4 \
-            else self.disassemble_generic
-        return fn(co, classname, code_objects=code_objects)
-
-    def disassemble_built_in(self, co, classname=None,
-                             code_objects={}):
+    # Note: we can't use built-in disassembly routines, unless
+    # we do post-processing like we do here.
+    def disassemble(self, co, classname=None,
+                    code_objects={}):
         # Container for tokens
         tokens = []
         customize = {}
@@ -116,17 +112,11 @@ class Scanner35(scan3.Scanner3):
                     pattr = const
                     pass
             elif opname in ('BUILD_LIST', 'BUILD_TUPLE', 'BUILD_SET', 'BUILD_SLICE',
+                            'BUILD_MAP',
                             'UNPACK_SEQUENCE',
                             'MAKE_FUNCTION', 'MAKE_CLOSURE',
                             'DUP_TOPX', 'RAISE_VARARGS'
                             ):
-                # if opname == 'BUILD_TUPLE' and \
-                #     self.code[self.prev[offset]] == LOAD_CLOSURE:
-                #     continue
-                # else:
-                #     op_name = '%s_%d' % (op_name, oparg)
-                #     if opname != BUILD_SLICE:
-                #         customize[op_name] = oparg
                 opname = '%s_%d' % (opname, inst.argval)
                 if inst.opname != 'BUILD_SLICE':
                     customize[opname] = inst.argval

uncompyle6/semantics/fragments.py

@@ -973,13 +973,29 @@ class FragmentsWalker(pysource.SourceWalker, object):
         else:
             raise RuntimeError('Internal Error: n_build_list expects list or tuple')
 
+        flat_elems = []
+        for elem in node:
+            if elem == 'expr1024':
+                for subelem in elem:
+                        for subsubelem in subelem:
+                            flat_elems.append(subsubelem)
+            elif elem == 'expr32':
+                for subelem in elem:
+                    flat_elems.append(subelem)
+            else:
+                flat_elems.append(elem)
+
         self.indentMore(INDENT_PER_LEVEL)
         if len(node) > 3:
             line_separator = ',\n' + self.indent
         else:
             line_separator = ', '
         sep = INDENT_PER_LEVEL[:-1]
-        for elem in node:
+
+        # FIXME:
+        # if flat_elems > some_number, then group
+        # do automatic wrapping
+        for elem in flat_elems:
             if (elem == 'ROT_THREE'):
                 continue
 

uncompyle6/semantics/pysource.py

@@ -1148,29 +1148,42 @@ class SourceWalker(GenericASTTraversal, object):
         """
         p = self.prec
         self.prec = 100
-        assert node[-1] == 'kvlist'
-        node = node[-1] # goto kvlist
 
         self.indentMore(INDENT_PER_LEVEL)
         line_seperator = ',\n' + self.indent
         sep = INDENT_PER_LEVEL[:-1]
         self.write('{')
-        for kv in node:
-            assert kv in ('kv', 'kv2', 'kv3')
-            # kv ::= DUP_TOP expr ROT_TWO expr STORE_SUBSCR
-            # kv2 ::= DUP_TOP expr expr ROT_THREE STORE_SUBSCR
-            # kv3 ::= expr expr STORE_MAP
-            if kv == 'kv':
-                name = self.traverse(kv[-2], indent='')
-                value = self.traverse(kv[1], indent=self.indent+(len(name)+2)*' ')
-            elif kv == 'kv2':
-                name = self.traverse(kv[1], indent='')
-                value = self.traverse(kv[-3], indent=self.indent+(len(name)+2)*' ')
-            elif kv == 'kv3':
-                name = self.traverse(kv[-2], indent='')
-                value = self.traverse(kv[0], indent=self.indent+(len(name)+2)*' ')
-            self.write(sep, name, ': ', value)
-            sep = line_seperator
+
+        if node[0].type.startswith('kvlist'):
+            # Python 3.5 style key/value list in mapexpr
+            l = list(node[0])
+            i = 0
+            while i < len(l):
+                name = self.traverse(l[i], indent='')
+                value = self.traverse(l[i+1], indent=self.indent+(len(name)+2)*' ')
+                self.write(sep, name, ': ', value)
+                sep = line_seperator
+                i += 2
+        else:
+            assert node[-1] == 'kvlist'
+            node = node[-1] # goto kvlist
+
+            for kv in node:
+                assert kv in ('kv', 'kv2', 'kv3')
+                # kv ::= DUP_TOP expr ROT_TWO expr STORE_SUBSCR
+                # kv2 ::= DUP_TOP expr expr ROT_THREE STORE_SUBSCR
+                # kv3 ::= expr expr STORE_MAP
+                if kv == 'kv':
+                    name = self.traverse(kv[-2], indent='')
+                    value = self.traverse(kv[1], indent=self.indent+(len(name)+2)*' ')
+                elif kv == 'kv2':
+                    name = self.traverse(kv[1], indent='')
+                    value = self.traverse(kv[-3], indent=self.indent+(len(name)+2)*' ')
+                elif kv == 'kv3':
+                    name = self.traverse(kv[-2], indent='')
+                    value = self.traverse(kv[0], indent=self.indent+(len(name)+2)*' ')
+                    self.write(sep, name, ': ', value)
+                    sep = line_seperator
         self.write('}')
         self.indentLess(INDENT_PER_LEVEL)
         self.prec = p
@@ -1182,33 +1195,49 @@ class SourceWalker(GenericASTTraversal, object):
         """
         p = self.prec
         self.prec = 100
-        lastnode = node.pop().type
-        if lastnode.startswith('BUILD_LIST'):
+        lastnode = node.pop()
+        lastnodetype = lastnode.type
+        if lastnodetype.startswith('BUILD_LIST'):
             self.write('['); endchar = ']'
-        elif lastnode.startswith('BUILD_TUPLE'):
+        elif lastnodetype.startswith('BUILD_TUPLE'):
             self.write('('); endchar = ')'
-        elif lastnode.startswith('BUILD_SET'):
+        elif lastnodetype.startswith('BUILD_SET'):
             self.write('{'); endchar = '}'
-        elif lastnode.startswith('ROT_TWO'):
+        elif lastnodetype.startswith('ROT_TWO'):
             self.write('('); endchar = ')'
         else:
             raise 'Internal Error: n_build_list expects list or tuple'
 
+        flat_elems = []
+        for elem in node:
+            if elem == 'expr1024':
+                for subelem in elem:
+                        for subsubelem in subelem:
+                            flat_elems.append(subsubelem)
+            elif elem == 'expr32':
+                for subelem in elem:
+                    flat_elems.append(subelem)
+            else:
+                flat_elems.append(elem)
+
         self.indentMore(INDENT_PER_LEVEL)
-        if len(node) > 3:
+        if lastnode.attr > 3:
             line_separator = ',\n' + self.indent
         else:
             line_separator = ', '
         sep = INDENT_PER_LEVEL[:-1]
-        for elem in node:
-            if (elem == 'ROT_THREE'):
-                continue
 
+        # FIXME:
+        # if flat_elems > some_number, then group
+        # do automatic wrapping
+        for elem in flat_elems:
+            if elem == 'ROT_THREE':
+                continue
             assert elem == 'expr'
             value = self.traverse(elem)
             self.write(sep, value)
             sep = line_separator
-        if len(node) == 1 and lastnode.startswith('BUILD_TUPLE'):
+        if lastnode.attr == 1 and lastnodetype.startswith('BUILD_TUPLE'):
             self.write(',')
         self.write(endchar)
         self.indentLess(INDENT_PER_LEVEL)

uncompyle6/version.py

@@ -0,0 +1,3 @@
+# This file is suitable for sourcing inside bash as
+# well as importing into Python
+VERSION='2.3.3'