Administrivia

* Some grammar cleanup
* add more grammar debugging

.circleci/config.yml

@@ -1,11 +1,3 @@
-# This configuration was automatically generated from a CircleCI 1.0 config.
-# It should include any build commands you had along with commands that CircleCI
-# inferred from your project structure. We strongly recommend you read all the
-# comments in this file to understand the structure of CircleCI 2.0, as the idiom
-# for configuration has changed substantially in 2.0 to allow arbitrary jobs rather
-# than the prescribed lifecycle of 1.0. In general, we recommend using this generated
-# configuration as a reference rather than using it in production, though in most
-# cases it should duplicate the execution of your original 1.0 config.
 version: 2
 jobs:
   build:
@@ -42,7 +34,7 @@ jobs:
     # This is based on your 1.0 configuration file or project settings
     - run:
         working_directory: ~/rocky/python-uncompyle6
-        command: pyenv install 2.4.6 && pyenv local 2.4.6 && pyenv rehash && easy_install nose && pyenv rehash
+        command: pip install virtualenv && pip install nose && pip install pep8 && pyenv rehash
     # Dependencies
     #   This would typically go in either a build or a build-and-test job when using workflows
     # Restore the dependency cache
@@ -53,6 +45,7 @@ jobs:
           - v2-dependencies-
 
     # This is based on your 1.0 configuration file or project settings
+    - run: pip install --upgrade setuptools
     - run: pip install -e .
     - run: pip install -r requirements-dev.txt
 
@@ -72,8 +65,8 @@ jobs:
     # Test
     #   This would typically be a build job when using workflows, possibly combined with build
     # This is based on your 1.0 configuration file or project settings
-    - run: python ./setup.py develop && make check-2.4
-    - run: cd ./test/stdlib && pyenv local 2.4.6 && bash ./runtests.sh 'test_[p-z]*.py'
+    - run: python ./setup.py develop && make check-2.7
+    - run: cd ./test/stdlib && bash ./runtests.sh 'test_[p-z]*.py'
     # Teardown
     #   If you break your build into multiple jobs with workflows, you will probably want to do the parts of this that are relevant in each
     # Save test results

.gitignore

@@ -6,6 +6,7 @@
 /.eggs
 /.hypothesis
 /.idea
+/.mypy_cache
 /.pytest_cache
 /.python-version
 /.tox

.travis.yml

@@ -1,11 +1,15 @@
 language: python
 
 python:
-  - '2.7' # this is a cheat here because travis doesn't do 2.4-2.6
+  - '3.5'
+  - '2.7'
+  - '3.4'
+  - '3.6'
+  - '3.8'
 
 matrix:
   include:
-    - python: '2.7'
+    - python: '3.7'
       dist: xenial  # required for Python >= 3.7 (travis-ci/travis-ci#9069)
 
 install:

Makefile

@@ -40,16 +40,19 @@ check-3.0 check-3.1 check-3.2 check-3.6:
 check-3.7: pytest
 	$(MAKE) -C test check
 
-#:Tests for Python 2.4-2.5 (don't have pytest)
-check-2.4 check-2.5:
-	$(MAKE) -C test $@
+check-3.8:
+	$(MAKE) -C test check
 
 #:PyPy 2.6.1 PyPy 5.0.1, or PyPy 5.8.0-beta0
 # Skip for now
 2.6 5.0 5.3 5.6 5.8:
 
-#:PyPy pypy3-2.4.0 Python 3:
-pypy-3.2 2.4:
+#:PyPy pypy3-2.4.0 Python 3.6.1:
+7.1 pypy-3.2 2.4:
+	$(MAKE) -C test $@
+
+#:PyPy pypy3-2.4.0 Python 3.6.9:
+7.2:
 	$(MAKE) -C test $@
 
 #: Run py.test tests

NEWS.md

@@ -1,3 +1,34 @@
+3.5.1 2019-10-29 JNC
+====================
+
+- Pypy 3.3, 3.5, 3.6, and 3.6.9 support
+- Improve 3.0 decompilation
+    - no parse errors on stlib bytecode. However accurate translation in
+	  control-flow and and/or detection needs work
+- Remove extraneous iter() in "for" of list comprehension  Fixes #272
+- "for" block without a POP_BLOCK and confusing JUMP_BACK for CONTINUE. Fixes #293
+- Fix unmarshal incompletness detected in Pypy 3.6
+- Miscellaneous bugs fixed
+
+3.5.0 2019-10-12 Stony Brook Ride
+=================================
+
+- Fix fragment bugs
+   * missing RETURN_LAST introduced when adding transformation layer
+   * more parent entries on tokens
+- Preliminary support for decompiling Python 1.0, 1.1. 1.2 and 1.6
+   * Newer xdis version needed
+
+3.4.1 2019-10-02
+================
+
+- Correct assert{,2} transforms Fixes #289
+- Fragment parsing fixes:
+     * Wasn't handling 3-arg %p
+   	 * fielding error in code_deparse()
+- Use newer xdis to better track Python 3.8.0
+
+
 3.4.0 2019-08-24 Totoro
 =======================
 
@@ -105,7 +136,8 @@ Lots of decomplation bugs, especially in the 3.x series fixed. Don't worry thoug
 3.3.0 2019-04-14 Holy Week
 ==========================
 
-* First cut at Python 3.8 (many bug remain)
+* First cut at Python 3.8 (many bugs remain)
+* Reinstate -c | --compile (compile before disassembly) option
 * The usual smattering of bug and doc fixes
 
 3.2.6 2019-03-23 Mueller Report

README.rst

@@ -1,4 +1,4 @@
-|buildstatus| |Latest Version| |Supported Python Versions|
+|buildstatus|  |Pypi Installs| |Latest Version| |Supported Python Versions|
 
 |packagestatus|
 
@@ -13,7 +13,7 @@ Introduction
 ------------
 
 *uncompyle6* translates Python bytecode back into equivalent Python
-source code. It accepts bytecodes from Python version 1.3 to version
+source code. It accepts bytecodes from Python version 1.0 to version
 3.8, spanning over 24 years of Python releases. We include Dropbox's
 Python 2.5 bytecode and some PyPy bytecode.
 
@@ -88,9 +88,9 @@ This uses setup.py, so it follows the standard Python routine:
 
 ::
 
-    pip install -e .  # set up to run from source tree
-                      # Or if you want to install instead
-    python setup.py install # may need sudo
+    $ pip install -e .  # set up to run from source tree
+                        # Or if you want to install instead
+    $ python setup.py install # may need sudo
 
 A GNU makefile is also provided so :code:`make install` (possibly as root or
 sudo) will do the steps above.
@@ -244,3 +244,4 @@ See Also
 .. |Supported Python Versions| image:: https://img.shields.io/pypi/pyversions/uncompyle6.svg
 .. |Latest Version| image:: https://badge.fury.io/py/uncompyle6.svg
 		 :target: https://badge.fury.io/py/uncompyle6
+.. |Pypi Installs| image:: https://pepy.tech/badge/uncompyle6/month

__pkginfo__.py

@@ -58,7 +58,7 @@ entry_points       = {
     ]}
 ftp_url            = None
 install_requires   = ["spark-parser >= 1.8.9, < 1.9.0",
-                      "xdis >= 4.0.3, < 4.1.0"]
+                      "xdis >= 4.1.3, < 4.2.0"]
 
 license            = "GPL3"
 mailing_list       = "python-debugger@googlegroups.com"

admin-tools/how-to-make-a-release.md

@@ -56,19 +56,21 @@
 
     $ . ./admin-tools/make-dist-older.sh
     $ git tag release-python-2.4-$VERSION
-
+	$ twine check dist/uncompyle6-$VERSION*
     $ . ./admin-tools/make-dist-newer.sh
-
-Goto https://github.com/rocky/python-uncompyle6/releases
+	$ twine check dist/uncompyle6-$VERSION*
 
 # Upload single package and look at Rst Formating
 
+	$ twine check dist/uncompyle6-${VERSION}*
     $ twine upload dist/uncompyle6-${VERSION}-py3.3.egg
 
 # Upload rest of versions
 
     $ twine upload dist/uncompyle6-${VERSION}*
 
+Goto https://github.com/rocky/python-uncompyle6/releases
+
 # Push tags:
 
     $ git push --tags

admin-tools/how-to-make-a-release.txt

@@ -1,46 +0,0 @@
-git pull
-
-Change version in uncompyle6/version.py
-source uncompyle6/version.py
-echo $VERSION
-git commit -m"Get ready for release $VERSION" .
-
-Update ChangeLog:
-  make ChangeLog
-
-Update NEWS from ChangeLog
-make check
-
-git commit --amend .
-
-git push
-
-Make sure pyenv is running
-# Pyenv
-
-source admin-tools/check-newer-versions.sh
-
-
-# Switch to python-2.4 and build that first...
-source admin-tools/setup-python-2.4
-
-rm ChangeLog
-git merge master
-
-Update NEWS from master branch
-
-git commit -m"Get ready for release $VERSION" .
-
-source admin-tools/check-older-versions.sh
-source admin-tools/check-newer-versions.sh
-
-make-dist-older.sh
-
-git tag release-python-2.4-$VERSION
-
-./make-dist-newer.sh
-
-git tag release-$VERSION
-
-
-twine upload dist/uncompyle6-${VERSION}*

admin-tools/pyenv-newer-versions

@@ -5,4 +5,4 @@ if [[ $0 == ${BASH_SOURCE[0]} ]] ; then
     echo "This script should be *sourced* rather than run directly through bash"
     exit 1
 fi
-export PYVERSIONS='3.6.8 3.7.3 2.6.9 3.3.7 2.7.16 3.2.6 3.1.5 3.4.8'
+export PYVERSIONS='3.5.9 3.6.9 2.6.9 3.3.7 2.7.17 3.2.6 3.1.5 3.4.10 3.7.5'

admin-tools/pyenv-older-versions

@@ -6,4 +6,4 @@ if [[ $0 == ${BASH_SOURCE[0]} ]] ; then
     echo "This script should be *sourced* rather than run directly through bash"
     exit 1
 fi
-export PYVERSIONS='2.4.6 2.5.6'
+export PYVERSIONS='2.4.6 2.5.6 2.6.9'

admin-tools/setup-master.sh

@@ -1,5 +1,5 @@
 #!/bin/bash
-PYTHON_VERSION=3.6.8
+PYTHON_VERSION=3.7.5
 
 # FIXME put some of the below in a common routine
 function finish {

admin-tools/update-sources.sh

@@ -1,3 +0,0 @@
-#!/bin/bash
-cd $(dirname ${BASH_SOURCE[0]})/..
-git pull

pytest/test_deparse.py

@@ -30,7 +30,7 @@ def list_comp():
     [y for y in range(3)]
 
 def get_parsed_for_fn(fn):
-    code = fn.func_code
+    code = fn.__code__ if PYTHON3 else fn.func_code
     return deparse(code, version=PYTHON_VERSION)
 
 def check_expect(expect, parsed, fn_name):

pytest/test_fjt.py

@@ -20,7 +20,7 @@ def bug_loop(disassemble, tb=None):
     disassemble(tb)
 
 def test_if_in_for():
-    code = bug.func_code
+    code = bug.__code__
     scan = get_scanner(PYTHON_VERSION)
     if 2.7 <= PYTHON_VERSION <= 3.0 and not IS_PYPY:
         scan.build_instructions(code)

pytest/test_fstring.py

@@ -0,0 +1,158 @@
+# std
+# test
+import sys
+from uncompyle6 import PYTHON_VERSION, code_deparse
+import pytest
+
+pytestmark = pytest.mark.skipif(
+    PYTHON_VERSION <= 2.6, reason="hypothesis needs 2.7 or later"
+)
+if PYTHON_VERSION > 2.6:
+
+    import hypothesis
+    from hypothesis import strategies as st
+
+    # uncompyle6
+
+    @st.composite
+    def expressions(draw):
+        # todo : would be nice to generate expressions using hypothesis however
+        # this is pretty involved so for now just use a corpus of expressions
+        # from which to select.
+        return draw(
+            st.sampled_from(
+                (
+                    "abc",
+                    "len(items)",
+                    "x + 1",
+                    "lineno",
+                    "container",
+                    "self.attribute",
+                    "self.method()",
+                    # These expressions are failing, I think these are control
+                    # flow problems rather than problems with FORMAT_VALUE,
+                    # however I need to confirm this...
+                    #'sorted(items, key=lambda x: x.name)',
+                    #'func(*args, **kwargs)',
+                    #'text or default',
+                    #'43 if life_the_universe and everything else None'
+                )
+            )
+        )
+
+    @st.composite
+    def format_specifiers(draw):
+        """
+        Generate a valid format specifier using the rules:
+
+        format_spec ::=  [[fill]align][sign][#][0][width][,][.precision][type]
+        fill        ::=  <any character>
+        align       ::=  "<" | ">" | "=" | "^"
+        sign        ::=  "+" | "-" | " "
+        width       ::=  integer
+        precision   ::=  integer
+        type        ::=  "b" | "c" | "d" | "e" | "E" | "f" | "F" | "g" | "G" | "n" | "o" | "s" | "x" | "X" | "%"
+
+        See https://docs.python.org/2/library/string.html
+
+        :param draw: Let hypothesis draw from other strategies.
+
+        :return: An example format_specifier.
+        """
+        alphabet_strategy = st.characters(
+            min_codepoint=ord("a"), max_codepoint=ord("z")
+        )
+        fill = draw(st.one_of(alphabet_strategy, st.none()))
+        align = draw(st.sampled_from(list("<>=^")))
+        fill_align = (fill + align or "") if fill else ""
+
+        type_ = draw(st.sampled_from("bcdeEfFgGnosxX%"))
+        can_have_sign = type_ in "deEfFgGnoxX%"
+        can_have_comma = type_ in "deEfFgG%"
+        can_have_precision = type_ in "fFgG"
+        can_have_pound = type_ in "boxX%"
+        can_have_zero = type_ in "oxX"
+
+        sign = draw(st.sampled_from(list("+- ") + [""])) if can_have_sign else ""
+        pound = draw(st.sampled_from(("#", ""))) if can_have_pound else ""
+        zero = draw(st.sampled_from(("0", ""))) if can_have_zero else ""
+
+        int_strategy = st.integers(min_value=1, max_value=1000)
+
+        width = draw(st.one_of(int_strategy, st.none()))
+        width = str(width) if width is not None else ""
+
+        comma = draw(st.sampled_from((",", ""))) if can_have_comma else ""
+        if can_have_precision:
+            precision = draw(st.one_of(int_strategy, st.none()))
+            precision = "." + str(precision) if precision else ""
+        else:
+            precision = ""
+
+        return "".join((fill_align, sign, pound, zero, width, comma, precision, type_))
+
+    @st.composite
+    def fstrings(draw):
+        """
+        Generate a valid f-string.
+        See https://www.python.org/dev/peps/pep-0498/#specification
+
+        :param draw: Let hypothsis draw from other strategies.
+
+        :return: A valid f-string.
+        """
+        character_strategy = st.characters(
+            blacklist_characters="\r\n'\\s{}", min_codepoint=1, max_codepoint=1000
+        )
+        is_raw = draw(st.booleans())
+        integer_strategy = st.integers(min_value=0, max_value=3)
+        expression_count = draw(integer_strategy)
+        content = []
+        for _ in range(expression_count):
+            expression = draw(expressions())
+            conversion = draw(st.sampled_from(("", "!s", "!r", "!a")))
+            has_specifier = draw(st.booleans())
+            specifier = ":" + draw(format_specifiers()) if has_specifier else ""
+            content.append("{{{}{}}}".format(expression, conversion, specifier))
+            content.append(draw(st.text(character_strategy)))
+        content = "".join(content)
+        return "f{}'{}'".format("r" if is_raw else "", content)
+
+    @pytest.mark.skipif(PYTHON_VERSION != 3.6, reason="need Python 3.6")
+    @hypothesis.given(format_specifiers())
+    def test_format_specifiers(format_specifier):
+        """Verify that format_specifiers generates valid specifiers"""
+        try:
+            exec('"{:' + format_specifier + '}".format(0)')
+        except ValueError as e:
+            if "Unknown format code" not in str(e):
+                raise
+
+    def run_test(text):
+        hypothesis.assume(len(text))
+        hypothesis.assume("f'{" in text)
+        expr = text + "\n"
+        code = compile(expr, "<string>", "single")
+        deparsed = code_deparse(code, sys.stdout, PYTHON_VERSION, compile_mode="single")
+        recompiled = compile(deparsed.text, "<string>", "single")
+        if recompiled != code:
+            print(recompiled)
+            print("================")
+            print(code)
+            print("----------------")
+            assert (
+                "dis(" + deparsed.text.strip("\n") + ")"
+                == "dis(" + expr.strip("\n") + ")"
+            )
+
+    @pytest.mark.skipif(PYTHON_VERSION != 3.6, reason="need Python 3.6")
+    @hypothesis.given(fstrings())
+    def test_uncompyle_fstring(fstring):
+        """Verify uncompyling fstring bytecode"""
+        run_test(fstring)
+
+    @pytest.mark.skipif(PYTHON_VERSION != 3.6, reason="need Python 3.6+")
+    @pytest.mark.parametrize("fstring", ["f'{abc}{abc!s}'", "f'{abc}0'"])
+    def test_uncompyle_direct(fstring):
+        """useful for debugging"""
+        run_test(fstring)

pytest/test_function_call.py

@@ -0,0 +1,185 @@
+import string
+from uncompyle6 import PYTHON_VERSION
+import pytest
+pytestmark = pytest.mark.skip(PYTHON_VERSION < 2.7,
+                              reason="need at least Python 2.7")
+
+if PYTHON_VERSION > 2.6:
+    from hypothesis import given, assume, example, settings, strategies as st
+    from validate import validate_uncompyle
+    from test_fstring import expressions
+
+    alpha = st.sampled_from(string.ascii_lowercase)
+    numbers = st.sampled_from(string.digits)
+    alphanum = st.sampled_from(string.ascii_lowercase + string.digits)
+
+
+    @st.composite
+    def function_calls(draw,
+                       min_keyword_args=0, max_keyword_args=5,
+                       min_positional_args=0, max_positional_args=5,
+                       min_star_args=0, max_star_args=1,
+                       min_double_star_args=0, max_double_star_args=1):
+        """
+        Strategy factory for generating function calls.
+
+        :param draw: Callable which draws examples from other strategies.
+
+        :return: The function call text.
+        """
+        st_positional_args = st.lists(
+            alpha,
+            min_size=min_positional_args,
+            max_size=max_positional_args
+        )
+        st_keyword_args = st.lists(
+            alpha,
+            min_size=min_keyword_args,
+            max_size=max_keyword_args
+        )
+        st_star_args = st.lists(
+            alpha,
+            min_size=min_star_args,
+            max_size=max_star_args
+        )
+        st_double_star_args = st.lists(
+            alpha,
+            min_size=min_double_star_args,
+            max_size=max_double_star_args
+        )
+
+        positional_args = draw(st_positional_args)
+        keyword_args = draw(st_keyword_args)
+        st_values = st.lists(
+            expressions(),
+            min_size=len(keyword_args),
+            max_size=len(keyword_args)
+        )
+        keyword_args = [
+            x + '=' + e
+            for x, e in
+            zip(keyword_args, draw(st_values))
+        ]
+        star_args = ['*' + x for x in draw(st_star_args)]
+        double_star_args = ['**' + x for x in draw(st_double_star_args)]
+
+        arguments = positional_args + keyword_args + star_args + double_star_args
+        draw(st.randoms()).shuffle(arguments)
+        arguments = ','.join(arguments)
+
+        function_call = 'fn({arguments})'.format(arguments=arguments)
+        try:
+            # TODO: Figure out the exact rules for ordering of positional, keyword,
+            # star args, double star args and in which versions the various
+            # types of arguments are supported so we don't need to check that the
+            # expression compiles like this.
+            compile(function_call, '<string>', 'single')
+        except:
+            assume(False)
+        return function_call
+
+
+    def test_function_no_args():
+        validate_uncompyle("fn()")
+
+    @pytest.mark.skipif(PYTHON_VERSION < 2.7,
+                        reason="need at least Python 2.7")
+    def isolated_function_calls(which):
+        """
+        Returns a strategy for generating function calls, but isolated to
+        particular types of arguments, for example only positional arguments.
+
+        This can help reason about debugging errors in specific types of function
+        calls.
+
+        :param which: One of 'keyword', 'positional', 'star', 'double_star'
+
+        :return: Strategy for generating an function call isolated to specific
+                 argument types.
+        """
+        kwargs = dict(
+            max_keyword_args=0,
+            max_positional_args=0,
+            max_star_args=0,
+            max_double_star_args=0,
+        )
+        kwargs['_'.join(('min', which, 'args'))] = 1
+        kwargs['_'.join(('max', which, 'args'))] = 5 if 'star' not in which else 1
+        return function_calls(**kwargs)
+
+
+    with settings(max_examples=25):
+
+        @pytest.mark.skipif(PYTHON_VERSION < 2.7,
+                            reason="need at least Python 2.7")
+        @given(isolated_function_calls('positional'))
+        @example("fn(0)")
+        def test_function_positional_only(expr):
+            validate_uncompyle(expr)
+
+        @pytest.mark.skipif(PYTHON_VERSION < 2.7,
+                            reason="need at least Python 2.7")
+        @given(isolated_function_calls('keyword'))
+        @example("fn(a=0)")
+        def test_function_call_keyword_only(expr):
+            validate_uncompyle(expr)
+
+        @pytest.mark.skipif(PYTHON_VERSION < 2.7,
+                            reason="need at least Python 2.7")
+        @given(isolated_function_calls('star'))
+        @example("fn(*items)")
+        def test_function_call_star_only(expr):
+            validate_uncompyle(expr)
+
+        @pytest.mark.skipif(PYTHON_VERSION < 2.7,
+                            reason="need at least Python 2.7")
+        @given(isolated_function_calls('double_star'))
+        @example("fn(**{})")
+        def test_function_call_double_star_only(expr):
+            validate_uncompyle(expr)
+
+
+    @pytest.mark.xfail()
+    def test_BUILD_CONST_KEY_MAP_BUILD_MAP_UNPACK_WITH_CALL_BUILD_TUPLE_CALL_FUNCTION_EX():
+        validate_uncompyle("fn(w=0,m=0,**v)")
+
+
+    @pytest.mark.xfail()
+    def test_BUILD_MAP_BUILD_MAP_UNPACK_WITH_CALL_BUILD_TUPLE_CALL_FUNCTION_EX():
+        validate_uncompyle("fn(a=0,**g)")
+
+
+    @pytest.mark.xfail()
+    def test_CALL_FUNCTION_EX():
+        validate_uncompyle("fn(*g,**j)")
+
+
+    @pytest.mark.xfail()
+    def test_BUILD_MAP_CALL_FUNCTION_EX():
+        validate_uncompyle("fn(*z,u=0)")
+
+
+    @pytest.mark.xfail()
+    def test_BUILD_TUPLE_CALL_FUNCTION_EX():
+        validate_uncompyle("fn(**a)")
+
+
+    @pytest.mark.xfail()
+    def test_BUILD_MAP_BUILD_TUPLE_BUILD_TUPLE_UNPACK_WITH_CALL_CALL_FUNCTION_EX():
+        validate_uncompyle("fn(b,b,b=0,*a)")
+
+
+    @pytest.mark.xfail()
+    def test_BUILD_TUPLE_BUILD_TUPLE_UNPACK_WITH_CALL_CALL_FUNCTION_EX():
+        validate_uncompyle("fn(*c,v)")
+
+
+    @pytest.mark.xfail()
+    def test_BUILD_CONST_KEY_MAP_CALL_FUNCTION_EX():
+        validate_uncompyle("fn(i=0,y=0,*p)")
+
+
+    @pytest.mark.skip(reason='skipping property based test until all individual tests are passing')
+    @given(function_calls())
+    def test_function_call(function_call):
+        validate_uncompyle(function_call)

pytest/test_grammar.py

@@ -1,50 +1,54 @@
 import re
-from uncompyle6 import PYTHON_VERSION, PYTHON3, IS_PYPY # , PYTHON_VERSION
+from uncompyle6 import PYTHON_VERSION, PYTHON3, IS_PYPY  # , PYTHON_VERSION
 from uncompyle6.parser import get_python_parser, python_parser
 from uncompyle6.scanner import get_scanner
 
-def test_grammar():
 
+def test_grammar():
     def check_tokens(tokens, opcode_set):
         remain_tokens = set(tokens) - opcode_set
-        remain_tokens = set([re.sub(r'_\d+$','', t) for t in remain_tokens])
-        remain_tokens = set([re.sub('_CONT$','', t) for t in remain_tokens])
-        remain_tokens = set([re.sub('LOAD_CODE$','', t) for t in remain_tokens])
+        remain_tokens = set([re.sub(r"_\d+$", "", t) for t in remain_tokens])
+        remain_tokens = set([re.sub("_CONT$", "", t) for t in remain_tokens])
+        remain_tokens = set([re.sub("LOAD_CODE$", "", t) for t in remain_tokens])
         remain_tokens = set(remain_tokens) - opcode_set
-        assert remain_tokens == set([]), \
-            "Remaining tokens %s\n====\n%s" % (remain_tokens, p.dump_grammar())
+        assert remain_tokens == set([]), "Remaining tokens %s\n====\n%s" % (
+            remain_tokens,
+            p.dump_grammar(),
+        )
 
     p = get_python_parser(PYTHON_VERSION, is_pypy=IS_PYPY)
-    (lhs, rhs, tokens,
-     right_recursive, dup_rhs) = p.check_sets()
+    (lhs, rhs, tokens, right_recursive, dup_rhs) = p.check_sets()
 
     # We have custom rules that create the below
-    expect_lhs = set(['pos_arg', 'attribute'])
+    expect_lhs = set(["pos_arg", "attribute"])
     if PYTHON_VERSION < 3.8:
-        expect_lhs.add('get_iter')
+        expect_lhs.add("get_iter")
+    else:
+        expect_lhs.add("async_with_as_stmt")
+        expect_lhs.add("async_with_stmt")
 
+    unused_rhs = set(["list", "mkfunc", "mklambda", "unpack"])
 
-    unused_rhs = set(['list', 'mkfunc',
-                      'mklambda',
-                      'unpack',])
-
-    expect_right_recursive = set([('designList',
-                                   ('store', 'DUP_TOP', 'designList'))])
+    expect_right_recursive = set([("designList", ("store", "DUP_TOP", "designList"))])
 
     if PYTHON_VERSION < 3.7:
-        unused_rhs.add('call')
+        unused_rhs.add("call")
 
     if PYTHON_VERSION > 2.6:
-        expect_lhs.add('kvlist')
-        expect_lhs.add('kv3')
-        unused_rhs.add('dict')
+        expect_lhs.add("kvlist")
+        expect_lhs.add("kv3")
+        unused_rhs.add("dict")
 
     if PYTHON3:
-        expect_lhs.add('load_genexpr')
+        expect_lhs.add("load_genexpr")
 
-        unused_rhs = unused_rhs.union(set("""
+        unused_rhs = unused_rhs.union(
+            set(
+                """
         except_pop_except generator_exp
-        """.split()))
+        """.split()
+            )
+        )
         if PYTHON_VERSION >= 3.0:
             expect_lhs.add("annotate_arg")
             expect_lhs.add("annotate_tuple")
@@ -53,17 +57,19 @@ def test_grammar():
             unused_rhs.add("classdefdeco1")
             unused_rhs.add("tryelsestmtl")
             if PYTHON_VERSION >= 3.5:
-                expect_right_recursive.add((('l_stmts',
-                                             ('lastl_stmt', 'come_froms', 'l_stmts'))))
+                expect_right_recursive.add(
+                    (("l_stmts", ("lastl_stmt", "come_froms", "l_stmts")))
+                )
                 pass
             elif 3.0 < PYTHON_VERSION < 3.3:
-                expect_right_recursive.add((('l_stmts',
-                                             ('lastl_stmt', 'COME_FROM', 'l_stmts'))))
+                expect_right_recursive.add(
+                    (("l_stmts", ("lastl_stmt", "COME_FROM", "l_stmts")))
+                )
                 pass
             pass
         pass
     else:
-        expect_lhs.add('kwarg')
+        expect_lhs.add("kwarg")
 
     assert expect_lhs == set(lhs)
 
@@ -73,9 +79,16 @@ def test_grammar():
 
     assert expect_right_recursive == right_recursive
 
-    expect_dup_rhs = frozenset([('COME_FROM',), ('CONTINUE',), ('JUMP_ABSOLUTE',),
-                                ('LOAD_CONST',),
-                                ('JUMP_BACK',), ('JUMP_FORWARD',)])
+    expect_dup_rhs = frozenset(
+        [
+            ("COME_FROM",),
+            ("CONTINUE",),
+            ("JUMP_ABSOLUTE",),
+            ("LOAD_CONST",),
+            ("JUMP_BACK",),
+            ("JUMP_FORWARD",),
+        ]
+    )
     reduced_dup_rhs = dict((k, dup_rhs[k]) for k in dup_rhs if k not in expect_dup_rhs)
     for k in reduced_dup_rhs:
         print(k, reduced_dup_rhs[k])
@@ -83,7 +96,7 @@ def test_grammar():
 
     s = get_scanner(PYTHON_VERSION, IS_PYPY)
     ignore_set = set(
-            """
+        """
             JUMP_BACK CONTINUE
             COME_FROM COME_FROM_EXCEPT
             COME_FROM_EXCEPT_CLAUSE
@@ -92,22 +105,33 @@ def test_grammar():
             LOAD_GENEXPR LOAD_ASSERT LOAD_SETCOMP LOAD_DICTCOMP LOAD_STR LOAD_CODE
             LAMBDA_MARKER
             RETURN_END_IF RETURN_END_IF_LAMBDA RETURN_VALUE_LAMBDA RETURN_LAST
-            """.split())
+            """.split()
+    )
     if 2.6 <= PYTHON_VERSION <= 2.7:
         opcode_set = set(s.opc.opname).union(ignore_set)
         if PYTHON_VERSION == 2.6:
             opcode_set.add("THEN")
         check_tokens(tokens, opcode_set)
     elif PYTHON_VERSION == 3.4:
-        ignore_set.add('LOAD_CLASSNAME')
-        ignore_set.add('STORE_LOCALS')
+        ignore_set.add("LOAD_CLASSNAME")
+        ignore_set.add("STORE_LOCALS")
         opcode_set = set(s.opc.opname).union(ignore_set)
         check_tokens(tokens, opcode_set)
 
+
 def test_dup_rule():
     import inspect
-    python_parser(PYTHON_VERSION, inspect.currentframe().f_code,
-                  is_pypy=IS_PYPY,
-                  parser_debug={
-                      'dups': True, 'transition': False, 'reduce': False,
-                      'rules': False, 'errorstack': None, 'context': True})
+
+    python_parser(
+        PYTHON_VERSION,
+        inspect.currentframe().f_code,
+        is_pypy=IS_PYPY,
+        parser_debug={
+            "dups": True,
+            "transition": False,
+            "reduce": False,
+            "rules": False,
+            "errorstack": None,
+            "context": True,
+        },
+    )

pytest/test_pysource.py

@@ -8,8 +8,12 @@ from uncompyle6.semantics.consts import (
 
 if PYTHON3:
     from io import StringIO
+    def iteritems(d):
+        return d.items()
 else:
     from StringIO import StringIO
+    def iteritems(d):
+        return d.iteritems()
 
 from uncompyle6.semantics.pysource import (SourceWalker, deparse_code2str)
 
@@ -26,7 +30,7 @@ def test_template_engine():
     # FIXME: and so on...
 
 from uncompyle6.semantics.consts import (
-    TABLE_R, TABLE_DIRECT,
+    TABLE_DIRECT, TABLE_R,
     )
 
 from uncompyle6.semantics.fragments import (
@@ -40,7 +44,7 @@ def test_tables():
             (TABLE_DIRECT, 'TABLE_DIRECT', False),
             (TABLE_R, 'TABLE_R', False),
             (TABLE_DIRECT_FRAGMENT, 'TABLE_DIRECT_FRAGMENT', True)):
-        for k, entry in t.iteritems():
+        for k, entry in iteritems(t):
             if k in skip_for_now:
                 continue
             fmt = entry[0]

pytest/test_single_compile.py

@@ -1,4 +1,7 @@
+import pytest
 from uncompyle6 import PYTHON_VERSION, code_deparse
+pytestmark = pytest.mark.skip(PYTHON_VERSION < 2.7,
+                              reason="need at least Python 2.7")
 
 if PYTHON_VERSION > 2.6:
     def test_single_mode():

pytest/testdata/if-2.7.right

@@ -9,4 +9,4 @@
           12  JUMP_FORWARD          0  'to 15'
         15_0  COME_FROM            12  '12'
           15  LOAD_CONST               None
-          18  RETURN_VALUE
+          18  RETURN_VALUE     

pytest/testdata/ifelse-2.7.right

@@ -12,4 +12,4 @@
           18  STORE_NAME            2  'd'
         21_0  COME_FROM            12  '12'
           21  LOAD_CONST               None
-          24  RETURN_VALUE
+          24  RETURN_VALUE     

setup.cfg

@@ -1,6 +1,6 @@
 [bdist_rpm]
 release = 1
-packager = Mysterie <kajusska@gmail.com>
+packager = rocky <rb@dustyfeet.com>
 doc_files = README
 #		CHANGES.txt
 #		USAGE.txt
@@ -8,4 +8,4 @@ doc_files = README
 #		examples/
 
 [bdist_wheel]
-universal=1
+# universal=1

setup.py

@@ -1,16 +1,16 @@
 #!/usr/bin/env python
-"""Setup script for the 'uncompyle6' distribution."""
-
 import sys
 
+"""Setup script for the 'uncompyle6' distribution."""
+
 SYS_VERSION = sys.version_info[0:2]
-if not ((2, 4) <= SYS_VERSION  <= (2, 7)):
-    mess = "Python Release 2.4 .. 2.7 are supported in this code branch."
-    if ((3, 2) <= SYS_VERSION  <= (3, 8)):
-        mess += ("\nFor your Python, version %s, use the master code/branch." %
+if not ((2, 6) <= SYS_VERSION  <= (3, 8)):
+    mess = "Python Release 2.6 .. 3.8 are supported in this code branch."
+    if ((2, 4) <= SYS_VERSION <= (2, 7)):
+        mess += ("\nFor your Python, version %s, use the python-2.4 code/branch." %
                  sys.version[0:3])
-    else:
-        mess += ("\nThis package is not supported before Python 2.4. Your Python version is %s."
+    elif SYS_VERSION < (2, 4):
+        mess += ("\nThis package is not supported for Python version %s."
                  % sys.version[0:3])
     print(mess)
     raise Exception(mess)
@@ -32,6 +32,7 @@ setup(
        install_requires   = install_requires,
        license            = license,
        long_description   = long_description,
+       long_description_content_type = "text/x-rst",
        name               = modname,
        packages           = find_packages(),
        py_modules         = py_modules,

test-unit/test_grammar.py

@@ -1,55 +0,0 @@
-import re
-import unittest
-from uncompyle6 import PYTHON_VERSION, IS_PYPY # , PYTHON_VERSION
-from uncompyle6.parser import get_python_parser, python_parser
-
-class TestGrammar(unittest.TestCase):
-    def test_grammar(self):
-
-        def check_tokens(tokens, opcode_set):
-            remain_tokens = set(tokens) - opcode_set
-            remain_tokens = set([re.sub('_\d+$','', t) for t in remain_tokens])
-            remain_tokens = set([re.sub('_CONT$','', t) for t in remain_tokens])
-            remain_tokens = set(remain_tokens) - opcode_set
-            self.assertEqual(remain_tokens, set([]),
-                    "Remaining tokens %s\n====\n%s" % (remain_tokens, p.dump_grammar()))
-
-        p = get_python_parser(PYTHON_VERSION, is_pypy=IS_PYPY)
-        (lhs, rhs, tokens,
-         right_recursive, dup_rhs) = p.check_sets()
-        expect_lhs = set(['expr1024', 'pos_arg'])
-        unused_rhs = set(['list', 'call', 'mkfunc',
-                          'mklambda',
-                          'unpack',])
-
-        expect_right_recursive = frozenset([('designList',
-                                             ('store', 'DUP_TOP', 'designList'))])
-        expect_lhs.add('kwarg')
-
-        self.assertEqual(expect_lhs, set(lhs))
-        self.assertEqual(unused_rhs, set(rhs))
-        self.assertEqual(expect_right_recursive, right_recursive)
-
-        expect_dup_rhs = frozenset([('COME_FROM',), ('CONTINUE',), ('JUMP_ABSOLUTE',),
-                                    ('LOAD_CONST',),
-                                    ('JUMP_BACK',), ('JUMP_FORWARD',)])
-
-        reduced_dup_rhs = {}
-        for k in dup_rhs:
-            if k not in expect_dup_rhs:
-                reduced_dup_rhs[k] = dup_rhs[k]
-                pass
-            pass
-        for k in reduced_dup_rhs:
-            print(k, reduced_dup_rhs[k])
-        # assert not reduced_dup_rhs, reduced_dup_rhs
-
-    def test_dup_rule(self):
-        import inspect
-        python_parser(PYTHON_VERSION, inspect.currentframe().f_code,
-                      is_pypy=IS_PYPY,
-                      parser_debug={
-                          'dups': True, 'transition': False, 'reduce': False,
-                          'rules': False, 'errorstack': None, 'context': True})
-if __name__ == '__main__':
-    unittest.main()

test/Makefile

@@ -1,5 +1,6 @@
 PHONY=check clean dist distclean test test-unit test-functional rmChangeLog clean_pyc nosetests \
-      check-bytecode-1  check-bytecode-1.3 check-bytecode-1.4 check-bytecode-1.5 \
+      check-bytecode-1.0 check-bytecode-1.1 check-bytecode-1.2 check-bytecode-1.3 \
+      check-bytecode-1 check-bytecode-1.4 check-bytecode-1.5 check-bytecode-1.6 \
       check-bytecode-2 check-bytecode-3 check-bytecode-3-short \
       check-bytecode-2.2 check-byteocde-2.3 check-bytecode-2.4 \
       check-short check-2.6 check-2.7 check-3.0 check-3.1 check-3.2 check-3.3 \
@@ -29,7 +30,7 @@ check:
 	$(MAKE) check-$(PYTHON_VERSION)
 
 #: Run working tests from Python 2.6 or 2.7
-check-2.4 check-2.5 check-2.6 check-2.7: check-bytecode-2 check-bytecode-3 check-bytecode-1  check-native-short
+check-2.6 check-2.7: check-bytecode-2 check-bytecode-3 check-bytecode-1 check-native-short
 
 #: Run working tests from Python 3.0
 check-3.0: check-bytecode
@@ -71,10 +72,10 @@ check-3.7: check-bytecode
 	$(PYTHON) test_pythonlib.py --bytecode-3.7-run --verify-run
 	$(PYTHON) test_pythonlib.py --bytecode-3.7 --syntax-verify $(COMPILE)
 
-# #: Run working tests from Python 3.8
-# check-3.8: check-bytecode
-# 	$(PYTHON) test_pythonlib.py --bytecode-3.8-run --verify-run
-# 	$(PYTHON) test_pythonlib.py --bytecode-3.8 --syntax-verify $(COMPILE)
+#: Run working tests from Python 3.8
+check-3.8: check-bytecode
+	$(PYTHON) test_pythonlib.py --bytecode-3.8-run --verify-run
+	$(PYTHON) test_pythonlib.py --bytecode-3.8 --syntax-verify $(COMPILE)
 
 # FIXME
 #: this is called when running under pypy3.5-5.8.0 or pypy2-5.6.0
@@ -85,7 +86,7 @@ check-disasm:
 	$(PYTHON) dis-compare.py
 
 #: Check deparsing bytecode 1.x only
-check-bytecode-1: check-bytecode-1.4 check-bytecode-1.5
+check-bytecode-1: check-bytecode-1.0 check-bytecode-1.1 check-bytecode-1.2 check-bytecode-1.3 check-bytecode-1.4 check-bytecode-1.5 check-bytecode-1.6
 
 #: Check deparsing bytecode 2.x only
 check-bytecode-2:
@@ -98,8 +99,8 @@ check-bytecode-3:
 	$(PYTHON) test_pythonlib.py --bytecode-3.0 \
 	--bytecode-3.1 --bytecode-3.2 --bytecode-3.3 \
 	--bytecode-3.4 --bytecode-3.5 --bytecode-3.6 \
-        --bytecode-3.7 \
-	--bytecode-pypy3.2
+        --bytecode-3.7  \
+	--bytecode-pypy3.2 --bytecode-pypy3.6 --bytecode-3.8
 
 #: Check deparsing on selected bytecode 3.x
 check-bytecode-3-short:
@@ -109,6 +110,7 @@ check-bytecode-3-short:
 #: Check deparsing bytecode on all Python 2 and Python 3 versions
 check-bytecode: check-bytecode-3
 	$(PYTHON) test_pythonlib.py \
+	--bytecode-1.0 --bytecode-1.1 --bytecode-1.2 \
 	--bytecode-1.3 --bytecode-1.4 --bytecode-1.5 \
         --bytecode-2.2 --bytecode-2.3 --bytecode-2.4 \
 	--bytecode-2.1 --bytecode-2.2 --bytecode-2.3 --bytecode-2.4 \
@@ -122,6 +124,18 @@ check-bytecode-short: check-bytecode-3-short
         --bytecode-2.6 --bytecode-2.7 --bytecode-pypy2.7
 
 
+#: Check deparsing bytecode 1.0 only
+check-bytecode-1.0:
+	$(PYTHON) test_pythonlib.py --bytecode-1.0
+
+#: Check deparsing bytecode 1.1 only
+check-bytecode-1.1:
+	$(PYTHON) test_pythonlib.py --bytecode-1.1
+
+#: Check deparsing bytecode 1.2 only
+check-bytecode-1.2:
+	$(PYTHON) test_pythonlib.py --bytecode-1.2
+
 #: Check deparsing bytecode 1.3 only
 check-bytecode-1.3:
 	$(PYTHON) test_pythonlib.py --bytecode-1.3
@@ -134,6 +148,10 @@ check-bytecode-1.4:
 check-bytecode-1.5:
 	$(PYTHON) test_pythonlib.py --bytecode-1.5
 
+#: Check deparsing bytecode 1.6 only
+check-bytecode-1.6:
+	$(PYTHON) test_pythonlib.py --bytecode-1.6
+
 #: Check deparsing Python 2.1
 check-bytecode-2.1:
 	$(PYTHON) test_pythonlib.py --bytecode-2.1
@@ -148,7 +166,6 @@ check-bytecode-2.3:
 
 #: Check deparsing Python 2.4
 check-bytecode-2.4:
-	$(PYTHON) test_pythonlib.py --bytecode-2.4-run --verify-run
 	$(PYTHON) test_pythonlib.py --bytecode-2.4
 
 #: Check deparsing Python 2.5
@@ -277,20 +294,16 @@ check-bytecode-3.7:
 	$(PYTHON) test_pythonlib.py --bytecode-3.7-run --verify-run
 	$(PYTHON) test_pythonlib.py --bytecode-3.7 --syntax-verify
 
-# #: Check deparsing Python 3.8
-# check-bytecode-3.8:
-# 	$(PYTHON) test_pythonlib.py --bytecode-3.8-run --verify-run
-# 	$(PYTHON) test_pythonlib.py --bytecode-3.8 --syntax-verify
+#: Check deparsing Python 3.8
+check-bytecode-3.8:
+	$(PYTHON) test_pythonlib.py --bytecode-3.8-run --verify-run
+	$(PYTHON) test_pythonlib.py --bytecode-3.8 --syntax-verify
 
 #: short tests for bytecodes only for this version of Python
 check-native-short:
 	$(PYTHON) test_pythonlib.py --bytecode-$(PYTHON_VERSION) --syntax-verify $(COMPILE)
 	$(PYTHON) test_pythonlib.py --bytecode-$(PYTHON_VERSION)-run --verify-run $(COMPILE)
 
-#: Run longer Python 2.6's lib files known to be okay
-check-2.4-ok:
-	$(PYTHON) test_pythonlib.py --ok-2.4 --verify $(COMPILE)
-
 #: Run longer Python 2.6's lib files known to be okay
 check-2.6-ok:
 	$(PYTHON) test_pythonlib.py --ok-2.6 --syntax-verify $(COMPILE)
@@ -319,8 +332,16 @@ pypy-2.7 5.0 5.3 6.0:
 pypy-3.2 2.4:
 	$(PYTHON) test_pythonlib.py --bytecode-pypy3.2 --verify
 
-#: PyPy 5.0.x with Python 3.6 ...
+#: PyPy 5.0.x with Python 3.6.1 ...
+check-bytecode-pypy3.6: 7.1
 7.1:
+	$(PYTHON) test_pythonlib.py --bytecode-pypy3.6-run --verify-run
+	$(PYTHON) test_pythonlib.py --bytecode-pypy3.6 --verify
+
+#: PyPy 5.0.x with Python 3.6.9
+check-bytecode-pypy3.6: 7.2
+7.2:
+	$(PYTHON) test_pythonlib.py --bytecode-pypy3.6-run --verify-run
 	$(PYTHON) test_pythonlib.py --bytecode-pypy3.6 --verify
 
 

test/add-test.py

@@ -2,22 +2,23 @@
 """ Trivial helper program to bytecompile and run an uncompile
 """
 import os, sys, py_compile
+
 assert len(sys.argv) >= 2
 version = sys.version[0:3]
-if sys.argv[1] == '--run':
-    suffix = '_run'
+if sys.argv[1] in ("--run", "-r"):
+    suffix = "_run"
     py_source = sys.argv[2:]
 else:
-    suffix = ''
+    suffix = ""
     py_source = sys.argv[1:]
 
 for path in py_source:
     short = os.path.basename(path)
-    if hasattr(sys, 'pypy_version_info'):
-        cfile =  "bytecode_pypy%s%s/%s" % (version, suffix, short) + 'c'
+    if hasattr(sys, "pypy_version_info"):
+        cfile = "bytecode_pypy%s%s/%s" % (version, suffix, short) + "c"
     else:
-        cfile =  "bytecode_%s%s/%s" % (version, suffix, short) + 'c'
+        cfile = "bytecode_%s%s/%s" % (version, suffix, short) + "c"
     print("byte-compiling %s to %s" % (path, cfile))
     py_compile.compile(path, cfile)
     if isinstance(version, str) or version >= (2, 6, 0):
-        os.system("../bin/uncompyle6 -a -t %s" % cfile)
+        os.system("../bin/uncompyle6 -a -T %s" % cfile)

test/dis-compare.py

@@ -1,8 +1,9 @@
 #!/usr/bin/env python
 # Mode: -*- python -*-
 #
-# Copyright (c) 2015, 2017 by Rocky Bernstein <rb@dustyfeet.com>
+# Copyright (c) 2015 by Rocky Bernstein <rb@dustyfeet.com>
 #
+from __future__ import print_function
 
 
 import dis, os.path

test/simple-uncompyle-code-test.py

@@ -1,6 +1,7 @@
 #!/usr/bin/env python
 
-from uncompyle6 import uncompyle
+from __future__ import print_function
+
 from uncompyle6.main import decompile
 from xdis.magics import sysinfo2float
 import sys, inspect

test/simple_source/bug25/02_yield_bug.py

@@ -1,7 +0,0 @@
-# From 2.4 test_array.py
-# In Python 2.4 and earlier "yield" is not valid and instead
-# we must use "yield None". Bug was not adding "None"
-
-def yield_bug():
-    yield None
-    return

test/simple_source/bug25/04_try_else_confuse.py

@@ -1,19 +0,0 @@
-# From 2.4 test_sax.py
-# Bug was distinguishing try from try/else
-
-def verify_empty_attrs():
-    gvqk = 3
-    try:
-        gvk = 1/0
-    except ZeroDivisionError:
-        gvk = 1
-
-    try:
-        gvqk = 0
-    except KeyError:
-        gvqk = 1
-
-    # If try/else was used above the return will be 4
-    return gvk + gvqk
-
-assert 1 == verify_empty_attrs()

test/simple_source/bug30/02_try_except_except.py

@@ -7,3 +7,13 @@ def start_new_thread(function, args, kwargs={}):
         pass
     except:
         args()
+
+# Adapted from 3.0.1 code.py
+# Bug is again JUMP_FORWARD elimination compared
+# to earlier and later Pythons.
+def interact():
+    while 1:
+        try:
+            more = 1
+        except KeyboardInterrupt:
+            more = 0

test/simple_source/bug30/02_while1_if_while1.py

@@ -7,3 +7,8 @@ while 1:
             raise RuntimeError
     else:
         raise RuntimeError
+
+# Adapted from 3.0.1 cgi.py
+def _parseparam(s, end):
+    while end > 0 and s.count(''):
+        end = s.find(';')

test/simple_source/bug30/03_ifelse.py

@@ -0,0 +1,50 @@
+# Adapted from 3.0 base64
+# Problem was handling if/else which
+# needs to be like Python 2.6 (and not like 2.7 or 3.1)
+def main(args, f, func, sys):
+    """Small main program"""
+    if args and args[0] != '-':
+        func(f, sys.stdout.buffer)
+    else:
+        func(sys.stdin.buffer, sys.stdout.buffer)
+
+# From Python 3.0 _markupbase.py.
+#
+# The Problem was in the way "if"s are generated in 3.0 which is sort
+# of like a more optimized Python 2.6, with reduced extraneous jumps,
+# but still 2.6-ish and not 2.7- or 3.1-ish.
+def parse_marked_section(fn, i, rawdata, report=1):
+    if report:
+        j = 1
+        fn(rawdata[i: j])
+    return 10
+
+# From 3.0.1 _abcoll.py
+# Bug was in genexpr_func which doesn't have a JUMP_BACK but
+# in its gen_comp_body, we can use COME_FROM in its place.
+# As above omission of JUMPs is a feature of 3.0 that doesn't
+# seem to be in later versions (or earlier like 2.6).
+def __and__(self, other, Iterable):
+    if not isinstance(other, Iterable):
+        return NotImplemented
+    return self._from_iterable(value for value in other if value in self)
+
+# Adapted from 3.0.1 abc.py
+# Bug was in handling multiple COME_FROMs in return_if_stmt
+def __instancecheck__(subtype, subclass, cls):
+    if subtype:
+        if (cls and subclass):
+            return False
+
+
+# Adapted from 3.0.1 abc.py
+# Bug was rule in "jump_absolute_else" and disallowing
+# "else" to the wrong place.
+
+def _strptime(locale_time, found_zone, time):
+    for tz_values in locale_time:
+        if found_zone:
+            if (time and found_zone):
+                break
+            else:
+                break