Get ready for release 3.6.3

fixa

NEWS.md

@@ -1,3 +1,66 @@
+3.6.3: 2020-1-26 Martin and Susanne
+===================================
+
+Of late, every release fixes major gaps and embarrassments of the last release....
+
+And in some cases, like this one, exposes lacuna and rot.
+
+I now have [control] flow under control, even if it isn't the most optimal way.
+
+I now have greatly expanded automated testing.
+
+On the most recent Python versions I regularly decompile thousands of Python programs that are distributed with Python.  when it is possible, I then decompile Python's standard test suite distributed with Python and run the decompiled source code which basically checks itself. This amounts to about 250 test programs per version. This is in addition to the 3 CI testing services which do different things.
+
+Does this mean the decompiler works perfectly? No. There are still a dozen or so failing programs, although the actual number of bugs is probably smaller though.
+
+However, in perparation of a more major refactoring of the parser grammar, this release was born.
+
+In many cases, decompilation is better. But there are some cases where decompilation has gotten worse. For lack of time (and interest) 3.0 bytecode suffered a hit. Possibly some code in the 3.x range did too. In time and with cleaner refactored code, this will come back.
+
+Commit c90ff51 was a local maxiumum before, I started reworking the grammar to separate productions that were specific to loops versus those that are not in loops.
+In the middle of that I added another grammar simplication to remove singleton productions of the form `sstmts-> stmts`. These were always was a bit ugly, and complicated output.
+
+At any rate if decompilation fails, you can try c90ff51. Or another decompiler. `unpyc37` is pretty good for 3.7. wibiti `uncompyle2` is great for 2.7. `pycdc` is mediocre for Python before 3.5 or so, and not that good for the most recent Python. Geerally these programs will give some sort of answer even if it isn't correct.
+
+decompyle3 isn't that good for 3.7 and worse for 3.8, but right now it does things no other Python decompiler like `unpyc37` or `pycdc` does. For example, `decompyle3` handles variable annotations. As always, the issue trackers for the various programs will give you a sense for what needs to be done. For now, I've given up on reporting issues in the other decompilers because there are already enough issues reported, and they are just not getting fixed anyway.
+
+
+3.6.2: 2020-1-5 Samish
+======================
+
+Yet again the focus has been on just fixing bugs, mostly geared in the
+later 3.x range. To get some sense what sill needs fixing, consult
+test/stdlib/runtests.sh. And that only has a portion of what's known.
+
+`make_function.py` has gotten so complex that it was split out into 3 parts
+to handle different version ranges: Python <3, Python 3.0..3.6 and Python 3.7+.
+
+An important fix is that we had been dropping docstrings in Python 3 code as a result
+of a incomplete merge from the decompile3 base with respect to the transform phase.
+
+Also important (at least to me) is that we can now handle 3.6+
+variable type annotations.  Some of the decompile3 code uses that in
+its source code, and I now use variable annotations in conjunction
+with mypy in some of my other Python projects
+
+Code generation for imports, especially where the import is dotted
+changed a bit in 3.7; with this release are just now tracking that
+change better. For this I've added pseudo instruction
+`IMPORT_NAME_ATTR`, derived from the `IMPORT_NAME` instruction, to
+indicate when an import contains a dotted import. Similarly, code for
+3.7 `import .. as ` is basically the same as `from .. import`, the
+only difference is the target of the name changes to an "alias" in the
+former. As a result, the disambiguation is now done on the semantic
+action side, rathero than in parsing grammar rules.
+
+Some small specific fixes:
+
+* 3.7+ some chained compare parsing has been fixed. Other remain.
+* better if/else rule checking in the 3.4 and below range.
+* 3.4+ keyword-only parameter handling was fixed more generally
+* 3.3 .. 3.5 keyword-only parameter args in lambda was fixed
+
+
 3.6.1: 2019-12-10 Christmas Hannukah
 ====================================
 
@@ -190,8 +253,7 @@ Lots of decomplation bugs, especially in the 3.x series fixed. Don't worry thoug
 3.3.0 2019-04-14 Holy Week
 ==========================
 
-* First cut at Python 3.8 (many bugs remain)
-* Reinstate -c | --compile (compile before disassembly) option
+* First cut at Python 3.8 (many bug remain)
 * The usual smattering of bug and doc fixes
 
 3.2.6 2019-03-23 Mueller Report

README.rst

@@ -232,7 +232,7 @@ See Also
 * https://github.com/rocky/python-xdis : Cross Python version disassembler
 * https://github.com/rocky/python-xasm : Cross Python version assembler
 * https://github.com/rocky/python-uncompyle6/wiki : Wiki Documents which describe the code and aspects of it in more detail
-* https://github.com/zrax/pycdc : The README for this C++ code syas it aims to support all versions of Python. It is best for Python versions around 2.7 and 3.3 when the code was initially developed. Accuracy for current versions of Python3 and early versions of Python is lacking. Without major effort, it is unlikely it can be made to support current Python 3. See its `issue tracker <https://github.com/zrax/pycdc/issues>`_ for details. Currently lightly maintained.
+* https://github.com/zrax/pycdc : The README for this C++ code says it aims to support all versions of Python. It is best for Python versions around 2.7 and 3.3 when the code was initially developed. Accuracy for current versions of Python3 and early versions of Python is lacking. Without major effort, it is unlikely it can be made to support current Python 3. See its `issue tracker <https://github.com/zrax/pycdc/issues>`_ for details. Currently lightly maintained.
 
 
 .. _trepan: https://pypi.python.org/pypi/trepan2g

admin-tools/README.md

@@ -8,4 +8,15 @@ They are customized to my environment:
 - I have git repos for xdis, and spark parser at the same level as uncompyle6
 
 There may be other rocky-specific things that need customization.
-how-to-make-a-release.txt has overall how I make a release
+how-to-make-a-release.md has overall how I make a release
+
+Since this project uses python over a wide variety of release, some versions
+of projects that should be used for specific Python versions
+
+for 3.2.6:
+   pytest==2.9.2
+
+for 3.1.5
+   pytset==2.1.0
+   py=1.8.0 and comment out line 10 of _builtin.py # callable = callable
+   six==1.10.0

admin-tools/pyenv-newer-versions

@@ -5,4 +5,4 @@ if [[ $0 == ${BASH_SOURCE[0]} ]] ; then
     echo "This script should be *sourced* rather than run directly through bash"
     exit 1
 fi
-export PYVERSIONS='3.5.9 3.6.9 2.6.9 3.3.7 2.7.17 3.2.6 3.1.5 3.4.8 3.7.6 3.8.1'
+export PYVERSIONS='3.5.9 3.6.10 2.6.9 3.3.7 2.7.17 3.2.6 3.1.5 3.4.10 3.7.6 3.8.1'

admin-tools/setup-master.sh

@@ -1,5 +1,5 @@
 #!/bin/bash
-PYTHON_VERSION=3.7.5
+PYTHON_VERSION=3.7.6
 
 # FIXME put some of the below in a common routine
 function finish {

pytest/test_grammar.py

@@ -27,6 +27,9 @@ def test_grammar():
             expect_lhs.add("attribute")
 
         expect_lhs.add("get_iter")
+
+        if PYTHON_VERSION > 3.7 or PYTHON_VERSION < 3.0:
+            expect_lhs.add("stmts_opt")
     else:
         expect_lhs.add("async_with_as_stmt")
         expect_lhs.add("async_with_stmt")
@@ -35,7 +38,7 @@ def test_grammar():
 
     expect_right_recursive = set([("designList", ("store", "DUP_TOP", "designList"))])
 
-    if PYTHON_VERSION <= 3.7:
+    if PYTHON_VERSION <= 3.6:
         unused_rhs.add("call")
 
     if PYTHON_VERSION > 2.6:
@@ -43,6 +46,10 @@ def test_grammar():
         expect_lhs.add("kv3")
         unused_rhs.add("dict")
 
+        if PYTHON_VERSION < 3.7 and PYTHON_VERSION != 2.7:
+            # NOTE: this may disappear
+            expect_lhs.add("except_handler_else")
+
     if PYTHON3:
         expect_lhs.add("load_genexpr")
 

pytest/test_token.py

@@ -1,24 +1,25 @@
 from uncompyle6 import PYTHON_VERSION
 from uncompyle6.scanners.tok import Token
 
+
 def test_token():
     # Test token formatting of: LOAD_CONST None
-    t = Token('LOAD_CONST', offset=0, attr=None, pattr=None, has_arg=True)
-    expect = '           0  LOAD_CONST               None'
+    t = Token("LOAD_CONST", offset=0, attr=None, pattr=None, has_arg=True)
+    expect = "             0  LOAD_CONST               None"
     # print(t.format())
     assert t
     assert t.format() == expect
 
     # Make sure equality testing of tokens ignores offset
-    t2 = Token('LOAD_CONST', offset=2, attr=None, pattr=None, has_arg=True)
+    t2 = Token("LOAD_CONST", offset=2, attr=None, pattr=None, has_arg=True)
     assert t2 == t
 
-
     # Make sure formatting of: LOAD_CONST False. We assume False is the 0th index
     # of co_consts.
-    t = Token('LOAD_CONST', offset=1, attr=False, pattr=False, has_arg=True)
-    expect = '           1  LOAD_CONST               False'
+    t = Token("LOAD_CONST", offset=1, attr=False, pattr=False, has_arg=True)
+    expect = "             1  LOAD_CONST               False"
     assert t.format() == expect
 
-if __name__ == '__main__':
+
+if __name__ == "__main__":
     test_token()

pytest/testdata/if-2.7.right

@@ -1,12 +1,12 @@
 # Python 2.7
 # Embedded file name: simple_source/branching/05_if.py
 
-   6       0  LOAD_NAME             0  'True'
-           3  POP_JUMP_IF_FALSE    15  'to 15'
+   6         0  LOAD_NAME             0  'True'
+             3  POP_JUMP_IF_FALSE    15  'to 15'
 
-   7       6  LOAD_NAME             1  'False'
-           9  STORE_NAME            2  'b'
-          12  JUMP_FORWARD          0  'to 15'
-        15_0  COME_FROM            12  '12'
-          15  LOAD_CONST               None
-          18  RETURN_VALUE     
+   7         6  LOAD_NAME             1  'False'
+             9  STORE_NAME            2  'b'
+            12  JUMP_FORWARD          0  'to 15'
+          15_0  COME_FROM            12  '12'
+            15  LOAD_CONST               None
+            18  RETURN_VALUE     

pytest/testdata/ifelse-2.7.right

@@ -1,15 +1,15 @@
 # Python 2.7
 # Embedded file name: simple_source/branching/05_ifelse.py
 
-   3       0  LOAD_NAME             0  'True'
-           3  POP_JUMP_IF_FALSE    15  'to 15'
+   3         0  LOAD_NAME             0  'True'
+             3  POP_JUMP_IF_FALSE    15  'to 15'
 
-   4       6  LOAD_CONST               1
-           9  STORE_NAME            1  'b'
-          12  JUMP_FORWARD          6  'to 21'
+   4         6  LOAD_CONST               1
+             9  STORE_NAME            1  'b'
+            12  JUMP_FORWARD          6  'to 21'
 
-   6      15  LOAD_CONST               2
-          18  STORE_NAME            2  'd'
-        21_0  COME_FROM            12  '12'
-          21  LOAD_CONST               None
-          24  RETURN_VALUE     
+   6        15  LOAD_CONST               2
+            18  STORE_NAME            2  'd'
+          21_0  COME_FROM            12  '12'
+            21  LOAD_CONST               None
+            24  RETURN_VALUE     

test/.gitignore

@@ -1 +1,2 @@
+/.coverage
 /nohup.out

test/run-and-email.sh

@@ -14,27 +14,47 @@ function displaytime {
     printf '%d seconds\n' $S
 }
 
-PYVERSION=${PYVERSION:-"3.5.5 2.7.14 3.2.6 3.3.7 3.4.8 2.6.9 3.6.4"}
-# PYVERSION=${PYVERSION:-"3.5.5"}
+. ../admin-tools/pyenv-newer-versions
 
 USER=${USER:-rocky}
 EMAIL=${EMAIL:-rb@dustyfeet.com}
 MAX_TESTS=${MAX_TESTS:-800}
 typeset -i RUN_STARTTIME=$(date +%s)
 
-for VERSION in $PYVERSION ; do
+# PYVERSIONS="3.5.6"
+actual_versions=""
+for VERSION in $PYVERSIONS ; do
     typeset -i rc=0
     LOGFILE=/tmp/pyenvlib-$VERSION-$$.log
 
-    if [[ $VERSION == '3.5.5' ]] ; then
-	MAX_TESTS=224
-    elif [[ $VERSION == '3.2.6' ]] ; then
-	MAX_TESTS=700
-    elif [[ $VERSION == '3.6.4' ]] ; then
-	MAX_TESTS=400
-    else
-	MAX_TESTS=800
-    fi
+    case "$VERSION" in
+	3.7.6 | 3.8.1 | 3.1.5 | 3.0.1 )
+	    continue
+	    ;;
+	3.5.9 )
+	    MAX_TESTS=900
+	    ;;
+	3.2.6 )
+	    MAX_TESTS=900
+	    ;;
+	3.3.7 )
+	    MAX_TESTS=1300 # About 1256 exist
+	    ;;
+	3.4.10 )
+	    MAX_TESTS=800
+	    ;;
+	3.6.10 )
+	    MAX_TESTS=1300  # about 2139 exist
+	    ;;
+	2.6.9 )
+	    MAX_TESTS=1300
+	    ;;
+	* )
+	     MAX_TESTS=800
+	     ;;
+    esac
+
+    actual_versions="$actual_versions $VERSION"
 
     if ! pyenv local $VERSION ; then
 	rc=1
@@ -42,7 +62,9 @@ for VERSION in $PYVERSION ; do
       echo Python Version $(pyenv local) > $LOGFILE
       echo "" >> $LOGFILE
       typeset -i ALL_FILES_STARTTIME=$(date +%s)
-      python ./test_pyenvlib.py --max ${MAX_TESTS} --syntax-verify --$VERSION  >>$LOGFILE 2>&1
+      cmd="python ./test_pyenvlib.py --max ${MAX_TESTS} --syntax-verify --$VERSION"
+      echo "$cmd" >>$LOGFILE 2>&1
+      $cmd >>$LOGFILE 2>&1
       rc=$?
 
       echo Python Version $(pyenv local) >> $LOGFILE
@@ -66,4 +88,4 @@ done
 typeset -i RUN_ENDTIME=$(date +%s)
 (( time_diff =  RUN_ENDTIME - RUN_STARTTIME))
 elapsed_time=$(displaytime $time_diff)
-echo "Run complete $elapsed_time for versions $PYVERSION" | mail -s "pyenv weak verify in $elapsed_time" ${EMAIL}
+echo "Run complete $elapsed_time for versions $actual_versions" | mail -s "pyenv weak verify in $elapsed_time" ${EMAIL}

test/simple_source/bug26/04_if_and_bug.py

@@ -0,0 +1,52 @@
+def foo():
+    if julian == -1 and week_of_year != -1 and weekday != -1:
+        first_weekday = datetime_date(year, 1, 1).weekday()
+        preceeding_days = 7 - first_weekday
+        if preceeding_days == 7:
+            preceeding_days = 0
+        if weekday == 6 and week_of_year_start == 6:
+            week_of_year -= 1
+        if weekday == 0 and first_weekday == 0 and week_of_year_start == 6:
+            week_of_year += 1
+        if week_of_year == 0:
+            julian = 1 + weekday - first_weekday
+        else:
+            days_to_week = preceeding_days + 7 * (week_of_year - 1)
+            julian = 1 + days_to_week + weekday
+
+
+# 2.6 pstats.py
+# Bug is handling "for" with "elif" and "and"s.
+def eval_print_amount(a, b, c, d, list, msg=0):
+    if a:
+        for i in list:
+            msg = 1
+    elif b and c:
+        msg = 2
+    elif c and d:
+        msg = 3
+    return msg
+
+assert eval_print_amount(True, False, False, False, [1]) == 1
+assert eval_print_amount(True, False, False, False, []) == 0
+assert eval_print_amount(False, True, True, False, []) == 2
+assert eval_print_amount(False, False, True, True, []) == 3
+assert eval_print_amount(False, False, False, True, []) == 0
+
+
+# Bug in 2.6 was in including the part at x = value
+# at the end asa part of the "else"
+def eval_directive(a):
+    if a:
+        value = 2
+    else:
+        try:
+            value = 3
+        except:
+            pass
+
+    x = value
+    return x
+
+assert eval_directive(True)  == 2
+assert eval_directive(False) == 3

test/simple_source/bug30/06_listcomp.py

@@ -22,3 +22,22 @@ def columnize(l):
     return [i for i in range(len(l))
             if not isinstance(l[i], str)]
 assert [0, 2] == columnize([1, 'a', 2])
+
+# From 3.7.6 _collections_abc.py
+# Bug was handling "or" in listcomp
+def count(values, x):
+    return sum(1 for v in values if v or x)
+
+assert count([2, 2], False) == 2
+assert count([], False) == 0
+assert count([], True) == 0
+assert count([2], True) == 1
+assert count([0], False) == 0
+
+# From 3.7 test_generators
+# Bug was in handling the way list_if is optimized in 3.7+;
+# We need list_if37 and compare_chained37.
+def init_board(c):
+    return [io for io in c if 3 <= io < 5]
+
+assert init_board(list(range(6))) == [3, 4]

test/simple_source/bug31/04_def_annotate.py

@@ -150,3 +150,12 @@ ann2(1)
 assert test12(1, 2, 3, name='hi') == (1, (2, 3)), "a, *args, name"
 assert test13(1, 2, 3, name='hi') == ((1, 2, 3), 'hi'), "*args, name"
 assert test16('localhost', loop=2, limit=3, a='b') == ('localhost', None, 2, 3, {'a': 'b'})
+
+# From test 3.5 test_pydoc.py.
+# Bug was in 3.5 and earlier handling of the return type, typing.Tuple[...]
+try:
+    import typing
+    def foo() -> typing.Iterator[typing.Tuple[int, typing.Any]]:
+        ...
+except:
+    pass

test/simple_source/bug31/10_complex.py

@@ -1,4 +1,5 @@
 # Greatly simplified from from 3.3 test_complex.py
+from math import atan2
 
 # RUNNABLE!
 def assertCloseAbs(x, y, eps=1e-09):
@@ -38,14 +39,16 @@ def test_truediv():
             check_div(x, y)
 
 def test_plus_minus_0j():
-    z1, z2 = (0j, (-0 - 0j))
+    assert -0j == -0j == complex(0.0, 0.0)
+    assert -0-0j == -0j == complex(0.0, 0.0)
+    z1, z2 = (0j, -0j)
     assert atan2(z1.imag, -1.0) == atan2(0.0, -1.0)
-    assert atan2(z2.imag, -1.0), atan2(-0.0, -1.0)
+    # assert atan2(z2.imag, -1.0), atan2(-0.0, -1.0)
 
 # Check that we can handle -inf, and inf as a complex numbers.
 # And put it in a tuple and a list to make it harder.
 z1, z2 = (-1e1000j, 1e1000j)
 assert z1 in [-1e1000j, 1e1000j]
-assert z1 == z2
+assert z1 != z2
 test_truediv()
-test_plus_minus0j()
+test_plus_minus_0j()

test/simple_source/bug33/01_triple_compare.py

@@ -14,3 +14,23 @@ def _is_valid_netmask(netmask):
 
 # See in 2.6.9 quopri.py ishex():
 assert not '0' <= __file__ <= '9' or 'a' <= __file__ <= 'f' or 'A' <= __file__ <= 'F'
+
+# From 3.7 bug-grammar.py
+
+# Bug in 3.7 was handling the last line where compare_chained -> compare_chained37 and
+# therefore compare_chained has one child, not two as it normally does.
+
+def test_comparison():
+    ### comparison: expr (comp_op expr)*
+    ### comp_op: '<'|'>'|'=='|'>='|'<='|'!='|'in'|'not' 'in'|'is'|'is' 'not'
+    if 1: pass
+    x = (1 == 1)
+    if 1 == 1: pass
+    if 1 != 1: pass
+    if 1 < 1: pass
+    if 1 > 1: pass
+    if 1 <= 1: pass
+    if 1 >= 1: pass
+    if 1 in (): pass
+    if 1 not in (): pass
+    if 1 < 1 > 1 == 1 >= 1 <= 1 != 1 in 1 not in 1 is 1 is not 1: pass

test/simple_source/bug33/08_if_else.py

@@ -8,3 +8,33 @@ def init(modules=None):
 
 assert init() == set()
 assert init([1, 2, 3]) == set([1, 2, 3])
+
+# From 3.6 sre_parse
+# Bug was in handling multple COME_FROMS from nested if's
+def _escape(a, b, c, d, e):
+    if a:
+        if b:
+            if c:
+                if d:
+                    raise
+                return
+        if e:
+            if d:
+                raise
+            return
+        raise
+
+assert _escape(False, True, True,  True,  True) is None
+assert _escape(True,  True, True,  False, True) is None
+assert _escape(True,  True, False, False, True) is None
+
+for args in (
+        (True, True,  True, False, True),
+        (True, False, True, True,  True),
+        (True, False, True, True,  False),
+        ):
+    try:
+        _escape(*args)
+        assert False, args
+    except:
+        pass

test/simple_source/bug35/04_call_function.py

@@ -79,3 +79,7 @@ class ResultMixin(object):
 
 class SplitResult(namedtuple('SplitResult', 'scheme netloc path query fragment'), ResultMixin):
     pass
+
+# From 3.3.7 test_long.py
+# Bug was that we need parens around first "0"
+assert (0).bit_length() == 0

test/simple_source/bug35/04_importlist.py

@@ -1,7 +1,19 @@
+# From 3.7 test_cmath.py
 # Had bug in 3.x in not having semantic importlist rule
-def main(osp, Mfile, mainpyfile, dbg=None):
-    try:
-        from xdis import load_module, PYTHON_VERSION, IS_PYPY
-        return PYTHON_VERSION, IS_PYPY, load_module
-    except:
-        pass
+# bug is treating "import as" as "from xx import" while
+# still being able to hand "from xx import" properly
+
+# RUNNABLE!
+import os.path as osp
+from sys import platform
+from os import sep, name
+import collections.abc
+
+assert osp.basename("a") == "a"
+
+assert isinstance(platform, str)
+assert sep
+assert name
+assert collections.abc
+import os.path as path
+assert path

test/simple_source/bug36/01_conditional.py

@@ -0,0 +1,21 @@
+# From 3.7.6 test_buffer.py
+
+# RUNNABLE!
+def foo(n):
+    zero_stride = True if n >= 95 and n & 1 else False
+    return zero_stride
+
+assert foo(95)
+assert not foo(94)
+assert not foo(96)
+
+# from test_buffer.py
+# Bug was handling "or" inside a conditional
+def rslice(a, b):
+    minlen = 0 if a or b else 1
+    return minlen
+
+assert rslice(False, False) == 1
+assert rslice(False, True) == 0
+assert rslice(True, False) == 0
+assert rslice(True, True) == 0

test/simple_source/bug36/01_extended_arg.py

@@ -1,2 +1,82 @@
 if __file__:
     0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0
+
+
+# From 3.7 test_buffer.py
+# Bug is in dealing with EXTENDED_ARG instructions.
+# In reduction-rule tests where we are testing the offset,
+# getting *which* offset to test against, when there are two
+# possible offset, can mess us up.
+
+def five(a):
+    return 5
+
+def test_ndarray_slice_multidim(a, f, listerr):
+    for slices in a:
+        shape_t = 0; shape_t = 1; shape_t = 2; shape_t = 3; shape_t = 4; shape_t = 5; shape_t = 6; shape_t = 7; shape_t = 8; shape_t = 9
+        shape_t = 0; shape_t = 1; shape_t = 2; shape_t = 3; shape_t = 4; shape_t = 5; shape_t = 6; shape_t = 7; shape_t = 8; shape_t = 9
+        shape_t = 0; shape_t = 1; shape_t = 2; shape_t = 3; shape_t = 4; shape_t = 5; shape_t = 6; shape_t = 7; shape_t = 8; shape_t = 9
+        shape_t = 0; shape_t = 1; shape_t = 2; shape_t = 3; shape_t = 4; shape_t = 5; shape_t = 6; shape_t = 7; shape_t = 8; shape_t = 9
+        shape_t = 0; shape_t = 1; shape_t = 2; shape_t = 3; shape_t = 4; shape_t = 5; shape_t = 6; shape_t = 7; shape_t = 8; shape_t = 9
+        nderr = None
+        if nderr or listerr:
+            return f(5)
+        else:
+            return 2
+
+assert test_ndarray_slice_multidim([1], five, False) == 2
+assert test_ndarray_slice_multidim([1], five, True) == 5
+
+# From 3.7 test_builtin.py
+def test_pow(self, m, a, b, c, f):
+
+    shape_t = 0; shape_t = 1; shape_t = 2; shape_t = 3; shape_t = 4; shape_t = 5; shape_t = 6; shape_t = 7; shape_t = 8; shape_t = 9
+    shape_t = 0; shape_t = 1; shape_t = 2; shape_t = 3; shape_t = 4; shape_t = 5; shape_t = 6; shape_t = 7; shape_t = 8; shape_t = 9
+    shape_t = 0; shape_t = 1; shape_t = 2; shape_t = 3; shape_t = 4; shape_t = 5; shape_t = 6; shape_t = 7; shape_t = 8; shape_t = 9
+    shape_t = 0; shape_t = 1; shape_t = 2; shape_t = 3; shape_t = 4; shape_t = 5; shape_t = 6; shape_t = 7; shape_t = 8; shape_t = 9
+    shape_t = 0; shape_t = 1; shape_t = 2; shape_t = 3; shape_t = 4; shape_t = 5; shape_t = 6; shape_t = 7; shape_t = 8; shape_t = 9
+    shape_t = 0; shape_t = 1; shape_t = 2; shape_t = 3; shape_t = 4; shape_t = 5; shape_t = 6; shape_t = 7; shape_t = 8; shape_t = 9
+
+    shape_t = 0; shape_t = 1; shape_t = 2;
+
+    for z in m:
+        if a or \
+           b or \
+           c:
+            f(TypeError)
+        else:
+            x = 2
+
+    x = 3
+
+# From 3.7 test_exceptions.py
+# Bug is handling extended arg
+def testAttributes(exceptionList):
+    try:
+        x = 0
+    except:
+        pass
+
+    for exc in exceptionList:
+        x = 0; x = 1; x = 2; x = 3; x = 4; x = 5; x = 6; x = 7; x = 8; x = 9;
+        x = 0; x = 1; x = 2; x = 3; x = 4; x = 5; x = 6; x = 7; x = 8; x = 9;
+        x = 0; x = 1; x = 2; x = 3; x = 4; x = 5; x = 6; x = 7; x = 8; x = 9;
+        x = 0; x = 1; x = 2; x = 3; x = 4; x = 5; x = 6; x = 7; x = 8; x = 9;
+        x = 0; x = 1; x = 2; x = 3; x = 4; x = 5; x = 6; x = 7; x = 8; x = 9;
+        x = 0; x = 1; x = 2; x = 3; x = 4; x = 5; x = 6; x = 7; x = 8; x = 9;
+        x = 0
+
+# From test_urllibnet2.py
+# Bug was in detecting end of "if" flowing into an extended-arg "for".
+def _test_urls(retry, urls):
+    if retry:
+        urlopen = 1
+
+    for url in urls:
+        shape_t = 0; shape_t = 1; shape_t = 2; shape_t = 3; shape_t = 4; shape_t = 5; shape_t = 6; shape_t = 7; shape_t = 8; shape_t = 9
+        shape_t = 0; shape_t = 1; shape_t = 2; shape_t = 3; shape_t = 4; shape_t = 5; shape_t = 6; shape_t = 7; shape_t = 8; shape_t = 9
+        shape_t = 0; shape_t = 1; shape_t = 2; shape_t = 3; shape_t = 4; shape_t = 5; shape_t = 6; shape_t = 7; shape_t = 8; shape_t = 9
+        shape_t = 0; shape_t = 1; shape_t = 2; shape_t = 3; shape_t = 4; shape_t = 5; shape_t = 6; shape_t = 7; shape_t = 8; shape_t = 9
+        shape_t = 0; shape_t = 1; shape_t = 2; shape_t = 3; shape_t = 4; shape_t = 5; shape_t = 6; shape_t = 7; shape_t = 8; shape_t = 9
+        shape_t = 0; shape_t = 1; shape_t = 2; shape_t = 3; shape_t = 4; shape_t = 5; shape_t = 6; shape_t = 7; shape_t = 8; shape_t = 9
+        shape_t = 0

test/simple_source/bug36/02_call_ex_kw.py

@@ -53,3 +53,20 @@ from collections import namedtuple
 Point = namedtuple('Point', 'x y')
 p = Point(11, 22)
 assert p == Point(**dict(x=11, y=22))
+
+# From 3.7 test/test_keywordonlyarg.py
+# Bug was in handling {"4":4} as a dictionary needing **
+def posonly_sum(pos_arg1, *arg, **kwarg):
+    return pos_arg1 + sum(arg) + sum(kwarg.values())
+assert 1+2+3+4 == posonly_sum(1,*(2,3),**{"4":4})
+
+# From 3.7 test_grammar.py
+# Bug was in handling keyword-only parameters when there are annotations.
+# The stack order from least- to most-recent is:
+# default, keyword, annotation, closure
+# This changes in between Python 3.5 and 3.6.
+def f(a, b: 1, c: 2, d, e: 3=4, f=5, *g: 6, h: 7, i=8, j: 9=10,
+      **k: 11) -> 12: pass
+
+assert f.__annotations__ == {'b': 1, 'c': 2, 'e': 3, 'g': 6, 'h': 7, 'j': 9,
+                             'k': 11, 'return': 12}

test/simple_source/bug36/02_var_annotate.py

@@ -0,0 +1,12 @@
+# 3.6+ type annotations on variables
+from typing import List
+
+# RUNNABLE!
+y = 2
+x: bool
+z: int = 5
+x = (z == 5)
+assert x
+assert y == 2
+v: List[int] = [1, 2]
+assert v[1] == y

test/simple_source/bug36/10_fstring.py

@@ -67,3 +67,52 @@ def _repr_fn(fields):
 
 fields = ['a', 'b', 'c']
 assert _repr_fn(fields) == ['return xx + f"(a={self.a!r}, b={self.b!r}, c={self.c!r})"']
+
+
+#################################
+# From Python 3.7 test_fstring.py
+
+x = 5
+assert f'{(lambda y:x*y)("8")!r}' == "'88888'"
+assert f'{(lambda y:x*y)("8")!r:10}' == "'88888'   "
+assert f'{(lambda y:x*y)("8"):10}' == "88888     "
+
+try:
+    eval("f'{lambda x:x}'")
+except SyntaxError:
+    pass
+else:
+    assert False, "f'{lambda x:x}' should be a syntax error"
+
+(x, y, width) = ("foo", 2, 10)
+assert f'x={x*y:{width}}' == 'x=foofoo    '
+
+# Why the fact that the distinction of docstring versus stmt is a
+# string expression is important academic, but we will decompile an
+# equivalent thing. For compatiblity with older Python we'll use "%"
+# instead of a format string
+def f():
+    f'''Not a docstring'''
+def g():
+    '''Not a docstring''' \
+    f''
+
+assert f.__doc__ is None
+assert g.__doc__ is None
+
+import decimal
+width, precision, value = (10, 4, decimal.Decimal('12.34567'))
+
+# Make sure we don't have additional f'..' inside the format strings below.
+assert f'result: {value:{width}.{precision}}' == 'result:      12.35'
+assert f'result: {value:{width:0}.{precision:1}}' == 'result:      12.35'
+assert f'{2}\t' ==  '2\t'
+
+# But below we *do* need the additional f".."
+assert f'{f"{0}"*3}' == "000"
+
+# We need to make sure we have { {x:... not {{x: ...
+#                               ^
+# The former, {{ confuses the format strings so dictionary/set comprehensions
+# don't work.
+assert f'expr={ {x: y for x, y in [(1, 2), ]}}' == 'expr={1: 2}'

test/simple_source/bug37/01_assert2.py

@@ -6,3 +6,13 @@ def test_assert2(c):
         raise SyntaxError('Oops')
 
 test_assert2(5)
+
+# Bug is handling "assert" and confusing it with "or".
+# It is important that the assert be at the end of the loop.
+for x in (2, 4, 6):
+    assert x == x
+
+# Bug in 3.7 was not having a rule for 2-arg assert.
+# 2-arg assert code doesn't match "if not ... raise "
+for x in (1, 3, 5):
+    assert x == x, "foo"

test/simple_source/bug38/04_for_no_jump_back.py

@@ -1,6 +1,8 @@
 # from mult_by_const/instruction.py
 # Bug in 3.8 was handling no JUMP_BACK in "for" loop. It is
 # in the "if" instead
+
+# RUNNABLE!
 def instruction_sequence_value(instrs, a, b):
     for instr in instrs:
         if a:

test/simple_source/comprehension/05_dict_comp.py

@@ -3,12 +3,24 @@
 #
 # This code is RUNNABLE!
 def x(s):
-    return {k: v
-            for (k, v) in s
-            if not k.startswith('_')
-    }
+    return {k: v for (k, v) in s if not k.startswith("_")}
+
 
 # Yes, the print() is funny. This is
 # to test though a 2-arg assert where
 # the 2nd argument is not a string.
-assert x((('_foo', None),)) == {}, print("See issue #162")
+assert x((("_foo", None),)) == {}, print("See issue #162")
+
+# From 3.7 test_dictcomps.py
+assert {k: v for k in range(10) for v in range(10) if k == v} == {
+    0: 0,
+    1: 1,
+    2: 2,
+    3: 3,
+    4: 4,
+    5: 5,
+    6: 6,
+    7: 7,
+    8: 8,
+    9: 9,
+}

test/simple_source/def/01_class.py

@@ -8,8 +8,19 @@
 #   classdef ::= LOAD_CONST expr mkfunc CALL_FUNCTION_0 BUILD_CLASS store
 #   mkfunc ::= LOAD_CONST MAKE_FUNCTION_0
 
+# RUNNABLE!
 class A:
     pass
 
 class B(Exception):
     pass
+
+# From 3.x test_descr.py
+class MyInt(int):
+    class MyInt(int):
+        __slots__ = ()
+    try:
+        (1).__class__ = MyInt
+        assert False, "builtin types don't support __class__ assignment."
+    except TypeError:
+        pass

test/simple_source/looping/10_for_if_loopback.py

@@ -11,6 +11,7 @@
 #       101_0	COME_FROM         '50' statement: if name == ...to fictional "end if"
 #         101	JUMP_BACK         '16' jump as before to loop head
 
+# RUNNABLE!
 def _slotnames(cls):
     names = []
     for c in cls.__mro__:
@@ -21,3 +22,37 @@ def _slotnames(cls):
                     continue
                 else:
                     names.append(name) # 3.2 bug here jumping to outer for
+
+# From 3.2.6 pdb.py
+# There is no "come_from" after the "if" since the
+# if jumps back to the loop. For this we use
+# grammar rule "ifstmtl"
+def lasti2lineno(linestarts, a):
+    for i in linestarts:
+        if a:
+            return a
+    return -1
+
+assert lasti2lineno([], True) == -1
+assert lasti2lineno([], False) == -1
+assert lasti2lineno([1], False) == -1
+assert lasti2lineno([1], True) == 1
+
+# From 3.7 test_builtin.py
+# Bug was allowing if condition jump back to the
+# "for" loop as an acceptable "ifstmtl" rule.
+
+# RUNNABLE!
+def test_pow(m, b, c):
+    for a in m:
+        if a or \
+           b or \
+           c:
+            c = 1
+
+    return c
+
+assert test_pow([], 2, 3) == 3
+assert test_pow([1], 0, 5) == 1
+assert test_pow([1], 4, 2) == 1
+assert test_pow([0], 0, 0) == 0

test/simple_source/stmts/00_import.py

@@ -4,6 +4,7 @@ from os import path
 from os import *
 import time as time1, os as os1
 import http.client as httpclient
+from sys import stdin, stdout, stderr
 if len(__file__) == 0:
     # a.b.c should force consecutive LOAD_ATTRs
     import a.b.c as d

test/stdlib/2.5-exclude.sh

@@ -0,0 +1,41 @@
+SKIP_TESTS=(
+    [test_coercion.py]=1
+    [test_decimal.py]=1
+    [test_dis.py]=1        # We change line numbers - duh!
+    [test_file.py]=1       # test assertion failures
+    [test_generators.py]=1 # Investigate
+    [test_grammar.py]=1    # Too many stmts. Handle large stmts
+    [test_grp.py]=1        # Long test - might work Control flow?
+    [test_macfs.py]=1      # it fails on its own
+    [test_macostools.py]=1 # it fails on its own
+    [test_mailbox.py]=1
+    [test_nis.py]=1        # it fails on its own
+    [test_normalization.py]=1 # it fails on its own
+    [test_optparse.py]=1   # it fails on its own
+    [test_ossaudiodev.py]=1 # it fails on its own
+    [test_pdb.py]=1        # Line-number specific
+    [test_pep277.py]=1    # it fails on its own
+    [test_pep352.py]=1     # Investigate
+    [test_plistlib.py]=1 # it fails on its own
+    [test_pwd.py]=1 # Long test - might work? Control flow?
+    [test_pyclbr.py]=1 # Investigate
+    [test_rgbimg.py]=1 #  it fails on its own
+    [test_queue.py]=1 # Control flow?
+    [test_scriptpackages.py]=1 # it fails on its own
+    [test_socketserver.py]=1 # Too long to run - 42 seconds
+    [test_sqlite.py]=1 # it fails on its own
+    [test_startfile.py]=1 # it fails on its own
+    [test_struct.py]=1 # "if and" confused for if .. assert and
+    [test_sunaudiodev.py]=1 # it fails on its own
+    [test_support.py]=1 # it fails on its own
+    [test_tcl.py=1] # it fails on its own
+    [test_threading.py]=1 # test takes too long to run: 11 seconds
+    [test_thread.py]=1 # test takes too long to run: 36 seconds
+    [test_trace.py]=1  # Line numbers are expected to be different
+    [test_urllib2net.py]=1 # is interactive?
+    [test_urllibnet.py]=1 # it fails on its own
+    [test_winreg.py]=1 # it fails on its own
+    [test_winsound.py[=1 # it fails on its own
+    [test_zipfile64.py]=1  # Runs ok but takes 204 seconds
+    [test_zlib]=1  # fails on its own
+)

test/stdlib/2.6-exclude.sh

@@ -0,0 +1,101 @@
+SKIP_TESTS=(
+
+    [test___all__.py]=1  # it fails on its own
+    [test___all__.py]=1 # it fails on its own
+    [test_aepack.py]=1 # Fails on its own
+    [test_aepack.py]=1 # it fails on its own
+    [test_al.py]=1 # it fails on its own
+    [test_anydbm.py]=1 # it fails on its own
+    [test_applesingle.py]=1 # it fails on its own
+
+    [test_bsddb185.py]=1 # it fails on its own
+    [test_bsddb3.py]=1 # it fails on its own
+    [test_bsddb.py]=1 # it fails on its own
+
+    [test_cd.py]=1 # it fails on its own
+    [test_cl.py]=1 # it fails on its own
+    [test_codecmaps_cn.py]=1 # it fails on its own
+    [test_codecmaps_jp.py]=1 # it fails on its own
+    [test_codecmaps_kr.py]=1 # it fails on its own
+    [test_codecmaps_tw.py]=1 # it fails on its own
+    [test_commands.py]=1 # it fails on its own
+    [test_curses.py]=1 # it fails on its own
+
+    [test_dbm.py]=1 # it fails on its own
+    [test_descr.py]=1
+    [test_doctest.py]=1    #
+    [test_dis.py]=1        # We change line numbers - duh!
+    [test_dl.py]=1 # it fails on its own
+
+    [test_file.py]=1 # it fails on its own
+    [test_future5.py]=1 # it fails on its own
+
+    [test_generators.py]=1 # Investigate
+    [test_gl.py]=1 # it fails on its own
+    [test_grp.py]=1      # Long test - might work Control flow?
+
+    [test_imageop.py]=1 # it fails on its own
+    [test_imaplib.py]=1 # it fails on its own
+    [test_imgfile.py]=1 # it fails on its own
+    [test_ioctl.py]=1 # it fails on its own
+
+    [test_kqueue.py]=1 # it fails on its own
+
+    [test_linuxaudiodev.py]=1 # it fails on its own
+
+    [test_macos.py]=1 # it fails on its own
+    [test_macostools.py]=1 # it fails on its own
+    [test_mailbox.py]=1 # FIXME: release 3.6.2 may have worked
+
+    [test_normalization.py]=1 # it fails on its own
+
+    [test_ossaudiodev.py]=1 # it fails on its own
+
+    [test_pep277.py]=1 # it fails on its own
+    [test_pep352.py]=1     # Investigate
+    [test_pyclbr.py]=1 # Investigate
+    [test_pwd.py]=1 # Long test - might work? Control flow?
+    [test_py3kwarn.py]=1 # it fails on its own
+
+    [test_scriptpackages.py]=1 # it fails on its own
+    [test_select.py]=1 # test takes too long to run: 11 seconds
+    [test_socket.py]=1 # test takes too long to run: 12 seconds
+    [test_startfile.py]=1 # it fails on its own
+    [test_structmembers.py]=1 # it fails on its own
+    [test_sunaudiodev.py]=1 # it fails on its own
+    [test_support.py]=1 # # it fails on it s own
+
+    [test_threading.py]=1  # fails on its own?
+    [test_trace.py]=1  # Line numbers are expected to be different
+
+    [test_urllib2_localnet.py]=1 # test takes too long to run: 12 seconds
+    [test_urllib2net.py]=1 # test takes too long to run: 11 seconds
+    [test_urllibnet.py]=1 # it fails on its own
+
+
+    [test_whichdb.py]=1 # it fails on its own
+    [test_winreg.py]=1 # it fails on its own
+    [test_winsound.py]=1 # it fails on its own
+
+    [test_zipimport_support.py]=1
+    [test_zipfile64.py]=1  # Skip Long test
+    [test_zlib.py]=1  #
+    # .pyenv/versions/2.6.9/lib/python2.6/lib2to3/refactor.pyc
+    # .pyenv/versions/2.6.9/lib/python2.6/pyclbr.pyc
+)
+# About 305 unit-test files in about 12 minutes
+
+if (( batch )) ; then
+    # Fails in crontab environment?
+    # Figure out what's up here
+    SKIP_TESTS[test_aifc.py]=1
+    SKIP_TESTS[test_array.py]=1
+
+    # SyntaxError: Non-ASCII character '\xdd' in file test_base64.py on line 153, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details
+    SKIP_TESTS[test_base64.py]=1
+    SKIP_TESTS[test_decimal.py]=1 # Might be a POWER math thing
+
+    # output indicates expected == output, but this fails anyway.
+    # Maybe the underlying encoding is subtlely different so it
+    # looks the same?
+fi

test/stdlib/2.7-exclude.sh

@@ -0,0 +1,49 @@
+SKIP_TESTS=(
+    [test_bsddb3.py]=1 # test takes too long to run: 110 seconds
+    [test_compile.py]=1  # Code introspects on co_consts in a non-decompilable way
+    [test_curses.py]=1  # Possibly fails on its own but not detected
+    [test_cmd_line.py]=1 # Takes too long, maybe hangs, or looking for interactive input?
+    [test_dis.py]=1   # We change line numbers - duh!
+    [test_doctest.py]=1 # Fails on its own
+    [test_exceptions.py]=1
+    [test_format.py]=1 # Control flow "and" vs nested "if"
+    [test_grammar.py]=1     # Too many stmts. Handle large stmts
+    [test_grp.py]=1     # test takes to long, works interactively though
+    [test_io.py]=1 # Test takes too long to run
+    [test_ioctl.py]=1 # Test takes too long to run
+    [test_lib2to3.py]=1 # test takes too long to run: 28 seconds
+    [test_math.py]=1
+    [test_memoryio.py]=1 # FIX
+    [test_modulefinder.py]=1 # FIX
+    [test_multiprocessing.py]=1 # On uncompyle2, takes 24 secs
+    [test_poll.py]=1  # test takes too long to run: 11 seconds
+    [test_pwd.py]=1     # Takes too long
+    [test_regrtest.py]=1 #
+    [test_runpy.py]=1   # Long and fails on its own
+    [test_select.py]=1  # Runs okay but takes 11 seconds
+    [test_socket.py]=1  # Runs ok but takes 22 seconds
+    [test_ssl.py]=1  #
+    [test_subprocess.py]=1 # Runs ok but takes 22 seconds
+    [test_sys_settrace.py]=1 # Line numbers are expected to be different
+    [test_tokenize.py]=1 # test takes too long to run: 19 seconds
+    [test_traceback.py]=1 # Line numbers change - duh.
+    [test_unicode.py]=1  # Too long to run 11 seconds
+    [test_xpickle.py]=1 # Runs ok but takes 72 seconds
+    [test_zipfile64.py]=1  # Runs ok but takes 204 seconds
+    [test_zipimport.py]=1  #
+)
+# 334 unit-test files in about 15 minutes
+
+if (( batch )) ; then
+    # Fails in crontab environment?
+    # Figure out what's up here
+    SKIP_TESTS[test_array.py]=1
+    SKIP_TESTS[test_ast.py]=1
+    SKIP_TESTS[test_audioop.py]=1
+    SKIP_TESTS[test_doctest2.py]=1 # a POWER thing?
+    SKIP_TESTS[test_httplib.py]=1  # Ok, but POWER has problems with this
+    SKIP_TESTS[test_pdb.py]=1 # Ok, but POWER has problems with this
+
+    # SyntaxError: Non-ASCII character '\xdd' in file test_base64.py on line 153, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details
+    SKIP_TESTS[test_base64.py]=1
+fi

test/stdlib/3.2-exclude.sh

@@ -0,0 +1,48 @@
+SKIP_TESTS=(
+    [test_decorators.py]=1 # FIXME: Works on c90ff51
+    [test_optparse.py]=1 # FIXME: Works on c90ff51
+    [test_os.py]=1 # FIXME: Works on c90ff51
+    [test_pyclbr.py]=1 # FIXME: Works on c90ff51
+    [test_strftime.py]=1 # FIXME: Works on c90ff51
+
+    [test_binop.py]=1 # FIXME: Works on c90ff51?
+    [test_cgi.py]=1 # FIXME: Works on c90ff51?
+    [test_descr.py]=1 # FIXME: Works on c90ff51?
+    [test_doctest2.py]=1 # FIXME: Works on c90ff51?
+
+    [test_cmd_line.py]=1
+    [test_collections.py]=1
+    [test_concurrent_futures.py]=1 # too long to run over 46 seconds by itself
+    [test_datetimetester.py]=1
+    [test_decimal.py]=1
+    [test_dictcomps.py]=1 # FIXME: semantic error: actual = {k:v for k in }
+    [test_doctest.py]=1 # test failures
+    [test_dis.py]=1   # We change line numbers - duh!
+    [test_exceptions.py]=1 # parse error
+
+    [test_modulefinder.py]=1 # test failures
+    [test_multiprocessing.py]=1 # test takes too long to run: 35 seconds
+
+    [test_peepholer.py]=1
+    [test_pep352.py]=1
+
+    [test_quopri.py]=1 # TypeError: Can't convert 'bytes' object to str implicitly
+
+    [test_runpy.py]=1
+
+    [test_ssl.py]=1 # too installation specific
+    [test_startfile.py]=1 # fails on its own
+    [test_subprocess.py]=1  # fails on its own
+    [test_tcl.py]=1 # it fails on its own
+    [test_tk.py]=1 # it fails on its own
+    [test_traceback.py]=1 # it fails on its own
+    [test_zipfile64.py]=1 # Too long to run
+
+)
+
+if (( batch )) ; then
+    # Fails in crontab environment?
+    # Figure out what's up here
+    SKIP_TESTS[test_exception_variations.py]=1
+    SKIP_TESTS[test_quopri.py]=1
+fi

test/stdlib/3.3-exclude.sh

@@ -0,0 +1,80 @@
+SKIP_TESTS=(
+    [test_binop.py]=1 # FIXME: Works on c90ff51
+    [test_cgi.py]=1 # FIXME: Works on c90ff51
+    [test_decorators.py]=1 # FIXME: Works on c90ff51
+    [test_pyclbr.py]=1 # FIXME: Works on c90ff51
+    [test_optparse.py]=1 # FIXME: Works on c90ff51
+    [test_os.py]=1 # FIXME: Works on c90ff51
+    [test_pep352.py]=1 # FIXME: Works on c90ff51
+    [test_pyclbr.py]=1 # FIXME: Works on c90ff51
+    [test_shutil.py]=1 # FIXME: Works on c90ff51
+    [test_strftime.py]=1 # FIXME: Works on c90ff51
+    [test_symtable.py]=1 # FIXME: Works on c90ff51
+
+    [test_atexit.py]=1  # The atexit test starting at 3.3 looks for specific comments in error lines
+
+    [test_buffer.py]=1  # parse error
+
+    [test_cmd_line.py]=1 # too long?
+    [test_concurrent_futures.py]=1  # too long?
+
+    [test_decimal.py]=1 # test takes too long to run: 18 seconds
+    [test_descr.py]=1  # test assertion errors
+    [test_doctest.py]=1  # test assertion errors
+    [test_doctest2.py]=1  # test assertion errors
+    [test_dis.py]=1   # We change line numbers - duh!
+
+    [test_exceptions.py]=1   #
+    [test_faulthandler.py]=1
+    [test_fork1.py]=1 # test takes too long to run: 12 seconds
+
+    [test_grammar.py]=1
+
+    [test_io.py]=1  # test takes too long to run: 34 seconds
+
+    [test_lib2to3.py]=1
+    [test_logging.py]=1 # test takes too long to run: 13 seconds
+    [test_long.py]=1 # test assert failure AttributeError: 'Rat' object has no attribute 'd'
+
+    [test_math.py]=1
+    [test_modulefinder.py]=1
+    [test_multiprocessing.py]=1
+
+    [test_nntplib.py]=1
+
+    [test_peepholer.py]=1
+    [test_poll.py]=1  # test takes too long to run: 11 seconds
+    [test_pty.py]=1  # FIXME: Needs grammar loop isolation separation
+
+    [test_queue.py]=1
+
+    [test_re.py]=1
+    [test_resource.py]=1
+    [test_runpy.py]=1
+
+    [test_scope.py]=1
+    [test_select.py]=1
+    [test_signal.py]=1
+    [test_socket.py]=1
+    [test_ssl.py]=1 # too installation specific
+    [test_subprocess.py]=1  # test takes too long to run: 28 seconds
+    [test_sys_setprofile.py]=1 # test assertion errors
+    [test_sys_settrace.py]=1 # test assertion errors
+
+    [test_tcl.py]=1  # installation specific; it fails on its own
+    [test_timeout.py]=1 # Too long to run: 19 seconds
+    [test_traceback.py]=1 # Probably introspects code
+
+    [test_zipfile64.py]=1 # Too long to run
+)
+# About 300 unit-test files in about 20 minutes
+
+if (( batch )) ; then
+    SKIP_TESTS[test_ftplib.py]=1  # Runs too long on POWER; over 15 seconds
+    SKIP_TESTS[test_idle.py]=1  # No tk
+    SKIP_TESTS[test_pep352.py]=1  # UnicodeDecodeError may be funny on weird environments
+    SKIP_TESTS[test_pep380.py]=1  # test_delegating_generators_claim_to_be_running ?
+    # Fails in crontab environment?
+    # Figure out what's up here
+    # SKIP_TESTS[test_exception_variations.py]=1
+fi

test/stdlib/3.4-exclude.sh

@@ -0,0 +1,79 @@
+SKIP_TESTS=(
+    [test_buffer.py]=1 # FIXME: Works on c90ff51
+    [test_decorators.py]=1 # FIXME: Works on c90ff51
+    [test_faulthandler.py]=1 # FIXME: too long to run 20 seconds. Works on c90ff51
+    [test_decimal.py]=1 # FIXME: Works on c90ff51
+    [test_optparse.py]=1 # FIXME: Works on c90ff51
+    [test_os.py]=1 # FIXME: Works on c90ff51
+    [test_shutil.py]=1 # FIXME: Works on c90ff51
+    [test_strftime.py]=1 # FIXME: Works on c90ff51
+
+    [test___all__.py]=1  # it fails on its own
+    [test_atexit.py]=1  # The atexit test looks for specific comments in error lines
+    [test_cmd_line.py]=1 # takes too long to run
+    [test_concurrent_futures.py]=1  # too long?
+    [test_configparser.py]=1  # Doesn't terminate
+    [test_ctypes.py]=1 # it fails on its own
+    [test_curses.py]=1 # Investigate
+    [test_dbm_gnu.py]=1   # fails on its own
+    [test_devpoll.py]=1 # it fails on its own
+    [test_descr.py]=1   # test assertion errors
+    [test_dis.py]=1   # We change line numbers - duh!
+    [test_distutils.py]=1 # it fails on its own
+    [test_doctest2.py]=1
+    [test_doctest.py]=1  # test assert failures
+    [test_docxmlrpc.py]=1
+    [test_enum.py]=1 # compile syntax?
+    [test_exceptions.py]=1
+    [test_faulthandler.py]=1
+    [test_file_eintr.py]=1   # parse error
+    [test_fork1.py]=1 # too long
+    [test_gdb.py]=1 # it fails on its own
+    [test_grammar.py]=1 # parse error
+    [test_httplib.py]=1 # it fails on its own
+    [test_import.py]=1 # it fails on its own
+    [test_io.py]=1
+    [test_ioctl.py]=1 # it fails on its own
+    [test_inspect.py]=1 # Syntax error Investigate
+    [test_logging.py]=1 # Too long to run
+    [test_long.py]=1 # FIXME: Works on c90ff51
+    [test_modulefinder.py]=1  # test assertion error
+    [test_multiprocessing_fork.py]=1 # doesn't terminate
+    [test_multiprocessing_forkserver.py]=1 # doesn't terminate
+    [test_multiprocessing_main_handling.py]=1  # doesn't terminate
+    [test_multiprocessing_spawn.py]=1 # doesn't terminate
+    [test_nntplib.py]=1 # too long to run
+    [test_peepholer.py]=1 # control flow?
+    [test_pep352.py]=1 # test assert failures
+    [test_pickle.py]=1 # test assert failures
+    [test_pkgimport.py]=1 # long
+    [test_poll.py]=1 # Too long to run: 11 seconds
+    [test_pydoc.py]=1 # test assertion failures
+    [test_runpy.py]=1 # Too long:
+    [test_select.py]=1 # Too long: 11 seconds
+    [test_selectors.py]=1  # Too long: 11 seconds
+    [test_signal.py]=1 # Too long: 22 seconds
+    [test_sndhdr.py]=1
+    [test_socket.py]=1 # long 25 seconds
+    [test_socketserver.py]=1 # long 25 seconds
+    [test_subprocess.py]=1 # Too long
+    [test_symtable.py]=1 # Investigate bad output
+    [test_sys_settrace.py]=1 # test assert failures
+    [test_tcl.py]=1 # May be implementation specific. On POWER though it fails
+    [test_threading.py]=1 # Too long
+    [test_threadsignals.py]=1 # Too long to run: 12 seconds
+    [test_timeout.py]=1 # Too long to run: 19 seconds
+    [test_traceback.py]=1 # introspects on code
+    [test_urllib2net.py]=1 # Doesn't terminate
+    [test_zipfile64.py]=1
+    [test_zlib.py]=1
+)
+# 304 unit-test file in about 20 minutes
+
+if (( batch )) ; then
+    # Fails in crontab environment?
+    # Figure out what's up here
+    SKIP_TESTS[test_exception_variations.py]=1
+    SKIP_TESTS[test_mailbox.py]=1 # Takes to long on POWER; over 15 secs
+    SKIP_TESTS[test_quopri.py]=1
+fi