Get ready for release 2.7.0

I think this is a holdover from pre 2.3 days. Possibly it can be dropped altogether.

ChangeLog

@@ -1,6 +1,282 @@
+2016-07-15  rocky <rb@dustyfeet.com>
+
+	* README.rst, uncompyle6/version.py: Get ready for release 2.7.0
+
+2016-07-14  rocky <rb@dustyfeet.com>
+
+	* uncompyle6/parser.py, uncompyle6/parsers/parse2.py,
+	uncompyle6/scanners/scanner2.py, uncompyle6/scanners/scanner26.py,
+	uncompyle6/semantics/pysource.py: Isolate pseudo op IMPORT_NAME_CONT I think this is a holdover from pre 2.3 days. Possibly it can be
+	dropped altogether.
+
+2016-07-14  rocky <rb@dustyfeet.com>
+
+	* uncompyle6/parsers/parse3.py, uncompyle6/semantics/pysource.py: 
+	Attempt to get 3.5 RETURN_END_IF working This feels hacky and I'm not sure is quite right.  Untili we
+	understand better what to do though, we'll go with it.
+
+2016-07-14  rocky <rb@dustyfeet.com>
+
+	* test/Makefile, uncompyle6/semantics/pysource.py: 3.x __qualname__
+	= supression Class names have become more complicated so the pattern test needs
+	to be more complex as well. Sigh
+
+2016-07-14  rocky <rb@dustyfeet.com>
+
+	* README.rst, uncompyle6/semantics/fragments.py,
+	uncompyle6/semantics/pysource.py: 3.x list comprehenions
+	list_if_not, comp_ifnot bug Saw only list_if_not bug, but might also be applicable to comp_ifnot
+
+2016-07-14  rocky <rb@dustyfeet.com>
+
+	* test/simple_source/stmts/15_assert.py,
+	uncompyle6/scanners/scanner3.py: 3.x ""raise AssertionError" vs
+	"assert" Not sure this is totally correct yet.
+
+2016-07-14  rocky <rb@dustyfeet.com>
+
+	* uncompyle6/main.py, uncompyle6/parsers/parse3.py,
+	uncompyle6/parsers/parse34.py, uncompyle6/scanners/scanner2.py,
+	uncompyle6/scanners/scanner26.py, uncompyle6/scanners/scanner3.py,
+	uncompyle6/verify.py: Python 3.4 assertion handling. Improve verify 3.4 has jump optimization like 3.5.  verify.py: show mismatch on
+	verification mismatch
+
+2016-07-13  rocky <rb@dustyfeet.com>
+
+	* : Reinstate some code for verification
+
+2016-07-13  rocky <rb@dustyfeet.com>
+
+	* uncompyle6/main.py: Back off showing verification errors Pending why this failing in make.
+
+2016-07-13  rocky <rb@dustyfeet.com>
+
+	* uncompyle6/semantics/pysource.py: "return None" -> "pass" Python 3.4 classes have what looks like a return None, but this is
+	invalid syntax. So for now we'll remove all "return None"s.  There may be a "pass" in there instead.
+
+2016-07-13  rocky <rb@dustyfeet.com>
+
+	* : commit fd0d8d25672b9450bafc1d5c6781bedd7ec8bd1d Author: rocky
+	<rb@dustyfeet.com> Date:   Wed Jul 13 13:00:03 2016 -0400
+
+2016-07-13  rocky <rb@dustyfeet.com>
+
+	* test/simple_source/stmts/11_return_val.py,
+	uncompyle6/semantics/pysource.py: Debugging smutz caused verify
+	failures
+
+2016-07-12  rocky <rb@dustyfeet.com>
+
+	* uncompyle6/parsers/parse3.py, uncompyle6/semantics/fragments.py,
+	uncompyle6/semantics/pysource.py: Document/correct hide_internal and
+	store_locals
+
+2016-07-12  rocky <rb@dustyfeet.com>
+
+	* test/simple_source/bug33/05_store_locals.py,
+	test/simple_source/bug33/05_store_name.py,
+	uncompyle6/semantics/pysource.py: Better 3.2/33 store_locals
+	handling
+
+2016-07-11  rocky <rb@dustyfeet.com>
+
+	* test/simple_source/bug33/05_store_name.py,
+	test/simple_source/comprehension/05_3x_set_comphension.py,
+	uncompyle6/parsers/parse3.py, uncompyle6/semantics/pysource.py: 
+	Python 3.2 & 3.3 handle STORE_NAME better
+
+2016-07-11  rocky <rb@dustyfeet.com>
+
+	* ChangeLog, NEWS, README.rst, uncompyle6/version.py: Get ready for
+	release 2.6.2
+
+2016-07-11  rocky <rb@dustyfeet.com>
+
+	* test/add-test.py,
+	test/simple_source/comprehension/05_set_comprehension.py,
+	uncompyle6/semantics/pysource.py: Fix some Python set comprehension
+	bugs
+
+2016-07-11  rocky <rb@dustyfeet.com>
+
+	* test/simple_source/def/05_static_method.py: Add missing a test
+	source file
+
+2016-07-11  rocky <rb@dustyfeet.com>
+
+	* uncompyle6/parser.py, uncompyle6/parsers/parse26.py,
+	uncompyle6/parsers/parse27.py, uncompyle6/parsers/parse3.py,
+	uncompyle6/scanners/scanner3.py: Python 3.(4?) while1 bug Clean up while1 grammar a tad
+
+2016-07-10  rocky <rb@dustyfeet.com>
+
+	* uncompyle6/scanners/scanner2.py: Revert recent 2.x code which fail
+	verification
+
+2016-07-10  rocky <rb@dustyfeet.com>
+
+	* uncompyle6/parsers/parse3.py, uncompyle6/parsers/parse34.py,
+	uncompyle6/scanners/scanner2.py, uncompyle6/scanners/scanner26.py,
+	uncompyle6/scanners/scanner3.py: Structure detection bugs + Had borked 3.4 grammar rules in previous refactor
+
+2016-07-10  rocky <rb@dustyfeet.com>
+
+	* uncompyle6/scanner.py, uncompyle6/scanners/scanner2.py,
+	uncompyle6/scanners/scanner3.py: DRY and clean up code a little
+
+2016-07-10  rocky <rb@dustyfeet.com>
+
+	* uncompyle6/scanners/scanner2.py,
+	uncompyle6/scanners/scanner26.py, uncompyle6/scanners/scanner27.py,
+	uncompyle6/scanners/scanner3.py: Remove JA. Use standard
+	JUMP_ABSOLUTE instead
+
+2016-07-10  rocky <rb@dustyfeet.com>
+
+	* uncompyle6/scanners/scanner3.py: Python 2/3 compatibility bug
+
+2016-07-10  rocky <rb@dustyfeet.com>
+
+	* uncompyle6/parsers/parse3.py, uncompyle6/scanners/scanner3.py: 
+	Bugs caused by 3.x jump_forward misclasification
+
+2016-07-10  rocky <rb@dustyfeet.com>
+
+	* uncompyle6/parsers/parse3.py, uncompyle6/scanners/scanner2.py,
+	uncompyle6/scanners/scanner26.py, uncompyle6/scanners/scanner3.py: 
+	Python 3 better CONTINUE op classification Also document what's up with JUMP_ABSOLUTE classification
+
+2016-07-09  rocky <rb@dustyfeet.com>
+
+	* uncompyle6/parsers/parse34.py: 3.4 continue handling. More work is
+	needed
+
+2016-07-09  rocky <rb@dustyfeet.com>
+
+	* __pkginfo__.py: Need xdis 2.4 bugfix
+
+2016-07-09  rocky <rb@dustyfeet.com>
+
+	* uncompyle6/parser.py, uncompyle6/parsers/parse3.py,
+	uncompyle6/parsers/parse34.py: Start splitting off 3.4 grammar from
+	rest
+
+2016-07-09  rocky <rb@dustyfeet.com>
+
+	* test/simple_source/comprehension/01_list_comprehension.py,
+	uncompyle6/parsers/parse23.py: Python 2.3 list comprehensions
+
+2016-07-09  rocky <rb@dustyfeet.com>
+
+	* test/simple_source/comprehension/01_list_comprehension.py,
+	test/simple_source/operation_binary/add.py,
+	test/simple_source/operation_binary/and.py,
+	test/simple_source/operation_binary/divide_floor.py,
+	test/simple_source/operation_binary/divide_true.py,
+	test/simple_source/operation_binary/modulo.py,
+	test/simple_source/operation_binary/multiply.py,
+	test/simple_source/operation_binary/or.py,
+	test/simple_source/operation_binary/power.py,
+	test/simple_source/operation_binary/shift_left.py,
+	test/simple_source/operation_binary/shift_right.py,
+	test/simple_source/operation_binary/subscription.py,
+	test/simple_source/operation_binary/subtract.py,
+	test/simple_source/operation_binary/xor.py,
+	test/simple_source/operation_comparison/equal.py,
+	test/simple_source/operation_comparison/greater.py,
+	test/simple_source/operation_comparison/greater_equal.py,
+	test/simple_source/operation_comparison/less.py,
+	test/simple_source/operation_comparison/less_equal.py,
+	test/simple_source/operation_comparison/notequal.py,
+	test/simple_source/operation_inplace/add.py,
+	test/simple_source/operation_inplace/and.py,
+	test/simple_source/operation_inplace/divide_floor.py,
+	test/simple_source/operation_inplace/divide_true.py,
+	test/simple_source/operation_inplace/modulo.py,
+	test/simple_source/operation_inplace/multiply.py,
+	test/simple_source/operation_inplace/or.py,
+	test/simple_source/operation_inplace/power.py,
+	test/simple_source/operation_inplace/shift_left.py,
+	test/simple_source/operation_inplace/shift_right.py,
+	test/simple_source/operation_inplace/subtract.py,
+	test/simple_source/operation_inplace/xor.py,
+	test/simple_source/operation_unary/invert.py,
+	test/simple_source/operation_unary/iter.py,
+	test/simple_source/operation_unary/negative.py,
+	test/simple_source/operation_unary/not.py,
+	test/simple_source/operation_unary/positive.py,
+	test/simple_source/precedence/left.py,
+	test/simple_source/precedence/right.py,
+	test/simple_source/precedence/structure.py,
+	test/simple_source/stmts/00_import.py,
+	uncompyle6/parsers/parse2.py, uncompyle6/parsers/parse23.py,
+	uncompyle6/parsers/parse25.py, uncompyle6/scanners/scanner26.py: Fix
+	some 2.3 bugs; add more 2.3-6. tests
+
+2016-07-09  rocky <rb@dustyfeet.com>
+
+	* uncompyle6/parsers/parse23.py: 2.3 while1stmt bug
+
+2016-07-09  rocky <rb@dustyfeet.com>
+
+	* : commit ac45e5757c8c57af2c918c79ff4f6195268afec1 Author: rocky
+	<rocky@gnu.org> Date:   Sat Jul 9 05:59:02 2016 -0400
+
+2016-07-09  rocky <rb@dustyfeet.com>
+
+	* uncompyle6/parsers/parse23.py: Merge conflict
+
+2016-07-09  rocky <rocky@gnu.org>
+
+	* uncompyle6/scanners/scanner3.py: Python3 scanner code cleanup
+
 2016-07-08  rocky <rocky@gnu.org>
 
-	* README.rst, uncompyle6/version.py: Get ready for release 2.6.1
+	* uncompyle6/scanners/scanner3.py,
+	uncompyle6/scanners/scanner32.py, uncompyle6/scanners/scanner33.py,
+	uncompyle6/scanners/scanner34.py, uncompyle6/scanners/scanner35.py: 
+	Python 3 code cleanup
+
+2016-07-08  rocky <rb@dustyfeet.com>
+
+	* test/Makefile, uncompyle6/scanner.py,
+	uncompyle6/scanners/scanner24.py, uncompyle6/scanners/scanner25.py,
+	uncompyle6/scanners/scanner26.py: DRY scanner code. Allow 2.4
+	decompile from 3.x
+
+2016-07-08  rocky <rocky@gnu.org>
+
+	* : Add more 2.4 and 2.6 tests
+
+2016-07-08  rocky <rocky@gnu.org>
+
+	* uncompyle6/parsers/parse24.py: 2.4 "import *" grammar rule
+
+2016-07-08  rocky <rocky@gnu.org>
+
+	* uncompyle6/parsers/parse24.py, uncompyle6/scanners/scanner2.py,
+	uncompyle6/scanners/scanner24.py, uncompyle6/scanners/scanner26.py: 
+	Python 2.4 generator expressions and gen_comp_body
+
+2016-07-08  rocky <rocky@gnu.org>
+
+	* __pkginfo__.py, uncompyle6/parsers/parse24.py,
+	uncompyle6/scanners/scanner2.py: <2.5 grammar customizations for
+	imports and loops
+
+2016-07-08  rocky <rocky@gnu.org>
+
+	* __pkginfo__.py, test/Makefile, test/add-test.py,
+	test/test_pyenvlib.py, test/test_pythonlib.py,
+	uncompyle6/parser.py, uncompyle6/parsers/parse24.py,
+	uncompyle6/parsers/parse25.py, uncompyle6/scanner.py,
+	uncompyle6/scanners/scanner24.py, uncompyle6/scanners/scanner25.py,
+	uncompyle6/semantics/pysource.py: Start handling Pyton 2.4 bytecodes
+
+2016-07-08  rocky <rocky@gnu.org>
+
+	* ChangeLog, NEWS, README.rst, uncompyle6/version.py: Get ready for
+	release 2.6.1
 
 2016-07-08  rocky <rocky@gnu.org>
 

NEWS

@@ -1,3 +1,19 @@
+uncompyle6 2.7.0 2016-07-15
+
+- Many Syntax and verifification bugs removed
+  tested on standard libraries from 2.3.7 to 3.5.1
+  and they all decompile and verify fine.
+  I'm sure there are more bugs though.
+
+uncompyle6 2.6.2 2016-07-11 Manhattenhenge
+
+- Extend bytecodes back to 2.3
+- Fix bugs:
+  * 3.x and 2.7 set comprehensions,
+  * while1 loops
+  * continue statements
+- DRY and segregate grammar more
+
 uncompyle6 2.6.1 2016-07-08
 
 - Go over Python 2.5 bytecode deparsing

README.rst

@@ -11,9 +11,9 @@ Introduction
 ------------
 
 *uncompyle6* translates Python bytecode back into equivalent Python
-source code. It accepts bytecodes from Python version 2.5 to 3.5 or
+source code. It accepts bytecodes from Python version 2.3 to 3.5 or
 so. The code requires Python 2.6 or later and has been tested on Python
-running versions 2.6, 2.7, 3.2, 3.3, 3.4 and 3.5.
+running versions 2.3-2.7, and 3.2-3.5.
 
 Why this?
 ---------
@@ -84,25 +84,25 @@ for usage help.
 Known Bugs/Restrictions
 -----------------------
 
-Python 2 deparsing decompiles each and all the Python 2.7.10 and
-2.7.11 installed packages I have on my system, more than 90% verify
-ok. Some of these failures may be bugs in the verification process.
-At this point, 2.7 decompilation is better than uncompyle2. A number
-of bugs have been fixed over what was in uncompyle2.
+Python 2 deparsing decompiles and verifies from Python 2.3.7 to Python
+3.5.1 on the standard library packages I have on my system.
 
-All of the Python 3.2-3.5 Python standard lib packages that I have
-installed on my system deparse. Each Python version has about 200
-bytecode files. I'm not sure how well these verify though.
+(Verification is the process of decompiling bytecode, compiling with a
+Python for that byecode version, and then comparing the byetcode
+produced by the decompiled/compiled program. Some allowance is made
+for inessential differences.)
 
-There are a few constructs that still need to be added to Python 3.5
+Later distributions average about 200 files. At this point, 2.7
+decompilation is better than uncompyle2. A number of bugs have been
+fixed.
+
+That said, I'm sure down the line more problems will turn up. The
+decompilation process uses a number of heuristics that may prove
+faulty.
+
+There are a few constructs that still need to be added to Python 3.5.
 Python 3.6 changes things drastically by using word codes rather than
-byte codes. So that will be yet another challenge
-
-Python 2.5 and 2.6 installed library routines also decompile; it is
-probably the same as uncompyle2, although some 2.6 bugs from
-uncompyle2 have been fixed.  We handle 2.5 and 2.6 bytecode a little
-bit differently than the way uncompyle2 does so it is harder to assess
-quality.
+byte codes. So that will be yet another challenge.
 
 There is lots to do, so please dig in and help.
 

__pkginfo__.py

@@ -37,7 +37,7 @@ entry_points={
     ]}
 ftp_url            = None
 install_requires   = ['spark-parser >= 1.4.0',
-                      'xdis >= 1.1.4']
+                      'xdis >= 1.1.7']
 license            = 'MIT'
 mailing_list       = 'python-debugger@googlegroups.com'
 modname            = 'uncompyle6'

test/Makefile

@@ -30,20 +30,20 @@ check-3.2: check-bytecode
 check-3.3: check-bytecode
 	$(PYTHON) test_pythonlib.py --bytecode-3.3 --verify $(COMPILE)
 
-#: Run working tests from Python 3.5
-check-3.5: check-bytecode
-
 #: Run working tests from Python 3.4
 check-3.4: check-bytecode check-3.4-ok check-2.7-ok
 	$(PYTHON) test_pythonlib.py --bytecode-3.4 --verify $(COMPILE)
 
+#: Run working tests from Python 3.5
+check-3.5: check-bytecode
+
 #: Check deparsing only, but from a different Python version
 check-disasm:
 	$(PYTHON) dis-compare.py
 
 #: Check deparsing bytecode 2.x only
 check-bytecode-2:
-	$(PYTHON) test_pythonlib.py --bytecode-2.3 --bytecode-2.5 --bytecode-2.6 --bytecode-2.7
+	$(PYTHON) test_pythonlib.py --bytecode-2.3 --bytecode-2.4 --bytecode-2.5 --bytecode-2.6 --bytecode-2.7
 
 #: Check deparsing bytecode 3.x only
 check-bytecode-3:
@@ -51,12 +51,16 @@ check-bytecode-3:
 
 #: Check deparsing bytecode that works running Python 2 and Python 3
 check-bytecode: check-bytecode-3
-	$(PYTHON) test_pythonlib.py --bytecode-2.5 --bytecode-2.6 --bytecode-2.7
+	$(PYTHON) test_pythonlib.py --bytecode-2.3 --bytecode-2.4 --bytecode-2.5 --bytecode-2.6 --bytecode-2.7
 
 #: Check deparsing Python 2.3
 check-bytecode-2.3:
 	$(PYTHON) test_pythonlib.py --bytecode-2.3
 
+#: Check deparsing Python 2.4
+check-bytecode-2.4:
+	$(PYTHON) test_pythonlib.py --bytecode-2.4
+
 #: Check deparsing Python 2.5
 check-bytecode-2.5:
 	$(PYTHON) test_pythonlib.py --bytecode-2.5

test/add-test.py

@@ -2,12 +2,12 @@
 """ Trivial helper program to bytecompile and run an uncompile
 """
 import os, sys, py_compile
-assert len(sys.argv) == 2
-path = sys.argv[1]
-short = os.path.basename(path)
+assert len(sys.argv) >= 2
 version = sys.version[0:3]
-cfile =  "bytecode_%s/%s" % (version, short) + 'c'
-print("byte-compiling %s to %s" % (path, cfile))
-py_compile.compile(path, cfile)
-if sys.version >= (2, 6, 0):
-    os.system("../bin/uncompyle6 -a -t %s" % cfile)
+for path in sys.argv[1:]:
+    short = os.path.basename(path)
+    cfile =  "bytecode_%s/%s" % (version, short) + 'c'
+    print("byte-compiling %s to %s" % (path, cfile))
+    py_compile.compile(path, cfile)
+    if isinstance(version, str) or version >= (2, 6, 0):
+        os.system("../bin/uncompyle6 -a -t %s" % cfile)