Get ready for release 3.2.6

Add test for last change

.circleci/config.yml

@@ -0,0 +1,89 @@
+# This configuration was automatically generated from a CircleCI 1.0 config.
+# It should include any build commands you had along with commands that CircleCI
+# inferred from your project structure. We strongly recommend you read all the
+# comments in this file to understand the structure of CircleCI 2.0, as the idiom
+# for configuration has changed substantially in 2.0 to allow arbitrary jobs rather
+# than the prescribed lifecycle of 1.0. In general, we recommend using this generated
+# configuration as a reference rather than using it in production, though in most
+# cases it should duplicate the execution of your original 1.0 config.
+version: 2
+jobs:
+  build:
+    working_directory: ~/rocky/python-uncompyle6
+    parallelism: 1
+    shell: /bin/bash --login
+    # CircleCI 2.0 does not support environment variables that refer to each other the same way as 1.0 did.
+    # If any of these refer to each other, rewrite them so that they don't or see https://circleci.com/docs/2.0/env-vars/#interpolating-environment-variables-to-set-other-environment-variables .
+    environment:
+      CIRCLE_ARTIFACTS: /tmp/circleci-artifacts
+      CIRCLE_TEST_REPORTS: /tmp/circleci-test-results
+      COMPILE: --compile
+    # In CircleCI 1.0 we used a pre-configured image with a large number of languages and other packages.
+    # In CircleCI 2.0 you can now specify your own image, or use one of our pre-configured images.
+    # The following configuration line tells CircleCI to use the specified docker image as the runtime environment for you job.
+    # We have selected a pre-built image that mirrors the build environment we use on
+    # the 1.0 platform, but we recommend you choose an image more tailored to the needs
+    # of each job. For more information on choosing an image (or alternatively using a
+    # VM instead of a container) see https://circleci.com/docs/2.0/executor-types/
+    # To see the list of pre-built images that CircleCI provides for most common languages see
+    # https://circleci.com/docs/2.0/circleci-images/
+    docker:
+    - image: circleci/build-image:ubuntu-14.04-XXL-upstart-1189-5614f37
+      command: /sbin/init
+    steps:
+    # Machine Setup
+    #   If you break your build into multiple jobs with workflows, you will probably want to do the parts of this that are relevant in each
+    # The following `checkout` command checks out your code to your working directory. In 1.0 we did this implicitly. In 2.0 you can choose where in the course of a job your code should be checked out.
+    - checkout
+    # Prepare for artifact and test results  collection equivalent to how it was done on 1.0.
+    # In many cases you can simplify this from what is generated here.
+    # 'See docs on artifact collection here https://circleci.com/docs/2.0/artifacts/'
+    - run: mkdir -p $CIRCLE_ARTIFACTS $CIRCLE_TEST_REPORTS
+    # This is based on your 1.0 configuration file or project settings
+    - run:
+        working_directory: ~/rocky/python-uncompyle6
+        command: pyenv local 2.7.11 && pyenv rehash && pip install virtualenv && pip install nose && pip install pep8 && pip install six && pyenv rehash
+    # Dependencies
+    #   This would typically go in either a build or a build-and-test job when using workflows
+    # Restore the dependency cache
+    - restore_cache:
+        keys:
+        # This branch if available
+        - v1-dep-{{ .Branch }}-
+        # Default branch if not
+        - v1-dep-master-
+        # Any branch if there are none on the default branch - this should be unnecessary if you have your default branch configured correctly
+        - v1-dep-
+    # This is based on your 1.0 configuration file or project settings
+    - run: pip install --upgrade setuptools
+    - run: pip install -e .
+    - run: pip install pytest==3.2.5 hypothesis==3.0.0
+    # Save dependency cache
+    - save_cache:
+        key: v1-dep-{{ .Branch }}-{{ epoch }}
+        paths:
+        # This is a broad list of cache paths to include many possible development environments
+        # You can probably delete some of these entries
+        - vendor/bundle
+        - ~/virtualenvs
+        - ~/.m2
+        - ~/.ivy2
+        - ~/.bundle
+        - ~/.go_workspace
+        - ~/.gradle
+        - ~/.cache/bower
+    # Test
+    #   This would typically be a build job when using workflows, possibly combined with build
+    # This is based on your 1.0 configuration file or project settings
+    - run: python ./setup.py develop && make check-2.7
+    - run: cd ./test/stdlib && pyenv local 2.7.11 && bash ./runtests.sh 'test_[p-z]*.py'
+    # Teardown
+    #   If you break your build into multiple jobs with workflows, you will probably want to do the parts of this that are relevant in each
+    # Save test results
+    - store_test_results:
+        path: /tmp/circleci-test-results
+    # Save artifacts
+    - store_artifacts:
+        path: /tmp/circleci-artifacts
+    - store_artifacts:
+        path: /tmp/circleci-test-results

.github/ISSUE_TEMPLATE/bug-report.md

@@ -0,0 +1,77 @@
+---
+name: Bug report
+about: Tell us about uncompyle6 bugs
+
+---
+
+<!-- __Note:__ Have you read https://github.com/rocky/python-uncompyle6/blob/master/HOW-TO-REPORT-A-BUG.md ?
+
+Please remove any of the optional sections if they are not applicable.
+
+Prerequisites
+
+* Make sure the bytecode you have can be disassembled with a
+  disassembler.
+* Don't put bytecode and corresponding source code on any service that
+  requires registration to download.
+* When you open a bug report there is no privacy. If the legitimacy of
+  the activity is deemed suspicous, I may flag it as suspicious,
+  making the issue even more easy to detect.
+
+Bug reports that violate a prerequisite may be discarded.
+
+Note that there are way more bug-fix requestors than there are bug
+fixers. If you want you need more immediate, confidential or urgent
+assistance
+
+http://www.crazy-compilers.com/decompyle/ offers a byte-code
+decompiler service for versions of Python up to 2.6.
+
+-->
+
+## Description
+
+<!-- Add a clear and concise description of the bug. -->
+
+## How to Reproduce
+
+<!-- Please show both the input you gave and the
+output you got in describing how to reproduce the bug:
+
+or give a complete console log with input and output
+
+```console
+$ uncompyle6 <command-line-options>
+...
+$
+```
+
+Provide links to the Python bytecode. For example you can create a
+gist with the information. If you have the correct source code, you
+can add that too.
+
+-->
+
+## Expected behavior
+
+<!-- Add a clear and concise description of what you expected to happen. -->
+
+## Environment
+
+<!-- _This section sometimes is optional but helpful to us._
+
+Please modify for your setup
+
+- Uncompyle6 version: output from  `uncompyle6 --version` or `pip show uncompyle6`
+- Python version: `python -V`
+- OS and Version: [e.g. Ubuntu bionic]
+
+-->
+
+## Additional Environment or Context
+
+<!-- _This section is optional._
+
+Add any other context about the problem here or special environment setup.
+
+-->

.github/ISSUE_TEMPLATE/feature-request.md

@@ -0,0 +1,22 @@
+---
+name: Feature Request
+about: Tell us about a new feature that you would like to see in uncompyle6
+
+---
+
+## Description
+
+<!-- Add a short description of the feature. This might
+include same input and output. -->
+
+## Background
+
+<!-- Add any additional background for the
+feature, for example: user scenarios, or the value of the feature. -->
+
+## Tests
+<!-- _This section is optional._
+
+Add text with suggestions on how to test the feature,
+if it is not obvious.
+-->

.travis.yml

@@ -1,16 +1,17 @@
 language: python
 
-sudo: false
-
 python:
   - '3.5'
   - '2.7'
   - '2.6'
-  - '3.3'
   - '3.4'
-  - '3.2'
   - '3.6'
 
+matrix:
+  include:
+    - python: '3.7'
+      dist: xenial  # required for Python >= 3.7 (travis-ci/travis-ci#9069)
+
 install:
 - pip install -e .
 - pip install -r requirements-dev.txt

HOW-TO-REPORT-A-BUG.md

@@ -1,6 +1,21 @@
-# How to report a Bug
+<!-- markdown-toc start - Don't edit this section. Run M-x markdown-toc-refresh-toc -->
+**Table of Contents**
 
-## The difficulty of the problem
+- [The difficulty of the problem](#the-difficulty-of-the-problem)
+- [Is it really a bug?](#is-it-really-a-bug)
+    - [Do you have valid bytecode?](#do-you-have-valid-bytecode)
+    - [Semantic equivalence vs. exact source code](#semantic-equivalence-vs-exact-source-code)
+- [What to send (minimum requirements)](#what-to-send-minimum-requirements)
+- [What to send (additional helpful information)](#what-to-send-additional-helpful-information)
+    - [But I don't *have* the source code!](#but-i-dont-have-the-source-code)
+    - [But I don't *have* the source code and am incapable of figuring how how to do a hand disassembly!](#but-i-dont-have-the-source-code-and-am-incapable-of-figuring-how-how-to-do-a-hand-disassembly)
+- [Narrowing the problem](#narrowing-the-problem)
+- [Karma](#karma)
+- [Confidentiality of Bug Reports](#confidentiality-of-bug-reports)
+- [Ethics](#ethics)
+
+<!-- markdown-toc end -->
+# The difficulty of the problem
 
 This decompiler is a constant work in progress: Python keeps
 changing, and so does its code generation.
@@ -33,7 +48,7 @@ prescribed cases, the ill-defined amorphous cases as well will get
 handled as well.
 
 In sum, you may need to do some work to have the bug you have found
-handled before the hundreds of other bugs, and things I could be
+handled before the hundreds of other bugs, and other things I could be
 doing.
 
 No one is getting paid to work to work on this project, let alone the
@@ -41,10 +56,10 @@ bugs you may have an interest in. If you require decompiling bytecode
 immediately, consider using a decompilation service, listed further
 down in this document.
 
-## Is it really a bug?
+# Is it really a bug?
 
 
-### Do you have valid bytecode?
+## Do you have valid bytecode?
 
 As mentioned in README.rst, this project doesn't handle obfuscated
 code. See README.rst for suggestions for how to remove some kinds of
@@ -55,11 +70,11 @@ Python comes with a disassembly module called `dis`. A prerequisite
 module for this package, `xdis` has a cross-python version
 disassembler called `pydisasm`.
 
-### Semantic equivalence vs. exact source code
+## Semantic equivalence vs. exact source code
 
 Consider how Python compiles something like "(x*y) + 5". Early on
 Python creates an "abstract syntax tree" (AST) for this. And this is
-"abstract" in the sense that unimportant, redundant or unnecceary
+"abstract" in the sense that unimportant, redundant or unnecessary
 items have been removed. Here, this means that any notion that you
 wrote "x+y" in parenthesis is lost, since in this context they are
 unneeded. Also lost is the fact that the multiplication didn't have
@@ -132,7 +147,7 @@ Python will eliminate the entire "if" statement.
 So just because the text isn't the same, does not
 necessarily mean there's a bug.
 
-## What to send (minimum requirements)
+# What to send (minimum requirements)
 
 The basic requirement is pretty simple:
 
@@ -146,7 +161,7 @@ sending is too large.
 
 Also try to narrow the bug. See below.
 
-## What to send (additional helpful information)
+# What to send (additional helpful information)
 
 Some kind folks also give the invocation they used and the output
 which usually includes an error message produced. This is
@@ -159,7 +174,7 @@ provide the input command and the output from that, please give:
 * Python interpreter version used
 
 
-### But I don't *have* the source code!
+## But I don't *have* the source code!
 
 Sure, I get it. No problem. There is Python assembly code on parse
 errors, so simply by hand decompile that. To get a full disassembly,
@@ -167,7 +182,7 @@ use `pydisasm` from the [xdis](https://pypi.python.org/pypi/xdis)
 package. Opcodes are described in the documentation for
 the [dis](https://docs.python.org/3.6/library/dis.html) module.
 
-### But I don't *have* the source code and am incapable of figuring how how to do a hand disassembly!
+### But I don't *have* the source code and am incapable of figuring how to do a hand disassembly!
 
 Well, you could learn. No one is born into this world knowing how to
 disassemble Python bytecode. And as Richard Feynman once said, "What
@@ -179,7 +194,7 @@ Compilers](http://www.crazy-compilers.com/decompyle/) offers a
 byte-code decompiler service for versions of Python up to 2.6. (If
 there are others around let me know and I'll list them here.)
 
-## Narrowing the problem
+# Narrowing the problem
 
 I don't need or want the entire source code base for the file(s) or
 module(s) can't be decompiled. I just need those file(s) or module(s).
@@ -197,22 +212,53 @@ what doesn't. That is useful. Or maybe the same file will decompile
 properly on a neighboring version of Python. That is helpful too.
 
 In sum, the more you can isolate or narrow the problem, the more
-likley the problem will be fixed and fixed sooner.
+likely the problem will be fixed and fixed sooner.
 
-## Confidentiality of Bug Reports
+# Karma
+
+I realize that following the instructions given herein puts a bit of
+burden on the bug reporter. In my opinion, this is justified as
+attempts to balance somewhat the burden and effort needed to fix the
+bug and the attempts to balance number of would-be bug reporters with
+the number of bug fixers. Better bug reporters are more likely to move
+in the category of bug fixers.
+
+The barrier to reporting a big is pretty small: all you really need is
+a github account, and the ability to type something after clicking
+some buttons. So the reality is that many people just don't bother to
+read these instructions, let alone follow it to any simulacrum.
+
+And the reality is also that bugs sometimes get fixed even though
+these instructions are not followed.
+
+So one factors I may take into consideration is the bug reporter's karma.
+
+* Have you demonstrably contributed to open source? I may look at your
+  github profile to see what contributions you have made, how popular
+  those contributions are, or how popular you are.
+* How appreciative are you? Have you starred this project that you are
+  seeking help from? Have you starred _any_ github project? And the above
+  two kind of feed into ...
+* Attitude. Some people feel that they are doing me and the world a
+  great favor by just pointing out that there is a problem whose solution
+  would greatly benefit them. Perhaps this is why they feel that
+  instructions are not to be followed by them, nor any need for
+  showing evidence gratitude when help is offered them.
+
+# Confidentiality of Bug Reports
 
 When you report a bug, you are giving up confidentiality to the source
 code and the byte code. However, I would imagine that if you have
 narrowed the problem sufficiently, confidentiality of the little that
 remains would not be an issue.
 
-However feel free to remove any commments, and modify variable names
+However feel free to remove any comments, and modify variable names
 or constants in the source code.
 
-## Ethics
+# Ethics
 
 I do not condone using this program for unethical or illegal purposes.
-More detestful, at least to me, is asking for help to assist you in
+More detestable, at least to me, is asking for help to assist you in
 something that might not legitimate.
 
 Don't use the issue tracker for such solicitations. To try to stave

Makefile

@@ -38,6 +38,7 @@ check-3.0 check-3.1 check-3.2 check-3.6:
 	$(MAKE) -C test $@
 
 check-3.7: pytest
+	$(MAKE) -C test check
 
 #:PyPy 2.6.1 PyPy 5.0.1, or PyPy 5.8.0-beta0
 # Skip for now

NEWS.md

@@ -1,4 +1,74 @@
-uncompyle6 3.1.2 2018-04-08 Eastern Orthodox Easter
+3.2.6 2019-03-23 Mueller Report
+=======================================
+
+Mostly more of the same: bug fixes and pull requests.
+
+Bug Fixes
+-----------
+
+* [#155: Python 3.x bytecode confusing "try/else" with "try" in a loop](https://github.com/rocky/python-uncompyle6/issues/155),
+* [#200: Python 3 bug in not detecting end bounds of an "if" ... "elif"](https://github.com/rocky/python-uncompyle6/issues/200),
+* [#208: Comma placement in 3.6 and 3.7 **kwargs](https://github.com/rocky/python-uncompyle6/issues/208),
+* [#209: Fix "if" return boundary in 3.6+](https://github.com/rocky/python-uncompyle6/issues/209),
+* [#215: 2.7 can have two JUMP_BACKs at the end of a while loop](https://github.com/rocky/python-uncompyle6/issues/215)
+
+Pull Requests
+----------------
+
+* [#202: Better "assert" statement detemination in Python 2.7](https://github.com/rocky/python-uncompyle6/pull/211)
+* [#204: Python 3.7 testing](https://github.com/rocky/python-uncompyle6/pull/204)
+* [#205: Run more f-string tests on Python 3.7](https://github.com/rocky/python-uncompyle6/pull/205)
+* [#211: support utf-8 chars in Python 3 sourcecode](https://github.com/rocky/python-uncompyle6/pull/202)
+
+
+
+3.2.5 2018-12-30 Clearout sale
+======================================
+
+- 3.7.2 Remove deprecation warning on regexp string that isn't raw
+- main.main() parameter `codes` is not used - note that
+- Improve Python 3.6+ control flow detection
+- More complete fragment instruction annotation for `imports`
+
+3.2.4 2018-10-27 7x9 release
+===================================
+
+- Bug fixes #180, #182, #187, #192
+- Enhancements #189
+- Internal improvements
+
+3.2.3 2018-06-04 Michael Cohen flips and Fleetwood Redux
+======================================================================
+- Python 1.3 support 3.0 bug and
+- fix botched parameter ordering of 3.x in last release
+
+3.2.2 2018-06-04 When I'm 64
+===================================
+
+- Python 3.0 support and bug fixes
+
+3.2.1 2018-06-04 MF
+=======================
+
+- Python 1.4 and 1.5 bug fixes
+
+3.2.0 2018-05-19 Rocket Scientist
+=========================================
+
+- Add rudimentary 1.4 support (still a bit buggy)
+- add --tree+ option to show formatting rule, when it is constant
+- Python 2.7.15candidate1 support (via xdis)
+- bug fixes, especially for 3.7 (but 2.7 and 3.6 and others as well)
+
+3.1.3 2018-04-16
+====================
+
+- Add some Python 3.7 rules, such as for handling LOAD_METHOD (not complete)
+- Fix some fragment bugs
+- small doc changes
+
+3.1.2 2018-04-08 Eastern Orthodox Easter
+==================================================
 
 - Python 3.x subclass and call parsing fixes
 - Allow/note running on Python 3.1
@@ -6,7 +76,8 @@ uncompyle6 3.1.2 2018-04-08 Eastern Orthodox Easter
 - DRY instruction building code between 2.x and 3.x
 - expand testing
 
-uncompyle6 3.1.1 2018-04-01 Easter April Fool's
+3.1.1 2018-04-01 Easter April Fool's
+=============================================
 
 Jesus on Friday's New York Times puzzle: "I'm stuck on 2A"
 
@@ -19,7 +90,8 @@ Jesus on Friday's New York Times puzzle: "I'm stuck on 2A"
 - more runtime testing of decompiled code
 - more removal of parenthesis around calls via setting precidence
 
-uncompyle6 3.1.0 2018-03-21 Equinox
+3.1.0 2018-03-21 Equinox
+==============================
 
 - Add code_deparse_with_offset() fragment function.
 - Correct paramenter call fragment deparse_code()
@@ -27,7 +99,8 @@ uncompyle6 3.1.0 2018-03-21 Equinox
   About 5% of 3.6 fail parsing now. But
   semantics still needs much to be desired.
 
-uncompyle6 3.0.1 2018-02-17
+3.0.1 2018-02-17
+====================
 
 - All Python 2.6.9 standard library files weakly verify
 - Many 3.6 fixes. 84% of the first 200 standard library files weakly compile.
@@ -37,7 +110,8 @@ uncompyle6 3.0.1 2018-02-17
 - And more add tests target previous existing bugs more completely
 - sync recent license changes in metadata
 
-uncompyle6 3.0.0 2018-02-17
+3.0.0 2018-02-17
+====================
 
 - deparse_code() and lookalikes from the various semantic actions are
   now deprecated. Instead use new API code_deparse() which makes the
@@ -59,7 +133,8 @@ A bit more work is still needed for 3.6 especially in the area of
 function calls and definitions.
 
 
-uncompyle6 2.16.0 2018-02-17
+2.16.0 2018-02-17
+=====================
 
 - API additions:
   - add fragments.op_at_code_loc() and
@@ -71,18 +146,21 @@ uncompyle6 2.16.0 2018-02-17
 - Fix Python 3.5+ CALL_FUNCTION_VAR and BUILD_LIST_UNPACK in call; with this
   we can can handle 3.5+ f(a, b, *c, *d, *e) now
 
-uncompyle6 2.15.1 2018-02-05
+2.15.1 2018-02-05
+=====================
 
 - More bug fixes and revert an improper bug fix in 2.15.0
 
-uncompyle6 2.15.0 2018-02-05 pycon2018.co
+2.15.0 2018-02-05 pycon2018.co
+=====================================
 
 - Bug fixes
 - Code fragment improvements
 - Code cleanups
 - Expand testing
 
-uncompyle6 2.15.1 2018-01-27
+2.15.1 2018-01-27
+=====================
 
 - Add --linemap option to give line correspondences
   between original source lines and reconstructed line sources.
@@ -94,7 +172,8 @@ uncompyle6 2.15.1 2018-01-27
 - Correct 3.6+ calls with kwargs
 - Describe the difficulty of 3.6 in README
 
-uncompyle6 2.14.3 2018-01-19
+2.14.3 2018-01-19
+=====================
 
 - Fix bug in 3.5+ await stmt
 - Better version to magic handling; handle 3.5.2 .. 3.5.4 versions
@@ -106,7 +185,8 @@ uncompyle6 2.14.3 2018-01-19
 - better tests in setup.py for running the right version of Python
 - Fix 2.6- parsing of "for .. try/else" ...  with "continue"  inside
 
-uncompyle6 2.14.2 2018-01-09 Samish
+2.14.2 2018-01-09 Samish
+==============================
 
 Decompilation bug fixes, mostly 3.6 and pre 2.7
 
@@ -124,7 +204,8 @@ Decompilation bug fixes, mostly 3.6 and pre 2.7
   Python versions
 - Match Python AST names more closely when possible
 
-uncompyle6 2.14.1 2017-12-10 Dr. Gecko
+2.14.1 2017-12-10 Dr. Gecko
+===================================
 
 - Many decompilation bugfixes
 - Grammar rule reduction and version isolation
@@ -132,7 +213,8 @@ uncompyle6 2.14.1 2017-12-10 Dr. Gecko
   with Python AST
 - Start automated Python stdlib testing - full round trip
 
-uncompyle6 2.14.0 2017-11-26 johnnybamazing
+2.14.0 2017-11-26 johnnybamazing
+=========================================
 
 - Start to isolate grammar rules between versions
   and remove used grammar rules
@@ -140,7 +222,8 @@ uncompyle6 2.14.0 2017-11-26 johnnybamazing
   (many more remain)
 - Add stdlib/runtests.sh for even more rigorous testing
 
-uncompyle6 2.13.3 2017-11-13
+2.13.3 2017-11-13
+=====================
 
 Overall: better 3.6 decompiling and some much needed code refactoring and cleanup
 
@@ -166,22 +249,26 @@ Overall: better 3.6 decompiling and some much needed code refactoring and cleanu
 - reinstate some bytecode tests since decompiling has gotten better
 - Revise how to report a bug
 
-uncompyle6 2.13.2 2017-10-12
+2.13.2 2017-10-12
+=====================
 
 - Re-release using a more automated approach
 
-uncompyle6 2.13.1 2017-10-11
+2.13.1 2017-10-11
+=====================
 
 - Re-release because Python 2.4 source uploaded rather than 2.6-3.6
 
-uncompyle6 2.13.0 2017-10-10
+2.13.0 2017-10-10
+=====================
 
 - Fixes in deparsing lambda expressions
 - Improve table-semantics descriptions
 - Document hacky customize arg count better (until we can remove it)
 - Update to use xdis 3.7.0 or greater
 
-uncompyle6 2.12.0 2017-09-26
+2.12.0 2017-09-26
+=====================
 
 - Use xdis 3.6.0 or greater now
 - Small semantic table cleanups
@@ -189,11 +276,13 @@ uncompyle6 2.12.0 2017-09-26
 - Slightly more Python 3.7, but still failing a lot
 - Cross Python 2/3 compatibility with annotation arguments
 
-uncompyle6 2.11.5 2017-08-31
+2.11.5 2017-08-31
+=====================
 
 - Skeletal support for Python 3.7
 
-uncompyle6 2.11.4 2017-08-15
+2.11.4 2017-08-15
+=====================
 
 * scanner and parser now allow 3-part version string lookups,
   e.g. 2.7.1 We allow a float here, but if passed a string like '2.7'. or
@@ -205,7 +294,8 @@ uncompyle6 2.11.4 2017-08-15
 * Some PyPy tolerance in validate testing.
 * Some pyston tolerance
 
-uncompyle6 2.11.3 2017-08-09
+2.11.3 2017-08-09
+=====================
 
 Very minor changes
 
@@ -214,20 +304,24 @@ Very minor changes
 - use xdis opcode sets
 - xdis "exception match" is now "exception-match"
 
-uncompyle6 2.11.2 2017-07-09
+2.11.2 2017-07-09
+=====================
 
 - Start supporting Pypy 3.5 (5.7.1-beta)
 - use xdis 3.5.0's opcode sets and require xdis 3.5.0
 - Correct some Python 2.4-2.6 loop detection
 - guard against badly formatted bytecode
 
-uncompyle6 2.11.1 2017-06-25
+2.11.1 2017-06-25
+=====================
 
 - Python 3.x annotation and function signature fixes
 - Bump xdis version
 - Small pysource bug fixes
 
-uncompyle6 2.11.0 2017-06-18 Fleetwood
+2.11.0 2017-06-18 Fleetwood
+==================================
+
 - Major improvements in fragment tracking
   * Add nonterminal node in extractInfo
   * tag more offsets in expressions
@@ -237,14 +331,16 @@ uncompyle6 2.11.0 2017-06-18 Fleetwood
 - Fixes yet again for make_function node handling; document what's up here
 - Fix bug in snowflake Python 3.5 *args kwargs
 
-uncompyle6 2.10.1 2017-06-3 Marylin Frankel
+2.10.1 2017-06-3 Marylin Frankel
+========================================
 
 - fix some fragments parsing bugs
   - was returning the wrong type sometimes in deparse_code_around_offset()
   - capture function name in offsets
   - track changes to ifelstrmtr node from pysource into fragments
 
-uncompyle6 2.10.0 2017-05-30 Elaine Gordon
+2.10.0 2017-05-30 Elaine Gordon
+=======================================
 
 - Add fuzzy offset deparse look up
 - 3.6 bug fixes
@@ -264,19 +360,21 @@ uncompyle6 2.10.0 2017-05-30 Elaine Gordon
 - 2.3, 2.4 "if 1 .." fixes
 - 3.x annotation fixes
 
-uncompyle6 2.9.11 2017-04-06
+2.9.11 2017-04-06
+=====================
 
 - Better support for Python 3.5+ BUILD_MAP_UNPACK
 - Start 3.6 CALL_FUNCTION_EX support
 - Many decompilation bug fixes. (Many more remain). See ChangeLog
 
-uncompyle6 2.9.10 2017-02-25
+2.9.10 2017-02-25
+=====================
 
 - Python grammar rule fixes
 - Add ability to get grammar coverage on runs
 - Handle Python 3.6 opcode BUILD_CONST_KEYMAP
 
-uncompyle6 2.9.9 2016-12-16
+2.9.9 2016-12-16
 
 - Remaining Python 3.5 ops handled
   (this also means more Python 3.6 ops are handled)
@@ -286,7 +384,8 @@ uncompyle6 2.9.9 2016-12-16
 - Better control-flow detection
 - Code cleanups and misc bug fixes
 
-uncompyle6 2.9.8 2016-12-16
+2.9.8 2016-12-16
+====================
 
 - Better control-flow detection
 - pseudo instruction THEN in 2.x
@@ -299,7 +398,8 @@ uncompyle6 2.9.8 2016-12-16
 - verify call fixes for Python <= 2.4
 - more Python lint
 
-uncompyle6 2.9.7 2016-12-16
+2.9.7 2016-12-16
+====================
 
 - Start to handle 3.5/3.6 build_map_unpack_with_call
 - Some Python 3.6 bytecode to wordcode conversion fixes
@@ -309,7 +409,8 @@ uncompyle6 2.9.7 2016-12-16
 - some 3.2 compatibility
 - Better Python 3 control flow detection by adding Pseudo ELSE opcodes
 
-uncompyle6 2.9.6 2016-12-04
+2.9.6 2016-12-04
+====================
 
 - Shorten Python3 grammars with + and *
   this requires spark parser 1.5.1
@@ -317,7 +418,8 @@ uncompyle6 2.9.6 2016-12-04
   decompile accuracy. This too requires
   spark parser 1.5.1
 
-uncompyle6 2.9.6 2016-11-20
+2.9.6 2016-11-20
+====================
 
 - Correct MANIFEST.in
 - More AST grammar checking
@@ -334,7 +436,8 @@ uncompyle6 2.9.6 2016-11-20
 - Python 2 and 3 detect structure code is more similar
 - Handle Docstrings with embedded triple quotes (""")
 
-uncompyle6 2.9.5 2016-11-13
+2.9.5 2016-11-13
+====================
 
 - Fix Python 3 bugs:
   * improper while 1 else
@@ -344,13 +447,15 @@ uncompyle6 2.9.5 2016-11-13
 - Start grammar misparse checking
 
 
-uncompyle6 2.9.4 2016-11-02
+2.9.4 2016-11-02
+====================
 
 - Handle Python 3.x function annotations
 - track def keyword-parameter line-splitting in source code better
 - bump min xdis version to mask previous xdis bug
 
-uncompyle6 2.9.3 2016-10-26
+2.9.3 2016-10-26
+====================
 
 Release forced by incompatibility change in xdis 3.2.0.
 
@@ -365,7 +470,8 @@ Release forced by incompatibility change in xdis 3.2.0.
   * Handle 3.6 handle single and multiple fstring better
 
 
-uncompyle6 2.9.2 2016-10-15
+2.9.2 2016-10-15
+====================
 
 - use source-code line breaks to assist in where to break
   in tuples and maps
@@ -373,12 +479,14 @@ uncompyle6 2.9.2 2016-10-15
 - Fix some Python 2.6 and below bugs
 - DRY fragments.py code a little
 
-uncompyle6 2.9.1 2016-10-09
+2.9.1 2016-10-09
+====================
 
 - Improved Python 1.5 decompiling
 - Handle old-style pre Python 2.2 classes
 
-uncompyle6 2.9.0 2016-10-09
+2.9.0 2016-10-09
+====================
 
 - Use xdis 3.0.0 protocol load_module.
   this Forces change in requirements.txt and _pkg_info_.py
@@ -388,7 +496,8 @@ uncompyle6 2.9.0 2016-10-09
 - Fix bug with -t ...  Wasn't showing source text when -t option was given
 - Fix 2.1-2.6 bug in list comprehension
 
-uncompyle6 2.8.4 2016-10-08
+2.8.4 2016-10-08
+====================
 
 - Python 3 disassembly bug fixes
 - Python 3.6 fstring bug fixes (from moagstar)
@@ -396,7 +505,8 @@ uncompyle6 2.8.4 2016-10-08
 - COME_FROM suffixes added in Python3
 - use .py extension in verification disassembly
 
-uncompyle6 2.8.3 2016-09-11 live from NYC!
+2.8.3 2016-09-11 live from NYC!
+=======================================
 
 NOTE: this is possibly the last release before a major reworking of
 control-flow structure detection is done.
@@ -424,14 +534,16 @@ control-flow structure detection is done.
 - bump xdis requirement so we can deparse dropbox 2.5 code
 - Added H. Goebel's changes before 2.4 in DECOMPYLE-2.4-CHANGELOG.txt
 
-uncompyle6 2.8.2 2016-08-29
+2.8.2 2016-08-29
+====================
 
 - Handle Python 3.6 format string conversions !r, !s, !a
 - Start to handle 3.1 bytecode
 - Fix some PyPy translation bugs
 - We now only handle 3.6.0a3+ since that is incompatible with 3.6 before that
 
-uncompyle6 2.8.1 2016-08-20
+2.8.1 2016-08-20
+====================
 
 - Add Python 2.2 decompilation
 
@@ -439,7 +551,8 @@ uncompyle6 2.8.1 2016-08-20
  * PyPy LOOKUP_METHOD bug
  * Python 3.6 FORMAT_VALUE handles expressions now
 
-uncompyle6 2.8.0 2016-08-03
+2.8.0 2016-08-03
+====================
 
 - Start Python 3.6 support (moagstar)
   more work on PEP 498 needed
@@ -450,20 +563,23 @@ uncompyle6 2.8.0 2016-08-03
 - better grammar and semantic action segregation based
   on python bytecode version
 
-uncompyle6 2.7.1 2016-07-26
+2.7.1 2016-07-26
+====================
 
 - PyPy bytecodes for 2.7 and 3.2 added
 - Instruction formatting improved slightly
 - 2.7 bytecode "continue" bug fixed
 
-uncompyle6 2.7.0 2016-07-15
+2.7.0 2016-07-15
+====================
 
 - Many Syntax and verification bugs removed
   tested on standard libraries from 2.3.7 to 3.5.1
   and they all decompile and verify fine.
   I'm sure there are more bugs though.
 
-uncompyle6 2.6.2 2016-07-11 Manhattenhenge
+2.6.2 2016-07-11 Manhattenhenge
+=======================================
 
 - Extend bytecodes back to 2.3
 - Fix bugs:
@@ -472,13 +588,15 @@ uncompyle6 2.6.2 2016-07-11 Manhattenhenge
   * continue statements
 - DRY and segregate grammar more
 
-uncompyle6 2.6.1 2016-07-08
+2.6.1 2016-07-08
+====================
 
 - Go over Python 2.5 bytecode deparsing
   all library programs now deparse
 - Fix a couple bugs in 2.6 deparsing
 
-uncompyle6 2.6.0 2016-07-07
+2.6.0 2016-07-07
+====================
 
 - Improve Python 2.6 bytecode deparsing:
   stdlib now will deparse something
@@ -487,7 +605,8 @@ uncompyle6 2.6.0 2016-07-07
 - Fix bug in installing uncompyle6 script
 - Doc improvements
 
-uncompyle6 2.5.0 2016-06-22  Summer Solstice
+2.5.0 2016-06-22 Summer Solstice
+========================================
 
 - Much better Python 3.2-3.5 coverage.
   3.4.6 is probably the best;3.2 and 3.5 are weaker
@@ -496,7 +615,8 @@ uncompyle6 2.5.0 2016-06-22  Summer Solstice
 - Better fragment offset tracking
 - Some (much-needed) code refactoring
 
-uncompyle6 2.4.0 2016-05-18 (in memory of Lewis Bernstein)
+2.4.0 2016-05-18 (in memory of Lewis Bernstein)
+===========================================================
 
 - Many Python 3 bugs fixed:
   * Python 3.2 to 3.5 libraries largely
@@ -511,7 +631,8 @@ uncompyle6 2.4.0 2016-05-18 (in memory of Lewis Bernstein)
   * handle complex number unmarshaling
   * Running on Python 2 to works on Python 3.5 bytecodes now
 
-uncompyle6 2.3.5 and 2.3.6 2016-05-14
+2.3.5 and 2.3.6 2016-05-14
+=================================
 
 - Python 2 class decorator fix (thanks to Tey)
 - Fix fragment parsing bugs
@@ -523,20 +644,23 @@ uncompyle6 2.3.5 and 2.3.6 2016-05-14
 - Correct history based on info from Dan Pascu
 - Fix up pip packaging, ugh.
 
-uncompyle6 2.3.4 2016-05-5
+2.3.4 2016-05-5
+===================
 
 - More Python 3.5 parsing bugs addressed
 - decompiling Python 3.5 from other Python versions works
 - test from Python 3.2
 - remove "__module__ = __name__" in 3.0 <= Python 3.2
 
-uncompyle6 2.3.3 2016-05-3
+2.3.3 2016-05-3
+===================
 
 - Fix bug in running uncompyle6 script on Python 3
 - Speed up performance on deparsing long lists by grouping in chunks of 32 and 256 items
 - DRY Python expressions between Python 2 and 3
 
-uncompyle6 2.3.2 2016-05-1
+2.3.2 2016-05-1
+===================
 
 - Add --version option standalone scripts
 - Correct License information in package
@@ -545,17 +669,20 @@ uncompyle6 2.3.2 2016-05-1
   specific grammar code
 - Fix bug in 3.5+ constant map parsing
 
-uncompyle6 2.3.0, 2.3.1 2016-04-30
+2.3.0, 2.3.1 2016-04-30
+=============================
 
 - Require spark_parser  >= 1.1.0
 
-uncompyle6 2.2.0 2016-04-30
+2.2.0 2016-04-30
+====================
 
 - Spark is no longer here but pulled separate package spark_parse
 - Python 3 parsing fixes
 - More tests
 
-uncompyle6 2.2.0 2016-04-02
+2.2.0 2016-04-02
+====================
 
 - Support single-mode (in addition to exec-mode) compilation
 - Start to DRY Python 2 and Python 3 grammars
@@ -563,7 +690,8 @@ uncompyle6 2.2.0 2016-04-02
 - Fix bug in uncomplye6 -d and -r options (via lelicopter)
 
 
-uncompyle6 2.1.3 2016-01-02
+2.1.3 2016-01-02
+====================
 
 - Limited support for decompiling Python 3.5
 - Improve Python 3 class deparsing
@@ -572,18 +700,21 @@ uncompyle6 2.1.3 2016-01-02
 - increase test coverage
 - fix misc small bugs and some improvements
 
-uncompyle6 2.1.2 2015-12-31
+2.1.2 2015-12-31
+====================
 
 - Fix cross-version Marshal loading
 - Handle Python 3.3 . dotted class names
 - Limited 3.5 support: allows deparsing other versions
 - Refactor code more, misc bug fixes
 
-uncompyle6 2.1.1 2015-12-27
+2.1.1 2015-12-27
+====================
 
 - packaging issues
 
-uncompyle6 2.1.0 2015-12-27
+2.1.0 2015-12-27
+====================
 
 - Python 3.x deparsing much more solid
 - Better cross-version deparsing
@@ -592,7 +723,8 @@ Some bugs squashed while other run rampant. Some code cleanup while
 much more is yet needed. More tests added, but many more are needed.
 
 
-uncompyle6 2.0.0 2015-12-11
+2.0.0 2015-12-11
+====================
 
 Changes from uncompyle2
 

README.rst

@@ -1,4 +1,4 @@
-|buildstatus|
+|buildstatus| |Latest Version| |Supported Python Versions|
 
 uncompyle6
 ==========
@@ -11,8 +11,9 @@ Introduction
 ------------
 
 *uncompyle6* translates Python bytecode back into equivalent Python
-source code. It accepts bytecodes from Python version 1.5, and 2.1 to
+source code. It accepts bytecodes from Python version 1.3 to version
-3.7 or so, including PyPy bytecode and Dropbox's Python 2.5 bytecode.
+3.7, spanning over 22 years of Python releases. We include Dropbox's
+Python 2.5 bytecode and some PyPy bytecode.
 
 Why this?
 ---------
@@ -29,11 +30,11 @@ CPython bytecode decompilers is the ability to deparse just
 *fragments* of source code and give source-code information around a
 given bytecode offset.
 
-I use the tree fragments to deparse fragments of code inside my
+I use the tree fragments to deparse fragments of code *at run time*
-trepan_ debuggers_. For that, bytecode offsets are recorded and
+inside my trepan_ debuggers_. For that, bytecode offsets are recorded
-associated with fragments of the source code. This purpose, although
+and associated with fragments of the source code. This purpose,
-compatible with the original intention, is yet a little bit different.
+although compatible with the original intention, is yet a little bit
-See this_ for more information.
+different.  See this_ for more information.
 
 Python fragment deparsing given an instruction offset is useful in
 showing stack traces and can be encorporated into any program that
@@ -52,8 +53,17 @@ You get the idea. This code pulls all of these forks together and
 *moves forward*. There is some serious refactoring and cleanup in this
 code base over those old forks.
 
-This project has the most complete support for Python 3.3 and above
+This demonstrably does the best in decompiling Python across all
-and the best all-around Python support.
+Python versions. And even when there is another project that only
+provides decompilation for subset of Python versions, we generally do
+demonstrably better for those as well.
+
+How can we tell? By taking Python bytecode that comes distributed with
+that version of Python and decompiling these.  Among those that
+successfully decompile, we can then make sure the resulting programs
+are syntactically correct by running the Python interpreter for that
+bytecode version.  Finally, in cases where the program has a test for
+itself, we can run the check on the decompiled code.
 
 We are serious about testing, and use automated processes to find
 bugs. In the issue trackers for other decompilers, you will find a
@@ -66,7 +76,7 @@ Requirements
 The code here can be run on Python versions 2.6 or later, PyPy 3-2.4,
 or PyPy-5.0.1.  Python versions 2.4-2.7 are supported in the
 python-2.4 branch.  The bytecode files it can read have been tested on
-Python bytecodes from versions 1.5, 2.1-2.7, and 3.0-3.6 and the
+Python bytecodes from versions 1.4, 2.1-2.7, and 3.0-3.6 and the
 above-mentioned PyPy versions.
 
 Installation
@@ -136,26 +146,26 @@ All of the Python decompilers that I have looked at have problems
 decompiling Python's control flow. In some cases we can detect an
 erroneous decompilation and report that.
 
-*Verification* is the process of decompiling bytecode, compiling with
+In older versions of Python it was possible to verify bytecode by
-a Python for that bytecode version, and then comparing the bytecode
+decompiling bytecode, and then compiling using the Python interpreter
-produced by the decompiled/compiled program. Some allowance is made
+for that bytecode version. Having done this the bytecode produced
-for inessential differences. But other semantically equivalent
+could be compared with the original bytecode. However as Python's code
-differences are not caught. For example ``1 and 0`` is decompiled to
+generation got better, this is no longer feasible.
-the equivalent ``0``; remnants of the first true evaluation (1) is
-lost when Python compiles this. When Python next compiles ``0`` the
-resulting code is simpler.
 
-*Weak Verification*
+There verification that we use that doesn't check bytecode for
-on the other hand doesn't check bytecode for equivalence but does
+equivalence but does check to see if the resulting decompiled source
-check to see if the resulting decompiled source is a valid Python
+is a valid Python program by running the Python interpreter. Because
-program by running the Python interpreter. Because the Python language
+the Python language has changed so much, for best results you should
-has changed so much, for best results you should use the same Python
+use the same Python version in checking as was used in creating the
-Version in checking as used in the bytecode.
+bytecode.
 
-Finally, we have automated running the standard Python tests after
+There are however an interesting class of these programs that is
-first compiling and decompiling the test program. Results here are a
+readily available give stronger verification: those programs that
-bit weak (if not better than most other Python decompilers). But over
+when run check some computation, or even better themselves.
-time this will probably get better.
+
+And already Python has a set of programs like this: the test suite
+for the standard library that comes with Python. We have some
+code in `test/stdlib` to facilitate this kind of checking.
 
 Python support is strongest in Python 2 for 2.7 and drops off as you
 get further away from that. Support is also probably pretty good for
@@ -165,19 +175,21 @@ that era was minimal)
 
 There is some work to do on the lower end Python versions which is
 more difficult for us to handle since we don't have a Python
-interpreter for versions 1.5, 1.6, and 2.0.
+interpreter for versions 1.6, and 2.0.
 
 In the Python 3 series, Python support is is strongest around 3.4 or
 3.3 and drops off as you move further away from those versions. Python
-3.6 changes things drastically by using word codes rather than byte
+3.0 is weird in that it in some ways resembles 2.6 more than it does
-codes. As a result, the jump offset field in a jump instruction
+3.1 or 2.7. Python 3.6 changes things drastically by using word codes
-argument has been reduced. This makes the `EXTENDED_ARG` instructions
+rather than byte codes. As a result, the jump offset field in a jump
-are now more prevalent in jump instruction; previously they had been
+instruction argument has been reduced. This makes the `EXTENDED_ARG`
-rare.  Perhaps to compensate for the additional `EXTENDED_ARG`
+instructions are now more prevalent in jump instruction; previously
-instructions, additional jump optimization has been added. So in sum
+they had been rare.  Perhaps to compensate for the additional
-handling control flow by ad hoc means as is currently done is worse.
+`EXTENDED_ARG` instructions, additional jump optimization has been
+added. So in sum handling control flow by ad hoc means as is currently
+done is worse.
 
-Also, between Python 3.5, 3.6 and 3.7 there have been major changes to the
+Between Python 3.5, 3.6 and 3.7 there have been major changes to the
 `MAKE_FUNCTION` and `CALL_FUNCTION` instructions.
 
 Currently not all Python magic numbers are supported. Specifically in
@@ -203,11 +215,12 @@ There is lots to do, so please dig in and help.
 See Also
 --------
 
-* https://github.com/zrax/pycdc : supports all versions of Python and is written in C++. Support for later Python 3 versions is a bit lacking though.
+* https://github.com/zrax/pycdc : purports to support all versions of Python. It is written in C++ and is most accurate for Python versions around 2.7 and 3.3 when the code was more actively developed. Accuracy for more recent versions of Python 3 and early versions of Python are especially lacking. See its `issue tracker <https://github.com/zrax/pycdc/issues>`_ for details. Currently lightly maintained.
-* https://code.google.com/archive/p/unpyc3/ : supports Python 3.2 only. The above projects use a different decompiling technique than what is used here.
+* https://code.google.com/archive/p/unpyc3/ : supports Python 3.2 only. The above projects use a different decompiling technique than what is used here. Currently unmaintained.
-* https://github.com/figment/unpyc3/ : fork of above, but supports Python 3.3 only. Includes some fixes like supporting function annotations
+* https://github.com/figment/unpyc3/ : fork of above, but supports Python 3.3 only. Includes some fixes like supporting function annotations. Currently unmaintained.
-* The HISTORY_ file.
+* https://github.com/wibiti/uncompyle2 : supports Python 2.7 only, but does that fairly well. There are situtations where `uncompyle6` results are incorrect while `uncompyle2` results are not, but more often uncompyle6 is correct when uncompyle2 is not. Because `uncompyle6` adheres to accuracy over idiomatic Python, `uncompyle2` can produce more natural-looking code when it is correct. Currently `uncompyle2` is lightly maintained. See its issue `tracker <https://github.com/wibiti/uncompyle2/issues>`_ for more details
 * `How to report a bug <https://github.com/rocky/python-uncompyle6/blob/master/HOW-TO-REPORT-A-BUG.md>`_
+* The HISTORY_ file.
 * https://github.com/rocky/python-xdis : Cross Python version disassembler
 * https://github.com/rocky/python-xasm : Cross Python version assembler
 * https://github.com/rocky/python-uncompyle6/wiki : Wiki Documents which describe the code and aspects of it in more detail
@@ -225,3 +238,6 @@ See Also
 .. _PJOrion: http://www.koreanrandom.com/forum/topic/15280-pjorion-%D1%80%D0%B5%D0%B4%D0%B0%D0%BA%D1%82%D0%B8%D1%80%D0%BE%D0%B2%D0%B0%D0%BD%D0%B8%D0%B5-%D0%BA%D0%BE%D0%BC%D0%BF%D0%B8%D0%BB%D1%8F%D1%86%D0%B8%D1%8F-%D0%B4%D0%B5%D0%BA%D0%BE%D0%BC%D0%BF%D0%B8%D0%BB%D1%8F%D1%86%D0%B8%D1%8F-%D0%BE%D0%B1%D1%84
 .. _Deobfuscator: https://github.com/extremecoders-re/PjOrion-Deobfuscator
 .. _Py2EXE: https://en.wikipedia.org/wiki/Py2exe
+.. |Supported Python Versions| image:: https://img.shields.io/pypi/pyversions/uncompyle6.svg
+.. |Latest Version| image:: https://badge.fury.io/py/uncompyle6.svg
+		 :target: https://badge.fury.io/py/uncompyle6

__pkginfo__.py

@@ -35,6 +35,7 @@ classifiers =  ['Development Status :: 5 - Production/Stable',
                 'Programming Language :: Python :: 2.5',
                 'Programming Language :: Python :: 2.6',
                 'Programming Language :: Python :: 2.7',
+                'Programming Language :: Python :: 3.0',
                 'Programming Language :: Python :: 3.1',
                 'Programming Language :: Python :: 3.2',
                 'Programming Language :: Python :: 3.3',
@@ -55,8 +56,8 @@ entry_points       = {
         'pydisassemble=uncompyle6.bin.pydisassemble:main',
     ]}
 ftp_url            = None
-install_requires   = ['spark-parser >= 1.8.5, < 1.9.0',
+install_requires   = ['spark-parser >= 1.8.7, < 1.9.0',
-                      'xdis >= 3.8.0, < 3.9.0', 'six']
+                      'xdis >= 3.9.0, < 3.10.0']
 
 license            = 'GPL3'
 mailing_list       = 'python-debugger@googlegroups.com'

admin-tools/check-newer-versions.sh

@@ -16,6 +16,7 @@ if ! source ./setup-master.sh ; then
 fi
 cd ..
 for version in $PYVERSIONS; do
+    echo --- $version ---
     if ! pyenv local $version ; then
 	exit $?
     fi
@@ -23,4 +24,5 @@ for version in $PYVERSIONS; do
     if ! make check; then
 	exit $?
     fi
+    echo === $version ===
 done

admin-tools/check-older-versions.sh

@@ -15,6 +15,7 @@ fi
 
 cd ..
 for version in $PYVERSIONS; do
+    echo --- $version ---
     if ! pyenv local $version ; then
 	exit $?
     fi
@@ -22,4 +23,5 @@ for version in $PYVERSIONS; do
     if ! make check ; then
 	exit $?
     fi
+    echo === $version ===
 done

admin-tools/how-to-make-a-release.md

@@ -30,9 +30,9 @@
 
     $ make ChangeLog
 
-#  Update NEWS from ChangeLog:
+#  Update NEWS.md from ChangeLog:
 
-    $ emacs NEWS
+    $ emacs NEWS.md
     $ make check
     $ git commit --amend .
     $ git push   # get CI testing going early

admin-tools/pycdc-runtests.sh

@@ -0,0 +1,15 @@
+#!/bin/bash
+# Use pycdc to run our test/bytecode* test suite
+bs=${BASH_SOURCE[0]}
+testdir=$(dirname $bs)/../test
+fulldir=$(readlink -f $testdir)
+cd $fulldir
+for dir in bytecode_* ; do
+    echo ========= $dir ================
+    cd $fulldir/$dir
+    for file in *.pyc; do
+	if ! pycdc $file > /dev/null ; then
+	    echo -----  $dir/$file ------
+	fi
+    done
+done

admin-tools/pyenv-newer-versions

@@ -5,4 +5,4 @@ if [[ $0 == ${BASH_SOURCE[0]} ]] ; then
     echo "This script should be *sourced* rather than run directly through bash"
     exit 1
 fi
-export PYVERSIONS='3.5.5 3.6.4 2.6.9 3.3.7 2.7.14 3.2.6 3.1.5 3.4.8'
+export PYVERSIONS='3.2.6 3.6.8 3.7.2 2.6.9 3.3.7 2.7.15 3.2.6 3.1.5 3.4.8'

admin-tools/setup-master.sh

@@ -1,5 +1,5 @@
 #!/bin/bash
-PYTHON_VERSION=3.6.4
+PYTHON_VERSION=3.6.5
 
 # FIXME put some of the below in a common routine
 function finish {

admin-tools/uncompyle2-runtests.sh

@@ -0,0 +1,32 @@
+#!/bin/bash
+# Use pycdc to run our test/bytecode_2.7* test suite
+bs=${BASH_SOURCE[0]}
+topdir=$(dirname $bs)/..
+(cd $topdir && pyenv local 2.7.14)
+testdir=$topdir/test
+fulldir=$(readlink -f $testdir)
+cd $fulldir
+
+for bytecode in bytecode_2.7/*.pyc ; do
+    echo $bytecode
+    uncompyle2 $bytecode > /dev/null
+    echo ================ $bytecode rc: $? ==============
+done
+
+tmpdir=/tmp/test-2.7
+( cd bytecode_2.7_run &&
+      mkdir $tmpdir || true
+      for bytecode in *.pyc ; do
+	  shortname=$(basename $bytecode .pyc)
+	  echo $bytecode
+	  py_file=${tmpdir}/${shortname}.py
+	  typeset -i rc=0
+	  uncompyle2 $bytecode > $py_file
+	  rc=$?
+	  if (( rc == 0 )); then
+	      python $py_file
+	      rc=$?
+	  fi
+	  echo ================ $bytecode rc: $rc ==============
+      done
+)

appveyor.yml

@@ -47,7 +47,7 @@ install:
 
   # Upgrade to the latest version of pip to avoid it displaying warnings
   # about it being out of date.
-  - "pip install --disable-pip-version-check --user --upgrade pip"
+  - "%PYTHON%\\python.exe -m pip install --disable-pip-version-check --user --upgrade pip"
 
   # Install the build dependencies of the project. If some dependencies contain
   # compiled extensions and are not provided as pre-built wheel packages,

circle.yml

@@ -1,14 +0,0 @@
-machine:
-    python:
-        version: 2.7.10
-    environment:
-        COMPILE: --compile
-
-dependencies:
-    override:
-        - pip install -e .
-        - pip install pytest==3.2.5 hypothesis
-test:
-    override:
-        - python ./setup.py develop && make check-2.7
-        - cd ./test/stdlib && pyenv local 2.7.10 && bash ./runtests.sh 'test_[p-z]*.py'

pytest/test_fstring.py

@@ -1,150 +1,154 @@
 # std
-import os
 # test
-import pytest
-import hypothesis
-from hypothesis import strategies as st
-# uncompyle6
 from uncompyle6 import PYTHON_VERSION, deparse_code
+import pytest
+pytestmark = pytest.mark.skipif(PYTHON_VERSION <= 2.6,
+                                reason='hypothesis needs 2.7 or later')
+if PYTHON_VERSION > 2.6:
+
+    import hypothesis
+    from hypothesis import strategies as st
+
+    # uncompyle6
 
 
-@st.composite
+    @st.composite
-def expressions(draw):
+    def expressions(draw):
-    # todo : would be nice to generate expressions using hypothesis however
+        # todo : would be nice to generate expressions using hypothesis however
-    # this is pretty involved so for now just use a corpus of expressions
+        # this is pretty involved so for now just use a corpus of expressions
-    # from which to select.
+        # from which to select.
-    return draw(st.sampled_from((
+        return draw(st.sampled_from((
-        'abc',
+            'abc',
-        'len(items)',
+            'len(items)',
-        'x + 1',
+            'x + 1',
-        'lineno',
+            'lineno',
-        'container',
+            'container',
-        'self.attribute',
+            'self.attribute',
-        'self.method()',
+            'self.method()',
-        # These expressions are failing, I think these are control
+            # These expressions are failing, I think these are control
-        # flow problems rather than problems with FORMAT_VALUE,
+            # flow problems rather than problems with FORMAT_VALUE,
-        # however I need to confirm this...
+            # however I need to confirm this...
-        #'sorted(items, key=lambda x: x.name)',
+            #'sorted(items, key=lambda x: x.name)',
-        #'func(*args, **kwargs)',
+            #'func(*args, **kwargs)',
-        #'text or default',
+            #'text or default',
-        #'43 if life_the_universe and everything else None'
+            #'43 if life_the_universe and everything else None'
-    )))
+        )))
 
 
-@st.composite
+    @st.composite
-def format_specifiers(draw):
+    def format_specifiers(draw):
-    """
+        """
-    Generate a valid format specifier using the rules:
+        Generate a valid format specifier using the rules:
 
-    format_spec ::=  [[fill]align][sign][#][0][width][,][.precision][type]
+        format_spec ::=  [[fill]align][sign][#][0][width][,][.precision][type]
-    fill        ::=  <any character>
+        fill        ::=  <any character>
-    align       ::=  "<" | ">" | "=" | "^"
+        align       ::=  "<" | ">" | "=" | "^"
-    sign        ::=  "+" | "-" | " "
+        sign        ::=  "+" | "-" | " "
-    width       ::=  integer
+        width       ::=  integer
-    precision   ::=  integer
+        precision   ::=  integer
-    type        ::=  "b" | "c" | "d" | "e" | "E" | "f" | "F" | "g" | "G" | "n" | "o" | "s" | "x" | "X" | "%"
+        type        ::=  "b" | "c" | "d" | "e" | "E" | "f" | "F" | "g" | "G" | "n" | "o" | "s" | "x" | "X" | "%"
 
-    See https://docs.python.org/2/library/string.html
+        See https://docs.python.org/2/library/string.html
 
-    :param draw: Let hypothesis draw from other strategies.
+        :param draw: Let hypothesis draw from other strategies.
 
-    :return: An example format_specifier.
+        :return: An example format_specifier.
-    """
+        """
-    alphabet_strategy = st.characters(min_codepoint=ord('a'), max_codepoint=ord('z'))
+        alphabet_strategy = st.characters(min_codepoint=ord('a'), max_codepoint=ord('z'))
-    fill = draw(st.one_of(alphabet_strategy, st.none()))
+        fill = draw(st.one_of(alphabet_strategy, st.none()))
-    align = draw(st.sampled_from(list('<>=^')))
+        align = draw(st.sampled_from(list('<>=^')))
-    fill_align = (fill + align or '') if fill else ''
+        fill_align = (fill + align or '') if fill else ''
 
-    type_ = draw(st.sampled_from('bcdeEfFgGnosxX%'))
+        type_ = draw(st.sampled_from('bcdeEfFgGnosxX%'))
-    can_have_sign = type_ in 'deEfFgGnoxX%'
+        can_have_sign = type_ in 'deEfFgGnoxX%'
-    can_have_comma = type_ in 'deEfFgG%'
+        can_have_comma = type_ in 'deEfFgG%'
-    can_have_precision = type_ in 'fFgG'
+        can_have_precision = type_ in 'fFgG'
-    can_have_pound = type_ in 'boxX%'
+        can_have_pound = type_ in 'boxX%'
-    can_have_zero = type_ in 'oxX'
+        can_have_zero = type_ in 'oxX'
 
-    sign = draw(st.sampled_from(list('+- ') + [''])) if can_have_sign else ''
+        sign = draw(st.sampled_from(list('+- ') + [''])) if can_have_sign else ''
-    pound = draw(st.sampled_from(('#', '',))) if can_have_pound else ''
+        pound = draw(st.sampled_from(('#', '',))) if can_have_pound else ''
-    zero = draw(st.sampled_from(('0', '',))) if can_have_zero else ''
+        zero = draw(st.sampled_from(('0', '',))) if can_have_zero else ''
 
-    int_strategy = st.integers(min_value=1, max_value=1000)
+        int_strategy = st.integers(min_value=1, max_value=1000)
 
-    width = draw(st.one_of(int_strategy, st.none()))
+        width = draw(st.one_of(int_strategy, st.none()))
-    width = str(width) if width is not None else ''
+        width = str(width) if width is not None else ''
 
-    comma = draw(st.sampled_from((',', '',))) if can_have_comma else ''
+        comma = draw(st.sampled_from((',', '',))) if can_have_comma else ''
-    if can_have_precision:
+        if can_have_precision:
-        precision = draw(st.one_of(int_strategy, st.none()))
+            precision = draw(st.one_of(int_strategy, st.none()))
-        precision = '.' + str(precision) if precision else ''
+            precision = '.' + str(precision) if precision else ''
-    else:
+        else:
-        precision = ''
+            precision = ''
 
-    return ''.join((fill_align, sign, pound, zero, width, comma, precision, type_,))
+        return ''.join((fill_align, sign, pound, zero, width, comma, precision, type_,))
 
 
-@st.composite
+    @st.composite
-def fstrings(draw):
+    def fstrings(draw):
-    """
+        """
-    Generate a valid f-string.
+        Generate a valid f-string.
-    See https://www.python.org/dev/peps/pep-0498/#specification
+        See https://www.python.org/dev/peps/pep-0498/#specification
 
-    :param draw: Let hypothsis draw from other strategies.
+        :param draw: Let hypothsis draw from other strategies.
 
-    :return: A valid f-string.
+        :return: A valid f-string.
-    """
+        """
-    character_strategy = st.characters(
+        character_strategy = st.characters(
-        blacklist_characters='\r\n\'\\s{}',
+            blacklist_characters='\r\n\'\\s{}',
-        min_codepoint=1,
+            min_codepoint=1,
-        max_codepoint=1000,
+            max_codepoint=1000,
-    )
+        )
-    is_raw = draw(st.booleans())
+        is_raw = draw(st.booleans())
-    integer_strategy = st.integers(min_value=0, max_value=3)
+        integer_strategy = st.integers(min_value=0, max_value=3)
-    expression_count = draw(integer_strategy)
+        expression_count = draw(integer_strategy)
-    content = []
+        content = []
-    for _ in range(expression_count):
+        for _ in range(expression_count):
-        expression = draw(expressions())
+            expression = draw(expressions())
-        conversion = draw(st.sampled_from(('', '!s', '!r', '!a',)))
+            conversion = draw(st.sampled_from(('', '!s', '!r', '!a',)))
-        has_specifier = draw(st.booleans())
+            has_specifier = draw(st.booleans())
-        specifier = ':' + draw(format_specifiers()) if has_specifier else ''
+            specifier = ':' + draw(format_specifiers()) if has_specifier else ''
-        content.append('{{{}{}}}'.format(expression, conversion, specifier))
+            content.append('{{{}{}}}'.format(expression, conversion, specifier))
-        content.append(draw(st.text(character_strategy)))
+            content.append(draw(st.text(character_strategy)))
-    content = ''.join(content)
+        content = ''.join(content)
-    return "f{}'{}'".format('r' if is_raw else '', content)
+        return "f{}'{}'".format('r' if is_raw else '', content)
 
 
-@pytest.mark.skipif(PYTHON_VERSION < 3.6, reason='need at least python 3.6')
+    @pytest.mark.skipif(PYTHON_VERSION != 3.6, reason='need Python 3.6')
-@hypothesis.given(format_specifiers())
+    @hypothesis.given(format_specifiers())
-def test_format_specifiers(format_specifier):
+    def test_format_specifiers(format_specifier):
-    """Verify that format_specifiers generates valid specifiers"""
+        """Verify that format_specifiers generates valid specifiers"""
-    try:
+        try:
-        exec('"{:' + format_specifier + '}".format(0)')
+            exec('"{:' + format_specifier + '}".format(0)')
-    except ValueError as e:
+        except ValueError as e:
-        if 'Unknown format code' not in str(e):
+            if 'Unknown format code' not in str(e):
-            raise
+                raise
 
 
-def run_test(text):
+    def run_test(text):
-    hypothesis.assume(len(text))
+        hypothesis.assume(len(text))
-    hypothesis.assume("f'{" in text)
+        hypothesis.assume("f'{" in text)
-    expr = text + '\n'
+        expr = text + '\n'
-    code = compile(expr, '<string>', 'single')
+        code = compile(expr, '<string>', 'single')
-    deparsed = deparse_code(PYTHON_VERSION, code, compile_mode='single')
+        deparsed = deparse_code(PYTHON_VERSION, code, compile_mode='single')
-    recompiled = compile(deparsed.text, '<string>', 'single')
+        recompiled = compile(deparsed.text, '<string>', 'single')
-    if recompiled != code:
+        if recompiled != code:
-        assert 'dis(' + deparsed.text.strip('\n') + ')' == 'dis(' + expr.strip('\n') + ')'
+            assert 'dis(' + deparsed.text.strip('\n') + ')' == 'dis(' + expr.strip('\n') + ')'
 
 
-@pytest.mark.skipif(PYTHON_VERSION < 3.6, reason='need at least python 3.6')
+    @pytest.mark.skipif(PYTHON_VERSION != 3.6, reason='need Python 3.6')
-@hypothesis.given(fstrings())
+    @hypothesis.given(fstrings())
-def test_uncompyle_fstring(fstring):
+    def test_uncompyle_fstring(fstring):
-    """Verify uncompyling fstring bytecode"""
+        """Verify uncompyling fstring bytecode"""
-    run_test(fstring)
+        run_test(fstring)
 
 
-@pytest.mark.skipif(PYTHON_VERSION < 3.6, reason='need at least python 3.6')
+    @pytest.mark.skipif(PYTHON_VERSION < 3.6, reason='need Python 3.6+')
-@pytest.mark.parametrize('fstring', [
+    @pytest.mark.parametrize('fstring', [
-    "f'{abc}{abc!s}'",
+        "f'{abc}{abc!s}'",
-    "f'{abc}0'",
+        "f'{abc}0'",
-])
+    ])
-def test_uncompyle_direct(fstring):
+    def test_uncompyle_direct(fstring):
-    """useful for debugging"""
+        """useful for debugging"""
-    run_test(fstring)
+        run_test(fstring)

pytest/test_function_call.py

@@ -1,184 +1,185 @@
-# std
 import string
-# 3rd party
-from hypothesis import given, assume, example, settings, strategies as st
-import pytest
-# uncompyle
-from validate import validate_uncompyle
-from test_fstring import expressions
 from uncompyle6 import PYTHON_VERSION
+import pytest
+pytestmark = pytest.mark.skip(PYTHON_VERSION < 2.7,
+                              reason="need at least Python 2.7")
 
-alpha = st.sampled_from(string.ascii_lowercase)
+if PYTHON_VERSION > 2.6:
-numbers = st.sampled_from(string.digits)
+    from hypothesis import given, assume, example, settings, strategies as st
-alphanum = st.sampled_from(string.ascii_lowercase + string.digits)
+    from validate import validate_uncompyle
+    from test_fstring import expressions
+
+    alpha = st.sampled_from(string.ascii_lowercase)
+    numbers = st.sampled_from(string.digits)
+    alphanum = st.sampled_from(string.ascii_lowercase + string.digits)
 
 
-@st.composite
+    @st.composite
-def function_calls(draw,
+    def function_calls(draw,
-                   min_keyword_args=0, max_keyword_args=5,
+                       min_keyword_args=0, max_keyword_args=5,
-                   min_positional_args=0, max_positional_args=5,
+                       min_positional_args=0, max_positional_args=5,
-                   min_star_args=0, max_star_args=1,
+                       min_star_args=0, max_star_args=1,
-                   min_double_star_args=0, max_double_star_args=1):
+                       min_double_star_args=0, max_double_star_args=1):
-    """
+        """
-    Strategy factory for generating function calls.
+        Strategy factory for generating function calls.
 
-    :param draw: Callable which draws examples from other strategies.
+        :param draw: Callable which draws examples from other strategies.
 
-    :return: The function call text.
+        :return: The function call text.
-    """
+        """
-    st_positional_args = st.lists(
+        st_positional_args = st.lists(
-        alpha,
+            alpha,
-        min_size=min_positional_args,
+            min_size=min_positional_args,
-        max_size=max_positional_args
+            max_size=max_positional_args
-    )
+        )
-    st_keyword_args = st.lists(
+        st_keyword_args = st.lists(
-        alpha,
+            alpha,
-        min_size=min_keyword_args,
+            min_size=min_keyword_args,
-        max_size=max_keyword_args
+            max_size=max_keyword_args
-    )
+        )
-    st_star_args = st.lists(
+        st_star_args = st.lists(
-        alpha,
+            alpha,
-        min_size=min_star_args,
+            min_size=min_star_args,
-        max_size=max_star_args
+            max_size=max_star_args
-    )
+        )
-    st_double_star_args = st.lists(
+        st_double_star_args = st.lists(
-        alpha,
+            alpha,
-        min_size=min_double_star_args,
+            min_size=min_double_star_args,
-        max_size=max_double_star_args
+            max_size=max_double_star_args
-    )
+        )
 
-    positional_args = draw(st_positional_args)
+        positional_args = draw(st_positional_args)
-    keyword_args = draw(st_keyword_args)
+        keyword_args = draw(st_keyword_args)
-    st_values = st.lists(
+        st_values = st.lists(
-        expressions(),
+            expressions(),
-        min_size=len(keyword_args),
+            min_size=len(keyword_args),
-        max_size=len(keyword_args)
+            max_size=len(keyword_args)
-    )
+        )
-    keyword_args = [
+        keyword_args = [
-        x + '=' + e
+            x + '=' + e
-        for x, e in
+            for x, e in
-        zip(keyword_args, draw(st_values))
+            zip(keyword_args, draw(st_values))
-    ]
+        ]
-    star_args = ['*' + x for x in draw(st_star_args)]
+        star_args = ['*' + x for x in draw(st_star_args)]
-    double_star_args = ['**' + x for x in draw(st_double_star_args)]
+        double_star_args = ['**' + x for x in draw(st_double_star_args)]
 
-    arguments = positional_args + keyword_args + star_args + double_star_args
+        arguments = positional_args + keyword_args + star_args + double_star_args
-    draw(st.randoms()).shuffle(arguments)
+        draw(st.randoms()).shuffle(arguments)
-    arguments = ','.join(arguments)
+        arguments = ','.join(arguments)
 
-    function_call = 'fn({arguments})'.format(arguments=arguments)
+        function_call = 'fn({arguments})'.format(arguments=arguments)
-    try:
+        try:
-        # TODO: Figure out the exact rules for ordering of positional, keyword,
+            # TODO: Figure out the exact rules for ordering of positional, keyword,
-        # star args, double star args and in which versions the various
+            # star args, double star args and in which versions the various
-        # types of arguments are supported so we don't need to check that the
+            # types of arguments are supported so we don't need to check that the
-        # expression compiles like this.
+            # expression compiles like this.
-        compile(function_call, '<string>', 'single')
+            compile(function_call, '<string>', 'single')
-    except:
+        except:
-        assume(False)
+            assume(False)
-    return function_call
+        return function_call
 
 
-def test_function_no_args():
+    def test_function_no_args():
-    validate_uncompyle("fn()")
+        validate_uncompyle("fn()")
-
-@pytest.mark.skipif(PYTHON_VERSION < 2.7,
-                    reason="need at least Python 2.7")
-def isolated_function_calls(which):
-    """
-    Returns a strategy for generating function calls, but isolated to
-    particular types of arguments, for example only positional arguments.
-
-    This can help reason about debugging errors in specific types of function
-    calls.
-
-    :param which: One of 'keyword', 'positional', 'star', 'double_star'
-
-    :return: Strategy for generating an function call isolated to specific
-             argument types.
-    """
-    kwargs = dict(
-        max_keyword_args=0,
-        max_positional_args=0,
-        max_star_args=0,
-        max_double_star_args=0,
-    )
-    kwargs['_'.join(('min', which, 'args'))] = 1
-    kwargs['_'.join(('max', which, 'args'))] = 5 if 'star' not in which else 1
-    return function_calls(**kwargs)
-
-
-with settings(max_examples=25):
 
     @pytest.mark.skipif(PYTHON_VERSION < 2.7,
                         reason="need at least Python 2.7")
-    @given(isolated_function_calls('positional'))
+    def isolated_function_calls(which):
-    @example("fn(0)")
+        """
-    def test_function_positional_only(expr):
+        Returns a strategy for generating function calls, but isolated to
-        validate_uncompyle(expr)
+        particular types of arguments, for example only positional arguments.
 
-    @pytest.mark.skipif(PYTHON_VERSION < 2.7,
+        This can help reason about debugging errors in specific types of function
-                        reason="need at least Python 2.7")
+        calls.
-    @given(isolated_function_calls('keyword'))
-    @example("fn(a=0)")
-    def test_function_call_keyword_only(expr):
-        validate_uncompyle(expr)
 
-    @pytest.mark.skipif(PYTHON_VERSION < 2.7,
+        :param which: One of 'keyword', 'positional', 'star', 'double_star'
-                        reason="need at least Python 2.7")
-    @given(isolated_function_calls('star'))
-    @example("fn(*items)")
-    def test_function_call_star_only(expr):
-        validate_uncompyle(expr)
 
-    @pytest.mark.skipif(PYTHON_VERSION < 2.7,
+        :return: Strategy for generating an function call isolated to specific
-                        reason="need at least Python 2.7")
+                 argument types.
-    @given(isolated_function_calls('double_star'))
+        """
-    @example("fn(**{})")
+        kwargs = dict(
-    def test_function_call_double_star_only(expr):
+            max_keyword_args=0,
-        validate_uncompyle(expr)
+            max_positional_args=0,
+            max_star_args=0,
+            max_double_star_args=0,
+        )
+        kwargs['_'.join(('min', which, 'args'))] = 1
+        kwargs['_'.join(('max', which, 'args'))] = 5 if 'star' not in which else 1
+        return function_calls(**kwargs)
 
 
-@pytest.mark.xfail()
+    with settings(max_examples=25):
-def test_BUILD_CONST_KEY_MAP_BUILD_MAP_UNPACK_WITH_CALL_BUILD_TUPLE_CALL_FUNCTION_EX():
+
-    validate_uncompyle("fn(w=0,m=0,**v)")
+        @pytest.mark.skipif(PYTHON_VERSION < 2.7,
+                            reason="need at least Python 2.7")
+        @given(isolated_function_calls('positional'))
+        @example("fn(0)")
+        def test_function_positional_only(expr):
+            validate_uncompyle(expr)
+
+        @pytest.mark.skipif(PYTHON_VERSION < 2.7,
+                            reason="need at least Python 2.7")
+        @given(isolated_function_calls('keyword'))
+        @example("fn(a=0)")
+        def test_function_call_keyword_only(expr):
+            validate_uncompyle(expr)
+
+        @pytest.mark.skipif(PYTHON_VERSION < 2.7,
+                            reason="need at least Python 2.7")
+        @given(isolated_function_calls('star'))
+        @example("fn(*items)")
+        def test_function_call_star_only(expr):
+            validate_uncompyle(expr)
+
+        @pytest.mark.skipif(PYTHON_VERSION < 2.7,
+                            reason="need at least Python 2.7")
+        @given(isolated_function_calls('double_star'))
+        @example("fn(**{})")
+        def test_function_call_double_star_only(expr):
+            validate_uncompyle(expr)
 
 
-@pytest.mark.xfail()
+    @pytest.mark.xfail()
-def test_BUILD_MAP_BUILD_MAP_UNPACK_WITH_CALL_BUILD_TUPLE_CALL_FUNCTION_EX():
+    def test_BUILD_CONST_KEY_MAP_BUILD_MAP_UNPACK_WITH_CALL_BUILD_TUPLE_CALL_FUNCTION_EX():
-    validate_uncompyle("fn(a=0,**g)")
+        validate_uncompyle("fn(w=0,m=0,**v)")
 
 
-@pytest.mark.xfail()
+    @pytest.mark.xfail()
-def test_CALL_FUNCTION_EX():
+    def test_BUILD_MAP_BUILD_MAP_UNPACK_WITH_CALL_BUILD_TUPLE_CALL_FUNCTION_EX():
-    validate_uncompyle("fn(*g,**j)")
+        validate_uncompyle("fn(a=0,**g)")
 
 
-@pytest.mark.xfail()
+    @pytest.mark.xfail()
-def test_BUILD_MAP_CALL_FUNCTION_EX():
+    def test_CALL_FUNCTION_EX():
-    validate_uncompyle("fn(*z,u=0)")
+        validate_uncompyle("fn(*g,**j)")
 
 
-@pytest.mark.xfail()
+    @pytest.mark.xfail()
-def test_BUILD_TUPLE_CALL_FUNCTION_EX():
+    def test_BUILD_MAP_CALL_FUNCTION_EX():
-    validate_uncompyle("fn(**a)")
+        validate_uncompyle("fn(*z,u=0)")
 
 
-@pytest.mark.xfail()
+    @pytest.mark.xfail()
-def test_BUILD_MAP_BUILD_TUPLE_BUILD_TUPLE_UNPACK_WITH_CALL_CALL_FUNCTION_EX():
+    def test_BUILD_TUPLE_CALL_FUNCTION_EX():
-    validate_uncompyle("fn(b,b,b=0,*a)")
+        validate_uncompyle("fn(**a)")
 
 
-@pytest.mark.xfail()
+    @pytest.mark.xfail()
-def test_BUILD_TUPLE_BUILD_TUPLE_UNPACK_WITH_CALL_CALL_FUNCTION_EX():
+    def test_BUILD_MAP_BUILD_TUPLE_BUILD_TUPLE_UNPACK_WITH_CALL_CALL_FUNCTION_EX():
-    validate_uncompyle("fn(*c,v)")
+        validate_uncompyle("fn(b,b,b=0,*a)")
 
 
-@pytest.mark.xfail()
+    @pytest.mark.xfail()
-def test_BUILD_CONST_KEY_MAP_CALL_FUNCTION_EX():
+    def test_BUILD_TUPLE_BUILD_TUPLE_UNPACK_WITH_CALL_CALL_FUNCTION_EX():
-    validate_uncompyle("fn(i=0,y=0,*p)")
+        validate_uncompyle("fn(*c,v)")
 
 
-@pytest.mark.skip(reason='skipping property based test until all individual tests are passing')
+    @pytest.mark.xfail()
-@given(function_calls())
+    def test_BUILD_CONST_KEY_MAP_CALL_FUNCTION_EX():
-def test_function_call(function_call):
+        validate_uncompyle("fn(i=0,y=0,*p)")
-    validate_uncompyle(function_call)
+
+
+    @pytest.mark.skip(reason='skipping property based test until all individual tests are passing')
+    @given(function_calls())
+    def test_function_call(function_call):
+        validate_uncompyle(function_call)

pytest/test_grammar.py

@@ -7,7 +7,7 @@ def test_grammar():
 
     def check_tokens(tokens, opcode_set):
         remain_tokens = set(tokens) - opcode_set
-        remain_tokens = set([re.sub('_\d+$','', t) for t in remain_tokens])
+        remain_tokens = set([re.sub(r'_\d+$','', t) for t in remain_tokens])
         remain_tokens = set([re.sub('_CONT$','', t) for t in remain_tokens])
         remain_tokens = set(remain_tokens) - opcode_set
         assert remain_tokens == set([]), \
@@ -26,6 +26,9 @@ def test_grammar():
     expect_right_recursive = set([('designList',
                                    ('store', 'DUP_TOP', 'designList'))])
 
+    if PYTHON_VERSION != 3.7:
+        unused_rhs.add('call')
+
     if PYTHON_VERSION > 2.6:
         expect_lhs.add('kvlist')
         expect_lhs.add('kv3')
@@ -41,19 +44,14 @@ def test_grammar():
             expect_lhs.add("annotate_arg")
             expect_lhs.add("annotate_tuple")
             unused_rhs.add("mkfunc_annotate")
-            unused_rhs.add('call')
             unused_rhs.add("dict_comp")
             unused_rhs.add("classdefdeco1")
-            if PYTHON_VERSION < 3.6:
+            unused_rhs.add("tryelsestmtl")
-                # 3.6 has at least one non-custom call rule
+            if PYTHON_VERSION >= 3.5:
-                # the others don't
+                expect_right_recursive.add((('l_stmts',
-                unused_rhs.add('call')
+                                             ('lastl_stmt', 'come_froms', 'l_stmts'))))
-                if PYTHON_VERSION == 3.5:
-                    expect_right_recursive.add((('l_stmts',
-                                                 ('lastl_stmt', 'come_froms', 'l_stmts'))))
-                    pass
                 pass
-            else:
+            elif 3.0 < PYTHON_VERSION < 3.3:
                 expect_right_recursive.add((('l_stmts',
                                              ('lastl_stmt', 'COME_FROM', 'l_stmts'))))
                 pass
@@ -61,7 +59,6 @@ def test_grammar():
         pass
     else:
         expect_lhs.add('kwarg')
-        unused_rhs.add('call')
 
     assert expect_lhs == set(lhs)
     assert unused_rhs == set(rhs)

pytest/test_single_compile.py

@@ -1,21 +1,22 @@
 import pytest
-from uncompyle6 import PYTHON_VERSION, deparse_code
+from uncompyle6 import PYTHON_VERSION, code_deparse
+pytestmark = pytest.mark.skip(PYTHON_VERSION < 2.7,
+                              reason="need at least Python 2.7")
 
-@pytest.mark.skip(PYTHON_VERSION < 2.7,
+if PYTHON_VERSION > 2.6:
-                  reason="need at least Python 2.7")
+    def test_single_mode():
-def test_single_mode():
+        single_expressions = (
-    single_expressions = (
+            'i = 1',
-        'i = 1',
+            'i and (j or k)',
-        'i and (j or k)',
+            'i += 1',
-        'i += 1',
+            'i = j % 4',
-        'i = j % 4',
+            'i = {}',
-        'i = {}',
+            'i = []',
-        'i = []',
+            'for i in range(10):\n    i\n',
-        'for i in range(10):\n    i\n',
+            'for i in range(10):\n    for j in range(10):\n        i + j\n',
-        'for i in range(10):\n    for j in range(10):\n        i + j\n',
+            'try:\n    i\nexcept Exception:\n    j\nelse:\n    k\n'
-        'try:\n    i\nexcept Exception:\n    j\nelse:\n    k\n'
+        )
-    )
 
-    for expr in single_expressions:
+        for expr in single_expressions:
-        code = compile(expr + '\n', '<string>', 'single')
+            code = compile(expr + '\n', '<string>', 'single')
-        assert deparse_code(PYTHON_VERSION, code, compile_mode='single').text == expr + '\n'
+            assert code_deparse(code, compile_mode='single').text == expr + '\n'

pytest/validate.py

@@ -6,16 +6,19 @@ import difflib
 import subprocess
 import tempfile
 import functools
-# compatability
-import six
 # uncompyle6 / xdis
-from uncompyle6 import PYTHON_VERSION, IS_PYPY, deparse_code
+from uncompyle6 import PYTHON_VERSION, PYTHON3, IS_PYPY, deparse_code
 # TODO : I think we can get xdis to support the dis api (python 3 version) by doing something like this there
 from xdis.bytecode import Bytecode
 from xdis.main import get_opcode
 opc = get_opcode(PYTHON_VERSION, IS_PYPY)
 Bytecode = functools.partial(Bytecode, opc=opc)
+import six
 
+if PYTHON3:
+    from io import StringIO
+else:
+    from StringIO import StringIO
 
 def _dis_to_text(co):
     return Bytecode(co).dis()

requirements-dev.txt

@@ -1,3 +1,2 @@
-pytest>=3.0.0,<=3.0.1
 flake8
 hypothesis<=3.0.0

requirements.txt

@@ -1,2 +1,4 @@
 # Pick up stuff from setup.py
+hypothesis==2.0.0
+pytest
 -e .

setup.py

@@ -4,12 +4,12 @@ import sys
 """Setup script for the 'uncompyle6' distribution."""
 
 SYS_VERSION = sys.version_info[0:2]
-if not ((2, 6) <= SYS_VERSION  <= (3, 7)) or ((3, 0) <= SYS_VERSION <= (3, 0)):
+if not ((2, 6) <= SYS_VERSION  <= (3, 7)):
-    mess = "Python Release 2.6 .. 3.7 excluding 3.0 are supported in this code branch."
+    mess = "Python Release 2.6 .. 3.7 are supported in this code branch."
     if ((2, 4) <= SYS_VERSION <= (2, 7)):
         mess += ("\nFor your Python, version %s, use the python-2.4 code/branch." %
                  sys.version[0:3])
-    elif SYS_VERSION < (2, 4) or (3, 0) <= SYS_VERSION:
+    elif SYS_VERSION < (2, 4):
         mess += ("\nThis package is not supported for Python version %s."
                  % sys.version[0:3])
     print(mess)

test/Makefile

@@ -1,12 +1,13 @@
 PHONY=check clean dist distclean test test-unit test-functional rmChangeLog clean_pyc nosetests \
-      check-bytecode-1.5 check-bytecode-1 check-bytecode-2 check-bytecode-3 \
+      check-bytecode-1  check-bytecode-1.3 check-bytecode-1.4 check-bytecode-1.5 \
+      check-bytecode-2 check-bytecode-3 check-bytecode-3-short \
       check-bytecode-2.2 check-byteocde-2.3 check-bytecode-2.4 \
       check-short check-2.6 check-2.7 check-3.0 check-3.1 check-3.2 check-3.3 \
-      check-3.4 check-3.5 check-5.6 5.6 5.8 \
+      check-3.4 check-3.5 check-3.6 check-3.7 check-5.6 5.6 5.8 \
       grammar-coverage-2.5 grammar-coverage-2.6 grammar-coverage-2.7 \
       grammar-coverage-3.1 grammar-coverage-3.2 grammar-coverage-3.3 \
-      grammar-coverage-3.4 grammar-coverage-3.5 grammar-coverage-3.6
+      grammar-coverage-3.4 grammar-coverage-3.5 grammar-coverage-3.6 \
-
+      grammar-coverage-3.7
 
 GIT2CL ?= git2cl
 PYTHON ?= python
@@ -21,7 +22,7 @@ COVER_DIR=../tmp/grammar-cover
 # Run short tests
 check-short:
 	@$(PYTHON) -V && PYTHON_VERSION=`$(PYTHON) -V 2>&1 | cut -d ' ' -f 2 | cut -d'.' -f1,2`; \
-	$(MAKE) check-bytecode
+	$(MAKE) check-bytecode-short
 
 # Run all tests
 check:
@@ -33,6 +34,7 @@ check-2.6 check-2.7: check-bytecode-2 check-bytecode-3 check-bytecode-1 check-na
 #: Run working tests from Python 3.0
 check-3.0: check-bytecode
 	$(PYTHON) test_pythonlib.py --bytecode-3.0 --weak-verify $(COMPILE)
+	$(PYTHON) test_pythonlib.py --bytecode-3.0-run --verify-run
 
 #: Run working tests from Python 3.1
 check-3.1: check-bytecode
@@ -62,6 +64,12 @@ check-3.5: check-bytecode
 #: Run working tests from Python 3.6
 check-3.6: check-bytecode
 	$(PYTHON) test_pythonlib.py --bytecode-3.6 --weak-verify $(COMPILE)
+	$(PYTHON) test_pythonlib.py --bytecode-3.6-run --verify-run
+
+#: Run working tests from Python 3.7
+check-3.7: check-bytecode
+	$(PYTHON) test_pythonlib.py --bytecode-3.7 --weak-verify $(COMPILE)
+	$(PYTHON) test_pythonlib.py --bytecode-3.7-run --verify-run
 
 # FIXME
 #: this is called when running under pypy3.5-5.8.0 or pypy2-5.6.0
@@ -72,7 +80,7 @@ check-disasm:
 	$(PYTHON) dis-compare.py
 
 #: Check deparsing bytecode 1.x only
-check-bytecode-1: check-bytecode-1.5
+check-bytecode-1: check-bytecode-1.4 check-bytecode-1.5
 
 #: Check deparsing bytecode 2.x only
 check-bytecode-2:
@@ -83,17 +91,39 @@ check-bytecode-2:
 #: Check deparsing bytecode 3.x only
 check-bytecode-3:
 	$(PYTHON) test_pythonlib.py --bytecode-3.0 \
-       --bytecode-3.1 --bytecode-3.2 --bytecode-3.3 \
+	--bytecode-3.1 --bytecode-3.2 --bytecode-3.3 \
-       --bytecode-3.4 --bytecode-3.5 --bytecode-3.6 --bytecode-pypy3.2
+	--bytecode-3.4 --bytecode-3.5 --bytecode-3.6 --bytecode-3.7 \
+	--bytecode-pypy3.2
 
-#: Check deparsing bytecode that works running Python 2 and Python 3
+#: Check deparsing on selected bytecode 3.x
+check-bytecode-3-short:
+	$(PYTHON) test_pythonlib.py \
+       --bytecode-3.4 --bytecode-3.5 --bytecode-3.6
+
+#: Check deparsing bytecode on all Python 2 and Python 3 versions
 check-bytecode: check-bytecode-3
 	$(PYTHON) test_pythonlib.py \
+	--bytecode-1.3 --bytecode-1.4 --bytecode-1.5 \
+        --bytecode-2.2 --bytecode-2.3 --bytecode-2.4 \
 	--bytecode-2.1 --bytecode-2.2 --bytecode-2.3 --bytecode-2.4 \
         --bytecode-2.5 --bytecode-2.6 --bytecode-2.7 \
         --bytecode-pypy2.7
 
 
+#: Check deparsing bytecode on selected Python 2 and Python 3 versions
+check-bytecode-short: check-bytecode-3-short
+	$(PYTHON) test_pythonlib.py \
+        --bytecode-2.6 --bytecode-2.7 --bytecode-pypy2.7
+
+
+#: Check deparsing bytecode 1.3 only
+check-bytecode-1.3:
+	$(PYTHON) test_pythonlib.py --bytecode-1.3
+
+#: Check deparsing bytecode 1.4 only
+check-bytecode-1.4:
+	$(PYTHON) test_pythonlib.py --bytecode-1.4
+
 #: Check deparsing bytecode 1.5 only
 check-bytecode-1.5:
 	$(PYTHON) test_pythonlib.py --bytecode-1.5
@@ -198,6 +228,7 @@ check-bytecode-2.7:
 #: Check deparsing Python 3.0
 check-bytecode-3.0:
 	$(PYTHON) test_pythonlib.py --bytecode-3.0 --weak-verify
+	$(PYTHON) test_pythonlib.py --bytecode-3.0-run --verify-run
 
 #: Check deparsing Python 3.1
 check-bytecode-3.1:
@@ -229,6 +260,10 @@ check-bytecode-3.6:
 	$(PYTHON) test_pythonlib.py --bytecode-3.6 --weak-verify
 	$(PYTHON) test_pythonlib.py --bytecode-3.6-run --verify-run
 
+#: Check deparsing Python 3.7
+check-bytecode-3.7:
+	$(PYTHON) test_pythonlib.py --bytecode-3.7 --weak-verify
+
 #: short tests for bytecodes only for this version of Python
 check-native-short:
 	$(PYTHON) test_pythonlib.py --bytecode-$(PYTHON_VERSION) --weak-verify $(COMPILE)
@@ -255,7 +290,7 @@ check-3.4-ok:
 2.6:
 
 #: PyPy 5.0.x with Python 2.7 ...
-pypy-2.7 5.0 5.3:
+pypy-2.7 5.0 5.3 6.0:
 	$(PYTHON) test_pythonlib.py --bytecode-pypy2.7 --verify
 
 #: PyPy 2.4.x with Python 3.2 ...

test/bytecode_2.4_run/README

@@ -1,5 +1,5 @@
 These are byte-compiled programs compiled by Python 2.4
 
-Furthrmore the programs here are self-checking: when decompiled and
+Furthermore, the programs here are self-checking: when decompiled and
-then run again in a 2.4 interpreter, they will give an error if they
+then run again in a 2.4 interpreter, they are likely to give an error when they
 are miscompiled.

test/bytecode_2.5_run/README

@@ -1,5 +1,5 @@
 These are byte-compiled programs compiled by Python 2.5.
 
-Furthrmore the programs here are self-checking: when decompiled and
+Furthermore the programs here are self-checking: when decompiled and
-then run again in a 2.5 interpreter, they will give an error if they
+then run again in a 2.5 interpreter, they are likely to give an error
-are miscompiled.
+when they are miscompiled.

test/bytecode_2.6_run/README

@@ -1,5 +1,5 @@
 These are byte-compiled programs compiled by Python 2.6.
 
-Furthrmore the programs here are self-checking: when decompiled and
+Furthermore the programs here are self-checking: when decompiled and
-then run again in a 2.6 interpreter, they will give an error if they
+then run again in a 2.6 interpreter, they are likely to give an error
-are miscompiled.
+when they are miscompiled.

test/bytecode_2.7_run/README

@@ -1,5 +1,5 @@
 These are byte-compiled programs compiled by Python 2.7.
 
-Furthrmore the programs here are self-checking: when decompiled and
+Furthermore, the programs here are self-checking: when decompiled and
-then run again in a 2.7 interpreter, they will give an error if they
+then run again in a 2.7 interpreter, they are likely to give an error
-are miscompiled.
+when they are miscompiled.

test/bytecode_3.0_run/README

@@ -1,5 +1,5 @@
 These are byte-compiled programs compiled by Python 3.0.
 
-Furthrmore the programs here are self-checking: when decompiled and
+Furthermore, the programs here are self-checking: when decompiled and
-then run again in a 3.0 interpreter, they will give an error if they
+then run again in a 3.0 interpreter, they are likely to give an error
-are miscompiled.
+when they are miscompiled.

test/bytecode_3.1_run/README

@@ -1,5 +1,5 @@
 These are byte-compiled programs compiled by Python 3.1.
 
-Furthrmore the programs here are self-checking: when decompiled and
+Furthrmore, the programs here are self-checking: when decompiled and
-then run again in a 3.1 interpreter, they will give an error if they
+then run again in a 3.1 interpreter, they are likely to give an error
-are miscompiled.
+when they are miscompiled.

test/bytecode_3.2_run/README

@@ -1,5 +1,5 @@
 These are byte-compiled programs compiled by Python 3.2.
 
-Furthrmore the programs here are self-checking: when decompiled and
+Furthermore, the programs here are self-checking: when decompiled and
-then run again in a 3.2 interpreter, they will give an error if they
+then run again in a 3.2 interpreter, they are likely to give an error
-are miscompiled.
+when they are miscompiled.

test/bytecode_3.3_run/README

@@ -1,5 +1,5 @@
 These are byte-compiled programs compiled by Python 3.3.
 
-Furthrmore the programs here are self-checking: when decompiled and
+Furthermore, the programs here are self-checking: when decompiled and
-then run again in a 3.3 interpreter, they will give an error if they
+then run again in a 3.3 interpreter, they are likely to give an error
-are miscompiled.
+when they are miscompiled.

test/bytecode_3.4_run/README

@@ -1,5 +1,5 @@
 These are byte-compiled programs compiled by Python 3.4.
 
-Furthrmore the programs here are self-checking: when decompiled and
+Furthermore, the programs here are self-checking: when decompiled and
-then run again in a 3.4 interpreter, they will give an error if they
+then run again in a 3.4 interpreter, they are likely to give an error
-are miscompiled.
+when they are miscompiled.

test/bytecode_3.5_run/README

@@ -1,5 +1,5 @@
 These are byte-compiled programs compiled by Python 3.5.
 
-Furthrmore the programs here are self-checking: when decompiled and
+Furthrmore, the programs here are self-checking: when decompiled and
-then run again in a 3.5 interpreter, they will give an error if they
+then run again in a 3.5 interpreter, they are likely to give an error
-are miscompiled.
+when they are miscompiled.