Compare commits

...

36 Commits

Author SHA1 Message Date
rocky
053270483c Get ready for release 3.2.6 2019-03-23 18:17:53 -04:00
rocky
4a354269bc Adjust 3.7 chained compare for adjusted grammar
Add test for last change
2019-03-23 17:06:50 -04:00
rocky
cd64156708 Fix else detection bug in Python 3.6+ 2019-03-23 08:57:34 -04:00
rocky
3d49b499fb Move 3.6 return_if_lambda rule back to 3.5 2019-03-10 14:01:57 -04:00
rocky
dcad6cf6ce Fix if return boundary in 3.6+
Fixes #209
2019-03-10 05:59:15 -04:00
rocky
bfceeac6c8 2.7 can have two JUMP_BACKs at the end of a while loop
Fixes #215
2019-01-27 21:41:17 -05:00
rocky
47448e7ce4 Merge branch 'master' of github.com:rocky/python-uncompyle6 2019-01-26 18:19:12 -05:00
rocky
e1628d4d3a Possibly addresses issue #215 2019-01-26 18:18:21 -05:00
R. Bernstein
3328ed494e Merge pull request #211 from byehack/master
support utf-8 chars
2019-01-17 13:35:22 -05:00
byehack
0c5f0dfc7a support utf-8 in py3 2019-01-17 18:52:40 +03:30
byehack
138b2ac5ee support utf-8 encoding for PYTHON>=3 2019-01-17 18:43:13 +03:30
byehack
ceae035c70 support utf-8 chars 2019-01-16 19:36:13 +03:30
rocky
763c599c16 I said use Python 2.7.11 2019-01-14 22:28:26 -05:00
rocky
14111d9341 Need hypothesis 3.0.0? 2019-01-14 22:21:59 -05:00
rocky
739ba48f61 Go with Python 2.7.11 which seems to be installed already 2019-01-14 22:18:52 -05:00
rocky
9f1a7fa7ff Go with Python 2.7.11 which seems to be installed already 2019-01-14 22:18:07 -05:00
rocky
fb117713cf Let's try Python 2.7.15 on CI 2019-01-14 22:10:20 -05:00
rocky
43646b3c71 Comma placement in 3.6 and 3.7 **kwargs
fixes #208
2019-01-14 17:41:54 -05:00
rocky
9a14db567b Merge branch 'master' of github.com:rocky/python-uncompyle6 2019-01-13 19:39:54 -05:00
rocky
a97d4003c7 Python 3.7 changes chained comparison code
fixes #206
2019-01-13 19:37:41 -05:00
R. Bernstein
acb96deba5 Merge pull request #205 from cclauss/patch-1
Travis CI: Run more f-string tests on Python 3.7
2019-01-12 17:40:12 -05:00
cclauss
6cbaef4ba5 Travis CI: Run more f-string tests on Python 3.7 2019-01-12 20:24:08 +01:00
R. Bernstein
7c9691b5a7 Merge pull request #204 from rocky/python-3.7-testing
Python 3.7 testing
2019-01-12 11:57:48 -05:00
rocky
0fa45301fa Python 3.7 testing fixes 2019-01-12 11:51:01 -05:00
cclauss
3b43801067 Travis CI: Add Python 3.7 to the testing
Also, [Travis are now recommending removing the __sudo__ tag](https://blog.travis-ci.com/2018-11-19-required-linux-infrastructure-migration).
2019-01-09 19:52:56 +01:00
R. Bernstein
e41ef897d1 Merge pull request #202 from rocky/assert-fix
Better handling of AssertError.  I expect fewer asserts to be recognized in favor of if AssertError. Also, this fixes one thing but breaks 04_assert_continue.pyc
2019-01-05 16:59:00 -05:00
rocky
d0dc26caf7 Another test 2019-01-05 16:48:55 -05:00
rocky
df105fbfb2 Fixed one thing in Python 2.7 and break another.
We'll go with this until we get to a more serious refactoring.
2019-01-05 16:38:07 -05:00
Yiming Wang
fbf51a0ae3 Fix when offset like 47_0 2019-01-05 13:33:31 -05:00
Yiming Wang
5d99322078 Better assert and AssertionError determine for Python 2.7 2019-01-05 13:32:08 -05:00
rocky
1a70f75ffc Pypy 2.7 fixes
* pypy doesn't seem to grok sys.stdout.flush() sometimes?
* pypy has extra come_froms for return_if_stmt
2019-01-05 13:28:29 -05:00
rocky
37750814b9 Adjust grammar checking...
More conditional rules were added
2019-01-01 23:03:17 -05:00
rocky
a2321773d7 Fix Python 3.x try/else detection
Fixes #155
2019-01-01 22:50:28 -05:00
rocky
acd0e5fea6 Note weirdness in try/else 2019-01-01 09:40:59 -05:00
rocky
d443295df6 Check range of _come_froms on ifelsestmt reduction
Fixes #200
2018-12-31 08:39:08 -05:00
rocky
296a2129eb Bump 3.2.6 version 2018-12-30 12:35:25 -05:00
46 changed files with 534 additions and 167 deletions

View File

@@ -42,7 +42,7 @@ jobs:
# This is based on your 1.0 configuration file or project settings
- run:
working_directory: ~/rocky/python-uncompyle6
command: pyenv install 2.7.13 && pyenv local 2.7.13 && pyenv rehash && pip install virtualenv && pip install nose && pip install pep8 && pip install six && pyenv rehash
command: pyenv local 2.7.11 && pyenv rehash && pip install virtualenv && pip install nose && pip install pep8 && pip install six && pyenv rehash
# Dependencies
# This would typically go in either a build or a build-and-test job when using workflows
# Restore the dependency cache
@@ -57,7 +57,7 @@ jobs:
# This is based on your 1.0 configuration file or project settings
- run: pip install --upgrade setuptools
- run: pip install -e .
- run: pip install pytest==3.2.5 hypothesis
- run: pip install pytest==3.2.5 hypothesis==3.0.0
# Save dependency cache
- save_cache:
key: v1-dep-{{ .Branch }}-{{ epoch }}
@@ -76,7 +76,7 @@ jobs:
# This would typically be a build job when using workflows, possibly combined with build
# This is based on your 1.0 configuration file or project settings
- run: python ./setup.py develop && make check-2.7
- run: cd ./test/stdlib && pyenv local 2.7.13 && bash ./runtests.sh 'test_[p-z]*.py'
- run: cd ./test/stdlib && pyenv local 2.7.11 && bash ./runtests.sh 'test_[p-z]*.py'
# Teardown
# If you break your build into multiple jobs with workflows, you will probably want to do the parts of this that are relevant in each
# Save test results

View File

@@ -1,7 +1,5 @@
language: python
sudo: false
python:
- '3.5'
- '2.7'
@@ -9,6 +7,11 @@ python:
- '3.4'
- '3.6'
matrix:
include:
- python: '3.7'
dist: xenial # required for Python >= 3.7 (travis-ci/travis-ci#9069)
install:
- pip install -e .
- pip install -r requirements-dev.txt

View File

@@ -1,43 +1,74 @@
uncompyle6 3.2.5 2019-12-30 Clearout sale
3.2.6 2019-03-23 Mueller Report
=======================================
Mostly more of the same: bug fixes and pull requests.
Bug Fixes
-----------
* [#155: Python 3.x bytecode confusing "try/else" with "try" in a loop](https://github.com/rocky/python-uncompyle6/issues/155),
* [#200: Python 3 bug in not detecting end bounds of an "if" ... "elif"](https://github.com/rocky/python-uncompyle6/issues/200),
* [#208: Comma placement in 3.6 and 3.7 **kwargs](https://github.com/rocky/python-uncompyle6/issues/208),
* [#209: Fix "if" return boundary in 3.6+](https://github.com/rocky/python-uncompyle6/issues/209),
* [#215: 2.7 can have two JUMP_BACKs at the end of a while loop](https://github.com/rocky/python-uncompyle6/issues/215)
Pull Requests
----------------
* [#202: Better "assert" statement detemination in Python 2.7](https://github.com/rocky/python-uncompyle6/pull/211)
* [#204: Python 3.7 testing](https://github.com/rocky/python-uncompyle6/pull/204)
* [#205: Run more f-string tests on Python 3.7](https://github.com/rocky/python-uncompyle6/pull/205)
* [#211: support utf-8 chars in Python 3 sourcecode](https://github.com/rocky/python-uncompyle6/pull/202)
3.2.5 2018-12-30 Clearout sale
======================================
- 3.7.2 Remove deprecation warning on regexp string that isn't raw
- main.main() parameter `codes` is not used - note that
- Improve Python 3.6+ control flow detection
- More complete fragment instruction annotation for `imports`
uncompyle6 3.2.4 2018-10-27 7x9 release
3.2.4 2018-10-27 7x9 release
===================================
- Bug fixes #180, #182, #187, #192
- Enhancements #189
- Internal improvements
uncompyle6 3.2.3 2018-06-04 Michael Cohen flips and Fleetwood Redux
3.2.3 2018-06-04 Michael Cohen flips and Fleetwood Redux
======================================================================
- Python 1.3 support 3.0 bug and
- fix botched parameter ordering of 3.x in last release
uncompyle6 3.2.2 2018-06-04 When I'm 64
3.2.2 2018-06-04 When I'm 64
===================================
- Python 3.0 support and bug fixes
uncompyle6 3.2.1 2018-06-04 MF
3.2.1 2018-06-04 MF
=======================
- Python 1.4 and 1.5 bug fixes
uncompyle6 3.2.0 2018-05-19 Rocket Scientist
3.2.0 2018-05-19 Rocket Scientist
=========================================
- Add rudimentary 1.4 support (still a bit buggy)
- add --tree+ option to show formatting rule, when it is constant
- Python 2.7.15candidate1 support (via xdis)
- bug fixes, especially for 3.7 (but 2.7 and 3.6 and others as well)
uncompyle6 3.1.3 2018-04-16
3.1.3 2018-04-16
====================
- Add some Python 3.7 rules, such as for handling LOAD_METHOD (not complete)
- Fix some fragment bugs
- small doc changes
uncompyle6 3.1.2 2018-04-08 Eastern Orthodox Easter
3.1.2 2018-04-08 Eastern Orthodox Easter
==================================================
- Python 3.x subclass and call parsing fixes
- Allow/note running on Python 3.1
@@ -45,7 +76,8 @@ uncompyle6 3.1.2 2018-04-08 Eastern Orthodox Easter
- DRY instruction building code between 2.x and 3.x
- expand testing
uncompyle6 3.1.1 2018-04-01 Easter April Fool's
3.1.1 2018-04-01 Easter April Fool's
=============================================
Jesus on Friday's New York Times puzzle: "I'm stuck on 2A"
@@ -58,7 +90,8 @@ Jesus on Friday's New York Times puzzle: "I'm stuck on 2A"
- more runtime testing of decompiled code
- more removal of parenthesis around calls via setting precidence
uncompyle6 3.1.0 2018-03-21 Equinox
3.1.0 2018-03-21 Equinox
==============================
- Add code_deparse_with_offset() fragment function.
- Correct paramenter call fragment deparse_code()
@@ -66,7 +99,8 @@ uncompyle6 3.1.0 2018-03-21 Equinox
About 5% of 3.6 fail parsing now. But
semantics still needs much to be desired.
uncompyle6 3.0.1 2018-02-17
3.0.1 2018-02-17
====================
- All Python 2.6.9 standard library files weakly verify
- Many 3.6 fixes. 84% of the first 200 standard library files weakly compile.
@@ -76,7 +110,8 @@ uncompyle6 3.0.1 2018-02-17
- And more add tests target previous existing bugs more completely
- sync recent license changes in metadata
uncompyle6 3.0.0 2018-02-17
3.0.0 2018-02-17
====================
- deparse_code() and lookalikes from the various semantic actions are
now deprecated. Instead use new API code_deparse() which makes the
@@ -98,7 +133,8 @@ A bit more work is still needed for 3.6 especially in the area of
function calls and definitions.
uncompyle6 2.16.0 2018-02-17
2.16.0 2018-02-17
=====================
- API additions:
- add fragments.op_at_code_loc() and
@@ -110,18 +146,21 @@ uncompyle6 2.16.0 2018-02-17
- Fix Python 3.5+ CALL_FUNCTION_VAR and BUILD_LIST_UNPACK in call; with this
we can can handle 3.5+ f(a, b, *c, *d, *e) now
uncompyle6 2.15.1 2018-02-05
2.15.1 2018-02-05
=====================
- More bug fixes and revert an improper bug fix in 2.15.0
uncompyle6 2.15.0 2018-02-05 pycon2018.co
2.15.0 2018-02-05 pycon2018.co
=====================================
- Bug fixes
- Code fragment improvements
- Code cleanups
- Expand testing
uncompyle6 2.15.1 2018-01-27
2.15.1 2018-01-27
=====================
- Add --linemap option to give line correspondences
between original source lines and reconstructed line sources.
@@ -133,7 +172,8 @@ uncompyle6 2.15.1 2018-01-27
- Correct 3.6+ calls with kwargs
- Describe the difficulty of 3.6 in README
uncompyle6 2.14.3 2018-01-19
2.14.3 2018-01-19
=====================
- Fix bug in 3.5+ await stmt
- Better version to magic handling; handle 3.5.2 .. 3.5.4 versions
@@ -145,7 +185,8 @@ uncompyle6 2.14.3 2018-01-19
- better tests in setup.py for running the right version of Python
- Fix 2.6- parsing of "for .. try/else" ... with "continue" inside
uncompyle6 2.14.2 2018-01-09 Samish
2.14.2 2018-01-09 Samish
==============================
Decompilation bug fixes, mostly 3.6 and pre 2.7
@@ -163,7 +204,8 @@ Decompilation bug fixes, mostly 3.6 and pre 2.7
Python versions
- Match Python AST names more closely when possible
uncompyle6 2.14.1 2017-12-10 Dr. Gecko
2.14.1 2017-12-10 Dr. Gecko
===================================
- Many decompilation bugfixes
- Grammar rule reduction and version isolation
@@ -171,7 +213,8 @@ uncompyle6 2.14.1 2017-12-10 Dr. Gecko
with Python AST
- Start automated Python stdlib testing - full round trip
uncompyle6 2.14.0 2017-11-26 johnnybamazing
2.14.0 2017-11-26 johnnybamazing
=========================================
- Start to isolate grammar rules between versions
and remove used grammar rules
@@ -179,7 +222,8 @@ uncompyle6 2.14.0 2017-11-26 johnnybamazing
(many more remain)
- Add stdlib/runtests.sh for even more rigorous testing
uncompyle6 2.13.3 2017-11-13
2.13.3 2017-11-13
=====================
Overall: better 3.6 decompiling and some much needed code refactoring and cleanup
@@ -205,22 +249,26 @@ Overall: better 3.6 decompiling and some much needed code refactoring and cleanu
- reinstate some bytecode tests since decompiling has gotten better
- Revise how to report a bug
uncompyle6 2.13.2 2017-10-12
2.13.2 2017-10-12
=====================
- Re-release using a more automated approach
uncompyle6 2.13.1 2017-10-11
2.13.1 2017-10-11
=====================
- Re-release because Python 2.4 source uploaded rather than 2.6-3.6
uncompyle6 2.13.0 2017-10-10
2.13.0 2017-10-10
=====================
- Fixes in deparsing lambda expressions
- Improve table-semantics descriptions
- Document hacky customize arg count better (until we can remove it)
- Update to use xdis 3.7.0 or greater
uncompyle6 2.12.0 2017-09-26
2.12.0 2017-09-26
=====================
- Use xdis 3.6.0 or greater now
- Small semantic table cleanups
@@ -228,11 +276,13 @@ uncompyle6 2.12.0 2017-09-26
- Slightly more Python 3.7, but still failing a lot
- Cross Python 2/3 compatibility with annotation arguments
uncompyle6 2.11.5 2017-08-31
2.11.5 2017-08-31
=====================
- Skeletal support for Python 3.7
uncompyle6 2.11.4 2017-08-15
2.11.4 2017-08-15
=====================
* scanner and parser now allow 3-part version string lookups,
e.g. 2.7.1 We allow a float here, but if passed a string like '2.7'. or
@@ -244,7 +294,8 @@ uncompyle6 2.11.4 2017-08-15
* Some PyPy tolerance in validate testing.
* Some pyston tolerance
uncompyle6 2.11.3 2017-08-09
2.11.3 2017-08-09
=====================
Very minor changes
@@ -253,20 +304,24 @@ Very minor changes
- use xdis opcode sets
- xdis "exception match" is now "exception-match"
uncompyle6 2.11.2 2017-07-09
2.11.2 2017-07-09
=====================
- Start supporting Pypy 3.5 (5.7.1-beta)
- use xdis 3.5.0's opcode sets and require xdis 3.5.0
- Correct some Python 2.4-2.6 loop detection
- guard against badly formatted bytecode
uncompyle6 2.11.1 2017-06-25
2.11.1 2017-06-25
=====================
- Python 3.x annotation and function signature fixes
- Bump xdis version
- Small pysource bug fixes
uncompyle6 2.11.0 2017-06-18 Fleetwood
2.11.0 2017-06-18 Fleetwood
==================================
- Major improvements in fragment tracking
* Add nonterminal node in extractInfo
* tag more offsets in expressions
@@ -276,14 +331,16 @@ uncompyle6 2.11.0 2017-06-18 Fleetwood
- Fixes yet again for make_function node handling; document what's up here
- Fix bug in snowflake Python 3.5 *args kwargs
uncompyle6 2.10.1 2017-06-3 Marylin Frankel
2.10.1 2017-06-3 Marylin Frankel
========================================
- fix some fragments parsing bugs
- was returning the wrong type sometimes in deparse_code_around_offset()
- capture function name in offsets
- track changes to ifelstrmtr node from pysource into fragments
uncompyle6 2.10.0 2017-05-30 Elaine Gordon
2.10.0 2017-05-30 Elaine Gordon
=======================================
- Add fuzzy offset deparse look up
- 3.6 bug fixes
@@ -303,19 +360,21 @@ uncompyle6 2.10.0 2017-05-30 Elaine Gordon
- 2.3, 2.4 "if 1 .." fixes
- 3.x annotation fixes
uncompyle6 2.9.11 2017-04-06
2.9.11 2017-04-06
=====================
- Better support for Python 3.5+ BUILD_MAP_UNPACK
- Start 3.6 CALL_FUNCTION_EX support
- Many decompilation bug fixes. (Many more remain). See ChangeLog
uncompyle6 2.9.10 2017-02-25
2.9.10 2017-02-25
=====================
- Python grammar rule fixes
- Add ability to get grammar coverage on runs
- Handle Python 3.6 opcode BUILD_CONST_KEYMAP
uncompyle6 2.9.9 2016-12-16
2.9.9 2016-12-16
- Remaining Python 3.5 ops handled
(this also means more Python 3.6 ops are handled)
@@ -325,7 +384,8 @@ uncompyle6 2.9.9 2016-12-16
- Better control-flow detection
- Code cleanups and misc bug fixes
uncompyle6 2.9.8 2016-12-16
2.9.8 2016-12-16
====================
- Better control-flow detection
- pseudo instruction THEN in 2.x
@@ -338,7 +398,8 @@ uncompyle6 2.9.8 2016-12-16
- verify call fixes for Python <= 2.4
- more Python lint
uncompyle6 2.9.7 2016-12-16
2.9.7 2016-12-16
====================
- Start to handle 3.5/3.6 build_map_unpack_with_call
- Some Python 3.6 bytecode to wordcode conversion fixes
@@ -348,7 +409,8 @@ uncompyle6 2.9.7 2016-12-16
- some 3.2 compatibility
- Better Python 3 control flow detection by adding Pseudo ELSE opcodes
uncompyle6 2.9.6 2016-12-04
2.9.6 2016-12-04
====================
- Shorten Python3 grammars with + and *
this requires spark parser 1.5.1
@@ -356,7 +418,8 @@ uncompyle6 2.9.6 2016-12-04
decompile accuracy. This too requires
spark parser 1.5.1
uncompyle6 2.9.6 2016-11-20
2.9.6 2016-11-20
====================
- Correct MANIFEST.in
- More AST grammar checking
@@ -373,7 +436,8 @@ uncompyle6 2.9.6 2016-11-20
- Python 2 and 3 detect structure code is more similar
- Handle Docstrings with embedded triple quotes (""")
uncompyle6 2.9.5 2016-11-13
2.9.5 2016-11-13
====================
- Fix Python 3 bugs:
* improper while 1 else
@@ -383,13 +447,15 @@ uncompyle6 2.9.5 2016-11-13
- Start grammar misparse checking
uncompyle6 2.9.4 2016-11-02
2.9.4 2016-11-02
====================
- Handle Python 3.x function annotations
- track def keyword-parameter line-splitting in source code better
- bump min xdis version to mask previous xdis bug
uncompyle6 2.9.3 2016-10-26
2.9.3 2016-10-26
====================
Release forced by incompatibility change in xdis 3.2.0.
@@ -404,7 +470,8 @@ Release forced by incompatibility change in xdis 3.2.0.
* Handle 3.6 handle single and multiple fstring better
uncompyle6 2.9.2 2016-10-15
2.9.2 2016-10-15
====================
- use source-code line breaks to assist in where to break
in tuples and maps
@@ -412,12 +479,14 @@ uncompyle6 2.9.2 2016-10-15
- Fix some Python 2.6 and below bugs
- DRY fragments.py code a little
uncompyle6 2.9.1 2016-10-09
2.9.1 2016-10-09
====================
- Improved Python 1.5 decompiling
- Handle old-style pre Python 2.2 classes
uncompyle6 2.9.0 2016-10-09
2.9.0 2016-10-09
====================
- Use xdis 3.0.0 protocol load_module.
this Forces change in requirements.txt and _pkg_info_.py
@@ -427,7 +496,8 @@ uncompyle6 2.9.0 2016-10-09
- Fix bug with -t ... Wasn't showing source text when -t option was given
- Fix 2.1-2.6 bug in list comprehension
uncompyle6 2.8.4 2016-10-08
2.8.4 2016-10-08
====================
- Python 3 disassembly bug fixes
- Python 3.6 fstring bug fixes (from moagstar)
@@ -435,7 +505,8 @@ uncompyle6 2.8.4 2016-10-08
- COME_FROM suffixes added in Python3
- use .py extension in verification disassembly
uncompyle6 2.8.3 2016-09-11 live from NYC!
2.8.3 2016-09-11 live from NYC!
=======================================
NOTE: this is possibly the last release before a major reworking of
control-flow structure detection is done.
@@ -463,14 +534,16 @@ control-flow structure detection is done.
- bump xdis requirement so we can deparse dropbox 2.5 code
- Added H. Goebel's changes before 2.4 in DECOMPYLE-2.4-CHANGELOG.txt
uncompyle6 2.8.2 2016-08-29
2.8.2 2016-08-29
====================
- Handle Python 3.6 format string conversions !r, !s, !a
- Start to handle 3.1 bytecode
- Fix some PyPy translation bugs
- We now only handle 3.6.0a3+ since that is incompatible with 3.6 before that
uncompyle6 2.8.1 2016-08-20
2.8.1 2016-08-20
====================
- Add Python 2.2 decompilation
@@ -478,7 +551,8 @@ uncompyle6 2.8.1 2016-08-20
* PyPy LOOKUP_METHOD bug
* Python 3.6 FORMAT_VALUE handles expressions now
uncompyle6 2.8.0 2016-08-03
2.8.0 2016-08-03
====================
- Start Python 3.6 support (moagstar)
more work on PEP 498 needed
@@ -489,20 +563,23 @@ uncompyle6 2.8.0 2016-08-03
- better grammar and semantic action segregation based
on python bytecode version
uncompyle6 2.7.1 2016-07-26
2.7.1 2016-07-26
====================
- PyPy bytecodes for 2.7 and 3.2 added
- Instruction formatting improved slightly
- 2.7 bytecode "continue" bug fixed
uncompyle6 2.7.0 2016-07-15
2.7.0 2016-07-15
====================
- Many Syntax and verification bugs removed
tested on standard libraries from 2.3.7 to 3.5.1
and they all decompile and verify fine.
I'm sure there are more bugs though.
uncompyle6 2.6.2 2016-07-11 Manhattenhenge
2.6.2 2016-07-11 Manhattenhenge
=======================================
- Extend bytecodes back to 2.3
- Fix bugs:
@@ -511,13 +588,15 @@ uncompyle6 2.6.2 2016-07-11 Manhattenhenge
* continue statements
- DRY and segregate grammar more
uncompyle6 2.6.1 2016-07-08
2.6.1 2016-07-08
====================
- Go over Python 2.5 bytecode deparsing
all library programs now deparse
- Fix a couple bugs in 2.6 deparsing
uncompyle6 2.6.0 2016-07-07
2.6.0 2016-07-07
====================
- Improve Python 2.6 bytecode deparsing:
stdlib now will deparse something
@@ -526,7 +605,8 @@ uncompyle6 2.6.0 2016-07-07
- Fix bug in installing uncompyle6 script
- Doc improvements
uncompyle6 2.5.0 2016-06-22 Summer Solstice
2.5.0 2016-06-22 Summer Solstice
========================================
- Much better Python 3.2-3.5 coverage.
3.4.6 is probably the best;3.2 and 3.5 are weaker
@@ -535,7 +615,8 @@ uncompyle6 2.5.0 2016-06-22 Summer Solstice
- Better fragment offset tracking
- Some (much-needed) code refactoring
uncompyle6 2.4.0 2016-05-18 (in memory of Lewis Bernstein)
2.4.0 2016-05-18 (in memory of Lewis Bernstein)
===========================================================
- Many Python 3 bugs fixed:
* Python 3.2 to 3.5 libraries largely
@@ -550,7 +631,8 @@ uncompyle6 2.4.0 2016-05-18 (in memory of Lewis Bernstein)
* handle complex number unmarshaling
* Running on Python 2 to works on Python 3.5 bytecodes now
uncompyle6 2.3.5 and 2.3.6 2016-05-14
2.3.5 and 2.3.6 2016-05-14
=================================
- Python 2 class decorator fix (thanks to Tey)
- Fix fragment parsing bugs
@@ -562,20 +644,23 @@ uncompyle6 2.3.5 and 2.3.6 2016-05-14
- Correct history based on info from Dan Pascu
- Fix up pip packaging, ugh.
uncompyle6 2.3.4 2016-05-5
2.3.4 2016-05-5
===================
- More Python 3.5 parsing bugs addressed
- decompiling Python 3.5 from other Python versions works
- test from Python 3.2
- remove "__module__ = __name__" in 3.0 <= Python 3.2
uncompyle6 2.3.3 2016-05-3
2.3.3 2016-05-3
===================
- Fix bug in running uncompyle6 script on Python 3
- Speed up performance on deparsing long lists by grouping in chunks of 32 and 256 items
- DRY Python expressions between Python 2 and 3
uncompyle6 2.3.2 2016-05-1
2.3.2 2016-05-1
===================
- Add --version option standalone scripts
- Correct License information in package
@@ -584,17 +669,20 @@ uncompyle6 2.3.2 2016-05-1
specific grammar code
- Fix bug in 3.5+ constant map parsing
uncompyle6 2.3.0, 2.3.1 2016-04-30
2.3.0, 2.3.1 2016-04-30
=============================
- Require spark_parser >= 1.1.0
uncompyle6 2.2.0 2016-04-30
2.2.0 2016-04-30
====================
- Spark is no longer here but pulled separate package spark_parse
- Python 3 parsing fixes
- More tests
uncompyle6 2.2.0 2016-04-02
2.2.0 2016-04-02
====================
- Support single-mode (in addition to exec-mode) compilation
- Start to DRY Python 2 and Python 3 grammars
@@ -602,7 +690,8 @@ uncompyle6 2.2.0 2016-04-02
- Fix bug in uncomplye6 -d and -r options (via lelicopter)
uncompyle6 2.1.3 2016-01-02
2.1.3 2016-01-02
====================
- Limited support for decompiling Python 3.5
- Improve Python 3 class deparsing
@@ -611,18 +700,21 @@ uncompyle6 2.1.3 2016-01-02
- increase test coverage
- fix misc small bugs and some improvements
uncompyle6 2.1.2 2015-12-31
2.1.2 2015-12-31
====================
- Fix cross-version Marshal loading
- Handle Python 3.3 . dotted class names
- Limited 3.5 support: allows deparsing other versions
- Refactor code more, misc bug fixes
uncompyle6 2.1.1 2015-12-27
2.1.1 2015-12-27
====================
- packaging issues
uncompyle6 2.1.0 2015-12-27
2.1.0 2015-12-27
====================
- Python 3.x deparsing much more solid
- Better cross-version deparsing
@@ -631,7 +723,8 @@ Some bugs squashed while other run rampant. Some code cleanup while
much more is yet needed. More tests added, but many more are needed.
uncompyle6 2.0.0 2015-12-11
2.0.0 2015-12-11
====================
Changes from uncompyle2

View File

@@ -57,7 +57,7 @@ entry_points = {
]}
ftp_url = None
install_requires = ['spark-parser >= 1.8.7, < 1.9.0',
'xdis >= 3.8.9, < 3.9.0']
'xdis >= 3.9.0, < 3.10.0']
license = 'GPL3'
mailing_list = 'python-debugger@googlegroups.com'

View File

@@ -30,9 +30,9 @@
$ make ChangeLog
# Update NEWS from ChangeLog:
# Update NEWS.md from ChangeLog:
$ emacs NEWS
$ emacs NEWS.md
$ make check
$ git commit --amend .
$ git push # get CI testing going early

View File

@@ -5,4 +5,4 @@ if [[ $0 == ${BASH_SOURCE[0]} ]] ; then
echo "This script should be *sourced* rather than run directly through bash"
exit 1
fi
export PYVERSIONS='3.5.5 3.6.8 3.7.2 2.6.9 3.3.7 2.7.15 3.2.6 3.1.5 3.4.8'
export PYVERSIONS='3.2.6 3.6.8 3.7.2 2.6.9 3.3.7 2.7.15 3.2.6 3.1.5 3.4.8'

View File

@@ -115,7 +115,7 @@ if PYTHON_VERSION > 2.6:
return "f{}'{}'".format('r' if is_raw else '', content)
@pytest.mark.skipif(PYTHON_VERSION < 3.6, reason='need at least python 3.6')
@pytest.mark.skipif(PYTHON_VERSION != 3.6, reason='need Python 3.6')
@hypothesis.given(format_specifiers())
def test_format_specifiers(format_specifier):
"""Verify that format_specifiers generates valid specifiers"""
@@ -137,14 +137,14 @@ if PYTHON_VERSION > 2.6:
assert 'dis(' + deparsed.text.strip('\n') + ')' == 'dis(' + expr.strip('\n') + ')'
@pytest.mark.skipif(PYTHON_VERSION < 3.6, reason='need at least python 3.6')
@pytest.mark.skipif(PYTHON_VERSION != 3.6, reason='need Python 3.6')
@hypothesis.given(fstrings())
def test_uncompyle_fstring(fstring):
"""Verify uncompyling fstring bytecode"""
run_test(fstring)
@pytest.mark.skipif(PYTHON_VERSION < 3.6, reason='need at least python 3.6')
@pytest.mark.skipif(PYTHON_VERSION < 3.6, reason='need Python 3.6+')
@pytest.mark.parametrize('fstring', [
"f'{abc}{abc!s}'",
"f'{abc}0'",

View File

@@ -7,7 +7,7 @@ def test_grammar():
def check_tokens(tokens, opcode_set):
remain_tokens = set(tokens) - opcode_set
remain_tokens = set([re.sub('_\d+$','', t) for t in remain_tokens])
remain_tokens = set([re.sub(r'_\d+$','', t) for t in remain_tokens])
remain_tokens = set([re.sub('_CONT$','', t) for t in remain_tokens])
remain_tokens = set(remain_tokens) - opcode_set
assert remain_tokens == set([]), \
@@ -46,6 +46,7 @@ def test_grammar():
unused_rhs.add("mkfunc_annotate")
unused_rhs.add("dict_comp")
unused_rhs.add("classdefdeco1")
unused_rhs.add("tryelsestmtl")
if PYTHON_VERSION >= 3.5:
expect_right_recursive.add((('l_stmts',
('lastl_stmt', 'come_froms', 'l_stmts'))))

View File

@@ -64,10 +64,12 @@ check-3.5: check-bytecode
#: Run working tests from Python 3.6
check-3.6: check-bytecode
$(PYTHON) test_pythonlib.py --bytecode-3.6 --weak-verify $(COMPILE)
$(PYTHON) test_pythonlib.py --bytecode-3.6-run --verify-run
#: Run working tests from Python 3.7
check-3.7: check-bytecode
$(PYTHON) test_pythonlib.py --bytecode-3.7 --weak-verify $(COMPILE)
$(PYTHON) test_pythonlib.py --bytecode-3.7-run --verify-run
# FIXME
#: this is called when running under pypy3.5-5.8.0 or pypy2-5.6.0
@@ -90,7 +92,8 @@ check-bytecode-2:
check-bytecode-3:
$(PYTHON) test_pythonlib.py --bytecode-3.0 \
--bytecode-3.1 --bytecode-3.2 --bytecode-3.3 \
--bytecode-3.4 --bytecode-3.5 --bytecode-3.6 --bytecode-pypy3.2
--bytecode-3.4 --bytecode-3.5 --bytecode-3.6 --bytecode-3.7 \
--bytecode-pypy3.2
#: Check deparsing on selected bytecode 3.x
check-bytecode-3-short:

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

View File

@@ -0,0 +1,10 @@
# Python 2.4 (and before?) bug in handling unconditional "else if true"
# Doesn't occur in Python > 2.4
# From Issue #187
def unconditional_if_true_24(foo):
if not foo:
pass
elif 1:
pass
else:
return None

View File

@@ -0,0 +1,16 @@
# Bug in Python 2.7. Bug is bytecode for while loop having
# two consecutive JUMP_BACKS at the end of 'elif' and 'while'
# to the same place
def PreprocessConditionalStatement(self, IfList, ReplacedLine):
while self:
if self.__Token:
x = 1
elif not IfList:
if self <= 2:
continue
RegionSizeGuid = 3
if not RegionSizeGuid:
RegionLayoutLine = 5
continue
RegionLayoutLine = self.CurrentLineNumber
return 1

View File

@@ -0,0 +1,22 @@
# Bug in Python 2.7 is code creating a (useless) JUMP_ABSOLUTE to the instruction right after
# the "raise" which causes the
# RUNNABLE!
def testit(a, b):
if a:
if not b:
raise AssertionError("test JUMP_ABSOLUTE to next instruction")
def testit2(a, b):
if a:
if not b:
raise AssertionError("test with dead code after raise")
x = 10
testit(False, True)
testit(False, False)
testit(True, True)
testit2(False, True)
testit2(False, False)
testit2(True, True)

View File

@@ -0,0 +1,24 @@
# Bug based on 2.7 test_itertools.py but mis-decompiled in Python 3.x bytecode
# The bug is confusing "try/else" with "try" as a result of the loop which causes
# the end of the except to jump back to the beginning of the loop, outside of the
# try. In 3.x we not distinguising this jump out of the loop with a jump to the
# end of the "try".
# RUNNABLE!
def testit(stmts):
# Bug was confusing When the except jumps back to the beginning of the block
# to the beginning of this for loop
x = 1
results = []
for stmt in stmts:
try:
x = eval(stmt)
except SyntaxError:
results.append(1)
else:
results.append(x)
return results
results = testit(["1 + 2", "1 +"])
assert results == [3, 1], "try with else failed"

View File

@@ -0,0 +1,15 @@
# Python 3 bug in not detecting the end bounds of if elif.
def testit(b):
if b == 1:
a = 1
elif b == 2:
a = 2
else:
a = 4
return a
for x in (1, 2, 4):
x = testit(x)
assert x is not None, "Should have returned a value, not None"
assert x == x

View File

@@ -0,0 +1,10 @@
# Bug in 3.6 and above.
#Not detecting 2nd return is outside of
# if/then. Fix was to ensure COME_FROM
def return_return_bug(foo):
if foo =='say_hello':
return "hello"
return "world"
assert return_return_bug('say_hello') == 'hello'
assert return_return_bug('world') == 'world'

View File

@@ -0,0 +1,16 @@
# Bug in 3.6 was not taking "else" branch after compond "if"
# In earlier versions we had else detection needed here.
def f(a, b, c):
if a and b:
x = 1
else:
x = 2
if c:
x = 3
return(x)
assert f(True, True, True) == 3
assert f(True, True, False) == 1
assert f(True, False, True) == 3
assert f(True, False, False) == 2

View File

@@ -0,0 +1,8 @@
# The bug in python 3.6+ was in parsing that we
# add END_IF_THEN and using that inside "return results"
def whcms_license_info(md5hash, datahash, results):
if md5hash == datahash:
try:
return md5hash
except:
return results

View File

@@ -0,0 +1,9 @@
# Bug in Python 3.6 and 3.7 was getting comma before **kw
def fn(arg, *, kwarg='test', **kw):
assert arg == 1
assert kwarg == 'testing'
assert kw['foo'] == 'bar'
fn(1, kwarg='testing', foo='bar')

View File

@@ -0,0 +1,19 @@
# From Python 3.7 pickle.py
# Bug was different code generation for chained comparisons than prior Python versions
def chained_compare_a(protocol):
if not 0 <= protocol <= 7:
raise ValueError("pickle protocol must be <= %d" % 7)
def chained_compare_b(a, obj):
if a:
if -0x80000000 <= obj <= 0x7fffffff:
return 5
chained_compare_a(3)
try:
chained_compare_a(8)
except ValueError:
pass
chained_compare_b(True, 0x0)

View File

@@ -3,7 +3,7 @@
[flake8]
exclude = .tox,./build,./trepan/processor/command/tmp
filename = *.py
ignore = C901,E113,E121,E122,E123,E124,E125,E126,E127,E128,E129,E201,E202,E203,E221,E222,E225,E226,E241,E242,E251,E261,E271,E272,E302,E401,E501,F401,E701,E702
ignore = C901,E113,E121,E122,E123,E124,E125,E126,E127,E128,E129,E201,E202,E203,E221,E222,E225,E226,E241,E242,E251,E261,E271,E272,E302,E401,E402,E501,F401,E701,E702
[tox]
envlist = py27, py34, pypy

View File

@@ -1,4 +1,4 @@
# Copyright (c) 2015-2016, 2818 by Rocky Bernstein
# Copyright (c) 2015-2016, 2818-2019 by Rocky Bernstein
# Copyright (c) 2005 by Dan Pascu <dan@windowmaker.org>
# Copyright (c) 2000-2002 by hartmut Goebel <h.goebel@crazy-compilers.com>
# Copyright (c) 1999 John Aycock
@@ -77,18 +77,18 @@ def disco_loop(disasm, queue, real_out):
pass
pass
def disassemble_fp(fp, outstream=None):
"""
disassemble Python byte-code from an open file
"""
(version, timestamp, magic_int, co, is_pypy,
source_size) = load_from_fp(fp)
if type(co) == list:
for con in co:
disco(version, con, outstream)
else:
disco(version, co, outstream, is_pypy=is_pypy)
co = None
# def disassemble_fp(fp, outstream=None):
# """
# disassemble Python byte-code from an open file
# """
# (version, timestamp, magic_int, co, is_pypy,
# source_size) = load_from_fp(fp)
# if type(co) == list:
# for con in co:
# disco(version, con, outstream)
# else:
# disco(version, co, outstream, is_pypy=is_pypy)
# co = None
def disassemble_file(filename, outstream=None):
"""
@@ -120,5 +120,6 @@ def _test():
fn = sys.argv[1]
disassemble_file(fn)
if __name__ == "__main__":
_test()

View File

@@ -40,10 +40,9 @@ def _get_outstream(outfile):
except OSError:
pass
if PYTHON_VERSION < 3.0:
mode = 'wb'
return open(outfile, mode='wb')
else:
mode = 'w'
return open(outfile, mode)
return open(outfile, mode='w', encoding='utf-8')
def decompile(
bytecode_version, co, out=None, showasm=None, showast=False,
@@ -315,10 +314,19 @@ def main(in_base, out_base, files, codes, outfile=None,
sys.stdout.write("%s\r" %
status_msg(do_verify, tot_files, okay_files, failed_files,
verify_failed_files, do_verify))
try:
# FIXME: Something is weird with Pypy here
sys.stdout.flush()
except:
pass
if current_outfile:
sys.stdout.write("\n")
try:
# FIXME: Something is weird with Pypy here
sys.stdout.flush()
except:
pass
pass
return (tot_files, okay_files, failed_files, verify_failed_files)

View File

@@ -37,7 +37,9 @@ class ParserError(Exception):
return "Parse error at or near `%r' instruction at offset %s\n" % \
(self.token, self.offset)
nop_func = lambda self, args: None
def nop_func(self, args):
return None
class PythonParser(GenericASTBuilder):
@@ -792,6 +794,7 @@ def python_parser(version, co, out=sys.stdout, showasm=False,
p = get_python_parser(version, parser_debug)
return parse(p, tokens, customize)
if __name__ == '__main__':
def parse_test(co):
from uncompyle6 import PYTHON_VERSION, IS_PYPY

View File

@@ -533,14 +533,11 @@ class Python2Parser(PythonParser):
# Dead code testing...
# if lhs == 'while1elsestmt':
# from trepan.api import debug; debug()
if lhs in ('aug_assign1', 'aug_assign2') and ast[0] and ast[0][0] in ('and', 'or'):
return True
elif lhs in ('raise_stmt1',):
# We will assme 'LOAD_ASSERT' will be handled by an assert grammar rule
return (tokens[first] == 'LOAD_ASSERT' and
(last >= len(tokens) or tokens[last] not in
('COME_FROM', 'JUMP_BACK','JUMP_FORWARD')))
# We will assume 'LOAD_ASSERT' will be handled by an assert grammar rule
return (tokens[first] == 'LOAD_ASSERT' and (last >= len(tokens)))
elif rule == ('or', ('expr', 'jmp_true', 'expr', '\\e_come_from_opt')):
expr2 = ast[2]
return expr2 == 'expr' and expr2[0] == 'LOAD_ASSERT'

View File

@@ -85,15 +85,17 @@ class Python25Parser(Python26Parser):
super(Python25Parser, self).customize_grammar_rules(tokens, customize)
if self.version == 2.5:
self.check_reduce['try_except'] = 'tokens'
self.check_reduce['aug_assign1'] = 'AST'
## Don't need this for 2.5 yet..
# def reduce_is_invalid(self, rule, ast, tokens, first, last):
# invalid = super(Python25Parser,
# self).reduce_is_invalid(rule, ast,
# tokens, first, last)
# if invalid or tokens is None:
# return invalid
# return False
def reduce_is_invalid(self, rule, ast, tokens, first, last):
invalid = super(Python25Parser,
self).reduce_is_invalid(rule, ast,
tokens, first, last)
if invalid or tokens is None:
return invalid
if rule == ('aug_assign1', ('expr', 'expr', 'inplace_op', 'store')):
return ast[0][0] == 'and'
return False
class Python25ParserSingle(Python26Parser, PythonParserSingle):

View File

@@ -1,9 +1,10 @@
# Copyright (c) 2016-2018 Rocky Bernstein
# Copyright (c) 2016-2019 Rocky Bernstein
# Copyright (c) 2005 by Dan Pascu <dan@windowmaker.org>
# Copyright (c) 2000-2002 by hartmut Goebel <hartmut@goebel.noris.de>
from spark_parser import DEFAULT_DEBUG as PARSER_DEFAULT_DEBUG
from uncompyle6.parser import PythonParserSingle
from xdis import next_offset
from uncompyle6.parser import PythonParserSingle, nop_func
from uncompyle6.parsers.parse2 import Python2Parser
class Python27Parser(Python2Parser):
@@ -155,7 +156,13 @@ class Python27Parser(Python2Parser):
while1stmt ::= SETUP_LOOP returns bp_come_from
while1stmt ::= SETUP_LOOP l_stmts_opt JUMP_BACK POP_BLOCK COME_FROM
whilestmt ::= SETUP_LOOP testexpr l_stmts_opt JUMP_BACK POP_BLOCK _come_froms
# Should this be JUMP_BACK+ ?
# JUMP_BACK should all be to the same location
whilestmt ::= SETUP_LOOP testexpr l_stmts_opt JUMP_BACK JUMP_BACK POP_BLOCK _come_froms
while1elsestmt ::= SETUP_LOOP l_stmts JUMP_BACK POP_BLOCK
else_suitel COME_FROM
whileelsestmt ::= SETUP_LOOP testexpr l_stmts_opt JUMP_BACK POP_BLOCK
@@ -196,6 +203,13 @@ class Python27Parser(Python2Parser):
POP_BLOCK LOAD_CONST COME_FROM suite_stmts_opt
END_FINALLY
""")
if 'PyPy' in customize:
# PyPy-specific customizations
self.addRule("""
return_if_stmt ::= ret_expr RETURN_END_IF come_froms
""", nop_func)
super(Python27Parser, self).customize_grammar_rules(tokens, customize)
self.check_reduce['and'] = 'AST'
# self.check_reduce['or'] = 'AST'
@@ -203,14 +217,17 @@ class Python27Parser(Python2Parser):
self.check_reduce['list_if_not'] = 'AST'
self.check_reduce['list_if'] = 'AST'
self.check_reduce['conditional_true'] = 'AST'
self.check_reduce['whilestmt'] = 'tokens'
return
def reduce_is_invalid(self, rule, ast, tokens, first, last):
invalid = super(Python27Parser,
self).reduce_is_invalid(rule, ast,
tokens, first, last)
if invalid:
return invalid
if rule == ('and', ('expr', 'jmp_false', 'expr', '\\e_come_from_opt')):
# Test that jmp_false jumps to the end of "and"
# or that it jumps to the same place as the end of "and"
@@ -220,6 +237,12 @@ class Python27Parser(Python2Parser):
tokens[last].pattr == jmp_false.pattr)
elif rule[0] == ('raise_stmt1'):
return ast[0] == 'expr' and ast[0][0] == 'or'
elif rule[0] in ('assert', 'assert2'):
jump_inst = ast[1][0]
jump_target = jump_inst.attr
return not (last >= len(tokens)
or jump_target == tokens[last].offset
or jump_target == next_offset(ast[-1].op, ast[-1].opc, ast[-1].offset))
elif rule == ('list_if_not', ('expr', 'jmp_true', 'list_iter')):
jump_inst = ast[1][0]
jump_offset = jump_inst.attr
@@ -235,6 +258,16 @@ class Python27Parser(Python2Parser):
jmp_target = jmp_true.offset + jmp_true.attr + 3
return not (jmp_target == tokens[last].offset or
tokens[last].pattr == jmp_true.pattr)
elif (rule[0] == 'whilestmt' and
rule[1][0:-2] ==
('SETUP_LOOP', 'testexpr', 'l_stmts_opt',
'JUMP_BACK', 'JUMP_BACK')):
# Make sure that the jump backs all go to the same place
i = last-1
while (tokens[i] != 'JUMP_BACK'):
i -= 1
return tokens[i].attr != tokens[i-1].attr
# elif rule[0] == ('conditional_true'):
# # FIXME: the below is a hack: we check expr for
# # nodes that could have possibly been a been a Boolean.

View File

@@ -1,4 +1,4 @@
# Copyright (c) 2015-2018 Rocky Bernstein
# Copyright (c) 2015-2019 Rocky Bernstein
# Copyright (c) 2005 by Dan Pascu <dan@windowmaker.org>
# Copyright (c) 2000-2002 by hartmut Goebel <h.goebel@crazy-compilers.com>
# Copyright (c) 1999 John Aycock
@@ -26,6 +26,7 @@ If we succeed in creating a parse tree, then we have a Python program
that a later phase can turn into a sequence of ASCII text.
"""
from uncompyle6.scanners.tok import Token
from uncompyle6.parser import PythonParser, PythonParserSingle, nop_func
from uncompyle6.parsers.treenode import SyntaxTree
from spark_parser import DEFAULT_DEBUG as PARSER_DEFAULT_DEBUG
@@ -184,23 +185,11 @@ class Python3Parser(PythonParser):
# one COME_FROM for Python 2.7 seems to associate the
# COME_FROM targets from the wrong places
try_except ::= SETUP_EXCEPT suite_stmts_opt POP_BLOCK
except_handler opt_come_from_except
# this is nested inside a try_except
tryfinallystmt ::= SETUP_FINALLY suite_stmts_opt
POP_BLOCK LOAD_CONST
COME_FROM_FINALLY suite_stmts_opt END_FINALLY
tryelsestmt ::= SETUP_EXCEPT suite_stmts_opt POP_BLOCK
except_handler else_suite come_from_except_clauses
tryelsestmt ::= SETUP_EXCEPT suite_stmts_opt POP_BLOCK
except_handler else_suite come_froms
tryelsestmtl ::= SETUP_EXCEPT suite_stmts_opt POP_BLOCK
except_handler else_suitel come_from_except_clauses
except_handler ::= jmp_abs COME_FROM except_stmts
END_FINALLY
except_handler ::= jmp_abs COME_FROM_EXCEPT except_stmts
@@ -320,7 +309,7 @@ class Python3Parser(PythonParser):
# FIXME: Common with 2.7
ret_and ::= expr JUMP_IF_FALSE_OR_POP ret_expr_or_cond COME_FROM
ret_or ::= expr JUMP_IF_TRUE_OR_POP ret_expr_or_cond COME_FROM
ret_cond ::= expr POP_JUMP_IF_FALSE expr RETURN_END_IF ret_expr_or_cond
ret_cond ::= expr POP_JUMP_IF_FALSE expr RETURN_END_IF COME_FROM ret_expr_or_cond
or ::= expr JUMP_IF_TRUE_OR_POP expr COME_FROM
and ::= expr JUMP_IF_FALSE_OR_POP expr COME_FROM
@@ -565,7 +554,8 @@ class Python3Parser(PythonParser):
customize_instruction_basenames = frozenset(
('BUILD', 'CALL', 'CONTINUE', 'DELETE', 'GET',
'JUMP', 'LOAD', 'LOOKUP', 'MAKE',
'RETURN', 'RAISE', 'UNPACK'))
'RETURN', 'RAISE', 'SETUP',
'UNPACK'))
# Opcode names in the custom_ops_processed set have rules that get added
# unconditionally and the rules are constant. So they need to be done
@@ -593,7 +583,6 @@ class Python3Parser(PythonParser):
stmt ::= assign2_pypy
assign3_pypy ::= expr expr expr store store store
assign2_pypy ::= expr expr store store
return_if_lambda ::= RETURN_END_IF_LAMBDA
stmt ::= conditional_lambda
stmt ::= conditional_not_lambda
conditional_lambda ::= expr jmp_false expr return_if_lambda
@@ -1118,6 +1107,26 @@ class Python3Parser(PythonParser):
raise_stmt2 ::= expr expr RAISE_VARARGS_2
""", nop_func)
custom_ops_processed.add(opname)
elif opname == 'SETUP_EXCEPT':
self.addRule("""
try_except ::= SETUP_EXCEPT suite_stmts_opt POP_BLOCK
except_handler opt_come_from_except
tryelsestmt ::= SETUP_EXCEPT suite_stmts_opt POP_BLOCK
except_handler else_suite come_from_except_clauses
tryelsestmt ::= SETUP_EXCEPT suite_stmts_opt POP_BLOCK
except_handler else_suite come_froms
tryelsestmtl ::= SETUP_EXCEPT suite_stmts_opt POP_BLOCK
except_handler else_suitel come_from_except_clauses
stmt ::= tryelsestmtl3
tryelsestmtl3 ::= SETUP_EXCEPT suite_stmts_opt POP_BLOCK
except_handler COME_FROM else_suitel
opt_come_from_except
""", nop_func)
custom_ops_processed.add(opname)
elif opname_base in ('UNPACK_EX',):
before_count, after_count = token.attr
rule = 'unpack ::= ' + opname + ' store' * (before_count + after_count + 1)
@@ -1136,8 +1145,11 @@ class Python3Parser(PythonParser):
self.check_reduce['aug_assign2'] = 'AST'
self.check_reduce['while1stmt'] = 'noAST'
self.check_reduce['while1elsestmt'] = 'noAST'
self.check_reduce['ifelsestmt'] = 'AST'
self.check_reduce['annotate_tuple'] = 'noAST'
self.check_reduce['kwarg'] = 'noAST'
self.check_reduce['try_except'] = 'AST'
# FIXME: remove parser errors caused by the below
# self.check_reduce['while1elsestmt'] = 'noAST'
return
@@ -1179,6 +1191,16 @@ class Python3Parser(PythonParser):
if last == n:
return False
return tokens[first].attr > tokens[last].offset
elif rule == ('try_except',
('SETUP_EXCEPT', 'suite_stmts_opt', 'POP_BLOCK',
'except_handler', 'opt_come_from_except')):
come_from_except = ast[-1]
if come_from_except[0] == 'COME_FROM':
# There should be at last two COME_FROMs, one from an
# exception handler and one from the try. Otherwise
# we have a try/else.
return True
pass
elif lhs == 'while1stmt':
# If there is a fall through to the COME_FROM_LOOP. then this is
@@ -1212,6 +1234,14 @@ class Python3Parser(PythonParser):
if offset != tokens[first].attr:
return True
return False
elif rule == ('ifelsestmt',
('testexpr', 'c_stmts_opt', 'jump_forward_else', 'else_suite', '_come_froms')):
# Make sure the highest/smallest "come from" offset comes inside the "if".
come_froms = ast[-1]
if not isinstance(come_froms, Token):
return tokens[first].offset > come_froms[-1].attr
return False
return False
class Python30Parser(Python3Parser):

View File

@@ -84,7 +84,7 @@ if __name__ == '__main__':
""".split()))
remain_tokens = set(tokens) - opcode_set
import re
remain_tokens = set([re.sub('_\d+$', '', t) for t in remain_tokens])
remain_tokens = set([re.sub(r'_\d+$', '', t) for t in remain_tokens])
remain_tokens = set([re.sub('_CONT$', '', t) for t in remain_tokens])
remain_tokens = set(remain_tokens) - opcode_set
print(remain_tokens)

View File

@@ -1,4 +1,4 @@
# Copyright (c) 2016-2017 Rocky Bernstein
# Copyright (c) 2016-2017, 2019 Rocky Bernstein
"""
spark grammar differences over Python 3.4 for Python 3.5.
"""
@@ -104,6 +104,8 @@ class Python35Parser(Python34Parser):
# In <.3.5 the below is a JUMP_FORWARD to a JUMP_ABSOLUTE.
return_if_stmt ::= ret_expr RETURN_END_IF POP_BLOCK
return_if_lambda ::= RETURN_END_IF_LAMBDA COME_FROM
jb_else ::= JUMP_BACK ELSE
ifelsestmtc ::= testexpr c_stmts_opt JUMP_FORWARD else_suitec
@@ -267,7 +269,7 @@ if __name__ == '__main__':
""".split()))
remain_tokens = set(tokens) - opcode_set
import re
remain_tokens = set([re.sub('_\d+$', '', t) for t in remain_tokens])
remain_tokens = set([re.sub(r'_\d+$', '', t) for t in remain_tokens])
remain_tokens = set([re.sub('_CONT$', '', t) for t in remain_tokens])
remain_tokens = set(remain_tokens) - opcode_set
print(remain_tokens)

View File

@@ -1,4 +1,4 @@
# Copyright (c) 2016-2018 Rocky Bernstein
# Copyright (c) 2016-2019 Rocky Bernstein
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -46,6 +46,8 @@ class Python36Parser(Python35Parser):
# RETURN_VALUE is meant. Specifcally this can happen in
# ifelsestmt -> ...else_suite _. suite_stmts... (last) stmt
return ::= ret_expr RETURN_END_IF
return ::= ret_expr RETURN_VALUE COME_FROM
return_stmt_lambda ::= ret_expr RETURN_VALUE_LAMBDA COME_FROM
# A COME_FROM is dropped off because of JUMP-to-JUMP optimization
and ::= expr jmp_false expr
@@ -122,6 +124,7 @@ class Python36Parser(Python35Parser):
# """)
super(Python36Parser, self).customize_grammar_rules(tokens, customize)
self.remove_rules("""
except_handler ::= JUMP_FORWARD COME_FROM_EXCEPT except_stmts END_FINALLY COME_FROM
async_for_stmt ::= SETUP_LOOP expr
GET_AITER
LOAD_CONST YIELD_FROM SETUP_EXCEPT GET_ANEXT LOAD_CONST
@@ -238,6 +241,8 @@ class Python36Parser(Python35Parser):
WITH_CLEANUP_START WITH_CLEANUP_FINISH END_FINALLY
"""
self.addRule(rules_str, nop_func)
pass
pass
def custom_classfunc_rule(self, opname, token, customize, next_token):
@@ -344,7 +349,8 @@ class Python36Parser(Python35Parser):
if nt[0] == 'call_kw':
return True
nt = nt[0]
pass
pass
return False
class Python36ParserSingle(Python36Parser, PythonParserSingle):
pass
@@ -365,7 +371,7 @@ if __name__ == '__main__':
""".split()))
remain_tokens = set(tokens) - opcode_set
import re
remain_tokens = set([re.sub('_\d+$', '', t) for t in remain_tokens])
remain_tokens = set([re.sub(r'_\d+$', '', t) for t in remain_tokens])
remain_tokens = set([re.sub('_CONT$', '', t) for t in remain_tokens])
remain_tokens = set(remain_tokens) - opcode_set
print(remain_tokens)

View File

@@ -1,4 +1,4 @@
# Copyright (c) 2017-2018 Rocky Bernstein
# Copyright (c) 2017-2019 Rocky Bernstein
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -67,6 +67,17 @@ class Python37Parser(Python36Parser):
# FIXME: generalize and specialize
call ::= expr CALL_METHOD_0
testtrue ::= compare_chained37
compare_chained37 ::= expr compare_chained1a_37
compare_chained37 ::= expr compare_chained1b_37
compare_chained2a_37 ::= expr COMPARE_OP POP_JUMP_IF_TRUE JUMP_FORWARD
compare_chained2b_37 ::= expr COMPARE_OP COME_FROM POP_JUMP_IF_FALSE JUMP_FORWARD ELSE
compare_chained1a_37 ::= expr DUP_TOP ROT_THREE COMPARE_OP POP_JUMP_IF_FALSE
compare_chained2a_37 ELSE POP_TOP COME_FROM
compare_chained1b_37 ::= expr DUP_TOP ROT_THREE COMPARE_OP POP_JUMP_IF_FALSE
compare_chained2b_37 POP_TOP JUMP_FORWARD COME_FROM
"""
def customize_grammar_rules(self, tokens, customize):
@@ -104,7 +115,7 @@ if __name__ == '__main__':
""".split()))
remain_tokens = set(tokens) - opcode_set
import re
remain_tokens = set([re.sub('_\d+$', '', t) for t in remain_tokens])
remain_tokens = set([re.sub(r'_\d+$', '', t) for t in remain_tokens])
remain_tokens = set([re.sub('_CONT$', '', t) for t in remain_tokens])
remain_tokens = set(remain_tokens) - opcode_set
print(remain_tokens)

View File

@@ -351,9 +351,9 @@ class Scanner(object):
return result
# FIXME: this is broken on 3.6+. Replace remaining (2.x-based) calls
# with inst_matches
def all_instr(self, start, end, instr, target=None, include_beyond_target=False):
"""
Find all `instr` in the block from start to end.
@@ -423,8 +423,8 @@ class Scanner(object):
last_was_extarg = False
n = len(instructions)
for i, inst in enumerate(instructions):
if (inst.opname == 'EXTENDED_ARG' and
i+1 < n and instructions[i+1].opname != 'MAKE_FUNCTION'):
if (inst.opname == 'EXTENDED_ARG'
and i+1 < n and instructions[i+1].opname != 'MAKE_FUNCTION'):
last_was_extarg = True
starts_line = inst.starts_line
is_jump_target = inst.is_jump_target
@@ -527,6 +527,7 @@ def get_scanner(version, is_pypy=False, show_asm=None):
raise RuntimeError("Unsupported Python version %s" % version)
return scanner
if __name__ == "__main__":
import inspect, uncompyle6
co = inspect.currentframe().f_code

View File

@@ -1,4 +1,4 @@
# Copyright (c) 2015-2018 by Rocky Bernstein
# Copyright (c) 2015-2019 by Rocky Bernstein
# Copyright (c) 2005 by Dan Pascu <dan@windowmaker.org>
# Copyright (c) 2000-2002 by hartmut Goebel <h.goebel@crazy-compilers.com>
#
@@ -886,9 +886,10 @@ class Scanner2(Scanner):
# or a conditional assignment like:
# x = 1 if x else 2
#
# There are other contexts we may need to consider
# like whether the target is "END_FINALLY"
# or if the condition jump is to a forward location
# There are other situations we may need to consider, like
# if the condition jump is to a forward location.
# Also the existence of a jump to the instruction after "END_FINALLY"
# will distinguish "try/else" from "try".
code_pre_rtarget = code[pre_rtarget]
if code_pre_rtarget in self.jump_forward:

View File

@@ -1,4 +1,4 @@
# Copyright (c) 2015-2018 by Rocky Bernstein
# Copyright (c) 2015-2019 by Rocky Bernstein
# Copyright (c) 2005 by Dan Pascu <dan@windowmaker.org>
# Copyright (c) 2000-2002 by hartmut Goebel <h.goebel@crazy-compilers.com>
#
@@ -609,7 +609,9 @@ class Scanner3(Scanner):
"""
code = self.code
op = self.insts[inst_index].opcode
inst = self.insts[inst_index]
op = inst.opcode
# Detect parent structure
parent = self.structs[0]
@@ -632,7 +634,7 @@ class Scanner3(Scanner):
# Try to find the jump_back instruction of the loop.
# It could be a return instruction.
start += instruction_size(op, self.opc)
start += inst.inst_size
target = self.get_target(offset)
end = self.restrict_to_parent(target, parent)
self.setup_loops[target] = offset
@@ -718,8 +720,8 @@ class Scanner3(Scanner):
'start': after_jump_offset,
'end': end})
elif op in self.pop_jump_tf:
start = offset + instruction_size(op, self.opc)
target = self.insts[inst_index].argval
start = offset + inst.inst_size
target = inst.argval
rtarget = self.restrict_to_parent(target, parent)
prev_op = self.prev_op
@@ -757,7 +759,7 @@ class Scanner3(Scanner):
pre_rtarget = prev_op[rtarget]
# Is it an "and" inside an "if" or "while" block
if op == self.opc.POP_JUMP_IF_FALSE and self.version < 3.6:
if op == self.opc.POP_JUMP_IF_FALSE:
# Search for another POP_JUMP_IF_FALSE targetting the same op,
# in current statement, starting from current offset, and filter
@@ -851,9 +853,10 @@ class Scanner3(Scanner):
# For 3.5, in addition the JUMP_FORWARD above we could have
# JUMP_BACK or CONTINUE
#
# There are other contexts we may need to consider
# like whether the target is "END_FINALLY"
# or if the condition jump is to a forward location
# There are other situations we may need to consider, like
# if the condition jump is to a forward location.
# Also the existence of a jump to the instruction after "END_FINALLY"
# will distinguish "try/else" from "try".
if self.is_jump_forward(pre_rtarget) or (rtarget_is_ja and self.version >= 3.5):
if_end = self.get_target(pre_rtarget)
@@ -919,6 +922,10 @@ class Scanner3(Scanner):
return
pass
pass
if self.version >= 3.4:
self.fixed_jumps[offset] = rtarget
if code[pre_rtarget] == self.opc.RETURN_VALUE:
# If we are at some sort of POP_JUMP_IF and the instruction before was
# COMPARE_OP exception-match, then pre_rtarget is not an end_if

View File

@@ -107,7 +107,7 @@ class Token():
pattr = self.opc.cmp_op[self.attr]
# And so on. See xdis/bytecode.py get_instructions_bytes
pass
elif re.search('_\d+$', self.kind):
elif re.search(r'_\d+$', self.kind):
return "%s%s%s" % (prefix, offset_opname, argstr)
else:
pattr = ''

View File

@@ -1,4 +1,4 @@
# Copyright (c) 2018 by Rocky Bernstein
# Copyright (c) 2018-2019 by Rocky Bernstein
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -263,6 +263,12 @@ def customize_for_version3(self, version):
self.prune() # stop recursing
self.n_mkfunc_annotate = n_mkfunc_annotate
TABLE_DIRECT.update({
'tryelsestmtl3': ( '%|try:\n%+%c%-%c%|else:\n%+%c%-',
(1, 'suite_stmts_opt'),
(3, 'except_handler'),
(5, 'else_suitel') ),
})
if version >= 3.4:
########################
# Python 3.4+ Additions
@@ -911,6 +917,16 @@ def customize_for_version3(self, version):
'async_for_stmt': (
'%|async for %c in %c:\n%+%c%-%-\n\n',
7, 1, 17),
'compare_chained1a_37': ( ' %[3]{pattr.replace("-", " ")} %p %p',
(0, 19),
(-4, 19)),
'compare_chained1b_37': ( ' %[3]{pattr.replace("-", " ")} %p %p',
(0, 19),
(-4, 19)),
'compare_chained2a_37': ( '%[1]{pattr.replace("-", " ")} %p', (0, 19)),
'compare_chained2b_37': ( '%[1]{pattr.replace("-", " ")} %p', (0, 19)),
})
pass
pass # version >= 3.6

View File

@@ -735,7 +735,7 @@ def make_function3(self, node, is_lambda, nested=1, code_node=None):
self.write(sep)
self.write("%s=%s" % (n, defaults[i]))
sep = ', '
ends_in_comma = True
ends_in_comma = False
pass
pass
pass

View File

@@ -12,4 +12,4 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# This file is suitable for sourcing inside bash as
# well as importing into Python
VERSION='3.2.5'
VERSION='3.2.6'