Compare commits

..

11 Commits

Author SHA1 Message Date
rocky
df8d253f78 2.4 doesn't do six 2017-06-03 06:00:47 -04:00
rocky
89b42e3696 Nope it (appveyor) doesn't. 2017-06-03 05:55:21 -04:00
rocky
22e5a4a283 Administrivia
See if appveyor will handle 2.5
2017-06-03 05:53:41 -04:00
rocky
61810172d1 Merge branch 'master' into python-2.4 2017-06-03 05:50:42 -04:00
rocky
7c299fbf37 Streamline .travis.yml a little bit 2017-06-03 05:38:05 -04:00
rocky
da695115b5 We need six 2017-06-03 05:36:50 -04:00
rocky
f1d9e194fe Go over administrivia 2017-06-03 05:31:46 -04:00
rocky
e727a437ea Get ready for release 2.10.1 2017-06-03 05:26:34 -04:00
rocky
9a3e11a957 Fragment bugs
fragment.py:
* deparse_code_aorund_offset: was sometimes returning the wrong type
* capture function name offset
* lint imports

pysource.py: use a clearer variable name
2017-06-03 05:18:40 -04:00
rocky
966a4bc7dc Track changes in ifelstmtr..
in fragments from pysource
2017-06-02 21:15:23 -04:00
rocky
658c8b4be7 No decorators in Python < 2.6 2017-05-30 02:30:56 -04:00
11 changed files with 298 additions and 206 deletions

View File

@@ -6,7 +6,7 @@ python:
- '2.7' # this is a cheat here because travis doesn't do 2.4-2.6
install:
- pip install -r requirements.txt
- pip install -e .
- pip install -r requirements-dev.txt
script:

258
ChangeLog
View File

@@ -1,8 +1,258 @@
2017-05-30 rocky <rb@dustyfeet.com>
* pytest/test_function_call.py: No decorators in Python < 2.6
2017-05-30 rocky <rb@dustyfeet.com>
* : commit ad98fae3d4b0b83f65b15da8201e33c0ee6dab17 Author: rocky
<rb@dustyfeet.com> Date: Tue May 30 01:26:52 2017 -0400
2017-05-30 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/fragments.py: Python 3.6 makefunction
handling for fragments
2017-05-23 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/pysource.py: Fix up 3.6 unmapexpr
2017-05-23 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/pysource.py: Fix up retreiving "async"
property on 3.6
2017-05-23 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/pysource.py: Fix bug in a 3.6 class name.
2017-05-23 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/fragments.py,
uncompyle6/semantics/pysource.py: Add fuzzy offset deparse lookup
2017-05-21 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner3.py: Correct EXTENDED_ARG handling on
Python 3.6... where it can appear several times and xdis may handle it as well.
It possibly in other versions bug since EXTENDED_ARG is used so
rarely there because it has such a high value 1<<16, it's hard to
test and determine that.
2017-05-20 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/pysource.py: Worse results. Revert some of
the last changes
2017-05-20 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse3.py, uncompyle6/semantics/pysource.py:
More explicit about 3.5 UNMAP_PACK Have to reduce 3.5 bytecode testing for now, code is more solid.
2017-05-19 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse3.py, uncompyle6/parsers/parse36.py,
uncompyle6/scanners/scanner3.py: Simplify EXTENDED_ARG on 3.x We largely remove them and fold them itno the next op.
MAKE_FUNCTION though before 3.6 is an exception as that indicates an
annotated function
2017-05-19 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner26.py: EXTENDED_ARG is implemented in
2.6
2017-05-19 rocky <rb@dustyfeet.com>
* test/simple_source/expression/06_huge_list.py,
uncompyle6/parsers/parse3.py, uncompyle6/semantics/pysource.py: Fix
EXTENDED_ARG for long lists, sets, maps
2017-05-19 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner3.py: Another attempt at getting
get_target() correct
2017-05-19 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse3.py: Bug in pypy JUMP_IF_NOT_DEBUG
handling
2017-05-19 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse36.py, uncompyle6/scanners/scanner3.py:
EXTENDED_ARG handling... get_target() wasn't taking into account EXTENDED_ARG before opcode. This is mostly relevant in Python 3.6 where the max size before
needing EXTENDED_ARG has been reduced to 256, but theoretically
possible in earlier versions.
2017-05-18 rocky <rb@dustyfeet.com>
* __pkginfo__.py: Enforce using xdis >=3.3.1 .. to pick up bug fixes to 3.6 in xdis
2017-05-17 rocky <rb@dustyfeet.com>
* __pkginfo__.py, uncompyle6/parsers/parse36.py,
uncompyle6/scanners/scanner3.py: Small changes.... * __pkginfo__.py: Need spark parser 1.6.1 for corrected
remove_rules() fn * parser36.py: remove replaced Python3 rules * scanner3.py: corrected comment. Thanks to moagstar here. *
2017-05-16 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse36.py: Fix broken CI on 3.6... Another grammar rule replacing SETUP_LOOP with setup_loop
2017-05-16 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse36.py: More EXTENDED_ARGS on 3.6
2017-05-16 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse36.py: extend use of EXTENDED_ARGS in 3.6 switching to a wordcode seems to have made opcode fields smaller so
we need EXTENDED_ARG more?
2017-05-16 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse36.py, uncompyle6/semantics/pysource.py:
Allow LOAD_CONST EXTENDED_ARG
2017-05-15 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse3.py: Reinstate 3.6 listcomp rule
2017-05-15 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse3.py: Bang on 3.6 MAKE_FUNCTION some more
2017-05-14 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/pysource.py: towards fixing a
3.5.CALL_FUNCTONI_VAR bug
2017-05-14 rocky <rb@dustyfeet.com>
* test/simple_source/bug35/04_CALL_FUNCTION_VAR_KW.py,
uncompyle6/parsers/parse3.py: Python 3.5 kw arg can be an expr Fixes Issue #95
2017-05-14 R. Bernstein <rocky@users.noreply.github.com>
* : Merge pull request #117 from rocky/3.6-MAKE_FUNCTION 3.6 make function
2017-05-13 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner3.py: MAKE_FUNCTION_FLAGS can be a
simpler tuple
2017-05-13 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse3.py: Grammar rules for Python 3.6
MAKE_FUNCTION
2017-05-13 rocky <rb@dustyfeet.com>
* README.rst, uncompyle6/parsers/parse3.py,
uncompyle6/parsers/parse36.py, uncompyle6/semantics/pysource.py:
Bang on 3.6 MAKE_FUNCTION a bit more parse3.py, parse36.py: adding return_closure rule tags what's going
on with this rule pysource.py: start changing semantic rules to support code changed
by new make_function semantics README.rst: typo
2017-05-13 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner3.py: Typo
2017-05-12 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse27.py: Bug in 2.7 decompiling ourself! Troublesome file was uncompyle6.semantics.pysource.engine()
2017-05-11 R. Bernstein <rocky@users.noreply.github.com>
* : Merge pull request #113 from grkov90/patch-1 Fixed out_base bug
2017-05-11 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse3.py, uncompyle6/scanners/scanner3.py,
uncompyle6/semantics/make_function.py: WIP: start 3.6 MAKE_FUNCTION
handling
2017-05-11 Daniel Bradburn <moagstar@gmail.com>
* : Merge pull request #116 from moagstar/function_call_keyword_only Added support for Python 3.6 CALL_FUNCTION_KW
2017-05-10 Daniel Bradburn <moagstar@gmail.com>
* uncompyle6/semantics/pysource.py: Fixed bug in compiling double
star arg only function calls where the closing parenthesis would be
missed
2017-05-10 Daniel Bradburn <moagstar@gmail.com>
* requirements-dev.txt: Adding requirement for pytest >= 3.0 to fix
strange INTERNALERROR in combination with hypothesis when using
pytest 2.6.4
2017-05-10 Daniel Bradburn <moagstar@gmail.com>
* pytest/test_CALL_FUNCTION_KW.sh, pytest/test_function_call.py,
uncompyle6/parsers/parse3.py, uncompyle6/parsers/parse36.py,
uncompyle6/scanners/scanner36.py, uncompyle6/semantics/pysource.py:
Added support for support for Python 3.6 CALL_FUNCTION_KW
2017-05-08 rocky <rb@dustyfeet.com>
* appveyor.yml, test/test_pyenvlib.py,
uncompyle6/semantics/pysource.py: pysource guard and another
appveyor test
2017-05-08 rocky <rb@dustyfeet.com>
* appveyor.yml: appveyor take 2
2017-05-08 rocky <rb@dustyfeet.com>
* appveyor.yml, appveyor/install.ps1, appveyor/run_with_env.cmd: Try
appveyor
2017-05-07 rocky <rb@dustyfeet.com>
* uncompyle6/semantics/pysource.py: More guarded CONTINUE deletion
2017-05-07 rocky <rb@dustyfeet.com>
* uncompyle6/scanner.py, uncompyle6/scanners/scanner3.py,
uncompyle6/semantics/pysource.py: Reduce spurious "continue"
statements
2017-05-07 rocky <rb@dustyfeet.com>
* test/Makefile: --weak-verify on 3.3 with inclusion of last commit Note that the result is sematically equivalent, so it is is correct.
2017-05-07 rocky <rb@dustyfeet.com>
* test/simple_source/looping/12_if_while_true_pass.py,
uncompyle6/scanners/scanner3.py: Python 3.x control-flow bug... "pass" statement inside "while True"
2017-05-07 rocky <rb@dustyfeet.com>
* HOW-TO-REPORT-A-BUG.md: Small typo
2017-05-07 rocky <rb@dustyfeet.com>
* uncompyle6/scanners/scanner3.py: Fix improper COME_FROM_EXCEPT in
Python 3.3+
2017-05-06 rocky <rb@dustyfeet.com>
* uncompyle6/parsers/parse33.py: python 3.3 while True parsing bug
2017-05-06 rocky <rb@dustyfeet.com>
* ChangeLog, NEWS, uncompyle6/version.py: Get ready for release
2.9.11
2017-05-06 rocky <rb@dustyfeet.com>
* ChangeLog, NEWS, uncompyle6/version.py: Get ready for release
2.9.11
2017-05-06 rocky <rb@dustyfeet.com>
* test/Makefile: fix PYTHON variable setting in test/Makefile
2017-05-06 rocky <rb@dustyfeet.com>
* test/Makefile, uncompyle6/scanners/scanner2.py,
@@ -23,6 +273,14 @@
* .travis.yml: Try CI testing on Python 3.6
2017-05-03 Gregory <grkov90@gmail.com>
* uncompyle6/main.py: Some fix
2017-05-03 Gregory <grkov90@gmail.com>
* uncompyle6/main.py: Fixed out_base bug Variable filename using in for tags uncompyle6 -o haven't worked argument -o haven't worked
2017-05-02 rocky <rb@dustyfeet.com>
* test/simple_source/bug35/01_map_unpack.py, uncompyle6/parser.py,

7
NEWS
View File

@@ -1,3 +1,10 @@
uncompyle6 2.10.1 2016-06-3 Marylin Frankel
- fix some fragments parsing bugs
- was returning the wrong type sometimes in deparse_code_around_offset()
- capture function name in offsets
- track changes to ifelstrmtr node from pysource into fragments
uncompyle6 2.10.0 2016-05-30 Elaine Gordon
- Add fuzzy offset deparse lookup

View File

@@ -56,7 +56,7 @@ This uses setup.py, so it follows the standard Python routine:
::
pip install -r requirements.txt
pip install -e setup.py
pip install -r requirements-dev.txt
python setup.py install # may need sudo
# or if you have pyenv:

View File

@@ -6,7 +6,7 @@ machine:
dependencies:
override:
- pip install -r requirements.txt
- pip install -e .
- pip install -r requirements-dev.txt
test:
override:

View File

@@ -1,175 +0,0 @@
# std
import string
# 3rd party
from hypothesis import given, assume, example, settings, strategies as st
import pytest
# uncompyle
from validate import validate_uncompyle
from test_fstring import expressions
alpha = st.sampled_from(string.ascii_lowercase)
numbers = st.sampled_from(string.digits)
alphanum = st.sampled_from(string.ascii_lowercase + string.digits)
@st.composite
def function_calls(draw,
min_keyword_args=0, max_keyword_args=5,
min_positional_args=0, max_positional_args=5,
min_star_args=0, max_star_args=1,
min_double_star_args=0, max_double_star_args=1):
"""
Strategy factory for generating function calls.
:param draw: Callable which draws examples from other strategies.
:return: The function call text.
"""
st_positional_args = st.lists(
alpha,
min_size=min_positional_args,
max_size=max_positional_args
)
st_keyword_args = st.lists(
alpha,
min_size=min_keyword_args,
max_size=max_keyword_args
)
st_star_args = st.lists(
alpha,
min_size=min_star_args,
max_size=max_star_args
)
st_double_star_args = st.lists(
alpha,
min_size=min_double_star_args,
max_size=max_double_star_args
)
positional_args = draw(st_positional_args)
keyword_args = draw(st_keyword_args)
st_values = st.lists(
expressions(),
min_size=len(keyword_args),
max_size=len(keyword_args)
)
keyword_args = [
x + '=' + e
for x, e in
zip(keyword_args, draw(st_values))
]
star_args = ['*' + x for x in draw(st_star_args)]
double_star_args = ['**' + x for x in draw(st_double_star_args)]
arguments = positional_args + keyword_args + star_args + double_star_args
draw(st.randoms()).shuffle(arguments)
arguments = ','.join(arguments)
function_call = 'fn({arguments})'.format(arguments=arguments)
try:
# TODO: Figure out the exact rules for ordering of positional, keyword,
# star args, double star args and in which versions the various
# types of arguments are supported so we don't need to check that the
# expression compiles like this.
compile(function_call, '<string>', 'single')
except:
assume(False)
return function_call
def test_function_no_args():
validate_uncompyle("fn()")
def isolated_function_calls(which):
"""
Returns a strategy for generating function calls, but isolated to
particular types of arguments, for example only positional arguments.
This can help reason about debugging errors in specific types of function
calls.
:param which: One of 'keyword', 'positional', 'star', 'double_star'
:return: Strategy for generating an function call isolated to specific
argument types.
"""
kwargs = dict(
max_keyword_args=0,
max_positional_args=0,
max_star_args=0,
max_double_star_args=0,
)
kwargs['_'.join(('min', which, 'args'))] = 1
kwargs['_'.join(('max', which, 'args'))] = 5 if 'star' not in which else 1
return function_calls(**kwargs)
with settings(max_examples=25):
@given(isolated_function_calls('positional'))
@example("fn(0)")
def test_function_positional_only(expr):
validate_uncompyle(expr)
@given(isolated_function_calls('keyword'))
@example("fn(a=0)")
def test_function_call_keyword_only(expr):
validate_uncompyle(expr)
@given(isolated_function_calls('star'))
@example("fn(*items)")
def test_function_call_star_only(expr):
validate_uncompyle(expr)
@given(isolated_function_calls('double_star'))
@example("fn(**{})")
def test_function_call_double_star_only(expr):
validate_uncompyle(expr)
@pytest.mark.xfail()
def test_BUILD_CONST_KEY_MAP_BUILD_MAP_UNPACK_WITH_CALL_BUILD_TUPLE_CALL_FUNCTION_EX():
validate_uncompyle("fn(w=0,m=0,**v)")
@pytest.mark.xfail()
def test_BUILD_MAP_BUILD_MAP_UNPACK_WITH_CALL_BUILD_TUPLE_CALL_FUNCTION_EX():
validate_uncompyle("fn(a=0,**g)")
@pytest.mark.xfail()
def test_CALL_FUNCTION_EX():
validate_uncompyle("fn(*g,**j)")
@pytest.mark.xfail()
def test_BUILD_MAP_CALL_FUNCTION_EX():
validate_uncompyle("fn(*z,u=0)")
@pytest.mark.xfail()
def test_BUILD_TUPLE_CALL_FUNCTION_EX():
validate_uncompyle("fn(**a)")
@pytest.mark.xfail()
def test_BUILD_MAP_BUILD_TUPLE_BUILD_TUPLE_UNPACK_WITH_CALL_CALL_FUNCTION_EX():
validate_uncompyle("fn(b,b,b=0,*a)")
@pytest.mark.xfail()
def test_BUILD_TUPLE_BUILD_TUPLE_UNPACK_WITH_CALL_CALL_FUNCTION_EX():
validate_uncompyle("fn(*c,v)")
@pytest.mark.xfail()
def test_BUILD_CONST_KEY_MAP_CALL_FUNCTION_EX():
validate_uncompyle("fn(i=0,y=0,*p)")
@pytest.mark.skip(reason='skipping property based test until all individual tests are passing')
@given(function_calls())
def test_function_call(function_call):
validate_uncompyle(function_call)

View File

@@ -6,8 +6,7 @@ import difflib
import subprocess
import tempfile
import functools
# compatability
import six
from StringIO import StringIO
# uncompyle6 / xdis
from uncompyle6 import PYTHON_VERSION, IS_PYPY, deparse_code
# TODO : I think we can get xdis to support the dis api (python 3 version) by doing something like this there
@@ -123,7 +122,7 @@ def validate_uncompyle(text, mode='exec'):
original_text = text
deparsed = deparse_code(PYTHON_VERSION, original_code,
compile_mode=mode, out=six.StringIO())
compile_mode=mode, out=StringIO())
uncompyled_text = deparsed.text
uncompyled_code = compile(uncompyled_text, '<string>', 'exec')

View File

@@ -1,4 +1,3 @@
pytest>=3.0.0
flake8
hypothesis
six

View File

@@ -53,36 +53,30 @@ The node position 0 will be associated with "import".
import re, sys
from uncompyle6 import PYTHON3, IS_PYPY, PYTHON_VERSION
from xdis.code import iscode
from uncompyle6.semantics import pysource
from uncompyle6 import parser
from uncompyle6.scanner import Token, Code, get_scanner
from uncompyle6.semantics.check_ast import checker
from uncompyle6.semantics.helper import print_docstring
from uncompyle6.show import (
maybe_show_asm,
maybe_show_ast,
maybe_show_ast_param_default,
)
from uncompyle6.parsers.astnode import AST
from uncompyle6.semantics.pysource import (
ParserError, find_globals, StringIO)
ParserError, StringIO)
from uncompyle6.semantics.consts import (
INDENT_PER_LEVEL, NONE, PRECEDENCE,
TABLE_DIRECT, escape, minint, MAP
)
from uncompyle6.semantics.make_function import (
find_all_globals, find_none, code_has_star_arg, code_has_star_star_arg
)
from spark_parser import DEFAULT_DEBUG as PARSER_DEFAULT_DEBUG
from uncompyle6 import PYTHON_VERSION
if PYTHON_VERSION < 2.6:
from xdis.namedtuple25 import namedtuple
else:
@@ -379,11 +373,18 @@ class FragmentsWalker(pysource.SourceWalker, object):
self.prune() # stop recursing
def n_ifelsestmtr(self, node):
if len(node[2]) != 2:
if node[2] == 'COME_FROM':
return_stmts_node = node[3]
node.type = 'ifelsestmtr2'
else:
return_stmts_node = node[2]
if len(return_stmts_node) != 2:
self.default(node)
if not (node[2][0][0][0] == 'ifstmt' and node[2][0][0][0][1][0] == 'return_if_stmts') \
and not (node[2][0][-1][0] == 'ifstmt' and node[2][0][-1][0][1][0] == 'return_if_stmts'):
if (not (return_stmts_node[0][0][0] == 'ifstmt'
and return_stmts_node[0][0][0][1][0] == 'return_if_stmts')
and not (return_stmts_node[0][-1][0] == 'ifstmt'
and return_stmts_node[0][-1][0][1][0] == 'return_if_stmts')):
self.default(node)
return
@@ -404,7 +405,7 @@ class FragmentsWalker(pysource.SourceWalker, object):
past_else = False
prev_stmt_is_if_ret = True
for n in node[2][0]:
for n in return_stmts_node[0]:
if (n[0] == 'ifstmt' and n[0][1][0] == 'return_if_stmts'):
if prev_stmt_is_if_ret:
n[0].type = 'elifstmt'
@@ -486,17 +487,20 @@ class FragmentsWalker(pysource.SourceWalker, object):
# LOAD_CONST code object ..
# LOAD_CONST 'x0' if >= 3.3
# MAKE_FUNCTION ..
code_index = -3
code_node = node[-3]
elif node[-2] == 'expr':
code_node = node[-2][0]
else:
# LOAD_CONST code object ..
# MAKE_FUNCTION ..
code_index = -2
code = node[code_index]
func_name = code.attr.co_name
code_node = node[-2]
func_name = code_node.attr.co_name
self.write(func_name)
self.set_pos_info(code_node, start, len(self.f.getvalue()))
self.indentMore()
self.make_function(node, isLambda=False, codeNode=code)
start = len(self.f.getvalue())
self.make_function(node, isLambda=False, codeNode=code_node)
self.set_pos_info(node, start, len(self.f.getvalue()))
@@ -1690,7 +1694,7 @@ def deparse_code_around_offset(name, offset, version, co, out=StringIO(),
deparsed = deparse_code(version, co, out, showasm, showast, showgrammar, is_pypy)
if (name, offset) in deparsed.offsets.keys():
# This is the easy case
return deparsed.offsets[name, offset]
return deparsed
valid_offsets = [t for t in deparsed.offsets if isinstance(t[1], int)]
offset_list = sorted([t[1] for t in valid_offsets if t[0] == name])

View File

@@ -898,19 +898,19 @@ class SourceWalker(GenericASTTraversal, object):
# LOAD_CONST code object ..
# LOAD_CONST 'x0' if >= 3.3
# MAKE_FUNCTION ..
code = node[-3]
code_node = node[-3]
elif node[-2] == 'expr':
code = node[-2][0]
code_node = node[-2][0]
else:
# LOAD_CONST code object ..
# MAKE_FUNCTION ..
code = node[-2]
code_node = node[-2]
func_name = code.attr.co_name
func_name = code_node.attr.co_name
self.write(func_name)
self.indentMore()
self.make_function(node, isLambda=False, codeNode=code)
self.make_function(node, isLambda=False, codeNode=code_node)
if len(self.param_stack) > 1:
self.write('\n\n')

View File

@@ -1,3 +1,3 @@
# This file is suitable for sourcing inside bash as
# well as importing into Python
VERSION='2.10.0'
VERSION='2.10.1'