This bug had possibly caused lots of grammar pollution which may need
addressing.
We want to process COME_FROMs to the same offset to be in *descending*
order so we have the larger range or biggest instruction interval
last. (I think they are sorted in increasing order, but for safety
we sort them). That way, specific COME_FROM tags will match up
properly. For example, a "loop" with an "if" nested in it should have
the "loop" tag last so the grammar rule matches that properly
Adjust Python 3 grammar for more COME_FROM -> COME_FROM_LOOP. And
remove optional COME_FROM_LOOP where possible. Previously, the
optional-ness was a result of inner nestings gobbling up the
COME_FROM.
We'll probably want to go back and fix this up in Python2.
Note: we have regressed in --verify and some tests,
but I believe that's because we are producing more equivalant
(if uglier) programs. That's a separate problem though.
Fix py.test pytest/test_grammar.py -
Comprehension sorts of things removed from parser that
don't exist in earlier pythons
scanners/tok.py Add back in "to" when needed
As part of tokenization for (de)parsing, we need to do something like a
disassembly, but is is really a little different.
Disassembly, strictly speaking, is done by the xdis module now.
What "ingestion" does is massage the instruction tokens to a form that is
more amenable for parsing.
In sum, ingestion is different than disassembly, although disassembly is
generally the first part of ingestion.
Note: it looks like COME_FROMs need to be classified better
with respect to whether they are from an except or not
we are getting if/else vs except nesting errors.
In Python 2.6 and possibly down to 2.3 we need to issue "return" not
"return None" inside a generator. So check for that "return None"
inside n_return and issue "return" for that.
If 2.6 or before POP_BLOCK after a JUMP_IF_FALSE does not
constitute a new statement. The POP_BLOCK is really part
of the JUMP_IF_FALSE. In Python 2.7+ it's a single op.
scanner3.py: bug in 3.x detecting "if" structure
Make scanner2.py look more like scanner3.py
verify.py: add weak-verify which tests Pytyon syntax, but not code
