[PATCH] C++ preprocessor fixes

[Dmitry Risenberg]

> The changes to the macro-expander are hard to review properly for me,
> I kind of hate that code, I've just spent a lot of time in it fixing
> bugs and adding smaller features, basically the same way you've done
> now, but it's hard for me to tell what's "right" or "wrong". Please
> make sure that you have a test for each single bug you've fixed, so
> things won't be broken again in future.

I could not add tests for my exact cases, because they involve lots of
macro code, but I did add some simple tests to test_parser.cpp that
check for the things that changed.

> About the one test you've commented out: Since the checked position is
> at the start of the merged token, it should be possible to map it. Why
> doesn't that work anymore?

I can correctly map the beginning of the token, but not the end of it,
because after it has been merged, further recursive macro expansion
treats it as one identifier of summary length. In this exact case, I
get (1, 5) -> (1, 15) range instead of required (1, 5) -> (1, 11).

-- 
Dmitry Risenberg