[rkward-devel] Request for feedback on plot history

[Prasenjit Kapat]

Hi,

Thanks for the quick inputs, Thomas.

To everyone: find if there is any "high-level-primary" plot function
which does not get recorded (technically, which does not call plot.new
()). For example: persp () does not call plot.new (), but we have
taken care of it by putting in a hook. Are there other such primary
functions? (My fear is, yes!) Browse through
__help(packages=graphics)__. In addition, plotting calls from
completely other systems (such as, may be, ggplot2) which does not
call either plot.new or print.trellis will get washed out completely.

On Sat, Sep 4, 2010 at 8:00 AM, Thomas Friedrichsmeier
<thomas.friedrichsmeier at ruhr-uni-bochum.de> wrote:
> Hi,
>
> gee, this whole affair sure is more complex than I would have imagined!

Indeed. I'll need to go back to drawing board and try to implement
things differently. Before getting into specific cases, I am
suggesting, in the interest of the upcoming release, that let us
ignore this plot history feature completely. Why? There is another
core problem:

par (mfrow = c(2,2))
plot (1,1); plot (2,1); plot (1,2); plot (2,2)

A similar problem occurs for screen calls. For example, follow one of
the examples in ?screen. I am not sure, how / if we can avoid this! Is
this an "acceptable" drawback for a released software? I would guess
not.

> I did not have a look at everything, so far, but here are a few issues and
> comments:

The main problem in the current implementation (of recording at every
first/prev/next/last.. action) is this: I can not handle a situation
where two devices have unsaved plots. And hence I have to call record
() over and over again, to make sure such a situation does not arise.
Of course, similar thing happens when "removing," which in turn, calls
replay () over and over again.

Getting rid of the "remove" action is not really an option, if we want
to impose a history length restriction. So, even if the user is not
provided with a visible "remove" action, it still has to be
implemented internally.

> On Saturday 04 September 2010, Prasenjit Kapat wrote:
>> All right, the current implementation records and replays liberally.
>> In some cases, you can "see the multiple replay processes." If it
>> feels too sluggish, then we will have to drop the plot history feature
>> for this release.
>
> One performance problem that I have hit occurs once the history limit is
> reached. Then, if you create a new plot (all on a single device!), what
> appears to happen is that the current plot is replayed. As far as I
> understand, this comes from remove(), and there pop.and.update(). This will
> always redraw plots in all current device windows, while actually the only
> windows that would really need an update are those whose plots have actually
> been popped. For the others it would be enough to just update the internal
> list of history positions. Not sure, how to do this, best, but I think this is
> the most severe performance problem ATM.
>
> Again, when looking into this, it might be worth considering whether a plot
> that has been popped out of the history should really be cleared from any
> other devices that show it. Perhaps it should rather gain the status of a new
> plot, instead, which has not yet been placed in the history. The use case I'm
> thinking of is this:
> 1) User creates a fancy plot in one device, wanting to keep it.
> 2) User opens a second device (so as not to overwrite the other plot), and
> starts creating a different plot in it. Since it's not that easy, it takes the
> user a good number of attempts to arrive at the desired result for this plot.
> Naturally, each attempt will take up a slot in the history...
> 3) As the history limit is reached, the plot created in 1 is popped out of the
> history, and is gone for good!

Right, you have a valid point here. And it is tied to the inability to
handle more than one new / unsaved plots. To be able to handle it,
things will have to be implemented differently (I have an idea,
similar to what I had earlier. But it will take some time and thinking
to combine the two strategies together.)

> While talking about all this, it would probably be a good idea to offer some
> configuration option for turning off the whole history mechanism in case of
> unforseen problems. Again, I'm not sure, how to implement this, best.

This was on my mind as well. And this is what windows does too,
slightly differently. To implement it in a way so that the user can
change it dynamically during a running rkward session will be tricky.
But, to do it statically (ie needs to restart rkward) will be easy.
I'll just have to, essentially, include the plot.new, dev.off, etc.
wrappers inside an __if (getOption ("rk.implement.plot.history"))__
block.

>> I have kept the "replace plot" feature because of trellis plots.
>
> All right. I suggest an additional "insert plot" for the wishlist, then. Use
> case: User wants to modify a plot, but wants to be able to revert to a
> previous state in case of mess up. So he "inserts" the previous state into the
> history, then proceeds with modifications.
>
> But of course this can wait until everything else works as expected.

Fair point. I'll keep this in mind when redoing this whole thing.

> On duplicated devices: I think we touched the subject before, but I don't
> remember the conclusion, and I can't find the mail right now: Is it really a
> good idea to treat the duplicate as the same history position? Isn't the point
> precisely to have a logically separate copy (i.e. like a regular new plot)?
> Somewhat similar to the issue of how to treat plots which have been removed in
> a different device.

Fair enough. But it would be difficult to avoid a duplicated plot in
history. For example consider duplicating device 2 to device 3 and the
status of the plot on device 3 is set to "new." Now, if the next
command from the user is a primary function call (as opposed to a
secondary one, such as: title, points, lines, grid, ...) such as:
plot/persp/hist/xyplot/... then the plot which was copied from device
2 will get recorded to the history at a new position, yet, it will be
the exact same plot as the already (?) recorded one from device 2.
Just another source of confusion / surprise ;)

I don't see how to avoid this without the "breakage-prone" hacks.

> Assorted other things:
> - Occasional "Error in gType[[n]] : subscript out of bounds" subscript out of
> bounds in gType[[...]]" while printing the plot properties. I'll try to find a
> way to reproduce this, when I have more time.

This, I know is coming from pop.and update because printPars () is
called after deleting one/more plots from history. This is an
artificial error imposed by calling printPars ()

> - I don't quite understand the meaning of "replacePositions" / "previous
> positions". Is this really needed? When exactly does this differ from the
> current history positions?

This actually is more meaning full for the lattice plots but applies
to standard graphics calls as well.

Suppose the user browses the history and stops at a lattice plot (say
the position is 5 and the current history length is 10). The user
updates this plot by calling:

update (trellis.last.object (), main = "Changed main") # see side note A below

Unlike the standard graphics system, this actually behaves as a
totally new plot and hence, is setup to be recorded at a new position
(at the end of the history). So, the history position of this device
is now set to 11. What "replacePosition" does is to hold the history
position of the plot that was displayed before the update (...) call,
ie 5. So, when the user opts for the replace action this new plot is
recorded at position 5 overwriting whatever was at position 5.
Position 11 is then set free.

This applies to standard graphics call as well, in a tad less
appealing way. Instead of the update (...) call, think of a primary
standard graphics function. Suppose the plot at position 5 is __hist
(X1)__ but the user, while browsing, realizes that the intended
variable was not X1 but X2; so calls:

hist (X2)

And now the same argument as above. In fact, think of it as a feature
to rectify an unintended plot call:

hist (x, sub = "time in minutes") # say this is a mistake, and the
time is in fact in seconds
hist (x, sub = "time in seconds") # try doing it with a title () call ;)

Actually, if the "insert" plot feature can be implemented in a proper
way, then the replace action can be ignored (or wrapped around) using
an insert() followed a remove() call.

Side note A:
The flexibility to allow updating a lattice plot this way, even when
it not strictly the last lattice plot, comes from the fact that the
lattice plots are not recorded/replayed using recordPlot () /
replayPlot () calls. I think this is a tremendous flexibility which is
missing from both windows ()'s and quartz ()'s implementation of plot
history.
Side note ends!

So, what next?
* One simpler strategy is to use separate history for individual
devices, the way it is implemented on Mac OSX's quartz device. This
way replaying / updating across devices is totally avoided.
* Provide only first / prev / next / last and clear history actions,
again, similar to quartz (), with no restriction on history length.
Thereby, no complications due to "remove" action.

Even with these, I do not see, without a more low level control on the
device, how we can avoid the par (mfrow) and screen () issues
mentioned on top. May be we can have a wrapper around screen ()! May
be we can check the mfrow parameter, before calling record ()! May be
there is "new.page" call/parameter somewhere which we can wrap around!
May be, may be, may be.....

Regards,
-- 
Prasenjit