[rkward-devel] Status update

[Thomas Friedrichsmeier]

Hi,

On Wednesday 22 July 2009, meik michalke wrote:
> i've added a missing test for the cronbach alpha dialog (using the "fuzzy
> output" option for the first time in history, it seems ;-)) and cleaned up
> a little:
>  o http://reaktanz.de/stuff/R/irt_tests.diff

I see you did not have SVN write access, yet. You do, now.

If you feel comfortable enough with SVN, go ahead and commit your tests 
yourself. Otherwise, of course it's ok to continue sending patches.

Some issues, though:
For the 3PL_parameter_estimation, I get a warning
"Hessian matrix at convergence is not positive definite; unstable solution." 
and noteable differences in the coefficients. Could you try to find something more 
"stable"?

For several of the other tests I got *slight* differences in the last one or 
two digits. I figured, instead of marking all those tests as "fuzzy_output", we 
could simply reduce the default printed precision. Therefore, I changed 
test_framework.R to set
	options (digits=5)
before the test runs. I hope 5 digits of precision will turn out to be a 
useful default for the purpose of plugin testing.

Of course this change means, that the output of most plugins has changed, and 
you will need to re-generate the standards.

> since one can't use the results of one test in another, i wonder what would
> be best practice to test all the plot and goodness-of-fit parts that need
> the estimated parameters produced by the tested functions?
>
> i see three possible ways:
>
> 1. recalculate the estimates a second time
> 2. save all needed objects from a test run and store them in
> tests/MYTESTSUITE as well somehow
> 3. pave the way in test suite design to make objects re-usable if necessary
>
> option 1 seems to me a waste of cpu time, and would slow down each test
> run. option 2 would produce some redundancies (saved objects and result
> outputs) and use up disk space. so i'd vote for option 3 ;-)

This limitation was by design. The idea was to make sure tests are as modular 
as possible, so, if test number 7 fails, you know the cause is in that 
particular test, and not - say - something got changed in test number 2 that 
causes this failure.

I can see your point, though, and I guess it may make sense to allow some way 
of carrying over results between tests, but only when this is explicitely 
intended. I'm not yet sure how to do that, best, and will need to focus on the 
0.5.1 release, next.

For now, there are two solutions:
1) Place all steps of this sequence into a single "test".
2) In the initCalls(), define an object that can act as a container for the 
parameter estimates (probably a list or an environment). Assign the estimates 
into that object, and fetch them back into .GlobalEnv in the tests where you 
need them. (Remember to call .rk.sync.global()).

Regards
Thomas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/rkward-devel/attachments/20090723/f4e1c6f3/attachment.sig>