[rkward-tracker] [ rkward-Bugs-3418483 ] FIXED IN SVN: Hang on big names arrays

SourceForge.net noreply at sourceforge.net
Tue Oct 4 16:03:09 UTC 2011


Bugs item #3418483, was opened at 2011-10-04 16:32
Message generated for change (Comment added) made by nalimilan
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=459007&aid=3418483&group_id=50231

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: Fixed
Priority: 5
Private: No
Submitted By: Milan Bouchet-Valat (nalimilan)
Assigned to: Thomas Friedrichsmeier (tfry)
Summary: FIXED IN SVN: Hang on big names arrays

Initial Comment:
Today I added a numeric variable to a big data frame, and rkward.frontend started eating GBs of memory and using all CPU, never finishing. I'm able to reproduce the hang with the attached .RData file, which contains that variable as an array. Loading this file always makes my RKWard hang. This is quite weird, since either converting the array to a vector, or setting dimnames to NULL is enough to fix the problem. But changing the names to 1:length(a) for example has no effect, nor changing the values of a.

You can download the .RData file from: http://ubuntuone.com/24J68sdk9kd6PUCtO6Hd1f

No need to say I'm very interested in debugging this, since I need to work with this data. ;-)

I'm using RKWard 0.5.6z+0.5.7+devel1 on Fedora 15 64-bits.

> str(a)
 num [1:1985324(1d)] 0.2619 0.7744 -0.0511 -0.0113 0.256 ...
 - attr(*, "dimnames")=List of 1
  ..$ : chr [1:1985324] "1969" "1969" "1969" "1969" ...


Backtrace of rkward while hanging on load:
(gdb) ba
#0  0x000000000048e990 in QList<RObject*>::indexOf (this=<optimized out>, t=
    @0x7fff99b712a8, from=<optimized out>) at /usr/include/QtCore/qlist.h:833
#1  0x00000000004e84a2 in RContainerObject::updateChildren (this=0x2b22d90, 
    new_children=<optimized out>)
    at /home/milan/Dev/rkward/rkward/rkward/core/rcontainerobject.cpp:180
#2  0x00000000004e8911 in RContainerObject::updateStructure (this=0x2b22d90, 
    new_data=<optimized out>)
    at /home/milan/Dev/rkward/rkward/rkward/core/rcontainerobject.cpp:93
#3  0x00000000004e7e11 in RContainerObject::updateChildStructure (this=
    0x2565260, child=0x2b22d90, new_data=0x2c83710, just_created=true)
    at /home/milan/Dev/rkward/rkward/rkward/core/rcontainerobject.cpp:55
#4  0x00000000004e7b8f in RContainerObject::createChildFromStructure (this=
    0x2565260, child_data=0x2c83710, child_name=..., position=0)
    at /home/milan/Dev/rkward/rkward/rkward/core/rcontainerobject.cpp:128
#5  0x00000000004e8014 in RContainerObject::updateChildStructure (this=
    0x2565260, child=0x2c85970, new_data=0x2c83710, 
    just_created=<optimized out>)
    at /home/milan/Dev/rkward/rkward/rkward/core/rcontainerobject.cpp:72
#6  0x00000000004e0477 in rCommandDone (command=0x2c83710, this=0x2c85970)
    at /home/milan/Dev/rkward/rkward/rkward/core/robject.cpp:263
#7  RObject::rCommandDone (this=0x2c85970, command=0x2c83710)
    at /home/milan/Dev/rkward/rkward/rkward/core/robject.cpp:253
#8  0x00000000004db7fb in RKVariable::rCommandDone (this=0x2c85970, command=
---Type <return> to continue, or q <return> to quit---
    0x2c83710) at /home/milan/Dev/rkward/rkward/rkward/core/rkvariable.cpp:120
#9  0x00000000004fdd81 in RCommand::finished (this=0x2c83710)
    at /home/milan/Dev/rkward/rkward/rkward/rbackend/rcommand.cpp:113
#10 0x00000000004f57a0 in RInterface::handleCommandOut (this=<optimized out>, 
    command=0x2c83710)
    at /home/milan/Dev/rkward/rkward/rkward/rbackend/rinterface.cpp:209
#11 0x00000000004fc80d in RInterface::handleRequest (this=0x24c6ea0, request=
    0x7fad6c07f2e0)
    at /home/milan/Dev/rkward/rkward/rkward/rbackend/rinterface.cpp:324
#12 0x00000035e4f70c0c in QObject::event (this=0x24c5cf0, e=<optimized out>)
    at kernel/qobject.cpp:1248
#13 0x00000035f1bb9324 in notify_helper (e=0x7fad6c10a500, receiver=0x24c5cf0, 
    this=0x21a7df0) at kernel/qapplication.cpp:4481
#14 QApplicationPrivate::notify_helper (this=0x21a7df0, receiver=0x24c5cf0, e=
    0x7fad6c10a500) at kernel/qapplication.cpp:4453
#15 0x00000035f1bbe1b1 in QApplication::notify (this=0x7fff99b721c0, receiver=
    0x24c5cf0, e=0x7fad6c10a500) at kernel/qapplication.cpp:4360
#16 0x00000035eb841d56 in KApplication::notify(QObject*, QEvent*) ()
   from /usr/lib64/libkdeui.so.5
#17 0x00000035e4f5a20c in QCoreApplication::notifyInternal (this=
    0x7fff99b721c0, receiver=0x24c5cf0, event=0x7fad6c10a500)
    at kernel/qcoreapplication.cpp:787
#18 0x00000035e4f5d7d4 in sendEvent (event=0x7fad6c10a500, receiver=0x24c5cf0)
---Type <return> to continue, or q <return> to quit---
    at kernel/qcoreapplication.h:215
#19 QCoreApplicationPrivate::sendPostedEvents (receiver=0x0, event_type=0, 
    data=0x2180070) at kernel/qcoreapplication.cpp:1428
#20 0x00000035e4f84973 in sendPostedEvents () at kernel/qcoreapplication.h:220
#21 postEventSourceDispatch (s=0x21ab8c0)
    at kernel/qeventdispatcher_glib.cpp:277
#22 0x00000035df6427ed in g_main_dispatch (context=0x21ab4e0) at gmain.c:2441
#23 g_main_context_dispatch (context=0x21ab4e0) at gmain.c:3014
#24 0x00000035df642fc8 in g_main_context_iterate (context=0x21ab4e0, 
    block=<optimized out>, dispatch=1, self=<optimized out>) at gmain.c:3092
#25 0x00000035df64325c in g_main_context_iteration (context=0x21ab4e0, 
    may_block=1) at gmain.c:3155
#26 0x00000035e4f84dcf in QEventDispatcherGlib::processEvents (this=0x21a7490, 
    flags=<optimized out>) at kernel/qeventdispatcher_glib.cpp:422
#27 0x00000035f1c5c12e in QGuiEventDispatcherGlib::processEvents (
    this=<optimized out>, flags=<optimized out>)
    at kernel/qguieventdispatcher_glib.cpp:207
#28 0x00000035e4f59722 in QEventLoop::processEvents (this=<optimized out>, 
    flags=...) at kernel/qeventloop.cpp:149
#29 0x00000035e4f5991f in QEventLoop::exec (this=0x7fff99b72110, flags=...)
    at kernel/qeventloop.cpp:201
#30 0x00000035e4f5da67 in QCoreApplication::exec ()
    at kernel/qcoreapplication.cpp:1064
---Type <return> to continue, or q <return> to quit---
#31 0x0000000000432634 in main (argc=<optimized out>, argv=<optimized out>)
    at /home/milan/Dev/rkward/rkward/rkward/main.cpp:177
(gdb) list
828	Q_OUTOFLINE_TEMPLATE int QList<T>::indexOf(const T &t, int from) const
829	{
830	    if (from < 0)
831	        from = qMax(from + p.size(), 0);
832	    if (from < p.size()) {
833	        Node *n = reinterpret_cast<Node *>(p.at(from -1));
834	        Node *e = reinterpret_cast<Node *>(p.end());
835	        while (++n != e)
836	            if (n->t() == t)
837	                return int(n - reinterpret_cast<Node *>(p.begin()));



----------------------------------------------------------------------

>Comment By: Milan Bouchet-Valat (nalimilan)
Date: 2011-10-04 18:03

Message:
Ah, cool, I'll try with a more recent SVN then. I would have updated before
reporting, but my setup is a little messed up ATM, so that's not as easy as
it sounds. ;-)

That 'a' object doesn't make any sense, it's just that I extracted the
problematic variable from a big data frame using a<-df$a. Since it was the
smallest unit that triggered the bug, I kept it, but it isn't useful by
itself. The names probably correspond to the first column of the data
frame, with contains the survey year, but is of course used in relation
with many more variables.

----------------------------------------------------------------------

Comment By: Thomas Friedrichsmeier (tfry)
Date: 2011-10-04 17:52

Message:
Hi!

The core problem with that object is that it looked like an object
containing 1985324 sub-objects to RKWard. I have addressed this in the
development version (http://p.sf.net/rkward/svn) by making sure that RKWard
will never try to look at more than 100000 children of any object, and
specifically will not treat arrays as hierarchical named objects (which it
never did quite right, anyway). That should solve the hang.

That said, that object of yours really is a bit unusual. To me it looks
more like you wanted to have two separate variables, as in:
   x <- data.frame (year=names (a), as.vector (a))
or, alternatively:
   x <- sapply (unique (names (a)), function (x) as.vector (a[x == names
(a)]))

While names() can contain duplicates for most R objects, this is rarely
ever useful, and typically it indicates a problem. E.g. what's the use of
  a["2009"]
?

Regards
Thomas

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=459007&aid=3418483&group_id=50231




More information about the rkward-tracker mailing list