[KimDaBa] Profiling Kimdaba startup

[Robert L Krawitz]

   From: "Jesper K. Pedersen" <blackie at blackie.dk>
   Date: Tue, 4 Jan 2005 13:52:56 +0100
   Cc: kimdaba at klaralvdalens-datakonsult.se

   On Tuesday 04 January 2005 13:48, Robert L Krawitz wrote:
   |    From: "Jesper K. Pedersen" <blackie at blackie.dk>
   |    Date: Tue, 4 Jan 2005 09:06:30 +0100
   |
   |    On Tuesday 04 January 2005 09:01, Eivind wrote:
   |    | On Tuesday 04 January 2005 07:46, Jesper K. Pedersen wrote:
   |    | > Robert, thx for your profiling.
   |    | > As it looks now, I can't see that there are any thing else I can do
   |    | > to speed up things, except rewritting to a real database.
   |    | >
   |    | > Rewritting to a real db would be a huge job, and currently I dont
   |    | > see it worth it to save at most 2.5+1=3.5 secs. People with 50.000
   |    | > images might disagree, but we aren't really there yet
   |    |
   |    | I agree with this. Spending lots of time to change to a "real db"
   |    | isn't likely to deserve being on the top of the priority list at
   |    | present. Though I do think that keeping the thougth is a good idea,
   |    | for example in trying to not needlessly further complicate a later
   |    | switch.
   |
   |    Sure, I've done so from the very start.
   |
   | The startup code doesn't appear to be very complicated.  It would be
   | interesting, as an exercise, to replace it with code using mxml (a
   | light weight XML parser).

   At some point I tried replacing the DOM parser with a SAX parser,
   and after a long night of work I gave up. One thing is the parser,
   the second thing is that you need to restructure the internals of
   kimdaba too.

The code looks very similar to MXML-based code, which we use in
Gutenprint.  We switched from libxml2 to MXML to get a lighter-weight
parser.  I believe MXML is also DOM-based.

   |    | > (at least that is not what I hear from most kimdaba users), and once
   |    | > we are there, computers are likely so much faster that the 25 secs
   |    | > has gone done.
   |    |
   |    | This however, I disagree with.
   |    |
   |    | A real db migth bring faster startup-times, which is important
   |    | for people with humongous collections of pictures. I think this
   |    | group of people is going to grow very quickly as the years with
   |    | digital cameras pass. I know that my picture-collection grows by
   |    | atleast a couple of thousand images a year, and I'm probably not
   |    | the only one.
   |
   |    See here I kind of disagree, Even with 2000 images per year, you
   |    will spent 12.5 year to get to 25000 images, by the computers will
   |    be able to read a billion image database (in its current XML
   |    format) in a fraction of a second.  But as I said before, lets see,
   |    if a huge amount of users get databases which are this size, I will
   |    reconsider.
   |
   | 2000 images per year isn't very many.  I'm not a professional, and yet
   | I shot 5000 images this year without even trying very hard.  I don't
   | think computers (and storage) will get that fast for single data and
   | instruction streams any time soon; the recent trend seems to be more
   | toward multi-processing chips to improve system performance.  However,
   | I do think that if you set 50K images to be a reasonable target, then
   | current machines can handle it without great difficulty.

   Lets just agree that for the time being we will just monitor the
   situation, and once a lot of kimdaba users complain, I'll have
   another look. In the meantime I'll waste my sparetime on other cool
   topics like the datebar improvement ;-)

I think this is the right plan.

-- 
Robert Krawitz                                     <rlk at alum.mit.edu>

Tall Clubs International  --  http://www.tall.org/ or 1-888-IM-TALL-2
Member of the League for Programming Freedom -- mail lpf at uunet.uu.net
Project lead for Gimp Print   --    http://gimp-print.sourceforge.net

"Linux doesn't dictate how I work, I dictate how Linux works."
--Eric Crampton