Patch/Thoughts - Konqueror's history loading..
Maks Orlovich
kde-optimize@mail.kde.org
Thu, 9 Jan 2003 23:30:01 -0500
--Boundary-00=_JxkH+IAVyiLaR/2
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Hi.. For some people, a substational part of Konqui's startup time (in order
of 10+%) is the loading of the history file (for some, since it obviously
depends on the size -- some have history files <20K, other's >100K). I've
profiled this as best as I could. The first big bottleneck is KURL parsing.
The attached patch fixes that (it also drops a redundant call to prettyURL),
by storing KURL's in the history file format (thus it introduces a new file
format version; it of course continues to load the old version). Once the
format conversion takes place, this cuts down upto nearly 40% or so of the
history loading time. Still, however, with the (admittedly twice as big as
I've had other people report) 200KB history file I tested on, we spend nearly
75ms on my Celery 950 loading the history...
Thus, I am not sure of whether to commit what I have right now, or whether
there can be some ways to speed up other parts. The 75 ms splits up roughly
as following:
Actual file I/O, CRC, closing down, etc - 9ms
Creating entries, deserializing them - 26ms
Calculating prettyURL - 13ms
Putting things into KCompletion - 8ms
Calculating KURL::url() - 15ms
Putting things into KParts::HistoryProvider - 4 ms
For the two KURL ones, I am not sure where the hot spots are -- the time seems
to be roughly spread out (although it's hard for me to visualize - I am
instrumenting mostly manually -- anyone know how to get kcachegrind to run
with glibc-2.3.1?)
Any suggestions/ideas/thoughts?
--Boundary-00=_JxkH+IAVyiLaR/2
Content-Type: text/x-diff;
charset="us-ascii";
name="opt_history_loader.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="opt_history_loader.diff"
Index: konq_historymgr.cc
===================================================================
RCS file: /home/kde/kdebase/libkonq/konq_historymgr.cc,v
retrieving revision 1.32
diff -u -3 -p -b -B -r1.32 konq_historymgr.cc
--- konq_historymgr.cc 11 Aug 2002 20:30:07 -0000 1.32
+++ konq_historymgr.cc 9 Jan 2003 23:53:31 -0000
@@ -30,7 +30,7 @@
#include <zlib.h>
-const Q_UINT32 KonqHistoryManager::s_historyVersion = 2;
+const Q_UINT32 KonqHistoryManager::s_historyVersion = 3;
KonqHistoryManager::KonqHistoryManager( QObject *parent, const char *name )
: KParts::HistoryProvider( parent, name ),
@@ -74,6 +74,7 @@ KonqHistoryManager::~KonqHistoryManager(
// loads the entire history
bool KonqHistoryManager::loadHistory()
{
+ KonqHistoryEntry::loadStringURL = false;
clearPending();
m_history.clear();
m_pCompletion->clear();
@@ -104,15 +105,11 @@ bool KonqHistoryManager::loadHistory()
bool crcChecked = false;
bool crcOk = false;
- if ( version == 2 ) {
+ if ( version == 2 || version == 3) {
Q_UINT32 crc;
-
crcChecked = true;
-
fileStream >> crc >> data;
-
crcOk = crc32( 0, reinterpret_cast<unsigned char *>( data.data() ), data.size() ) == crc;
-
stream = &crcStream; // pick up the right stream
}
else if ( version == 1 ) // fake, as we still support v1
@@ -118,6 +115,12 @@ bool KonqHistoryManager::loadHistory()
else if ( version == 1 ) // fake, as we still support v1
version = 2;
+ if ( version == 2 )
+ {
+ KonqHistoryEntry::loadStringURL = true;
+ version = 3;
+ }
+
if ( s_historyVersion != version || ( crcChecked && !crcOk ) ) {
kdWarning() << "The history version doesn't match, aborting loading" << endl;
file.close();
@@ -136,12 +139,11 @@ bool KonqHistoryManager::loadHistory()
KonqHistoryEntry *entry = new KonqHistoryEntry;
Q_CHECK_PTR( entry );
*stream >> *entry;
-
// kdDebug(1203) << "## loaded entry: " << entry->url << ", Title: " << entry->title << endl;
m_history.append( entry );
+ QString urlString2 = entry->url.prettyURL();
- // insert the completion item weighted
- m_pCompletion->addItem( entry->url.prettyURL(),
+ m_pCompletion->addItem( urlString2,
entry->numberOfTimesVisited );
m_pCompletion->addItem( entry->typedURL,
entry->numberOfTimesVisited );
@@ -151,7 +153,7 @@ bool KonqHistoryManager::loadHistory()
KParts::HistoryProvider::insert( urlString );
// DF: also insert the "pretty" version if different
// This helps getting 'visited' links on websites which don't use fully-escaped urls.
- QString urlString2 = entry->url.prettyURL();
+
if ( urlString != urlString2 )
KParts::HistoryProvider::insert( urlString2 );
}
Index: konq_historycomm.cc
===================================================================
RCS file: /home/kde/kdebase/libkonq/konq_historycomm.cc,v
retrieving revision 1.3
diff -u -3 -p -b -B -r1.3 konq_historycomm.cc
--- konq_historycomm.cc 16 Dec 2002 13:01:51 -0000 1.3
+++ konq_historycomm.cc 9 Jan 2003 23:53:31 -0000
@@ -1,11 +1,11 @@
#include "konq_historycomm.h"
+bool KonqHistoryEntry::loadStringURL;
+
// QDataStream operators (read and write a KonqHistoryEntry
// from/into a QDataStream)
QDataStream& operator<< (QDataStream& s, const KonqHistoryEntry& e) {
- // TODO (BIC) stream the KURL, not just its .url(). Takes a bit more space,
- // but prevents much reparsing when loading.
- s << e.url.url();
+ s << e.url;
s << e.typedURL;
s << e.title;
s << e.numberOfTimesVisited;
@@ -16,9 +16,16 @@ QDataStream& operator<< (QDataStream& s,
}
QDataStream& operator>> (QDataStream& s, KonqHistoryEntry& e) {
+ if (!KonqHistoryEntry::loadStringURL) //Backwards compatibility mode (V2 format)
+ {
+ s>>e.url;
+ }
+ else
+ {
QString url;
s >> url;
e.url = url;
+ }
s >> e.typedURL;
s >> e.title;
s >> e.numberOfTimesVisited;
Index: konq_historycomm.h
===================================================================
RCS file: /home/kde/kdebase/libkonq/konq_historycomm.h,v
retrieving revision 1.4
diff -u -3 -p -b -B -r1.4 konq_historycomm.h
--- konq_historycomm.h 15 Jan 2001 15:28:07 -0000 1.4
+++ konq_historycomm.h 9 Jan 2003 23:53:31 -0000
@@ -28,6 +28,8 @@
class KonqHistoryEntry
{
public:
+ //Backwards compatibility mode - set to true if loading V2 history files
+ static bool loadStringURL;
KonqHistoryEntry()
: numberOfTimesVisited(1) {}
--Boundary-00=_JxkH+IAVyiLaR/2--