Optimizing our URL usage

[Andreas Pakulat]

Hi,

On Thu, Nov 29, 2012 at 5:59 PM, Milian Wolff <mail at milianw.de> wrote:
> In the sharedurls branch of KDevplatform you can now find my take on an
> optimized data structure for URLs in KDevelop.
[...]
> There are a few open questions which I will investigate in the following days,
> but I'd like to see any kind of feedback even now:
>
> - what is a good name for this data type, I would personally prefer
> KDevelop::URL

I'd use KDevelop::ProjectPath (unless you intend to also use it for
arbitrary other urls which we use, i.e. files outside of a project),
see also further below on the why.

> - how often do we need an actual KUrl instance, the conversion migh be
> potentially slow. personally, I don't see too many instances where we need
> actual KUrl's though. Only when opening a file (which is far slower than the
> conversion) or similar.

I'd say we need a KUrl whenever code needs to access the file, i.e.
whenever we actually go to the filesystem. Anything else can work on
project-relative paths, be it listing files, checking if a file is in
a project, comparing two paths wether its the same file etc.

> - the API changes required might be quite pervasive... do we want something
> like that Maybe something we should tackle with 4.6?

I'd start the work in a branch and then see when this is done. Try to
get help with the porting from others. IMHO its ok if the branch does
not fully build for some time in this case, just start porting
"logical" units, i.e. one commit per plugin or framework lib being
ported.

> - while at it, we might also want to introduce an IndexedUrl which I think
> could be done in a more efficient manner compared to the usual Url -> String -
>> Indexed -> String -> Url roundtrip.

I'd keep that part separate and would probably first land the new type
and then let it settle a bit. Maybe even wait for 4.7 to introduce
this. This is mainly to avoid endangering the stability of KDevelop's
release by doing too many thing at once.

> Now the somewhat ugly issues
>
> What about more esoteric URLs, e.g. containing fragment (#...), queries
> (?...), passwords, ... you name it. Do we want to support them? Personally, I
> don't think so. We only operate on files here, and while they might be remote,
> they should not have any "logic" which would require fragments or queries. And
> passwords should imo be stored by KIO/KWallet.

I agree that we can ignore fragments and queries. You can't open such
things are projects, unless the project itself is already using a
query. Fragments are completely useless to us since its something
thats only interpreter on the client side and the server never changes
the reported page content depending on the fragement IIRC.

This also brings me to my suggestion from IRC and why I suggested
ProjectPath as classname, IMHO we should only handle paths relative to
the project root and the project pointer for these things. Getting a
full url can be done by asking the project for its KUrl and adding the
internally stored path to it via addPath. This would then even keep
queries in the projects url intact when done through the KUrl API.

I've seen you also did some measurements and compared KUrl and this
new class, what I'd be interested in is how much memory we could
actually save in KDevelop with this. Do you have any data on how many
KUrl instances would be affected, say for KDevelop as a project and
how much this is compared to the overal memory consumption of the app?

I'm wondering wether this is something really noticeable in the
overall memory consumption and hence should have relatively high
priority or more a nice-to-have and "makes huge projects scale a bit
better" thing...

Andreas