[Kdenlive-devel] relative paths in project files

[Mads Bondo Dydensborg]

mandag 17 November 2008 skrev Alberto Villa:
> On Monday 17 November 2008 16:02:11 Mads Bondo Dydensborg wrote:
> > This might be inspirational? What kphotoalbum appears to do, is to scan 
all
> > files below a certain path, and match against the size/md5 sum database
> > that it has. This is very quick - it scans my image collection of about
> > 18000 high resolution images in 20-30 seconds. If I should move things
> > around, it would be slower, of course. Again, for inspiration: If we 
stored
> > md5 sums of clips with other metadata in the project file, we could offer
> > the user to search (a path/tree) for a clip, if it was missing. That way,
> > you could move all your stuff somewhere else, open the project file, and
> > have kdenlive look up all the clips again. Or something.
> 
> nice idea, but calculating the hash of an entire video clip would take... 
> hours? i tried md5 on an 8 mb file: 40s...

(I think you made that measurement on an NFS drive, or something, but a few 
numbers):

I tried it on a 1GB file, attached on USB2, with a transfer rate of about 25 
MB/s. 

Cold cache :

cat file > /dev/null: 0m41.060s
md5sum file: 0m41.216s

On a hot cache (2GB ram), using time, real time

cat file > /dev/null: 0m0.538s
wc file: 2m26.590s
md5sum file: 0m5.230s
cksum file: 0m4.572s

Inspecting the user time, the conclusion (for this particular setup) seems 
reasonably clear: The overhead of doing a md5sum is negligble compared to the 
overhead of reading the entire file. I am not sure, but I think kdenlive 
already reads most of the file when it needs to do the thumbnails? If that is 
the case, calculating a md5sum should be doable. Likewise, when adding a 
single clip. However, it must be worth it: if relative paths can be supported 
in a good way without the need for stuff like this, obviously, there is no 
need to implement it.

> maybe we could get the hash only of the first bytes of the file, but that 
> sounds like a bad thing (if i then truncate the file i'll get a wrong 
match... 
> and there's also the risk of collisions). i'm thinking about this, maybe it 
> could work if we include the original file size in the hash, or something 
> else...

The original size would definitely be something that would need to be stored 
as information about a clip, as the md5sum trivially would not match in that 
case. So, for the "locate clips again", you can trivially reject all non-size 
matches. 

> 
> > Another thing KPhotoAlbum does (or at least did once) is that it allows
> > image collections on cdroms/moveable media that are not mounted to be
> > "used": that is, it is still searchable, and so on. It just appears with a
> > "placeholder" icon in the GUI. In Kdenlive that would amount to e.g. being
> > able to move clips around in the timeline, edit titles, all that kind of
> > stuff, without having access to the actual clips on disk at the moment. Of
> > course, it would not work for render, even though one could envision a
> > partial render of the available clips.
> 
> i can already edit projects with only placeholders in kdenlive: an old 
project 
> of mine includes some .westley which refers to lost files, but kdenlive 
isn't 
> aware of this and doesn't remove them... i think it could be easily done 
> rethinking the "you're file cannot be found and will be removed" dialog

I think there is some merit to this idea. All my footage is on an external 
harddisk to my laptop, but I think it would be great to be able to edit e.g. 
titles and transitions between them (think endtitles) without having to 
connect the external harddisk (on the road, stuff like that). Might not be 
worth the trouble though, if the number of people that would actually use 
this is small.

Regards

Mads


-- 
Mads Bondo Dydensborg   mads at dydensborg.dk   http://www.madsdydensborg.dk/

You may not use the Software in connection with any site that disparages
Microsoft, MSN, MSNBC, Expedia, or their products or services, infringe any
intellectual property or other rights of these parties, violate any state,
federal or international law, or promote racism, hatred or pornography. 
                                     - Part of MS Frontpage 2002 EULA