file_hash questions

[jb]

On jeudi, 24 septembre 2020 13.50:28 h CEST jb wrote:
> Le 23.09.20 à 23:26, Camille Moulin a écrit :
> > Hi Tom,
> 
> > On 22/09/2020 07:43, amindfv at mailbox.org wrote:
> Hi,
> 
> >> Apologies if this is answered in the docs - a quick search didn't uncover
> >> the answers.> 
> > Generally speaking, the documentation of the file format could be
> > improved :-(
> > 
> >> I'm wondering about the "kdenlive:file_hash" property in the file XML:
> >>    - Which hashing algorithm is used?
> >  
> >  From what I understand, it's seems to be md5, but, for performance
> > 
> > reasons,  it's calculated only on the first and last Mo of the file if
> > the file is bigger than 2Mo
> > 
> > In ./src/bin/projectclip.cpp, starting line 820:
> >>          QFile file(clipUrl());
> >>          if (file.open(QIODevice::ReadOnly)) { // write size and hash
> >> 
> >> only if resource points to a file
> >> 
> >>              /*
> >>               * 1 MB = 1 second per 450 files (or faster)
> >>               * 10 MB = 9 seconds per 450 files (or faster)
> >>               */
> >>              if (file.size() > 2000000) {
> >>                  fileData = file.read(1000000);
> >>                  if (file.seek(file.size() - 1000000)) {
> >>                      fileData.append(file.readAll());
> >>                  }
> >>              } else {
> >>                  fileData = file.readAll();
> >>              }
> >>              file.close();
> >> 
> >> ClipController::setProducerProperty(QStringLiteral("kdenlive:file_size"),
> >> QString::number(file.size()));
> >> 
> >>              fileHash = QCryptographicHash::hash(fileData,
> >> 
> >> QCryptographicHash::Md5);
> >> 
> >>          }
> 
> That's correct.
> 
> >>    - When are the hashes checked? E.g. if I save a kdenlive file, quit,
> >>    then reopen it, will the hashes of the files on disk be checked?> 
> > Yes, I think so
> 
> That's a very good question. In fact the hash not really checked during
> normal usage. It is updated when Kdenlive is running and detects that
> the file was changed or when the user triggers reloading a clip, but
> does not seem to be checked on project opening. Thinking of it, I think
> that if the user changes a clip when Kdenlive is not running, opening
> the project will contain an incorrect hash, leading to possible bugs.
> 
> 
> It would be much safer to check it on each project opening... I will
> investigate and come back to you.

So after some tests, I can confirm that the hash was not checked on project 
opening. As a result, a project could in some circumstances uses a wrong proxy 
clip or be unable to find a clip that had been moved to another location.

This is now fixed in git master and 20.08.x branch, with this commit:
https://invent.kde.org/multimedia/kdenlive/commit/
0430805f0a6ed6121900cb3f1d15766dc7e3f4a1


Thanks.
Jean-Baptiste

> 
> Regards,
> 
> Jean-Baptiste
> 
> >>   Are there other points at which it's checked? Can the user manually
> >>   start a check?> 
> > I guess that if you reload the clip, it will check it and update it if
> > modified.
> > 
> > HTH,
> > 
> > Camille