[Kmymoney-devel] Bug in database upgrade

[Christian Dávid]

Am Dienstag, 7. Oktober 2014, 22:02:24 schrieb Fernando Vilas:
> > > > _ Solution _
> > > >
> > > > I would recommend not to use readFileInfo() and writeFileInfo() in any
> > > > upgrade methods anymore and just call writeFileInfo() at the end of
> > > > the
> > > > upgrade process. Also the m_hi…Id vars need to be set somewhere.
> > >
> > > That sounds reasonable. The system was designed to make upgrades easy,
> > > but
> > > there are always cases you did not consider. I do not remember us
> > > considering  removing columns, just adding/changing them, so this is new
> > > territory.
> >
> > I think this is a misunderstanding. To solve that: Where/when exactly are
> > columns added to the database? I cannot find any piece of code that
> > triggers that
> 
> It looks like I was mistaken. There is code to create new tables, but it is
> in  the upgradeToV<num>() calls that the call to generateDDL on the new
> columns is executed. There is probably a good way to abstract it if we want
> to automate the adding of columns. If not, the calls look like what is in
> upgradeToV1().
> > The errors I got are not caused by removed columns (I do not drop any
> > column) but it is caused by columns which are going to be created (well,
> > actually won't be created, because KMyMoney crashes first…).

I have a suggestion how to solve this problem [1]. Maybe this explains the 
issue even better than my description from above.

> The upgradeDb() call that invokes upgradeToV<num>() should take care of
> adding  the columns. If it still crashes afterwards, then that is a
> different bug.

I think there is no such bug.

> > We should create some database files and .xml files for each version and
> > save them for later test of the upgrade routines.
> 
> Yes!!! That has been a gap in our testing until now. I usually just took a 
> snapshot of my database and then practiced upgrading until it worked, then
> did  it on the master copy. I tended to make a backup first, though. We
> definitely could use a version 0 file for testing the upgrade path as part
> of the test harness.

I guess that is only possible if someone extracts the database generation code 
from an old git version by hand. When was the database version 0 introduced 
(KMyMoney's version?)?

> Also, I hope you don't misinterpret my delays in responses or vague
> guidance.  I am in the middle of a stretch of long weeks at $dayjob and
> have not been able to dedicate the time that KMM deserves. I really
> appreciate that someone else is digging into the code and improving it.

Not at all! My KMyMoney-time gets more and more limited as well. Take your 
time, you should enjoy your work for KMyMoney!

A long time ago we wrote how to allow plugins to write into a database. My 
last commit to add-onlinebanking [2] implements that feature (for sepa credit 
transfers only at the moment). At some points it is a bit over engineered 
(maybe). Short walk through:

KMyMoneyPlugin::storagePlugin shall contain the DDL and upgrade routines.
onlineTask got four new methods: one to define the DDL plugin, the save, modify 
and remove routines.

A new table kmmPluginInfo contains information about installed plugins in the 
database.

Maybe someone wants to comment on that.

Greetings
Christian

[1] https://git.reviewboard.kde.org/r/120499/
[2] 
https://projects.kde.org/projects/extragear/office/kmymoney/repository/revisions/b2fa6231e248b0301ebad21d2152cd6425271102