Solved: disappearing data or correupt data file

[Jack]

[trimming lots of the starting history]

On 10/8/23 03:40, Thomas Baumgart via KMyMoney-devel wrote:
> I re-visited this issue and noticed a possibility that could cause the 
> problem.
> https://invent.kde.org/office/kmymoney/-/commit/edb22bd2be7dd7c29bd94e14f6a4632866c44ccc
> should fix it.
>
> The reason is when erroneous entries are detected during writing (see the
> code following the above change which contains continue statements that
> cause to reach the m_writer->writeEndElement() line again. Loop twice and you
> get </PRICES> in the first iteration and </KMYMONEY-FILE> during the second
> one. Only in that case another <PRICEPAIR> can be issued. Resetting the flag
> immediately prevents issuing the </PRICES> and </KMYMONEY-FILE> markers
> in this situation.
>
> @Jack: I expect those weird price entries to happen during OFX import. So maybe
> you can check in that area (OFX prices imported into the last good data file).

Bingo!  It took me a while to track it down and replicate it.  It turns 
out the problem was NOT created by an OFX import (at least not directly.)

As I think I've complained in the past, Merrill Lynch will sometimes 
download transactions for security activities using a security name 
different from the "real" name, such as by including a date or price in 
the name.  Usually, KMM just creates these securities, which I call 
"bogus."  I say usually because I've recently had cases where KMM said 
that the security being imported did not already exist in the investment 
account being imported, and do I want to create it or not?  The problem 
with saying no is that it apparently stops importing any further 
transactions for that account.

In this case, such a "bogus" security (E000522) had been created some 
time ago, along with an account for it in an investment account, as well 
as two prices, presumably from sell transactions.  I edited the 
transactions to use the correct security.  I then deleted the security 
from the Securities tab of the investments view.  Apparently, this 
neither deleted the security account nor the prices for it.  I'm 
guessing this isn't exactly what I did originally, as on saving, I now 
got a Consistency Check error "Security "E000522' not found, which I do 
not recall seeing before.  However, the file saved is essentially the 
same as what I've been calling the first bad file. (Note the consistency 
check result says "The consistency check has found no issues in your 
data.  Details are presented below." with "Consistency check failed:" 
the above error message, and the path to mymoneyfile.cpp:3103 (from the 
appimage I am curently using.)

I'm now pretty sure that the four "bad" auto backup files were saved 
sequentially, thus essentially showing the same error, but accumulating 
the changes I had made during reconciliation.  I then closed KMyMoney, 
and although I didn't notice any errors on reading the latest saved 
file, it clearly stopped reading at the early </KMYMONEY-FILE> thus 
missing many of the prices and all my custom reports.

How many separate bug reports should this generate?  (Even excluding the 
one Thomas has already fixed - which I will confirm works, now that I 
can replicate the problem.)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kmymoney-devel/attachments/20231009/030db6aa/attachment.htm>