OFX import issue: Was: Is ofx import buildable in master?

[Jonatan Cloutier]

Jonatan Cloutier

Le 2023-04-24 à 08 h 24, Thomas Baumgart via KMyMoney-devel a écrit :
> On Montag, 24. April 2023 00:38:27 CEST Jack via KMyMoney-devel wrote:
>
>> On 2023.04.23 11:35, Jonatan Cloutier via KMyMoney-devel wrote:
>> [snip.....]
>>> On the other side, I was able to confirm that there seems to be a
>>> real issue in regards to loan scheduled transaction, when matched to
>>> an import. My mortage just jump to a higher amount than the us debt!
>>> (by exactly 92 233 720 368 547 758,07 which is actually LLONG_MAX/100
>>> ??). I'll try reproducing that in a test file. Let me know if there
>>> is any specific information that might be valuable. I'll open a bug
>>> when I feel I have enough info and that I'm not the one that created
>>> the issue with either my custom import or some manual change I would
>>> have done in my file.
>> I was about to file a wishlist for matching the ending balance of the
>> brokerage account when importing an investment account (using libofx)
>> and noticed something strange.
>>
>> The ofxlog.txt included the snippets "<INVBAL><AVAILCASH>234.56" and
>> "<BAL><NAME>TotalMarketValue<DESC>Total Market
>> Value<BALTYPE>DOLLAR<VALUE>123.45"  (amounts edited) which match the
>> balance of the brokerage account and the value of the investment
>> account.  However, in kmm-statement-2023-04-23T22-16-58.913.txt, the
>> line beginning <STATEMENT includes
>> 'closingbalance="-9223372036854775807/100"' which is the same amount
>> you noticed.   It looks like some numeric conversion is getting
>> confused, but I don't know if it's libofx or KMM itself.
> No conversion or calculation gets confused. It's certainly not libofx
> but the KMyMoney statement reader. That value you see is a constant
> provided with MyMoneyMoney::autoCalc. Actually it is not LLONG_MAX/100
> but (LLONG_MIN+1)/100, but Jonatan's guess was close :)  I noticed
> it is used by the statement reader. Here are some references:
>
> https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/mymoney/mymoneymoney.cpp#L61
> https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/mymoney/mymoneymoney.cpp#L383-L386
>
> https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/mymoney/mymoneystatement.h#L101
> https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/converter/mymoneystatementreader.cpp#L575
>
> So for the case of the statement file it simply means, that no balance
> information was extracted from the statement.
>
> For Jonatan's loan case, it is used as a placeholder in the principal and
> interest splits of a scheduled loan repayment transaction. They should
> be replaced with the actual values when the scheduled transaction is
> entered. Maybe something failed at that point and the value got into
> a real transaction unchanged. For the test file scenario one probably
> needs to have a KMM file with a loan, a scheduled repayment transaction
> which is due and an import file that contains that transaction. Then
> importing the file should show the problem. I expect this to happen
> somewhere in the match of scheduled transactions in
>
> https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/converter/mymoneystatementreader.cpp#L1361-L1369
>
> which calls ScheduledTransactionMatchFinder::findMatchInMatchCandidatesList
>
> https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/converter/scheduledtransactionmatchfinder.cpp#L28-L44
>
> which then calls KMyMoneyUtils::scheduledTransaction()
>
> https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/kmymoneyutils.cpp#L340-L355
>
> and that should take care of calculating the distribution using calculateAutoLoan() at
>
> https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/kmymoneyutils.cpp#L270-L277
>
> using MyMoneyForecast::calculateAutoLoan() at
>
> https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/mymoney/mymoneyforecast.cpp#L1612-L1683
>
>
> Using the original data and a bit of debugging would be the easiest to find
> out what is going on. Not sure if Jonatan is capable of doing that.
>
> @Jonatan: if possible, keep a copy of the KMM file and the imported statement
> aside from your real data so that you can always re-run this.
>
>
I did take a backup of both the file before I manually fix all of the 
previous one and before I did the last import. I'm actually waiting for 
this week payment so that I can have a smaller import to debug. I'm not 
quite setup to do proper c++ debug, but with the pointer given above, I 
might try with some debug print. I'll let you know my finding, probably 
toward the weekend. I also confirm it's not ofx, since my custom import 
use the kmm statement import directly, although, I'll debug with real 
ofx file as this is the real supported way to do it.