Problem payee on OFX import, was: oddness with blank payee and missing category
Thomas Baumgart
thb at net-bembel.de
Sat Dec 30 09:55:57 UTC 2017
On Freitag, 29. Dezember 2017 18:10:52 CET Jack wrote:
> On 2017.08.25 18:53, Jack wrote:
> > I finally got time to start clearing up this old mess. I changed the
> > name of the Payee with no name (can I call it the Payee formerly
> > known as ... ?) to " missing payee" so I can distinguish that payee
> > from no payee on a transaction. More below.
> >
> > On 2017.07.28 08:19, Thomas Baumgart wrote:
> >> On Mittwoch, 26. Juli 2017 14:06:31 CEST Jack wrote:
> > [snip.....]
> >
> >>> Question - should it also be safe just to change Payee="P000845"
> >>> to Payee="" in the file directly without breaking anything else?
> >>> It might end up being the fastest way, but I'm not sure I trust
> >>> it. (I would definitely make several backups first :-))
> >>
> >> Yes, that can be done. It leaves the transaction with no payee then.
> >> Once all of them have been processed that way, you should be able to
> >> delete the payee completely (giving it a name like "no payee" before
> >> reduces the confusion). It's exactly what KMyMoney does internally
> >> if you
> >>
> >> a) rename the empty payee to "no payee"
> >> b) go to a transaction and get rid of "no payee" as payee
> >> c) repeat b) until no more transactions are assigned
> >>
> >> which is a bit more tedious if we talk about a lot of transactions
> >> compared to manipulating the file directly.
> >
> > From the Payee View, I selected the " missing payee" and then
> > selected the first listed transaction. It is a "Sell shares"
> > transaction. I see no mention of " missing payee" but if I right
> > click, I do see entries to "go to " both the brokerage account and
> > the " missing payee." From the brokerage account, I only see the
> > goto the investment account. I suppose this is because there is
> > generally no payee involved in a "sell shares" transaction, so there
> > is no point in displaying it. However, since a payee got created on
> > import (by matching the payee with no name) it may end up necessary
> > for me to do this by editing the XML file.
> >
> > Any other thoughts?
> >
> > Jack
>
> As fast as I clean up the old transactions, I keep getting new ones
> with a similar (but not exactly the same) issue. The underlying
> problem seems to be that on OFX import, KMM assigns a payee to the
> transaction, even for investment transactions such as buy and sell
> shares, and dividends, which I think should not have a payee. Here is
> one example transaction - this seems to be the relevant part from
> ofxlog.txt (I've altered the wrapping and indentation)
> <SELLMF><INVSELL><INVTRAN>
>
> <FITID>20171222TB191128310441298<DTTRADE>20171222110000.000[-5:EST]<DTSETTLE
> >20171227110000.000[-5:EST] <MEMO>Sale: FIRST EAGLE OVERSEAS CL C FRAC SHR
> QUANTITY .975
> UNSOLIC
> </INVTRAN>
> <SECID><UNIQUEID>32008F804<UNIQUEIDTYPE>CUSIP</SECID>
> <UNITS>-588.975<UNITPRICE>23.6<MARKDOWN>0<COMMISSION>0<FEES>5.35<TOTAL>13894
> .46 <SUBACCTSEC>CASH<SUBACCTFUND>CASH</INVSELL>
> <SELLTYPE>SELL</SELLMF>
>
> It appears the UNIQUEID is used to identify the actual security
> elswhere in the imported OFX.
> This is the transaction as saved to kmm-statement-1.txt
> <TRANSACTION
> payee="Sale: FIRST EAGLE OVERSEAS CL C FRAC SHR QUANTITY .975
> UNSOLIC"
> dateposted="2017-12-27"
> security="FIRST EAGLE OVERSEAS CL C FRAC SH"
> reconcile="0"
> shares="-23559/40"
> amount="694723/50"
> action="sell"
> bankid="ID 20171222TB191128310441298"
> brokerageaccount=""
> number=""
> memo="Sale: FIRST EAGLE OVERSEAS CL C FRAC SHR QUANTITY .975
> UNSOLIC"/>
>
> So - it seems to assign the payee from the memo field, since there is
> no payeeid field.
Yes, it appears you are bitten by this (I just include the comment here):
// Decide whether to use NAME, PAYEEID or MEMO to construct the payee
and in your case NAME and PAYEEID seem to be empty/invalid and then the OFX
importer uses the MEMO data instead as a last resort. More can be found here:
https://bugs.kde.org/show_bug.cgi?id=272712 (oh, this is another can of
worms)
> The main problem with all of this is that the payee
> cannot be altered in any way through the standard UI. Right clicking
> on the transaction in the investment account lets you go to either the
> brokerage account or the payee. From either of those, however, you can
> only go back to the investment account.
>
> The basic behavior is the same in both 4.8 and git master. I'm
> bringing this up again in hopes that 1) others agree this is not
> correct behavior, and 2) it can be fixed in time for 5.0. (I'm
> guessing the fix will not be a trivial backport from master to 4.8, so
> I am NOT requesting it be fixed in 4.8, as I seem to be the first
> person to raise this issue.
Once we have agreement that in case of investment transactions it does not
make sense to assign a payee based on the memo field we can start developing a
fix and from what I have spotted, the code is very similar so it can even be
fixed in the 4.8 branch.
This seems reasonable, as I found the following sequence
// If the payee or memo fields are blank, set them to
// the other one which is NOT blank. (acejones)
if (t.m_strPayee.isEmpty()) {
// But we only create a payee for non-investment transactions (ipwizard)
if (! t.m_strMemo.isEmpty() && data.invtransactiontype_valid == false)
t.m_strPayee = t.m_strMemo;
} else {
if (t.m_strMemo.isEmpty())
t.m_strMemo = t.m_strPayee;
}
which goes back into the CVS days.
> One particularly peculiar issue with the above example, is that in this
> one case, rather than create a new payee from the MEMO field, the "IRS"
> (in First) matches to my payee of US Treasury, because I do have it set
> to match on IRS (Internal Revenue Service, for those not familiar with
> the branch of the US government which collects income taxes.)
>
> Another aspect of this is that if I import again, these transactions
> get re-imported, and do not match the originals.
That is strange and might be a different bug. Can you check that the bankid is
the same in both cases (original import and re-import).
bankid="ID 20171222TB191128310441298"
In fact, I noticed something similar lately if I import twice without
accepting in between. Then I have the transaction in the ledger twice (which
is not correct). I need to check this.
> If I match the newly
> imported one to the "corrected" one, and import again, it still imports
> it as a new transacion. Shouldn't it match based on the trasnaction
> ID, or is something funny going on with that also?
It should detect the bankid and find that it is already present. Maybe it does
some funny stuff due to the payee change. But it could also be related to my
findings which I mentioned above regarding the duplicates.
> Is there any point in my creating an actual OFX download file with a
> small number of transactions which can be used to demonstrate this
> problem in a clean KMM file?
I don't think so.
> Any other thoughts?
Since I cannot test any of this I will create a Phab patch against master and
you can test it and in case it does not work I can update it. Did you ever use
'arc'? It comes in very handy when testing patches on Phabricator. Maybe, I
have to write up a blog entry about it.
--
Regards
Thomas Baumgart
https://www.telegram.org/ Telegram, the better WhatsApp
-------------------------------------------------------------
Two of the most famous products of Berkeley are LSD and UNIX.
I don't think that this is a coincidence. (anonymous)
-------------------------------------------------------------
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 846 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kmymoney-devel/attachments/20171230/f447e056/attachment-0001.sig>
More information about the KMyMoney-devel
mailing list