Revisiting problems with price updates

[Jack]

As this is getting rather long winded, I'm going to trim down to mainly  
stuff I'm directly replying to.

On 2022.02.22 02:34, Thomas Baumgart via KMyMoney-devel wrote:
> On Dienstag, 22. Februar 2022 01:47:21 CET Jack via KMyMoney-devel  
> wrote:
[snip....]
>> As one example, I have an equity E0000416 for symbol CARR (Carrier  
>> Global Corp.)
>> If I look in the Price Editor for CARR, I see two entries - one is  
>> from KMyMOney Currency and the other from Yahoo Finance.  It is not  
>> clear why there are two en tries for the same ID and Name.
> See below. What are the dates of these entries?
1/6/21 Currency and 2/4/22 Yahoo Finance.  Probable the most recent  
price of each of the two pricepairs.

>> If I show all stored prices, the sources include Transaction, User,  
>> Yahoo Finance, Finance::Quote yahoo_json, Statement Importer, and  
>> KMyMoney Currency. All make sense to me except for the last.  In  
>> addition, if I look in my kmy file, I have five entries for  
>> <PRICEPAIR from="USD" to="E000416"> including one User and four  
>> KMyMoney Currency.
> Those are the ones causing the trouble (I think).
Agreed, but then why would there be KMyMoney Currency prices in the  
other batch?  See below for info on those price entries.

> The User one seems to be entered by the user (either directly as  
> price or by a transaction). Do you see a relationship between this  
> price entry and a transaction? The ones showing KMyMoney Currency are  
> a cause of this if they have a later date. Can you confirm?
Here is a review of all the prices.

My first shares of this stock were obtained as a dividend from a  
different stock which I manually entered as "Add shares" on 4/7/20.   
The User entry on that date was my entry of the price indicated for the  
date in my broker statement.  That is NOT the User entry mentioned  
above.

The F::Q entries from 5/5 to 10/30/20 were normal online price updates.

There were sales in three accounts on 9/21/20, thus the transaction  
price, but I don't know why I would have manually entered the  
transactions instead of using ofx import.  I cannot think of why I  
would have manually entered a price on that date, but that is the User  
entry mentioned above.

KMyMoney Currency (KMMC) prices on 10/1 and 10/30/20 also had F::Q  
prices on the same dates.  KMM Currency price 12/1/20 had a Yahoo  
Finance price the same date.  These are the three KMMC prices mentioned  
above.

KMMC price 1/6/21 had Statement Importer price the same date, and was  
for sales in four accounts.  This and all further KMMC prices are in  
the "right" direction pricepair.

KMMC prices from 2/7/21 through 1/4/22 were probably normal online  
price updates.  Yahoo Finance update 2/4/22 was my most recent online  
price update.

I'm pretty sure I set Yahoo Finance set as the price source far longer  
than a month ago.  I do have a bunch of backups, but I'd rather not go  
hunting through them manually.  If there is specific info to look for,  
I can put together a shell script or maybe a Perl script using the XML  
parser.
> 
>> Then, in the Price Editor, the Stock name for CARR is "Carrier  
>> Global Corp,Carrier Global Corp,Carrier Global Corp,Carrier Global  
>> Corp,Carrier Global Corp".  How did the name get repeated five times?
> Good question. I don't have an answer (yet).
Leaving so we don't forget, although it's only a cosmetic issue (I  
hope.)
> 
>> Then, if I look in the Price Updater, I see an entry for ID: CARR  
>> with Internal ID: E000416 and Source: Yahoo Finance.  However, if I  
>> scroll down, I see ID: USD > E000416 with internal ID: "USE E000416"  
>> a price of "CARR0.0218 and Source KMyMOney Currency.
> The internal ID should probably be "USD E000416" and is a typo on  
> your end.
Sorry, that was a typo, the InternalID is USD E000614 (not USE).
> The price of "CARR0.0218" is certainly weird.
Also note, there is no "CARR0" string anywhere in my data file,  
according to grep.  There are four other similar entries, where the  
price is the stock symbol followed by "0." and a four digit number  
which doesn't match anything I can think of.
> 
> Regarding the names Yahoo Finance and KMyMoney Currency I can point  
> to https://invent.kde.org/office/kmymoney/-/commit/8559f9ec which we  
> changed when Yahoo was not available a few years ago. Still, I don't  
> understand why the currency exchange rate source is used for an  
> investment price update.
That's what we call the $64,000 question.  (There was long ago a TV  
quiz show in the US with that name, and that was the grand prize if you  
answered all their questions.)  I originally used Yaoo, and when it  
started failing, inconsistently changed between F::Q yahoo_json, Stooq,  
and a small use of Financial Times.

>> Searching my kmy file for E000416 shows only the Security  
>> definition, the five Accounts holding that stock, and the two  
>> PRICEPAIR entries already mentioned.  In particular, I don't see  
>> what generates that USD > E000416 entry in the price updater.  My  
>> currency list includes only USD and EUR.
> >
> > Any thoughts or suggestions for further troubleshooting?
> 
> I scanned the term "KMyMoney Currency" and only found it in a few  
> spots:
> 
> in WebPriceQuote::launchCSV it is only used as reference:
> 
>     // for historical exchange rates we switch to Stooq
>     if (sourcename == QLatin1String("KMyMoney Currency"))
>         sourcename = QLatin1String("Stooq Currency");
> 
> and for the generation of the source information to select from:
> 
>     result["KMyMoney Currency"] = WebPriceQuoteSource("KMyMoney  
> Currency",
>                                   "https://fx-rate.net/%1/%2",
>                                   QString(),
>                                   "https://fx-rate.net/([^/]+/[^/]+)",
> 
> the third location is the one introduced by above mentioned commit  
> and it is the only location that actively sets the name. From there  
> it seems to be used for price entries during the update.
Can you tell under what conditions it is used, or is that part of the  
mystery?

I also need to look again at bug I first referred to:  
https://bugs.kde.org/show_bug.cgi?id=450426 as he posted a small kmy  
file which also shows the problem.  It might be easier to dig throug  
than my 18MB xml data file.

Jack