Entering mutual fund in KMM 4.7.2 without finding online source

[Volker Paul]

On 03.04.2016 09:14, Thomas Baumgart wrote:
> Hi Louis-Philippe,
>
> On Saturday 02 April 2016 19:20:13 Louis-Philippe Allard wrote:
>
>> Hello,
>>
>> I am trying to enter several mutual funds in KMM but their trading
>> symbol is not on any of the provided online sources in KMM's security
>> dialob box.
>>
>> The only remaining choice is to update the prices manually, but this
>> is cumbersome and I'd rather rely on a reliable online source to
>> update currencies and securities.
>>
>> One example of mutual fund I am trying to integrate in KMM is
>> http://quote.morningstar.ca/quicktakes/fund/f_ca.aspx?t=F0CAN05NHD&region=ca
>> n&culture=en-CA
>>
>> This is a canadian MF.  The symbol is CIG2601 (if you google this you
>> will inevitably get either the investment firm's website or the
>> Morningstar quote site which lists all of my other MF as well..
>>
>> Is it possible to add morningstar to the list of online sources in
>> KMM, or if not, how can use morningstar's website as online update
>> service?
> Did you have a look at https://docs.kde.org/stable4/en/extragear-office/kmymoney/details.investments.prices.html#details.investments.onlinequotes
> already? At the bottom of this page we explain how to add new quote sources. A
> bit of regular expression foo is required to extract the price from a web-
> site. In case you don't have that, I am sure someone here on the user mailing
> list can help out.
>
> Hope that helps.
>
Some years ago I had a similar problem, experimented a bit, found a solution
and wrote a Howto for myself and others in similar situations.
If nothing works, try the Troubleshooting section.
You find regexp tutorials and manuals in the Internet, for example here:
https://docs.python.org/2/howto/regex.html

Hope this helps.

Volker


How to "develop" regular expressions for online price update
============================================================

Investment detail wizard settings:
Trading symbol: this symbol is inserted in the URL
Online Source:  this source is queried;
     define its syntax in Settings - Configure KMyMoney - Online quotes

The page produced by e.g. http://www.wallstreet-online.de,
after searching an online quote, is quite complicated
but we need only a few parts of it.
It is useful to analyze it (viewing its source) and
extract these main parts. They contain the symbol:

     ISIN: LU0311232911<br>

the price:

     <td >Schlusskurs:</td>
     <td style="text-align: right;">94,53 (07.07.)</td>

and the date:

     <div class="market">Fonds (EUR), 08.07.09 | 18:00</div>

We must build regular expressions that match these parts and extract
the needed information from them.

We write these regular expressions to the settings for Online Quotes.
To do this, we select Settings - Configure KMyMoney - Online Quotes,
then the preferred online source,
e.g. Wallstreet-Online.DE (Default).
There we enter the correct URL:
     http://www.wallstreet-online.de/si/?k=%1&spid=ws
the symbol regexp:
     ISIN:\s*(\w+)
the price regexp:
     Schlusskurs:\D*(\d+,\d+)\D
the date regexp:
     , (\d+\D+\d+\D+\d+)
and the date format:
     %d %m %y

http://www.wallstreet-online.de/si/?k=LU0311232911&spid=ws
which redirects to:
http://www.wallstreet-online.de/fonds/1408426

To test it, we select Investments, then an investment account, then
right-click on an investment and select "Online price update...".

If the update was successful, the Status message is as follows:
     Fetching URL ...
     Symbol found: ...
     Price found: ...
     Date found: ...
     Price for ... updated (...)

If the symbol can't be found (regexp didn't match), then the "Symbol found"
line is not displayed. The update can still be successful, however.

If the price regexp did not match, then the "Price found"
line is not displayed, and a red "Unable to update price for ..."
line is displayed instead. The user is then asked if he wants to
disable online price updates for this security.
This question can be turned off (check "Do not ask again").

The same happens if the date regexp did not match. Unlike the symbol,
the date seems to be essential and if it is missing, the price update fails.

So the status message tells you where the problem is.

Troubleshooting
---------------

To debug you regexps, forget the real price source in the Internet for a 
while,
and write really simple symbol, price and date lines and corresponding 
regexps.
(But make a backup of the settings!)
For example, "ISIN: LU0311232911" as symbol line and "ISIN: (\w+)" as 
symbol regexp.
For testing purposes, these lines
can be produced by a getquote.sh script:

     echo "ISIN: DE0008470337"
     echo "Schlusskurs: 12,34 "
     echo ", 02.05.11"

or:

     echo "ISIN: DE0008470337"
     echo 'field="ql">123,45<'
     echo '<div id="timeinfo">   So 15.02.2015'


As you see, the output of this script is constant, the script ignores 
its argument.
In Settings - Configure KMyMoney - Online Quotes, as URL, write:
     file:/home/yourname/bin/getquote.sh %1
     file:/home/volker/data/prj_priv/kmymoney/getquote/getquote.sh %1
or wherever your script is, instead of
http://www.wallstreet-online.de/%1


Also enter the regexps created above.
Do not forget to click "Update" in the "Online Quotes" configuration dialog,
otherwise old settings are still used.
Then try the online update again.

This time it should really work, because your input is specially 
tailored for the regexps.
Of course it is not really "online", but just for testing.

 From here, carefully adapt regexps and input until they correspond to 
the real online
input and corresponding regexps (backed up above).
Edit the script and the settings simultaneously for this.
Finally, replace the URL setting with the real URL again.


Important:
- Check "Skip HTML stripping"
- It is useful to save a HTML file and use cat file.html in getquote.sh.
   But then USE AN ABSOLUTE PATH!