Automated task plugin

[Thomas Baumgart]

On Mittwoch, 27. Juli 2022 04:52:04 CEST Jonatan Cloutier wrote:

> Thank for the follow-up
> 
> Le 2022-07-26 à 15 h 55, Thomas Baumgart a écrit :
> 
> > Jonatan,
> >
> > On Dienstag, 26. Juli 2022 04:57:31 CEST Jonatan Cloutier via KMyMoney-devel wrote:
> >
> >> Hello, I'm looking at a way of automatically pulling data in my KMyMoney
> >> files. I already have the source data, but I'm now trying to find a way
> >> to get it automatically into the KMyMoney file. The main requirement is
> >> that I would highly prefer not to take manual action from KMyMoney.
> > Which source format does your data have?
> 
> The original src is a bit of anything, mainly web scraping, but might be 
> REST API calls as well if I get this working, then I can produce 
> whatever format fit my need. As a quick workaround I'm doing csv, but 
> that still needs a lot of manual processing to avoid duplicate transaction.
> 
> All in all the reason for this is that my bank is getting worse every 
> year at producing meaningful ofx files that just need way too many 
> manual fix after import (more info available in that feature request: 
> https://bugs.kde.org/show_bug.cgi?id=452392 ) and furthermore, there is 
> no export feature for loans and investments which are quite cumbersome 
> to manually update, in particular the later. And to put a cherry on top, 
> exports are limited to 30 days, which I frequently miss the dead line!
> 
> woob doesn't really work as my bank isn't supported and having double 
> factor authentication, I'm not sure it can work out.
> 
> >> After all that being said, two main questions: 1. Is it possible to run
> >> a background listener/task in a plugin as described above? 2. Any other
> >> suggestion on how to script custom data modification automatically?
> >
> > I don't know how much knowledge of programming in C++/Qt you bring along.
> > The background listener is probably not so easy to implement. A way to solve
> > your problem might be to construct a KMyMoney statement file in an external
> > tool. It's XML formatted and used internally by KMyMoney by all the importers.
> > AFAIR, there is a mechanism to start KMyMoney with an importer file as argument
> > and it will import it into the last opened KMyMoney file. If KMyMoney is already
> > running it will use the so called WebConnect feature to import it into the
> > already running process. In any case, you would need to have KMyMoney running
> > and this is only supported with a graphical user interface.
> >
> > So, maybe you can elaborate a bit about your (programming) skills and the operating
> > system you are using.
> 
> I'm on archlinux, been developing for years mainly in java and pythons 
> but do have a bit of experience in c++ and did use QT for some very old 
> project.
> 
> I've seen the webconnect features from the main, since it's not limited 
> to just the ofx importer I've been thinking that a custom importer could 
> do the job, but haven't really thought of using the statement format as 
> it doesn't seem documented. But if that enable doing the transaction 
> import without modal, I suppose it could work. Closest to documentation 
> I found is MyMoneyStatement::read function and its surroundings. I also 
> see that there is a write method, is it possible to export in that 
> format? It would just be quicker to get all the right xml structure.

The MyMoneyStatement object is the interface used by all importers to transfer
the data to KMyMoney. The read/write functions are used to convert the
object from/to an XML format. The existing importers (ofx, csv, kbanking, ...)
fill in the data and pass the object to the MyMoneyStatementReader which
takes care of importing the data into the MyMoneyFile object.

Writing out the XML version and read it using the Web-Connect feature
basically does the same just through an external file.

> I will investigate more on that possibility, might be easier than 
> creating a new importer and in the end, I think it could be close to my 
> hopes in terms of usability.
> 
> Quickly looking at that read method, I do have two questions:
> 
> * Are the m_accountId in the statement and in the split required to be 
> internal KMyMoney id or they can be inferred like from ofx? I suppose 
> they are the internal Id

It is the internal ID of KMyMoney. When mapping the online account the
plugin stores information how to identify the account and KMyMoney stores
that in the onlinebanking kvp of the MyMoneyAccount object.

When the statement is read it uses that to extract the internal ID.
See e.g. the logic at the end of OFXImporter::ofxAccountCallback()
how this is done. For OFX it is set in KOnlineBankingSetupWizard::ofxAccountCallback()
as part of the mapping process.

In KBanking the information looks very different but the mechanism is the
same. See KBanking::setupAccountReference() and KBankingExt::importAccountInfo().

> * Similarly, are both part of the split needed to be filled or is there 
> a way in the import that the automatching of paye and categories and 
> scheduled transaction would work like an ofx import?

No, the auto matching is performed by the MyMoneyStatementReader. In fact,
the existing importers only fill in a single split (exceptions may be
investment transactions).

Regarding an import version that does not require user intervention some
of the parts in MyMoneyStatementReader may have to be tweaked/refactored.
Something, we certainly need to discuss.

-- 

Regards

Thomas Baumgart

https://www.signal.org/       Signal, the better WhatsApp
-------------------------------------------------------------
The Unix Guru's View of Sex:
============================
# unzip; strip; touch; finger; mount; fsck; more; yes; umount; sleep
-------------------------------------------------------------
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 868 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kmymoney-devel/attachments/20220727/3913d0da/attachment-0001.sig>