OFX Import Bug

Wed Jan 11 07:27:35 UTC 2017

Hi Brendan,

yes, the formats are very different, as one (downloaded OFX) represents the 
data exchanged between the bank and libOFX. The txt file saved in /home/thb 
OTOH is the intermediate statement format that is used by all importers (OFX, 
QIF, HBCI, ...) and the KMyMoney statement importer.

I wonder about that 1969 problem. Looks like a time_t of value 0 with a 
subtractions of a couple of seconds/minutes/hours or a day.  Remember: time_t 
represents the number of seconds since 01-01-1970  ;)

Is the example you provide correct? '<DTEND>20141212210000' is converted to 
'enddate="1969-12-31"' by the OFX importer? If that is the case we need to 
take a look. Or am I missing something?

Regards

Thomas

On Tuesday 10 January 2017 22:57:23 you  wrote:
------------------------------------------------------------
> When I said the formats are different I meant very different.
> 
> The manually downloaded OFX files look like this:
> 
> OFXHEADER:100
> DATA:OFXSGML
> VERSION:102
> SECURITY:NONE
> ...
> <DTSTART>20170109
> <DTEND>20141212210000
> ...
> 
> The txt files saved in /home/thb look like this:
> 
> <?xml version="1.0" encoding="utf-8"?>
> <!DOCTYPE KMYMONEY-STATEMENT>
> <KMYMONEY-STATEMENT>
>  <STATEMENT version=...begindate="2016-10-12" enddate="1969-12-31"
> routingnumber...>
> 
> I've copied Thomas (thb) so that he may be able to guide me one more time.
> 
> 
> 
> *----Brendan*
> 
> On Tue, Jan 10, 2017 at 1:04 PM, Jack <ostroffjh at users.sourceforge.net>
> 
> wrote:
> > Brendan,
> > 
> > I wish I knew more about the software the banks actually use, but that is
> > not likely to happen.  I'm not surprised that the manual download might be
> > different from the direct connect download - but that's just based on
> > experience.  I would think they were generated by the same software - but
> > obviously called with different parameters - or something.
> > 
> > I know of no essential difference between QFX and OFX, other than the "Q"
> > implying Intuit/Quicken had something to do with it.  My guess is perhaps
> > differences in max length of some of the fields, like the memo.  I also
> > wouldn't expect a single bank to produce both - I've seen them offer one
> > or
> > the other, probably depending on who they buy/lease their OFX server
> > software from.
> > 
> > I have no explanation for the dates you are seeing.  I also agree the
> > probability of a useful response from the bank's tech support line is slim
> > to none - but it might be interesting if they have any response when you
> > tell them you are getting very different end dates from the two different
> > download methods.  It would depend at least on whehter the person you
> > speak
> > to even knows what OFX is.  Otherwise they just tell you to call Quicken
> > support.
> > 
> > I doubt very much a server will produce different results depending on
> > what you send in the request header - I suspect it only affects whether
> > they accept the request or reject it outright (with a useless error
> > message, probably.)
> > 
> > Finally, I think the direct connect download is intended to be transient,
> > so other than mucking in the code, I don't know that you will (easily) be
> > able to save it - but I believe it lives briefly somewhere under /tmp.  I
> > suspect what you end up with under /home/thb is close, depending on the
> > original purpose for saving it there.  Also remember, direct connect does
> > not download any "files."  It is a communication specification, meant for
> > direct use.  If the software happens to save it, the extension is
> > essentially arbitrary.   Formatting clearly doesn't matter - although the
> > content within the <OFX> tags should be similar, expecially for
> > transaction
> > data, perhaps allowing for different timestamps.
> > 
> > 
> > Jack
> > 
> > On 2017.01.10 15:18, Brendan Coupe wrote:
> >> Jack,
> >> 
> >> Your suggestion is fine for testing but a non-starter for day to day use
> >> of
> >> KMM. I can live with bogus warnings about my account balance not matching
> >> the download balance.
> >> 
> >> I downloaded both the OFX and QFX files from both banks. I diff'ed them
> >> and
> >> they are pretty similar (OFX vs QFX).
> >> 
> >> The formats are completely different from the files that are saved in the
> >> /home/thb folder which have the extension "txt" so they may not be copies
> >> of the downloaded OFX files.
> >> 
> >> Is there an easy way to find the OFX files downloaded when they are
> >> downloaded from KMM?
> >> 
> >> There is possible good news and bad news. The bank that shows 1969-12-31
> >> has the correct date in the manually downloaded OFX & QFX files.
> >> 
> >> <DTSTART>20161013000000
> >> <DTEND>20170110000000
> >> 
> >> 
> >> The bank that appears to use the date prior to the last transaction date
> >> (based on 2 recent downloads) shows an end date in 2014.
> >> 
> >> <DTSTART>20170109
> >> <DTEND>20141212210000
> >> 
> >> Very strange.
> >> 
> >> I have tried several options for "Identify as" and "Header Version" and
> >> the
> >> txt file that appears in the /home/thb folder is the same.
> >> 
> >> 
> >> 
> >> *----Brendan*
> >> 
> >> 
> >> On Tue, Jan 10, 2017 at 7:38 AM, Jack <ostroffjh at users.sourceforge.net>
> >> 
> >> wrote:
> >> > On 2017.01.09 23:24, Brendan Coupe wrote:
> >> >> I'm having a problem with 2 accounts when I import the OFX using
> >> >> online
> >> >> banking.
> >> >> 
> >> >> One account shows the Online Statement Balance on 12/31/1969.
> >> >> 
> >> >> Another account shows the Online Statement Balance on 1/3/2017 while
> >> 
> >> the
> >> 
> >> >> most recent transaction is on 1/4/2017 and I just downloaded it today
> >> >> (1/9/2017).
> >> >> 
> >> >> Not surprisingly when I checked the OXF files for the respective
> >> 
> >> accounts
> >> 
> >> >> I
> >> >> found the following (/home/thb comes to the rescue again):
> >> >> 
> >> >> ...begindate="2016-10-11" enddate="1969-12-31"...
> >> >> 
> >> >> ...begindate="2016-12-09" enddate="2017-01-03"...
> >> >> 
> >> >> I'm almost certain I will not have much luck getting the banks to fix
> >> >> this.
> >> >> 
> >> >> Is it possible to have KMM use the current download date when the date
> >> 
> >> in
> >> 
> >> >> the OFX file does not make sense? KMM could also use the latest date
> >> 
> >> of a
> >> 
> >> >> transaction in the OFX file when it's later then the "enddate".
> >> >> 
> >> >> At the very least it should recognize a problem since the enddate is
> >> >> before
> >> >> the begindate at the first bank.
> >> >> 
> >> >> I did check a different bank at it uses today's date as the enddate.
> >> >> 
> >> >> 
> >> >> *----Brendan*
> >> > 
> >> > One thing you might try is to manually download an OFX file.  I have
> >> 
> >> oddly
> >> 
> >> > found that it can differ from what gets downloaded by direct connect.
> >> 
> >> Even
> >> 
> >> > if it doesn't, you can manuallyl correct the end date if it's still
> >> 
> >> wrong.
> >> 
> >> > Not a good long term solution, but a temporary workaround.
> >> > 
> >> > Jack
-- 

Regards

Thomas Baumgart

GPG-FP: E55E D592 F45F 116B 8429   4F99 9C59 DB40 B75D D3BA
-------------------------------------------------------------
Any sufficiently advanced bug is indistinguishable from a feature.
(Rich Kulawiec)
-------------------------------------------------------------
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 225 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kmymoney-devel/attachments/20170111/7fd6b62b/attachment-0001.sig>