OFX Import Bug

Wed Jan 11 07:54:53 UTC 2017

No, you reversed them. <DTEND>20170110000000 from the manually downloaded
OFX file is being converted to 1969.12.31.

>From the other bank <DTEND>20141212210000 from the manually downloaded OFX
file ends up being the day before the oldest transaction in the txt file.
This makes no sense and I have only seen this twice so there may be
something else going on.

Now I'm getting begindate="2016-12-10" enddate="2016-12-06" from the second
bank and the last transaction in the thb text file is 2016.12.07. The
begindate makes sense because it's set to 30 days. The enddate and last
trnsaction date make no sense because I already have more
recent transactions from previous downloads.

Is there any way to capture the data from the online OFX download by KMM?
It seems likely that it does not match the manual download from these two
banks websites.

I didn't notice this until recently when the icons on my home page (Assets
and Liabilities Summary) suddenly reappeared after not seeing them for at
least a couple of years. Both of these accounts are relatively new. The red
warning on the Online Statement Balance line is above the lines shown on
the ledger so I don't see it in the 1969 account. I have been compiling
from source for a long time and I have no idea why the icons were missing
or why they came back but the red circles with the lines through them
tipped me off to this problem. Maybe you should just get rid of the
icons for me:-)

*----Brendan*

On Tue, Jan 10, 2017 at 11:27 PM, Thomas Baumgart <thb at net-bembel.de> wrote:

> Hi Brendan,
>
> yes, the formats are very different, as one (downloaded OFX) represents the
> data exchanged between the bank and libOFX. The txt file saved in /home/thb
> OTOH is the intermediate statement format that is used by all importers
> (OFX,
> QIF, HBCI, ...) and the KMyMoney statement importer.
>
> I wonder about that 1969 problem. Looks like a time_t of value 0 with a
> subtractions of a couple of seconds/minutes/hours or a day.  Remember:
> time_t
> represents the number of seconds since 01-01-1970  ;)
>
> Is the example you provide correct? '<DTEND>20141212210000' is converted to
> 'enddate="1969-12-31"' by the OFX importer? If that is the case we need to
> take a look. Or am I missing something?
>
> Regards
>
> Thomas
>
>
>
> On Tuesday 10 January 2017 22:57:23 you  wrote:
> ------------------------------------------------------------
> > When I said the formats are different I meant very different.
> >
> > The manually downloaded OFX files look like this:
> >
> > OFXHEADER:100
> > DATA:OFXSGML
> > VERSION:102
> > SECURITY:NONE
> > ...
> > <DTSTART>20170109
> > <DTEND>20141212210000
> > ...
> >
> > The txt files saved in /home/thb look like this:
> >
> > <?xml version="1.0" encoding="utf-8"?>
> > <!DOCTYPE KMYMONEY-STATEMENT>
> > <KMYMONEY-STATEMENT>
> >  <STATEMENT version=...begindate="2016-10-12" enddate="1969-12-31"
> > routingnumber...>
> >
> > I've copied Thomas (thb) so that he may be able to guide me one more
> time.
> >
> >
> >
> > *----Brendan*
> >
> > On Tue, Jan 10, 2017 at 1:04 PM, Jack <ostroffjh at users.sourceforge.net>
> >
> > wrote:
> > > Brendan,
> > >
> > > I wish I knew more about the software the banks actually use, but that
> is
> > > not likely to happen.  I'm not surprised that the manual download
> might be
> > > different from the direct connect download - but that's just based on
> > > experience.  I would think they were generated by the same software -
> but
> > > obviously called with different parameters - or something.
> > >
> > > I know of no essential difference between QFX and OFX, other than the
> "Q"
> > > implying Intuit/Quicken had something to do with it.  My guess is
> perhaps
> > > differences in max length of some of the fields, like the memo.  I also
> > > wouldn't expect a single bank to produce both - I've seen them offer
> one
> > > or
> > > the other, probably depending on who they buy/lease their OFX server
> > > software from.
> > >
> > > I have no explanation for the dates you are seeing.  I also agree the
> > > probability of a useful response from the bank's tech support line is
> slim
> > > to none - but it might be interesting if they have any response when
> you
> > > tell them you are getting very different end dates from the two
> different
> > > download methods.  It would depend at least on whehter the person you
> > > speak
> > > to even knows what OFX is.  Otherwise they just tell you to call
> Quicken
> > > support.
> > >
> > > I doubt very much a server will produce different results depending on
> > > what you send in the request header - I suspect it only affects whether
> > > they accept the request or reject it outright (with a useless error
> > > message, probably.)
> > >
> > > Finally, I think the direct connect download is intended to be
> transient,
> > > so other than mucking in the code, I don't know that you will (easily)
> be
> > > able to save it - but I believe it lives briefly somewhere under
> /tmp.  I
> > > suspect what you end up with under /home/thb is close, depending on the
> > > original purpose for saving it there.  Also remember, direct connect
> does
> > > not download any "files."  It is a communication specification, meant
> for
> > > direct use.  If the software happens to save it, the extension is
> > > essentially arbitrary.   Formatting clearly doesn't matter - although
> the
> > > content within the <OFX> tags should be similar, expecially for
> > > transaction
> > > data, perhaps allowing for different timestamps.
> > >
> > >
> > > Jack
> > >
> > > On 2017.01.10 15:18, Brendan Coupe wrote:
> > >> Jack,
> > >>
> > >> Your suggestion is fine for testing but a non-starter for day to day
> use
> > >> of
> > >> KMM. I can live with bogus warnings about my account balance not
> matching
> > >> the download balance.
> > >>
> > >> I downloaded both the OFX and QFX files from both banks. I diff'ed
> them
> > >> and
> > >> they are pretty similar (OFX vs QFX).
> > >>
> > >> The formats are completely different from the files that are saved in
> the
> > >> /home/thb folder which have the extension "txt" so they may not be
> copies
> > >> of the downloaded OFX files.
> > >>
> > >> Is there an easy way to find the OFX files downloaded when they are
> > >> downloaded from KMM?
> > >>
> > >> There is possible good news and bad news. The bank that shows
> 1969-12-31
> > >> has the correct date in the manually downloaded OFX & QFX files.
> > >>
> > >> <DTSTART>20161013000000
> > >> <DTEND>20170110000000
> > >>
> > >>
> > >> The bank that appears to use the date prior to the last transaction
> date
> > >> (based on 2 recent downloads) shows an end date in 2014.
> > >>
> > >> <DTSTART>20170109
> > >> <DTEND>20141212210000
> > >>
> > >> Very strange.
> > >>
> > >> I have tried several options for "Identify as" and "Header Version"
> and
> > >> the
> > >> txt file that appears in the /home/thb folder is the same.
> > >>
> > >>
> > >>
> > >> *----Brendan*
> > >>
> > >>
> > >> On Tue, Jan 10, 2017 at 7:38 AM, Jack <ostroffjh at users.sourceforge.
> net>
> > >>
> > >> wrote:
> > >> > On 2017.01.09 23:24, Brendan Coupe wrote:
> > >> >> I'm having a problem with 2 accounts when I import the OFX using
> > >> >> online
> > >> >> banking.
> > >> >>
> > >> >> One account shows the Online Statement Balance on 12/31/1969.
> > >> >>
> > >> >> Another account shows the Online Statement Balance on 1/3/2017
> while
> > >>
> > >> the
> > >>
> > >> >> most recent transaction is on 1/4/2017 and I just downloaded it
> today
> > >> >> (1/9/2017).
> > >> >>
> > >> >> Not surprisingly when I checked the OXF files for the respective
> > >>
> > >> accounts
> > >>
> > >> >> I
> > >> >> found the following (/home/thb comes to the rescue again):
> > >> >>
> > >> >> ...begindate="2016-10-11" enddate="1969-12-31"...
> > >> >>
> > >> >> ...begindate="2016-12-09" enddate="2017-01-03"...
> > >> >>
> > >> >> I'm almost certain I will not have much luck getting the banks to
> fix
> > >> >> this.
> > >> >>
> > >> >> Is it possible to have KMM use the current download date when the
> date
> > >>
> > >> in
> > >>
> > >> >> the OFX file does not make sense? KMM could also use the latest
> date
> > >>
> > >> of a
> > >>
> > >> >> transaction in the OFX file when it's later then the "enddate".
> > >> >>
> > >> >> At the very least it should recognize a problem since the enddate
> is
> > >> >> before
> > >> >> the begindate at the first bank.
> > >> >>
> > >> >> I did check a different bank at it uses today's date as the
> enddate.
> > >> >>
> > >> >>
> > >> >> *----Brendan*
> > >> >
> > >> > One thing you might try is to manually download an OFX file.  I have
> > >>
> > >> oddly
> > >>
> > >> > found that it can differ from what gets downloaded by direct
> connect.
> > >>
> > >> Even
> > >>
> > >> > if it doesn't, you can manuallyl correct the end date if it's still
> > >>
> > >> wrong.
> > >>
> > >> > Not a good long term solution, but a temporary workaround.
> > >> >
> > >> > Jack
> --
>
> Regards
>
> Thomas Baumgart
>
> GPG-FP: E55E D592 F45F 116B 8429   4F99 9C59 DB40 B75D D3BA
> -------------------------------------------------------------
> Any sufficiently advanced bug is indistinguishable from a feature.
> (Rich Kulawiec)
> -------------------------------------------------------------
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kmymoney-devel/attachments/20170110/c4252276/attachment.html>