[Okular-devel] Engaging

[Yuval Levy]

Thank you Fabio for the short off-list insights that made me feel
confident to write this back:

On 13-03-30 09:39 AM, Fabio D'Urso wrote:
> I'm glad you liked the new annotation features :)

"like" is an understatement.  In my opinion *PDF* annotations are the
killer feature of Okular, and Okular as a *PDF* viewer is a killer app
to enable the survival of free desktops along proprietary desktops.


> I don't see inconsistency between "Save as" and "Save copy as" as 
> functionalities, but it could be because I "just know" what they do :)

OK, I was trying to bring this point as a usability argument for menu
simplicity.  I'll venture on thinner ice now:  Okular declares itself on
its home page as "universal" document reader.  Embedded *PDF*
annotations seem to me more universal than XML annotations because they
are more widely supported and they enable me to communicate
"universally", with more users.  Everything else follows.


> I see the "Annotations only (XML)" option a little hard to do:

Well, I suggested this one because I did not want to take away existing
functionality from users who might have a strong attachment to it.  I do
not want to impose my personal views on anybody.  That said, IMHO there
is no need to save *PDF* annotation to XML at all.  Newer versions of
Okular should reflect this and stop using XML for the PDF annotations.
Even better:  on first start they should check for existing XML
annotations to PDF files and offers a migration of all of my documents
(with three options: Yes, No, Remind me later).  If I had a choice 18
months ago between XML and embedded annotation, my choice would have
been for embedded.  Now I am stuck with XML annotations that I'd rather
convert all to embedded ones.


> what would you 
> do if the user chooses this option on a already annotated PDF file? Would you 
> only save modified annotations to XML? What about deleted ones?
> Currently we refuse to use the XML format on already annotated PDF documents 
> (that's why you get the "annotation chnages will not be saved automatically" 
> warning in such documents).

With all due respect, what Okular does now is unnecessarily complex.
Why not save annotation into PDF documents automatically?  Of course
with a backup copy of the original file, to account for data integrity
concerns such as <https://bugs.kde.org/show_bug.cgi?id=301774#c3>.  The
backup copy could be optional in the preference, but the option should
be on by default.


> Yuval, are you willing to work on it?

As I wrote to you privately, there are two constrains: skill and time.
Skill can be compensated for by training and mentoring.  Indeed with a
little of encouragement from you I was able to build Okular on my end.
Time is, unfortunately, a more critical constrain.  I almost completely
stopped contributing to (other) Open Source projects 18 months ago.  It
is very unlikely that my situation will change in the next 18 (or more
likely 30) months.  Despite all the good will I can put forward, my
contributions will be very limited.

Here an initial contribution.  Could somebody with edit privilege for
the website add the following two FAQs under "Compiling Okular" ?

Q: Cmake quits with 'ERROR: cmake/modules/FindKDE4Internal.cmake not found'

A: Missing development headers.  On Ubuntu (12.10) type the command
'sudo apt-get install kde-workspace-dev'

Q: Cmake ends stating that the following REQUIRED packages could NOT be
located on your system: QImageBlitz (kdesupport or higher)

A: Missing a required library.  On Ubuntu (12.10) type the command 'sudo
apt-get install libqimageblitz-dev'


I believe Okular is critically important to the survival of free
desktops alongside proprietary ones.  The ability to read documents in a
universal format (*PDF*) while respecting their integrity; to annotate
them; and to exchange these documents with annotations in a
standardized/agreed way across systems and organizational boundaries are
a critical functionality of an office machine.  Okular is, to my
knowledge, the best tool for the task in the free software world.


> A regular "Save" option has already been requested in bug 301774, so this 
> should probably be discussed there.

Thank you for pointing me to the bug report.  You will excuse me for
continuing the discussion here rather than there, for two reasons: (1)
that bug report is flawed; and (2) this discussion is about more than
just a mere functionality / bug report.

Let me first handle what is IMHO a flaw of the bug report.  Frankly (and
maybe this is a criticism of the KDE bug tracker) it is scary.  That bug
report (and an even worse example is bug report 151614) looks to me like
a thread in a web-based forum.  I had to read through flames and other
irrelevant stuff to distill the essence of the bug report.  This is bad
use of potential contributor time.  That bug tracker needs cleaning.
Anyway, with over 400 bugs, somebody should sift through that bug
tracker and clean it.

The first thing I would expect in a bug tracker is that the bug
description should be editable.  Does not seem to be the case on this
bug tracker, but it might be that I do not have sufficient access rights
to do so.  What is now the description would be comment #1 on the bug.
Whoever takes ownership of the report (usually the user who reports it)
should put a proper analysis in the body of the ticket, and that
analysis should be updated based on insights from the discussion thread
so that reading through the thread becomes optional.

That was easy.  Well, you could ask me to clean up the current bug
tracker, and I might agree and raise to the task, but there is a more
difficult issue to solve first.  Difficult one, because there are
trade-offs to be made, values and goals for the project to be decided.
You can decide them, I can't.

Reading the two bug threads, I see two conflicting type of universality:

*1* Document-Format Universality:
<https://bugs.kde.org/show_bug.cgi?id=151614#c23> stands for the
proposition that Okular needs the XML format as long as not all back-end
document formats support embedding annotations.  That means forever.
Even if all back-ends could process annotations, there will inevitably
be differences and incompatibilities.  How to add text or vector
annotation to a TIFF or a JPEG?  It is IMHO impossible to annotate all
document formats in the exact same way.  Okular's own XML format makes
sense because it enables to add a uniform annotation layer on top of
each document, no matter its original format.  This, however, comes at a
very high cost, since it is generally bound to the user's profile and as
such far from being "universal".

*2* User Universality:  For users, universality is the ability to share
documents with annotation across platforms and organizational
boundaries.  From this perspective, the cost imposed by document
universality are excessive since the result is that documents with
annotations are difficult to share even between two users of the same
machine (the XML annotations are stored in a user-specific folder!).

The question becomes: what is more important to you?  Universal support
of document formats or universal support of users across platforms?

<http://okular.kde.org/> states that Okular is universal in that it
works on multiple platforms.  IMHO this should not be read in a pure
technical way (i.e. works on Linux, Windows, Mac, etc.), but in a more
holistic way: it works for me and it works for you because we can both
access the same original document, in its integrity, and the comments we
each added to it.  The interchange is at the document level and that's
where universality should reside, IMHO.

Remember the Unix principle: single-purpose tools.  I don't need Okular
to support a plethora of document formats.  Some of them are nice to
have.  Some of them are plain redundant and a drag on development of
what IMHO is the key purpose of Okular: read and annotate PDF documents.
 For TIFFs and JPEG there is Gwenview (and if I wanted to annotate them
with Okular, I can use ImageMagick convert to embed them in a PDF).  ODF
is well taken care natively by LibreOffice (which can save to PDF as
well).  A bunch of the formats listed on
<http://okular.kde.org/formats.php> result in a drag on the one unique
proposition for which there is critical need that Okular seems designed
to satisfy: reading and annotating *PDF*.

I think I have already exposed too much of my own vision and it is not
up to me, an outsider, to express a vision for Okular.  It is up to you,
the insiders.  What do you want Okular to be?

I would recommend taking a hard look at what kind of universality you
want.  I would suggest defining three tiers of support for document types:
(1) fully supported
(2) partially supported
(3) not supported

Fully supported would be with embedded annotations ONLY (migrate from
XML to embedded annotation).

Partially supported would be read-only, or maybe with XML annotations.

Not supported: make a conscious decision that Okular does not add much
value to supporting the format; and that support for the format does not
add value to Okular either. Drop them.  Make Oklar leaner and focus your
limited resources on the top two tiers.

Personally I would put ODT, TIFF, and JPEG in (3).  And PDF and epub in (1).

Full support would mean *migrating* from XML annotation to embedded
annotation whenever possible.  It means stepping up to a better level of
support: now the annotation are truly universally available, as opposed
to only within Okular, and only within the one user account with the XML
file.

And to prevent disasters, Okular can (and should), on save, rename the
original document to something like document.timestamp.bak.pdf and save
the new document with the new embedded comments as document.pdf.
Without asking any question.  With an optional preference setting to
take risks and not to save a backup copy.

Depending on the view you take regarding the purpose of Okular, you may
like my proposal or you may find it heretically subvertive.  I am not
suggesting you should do it, I am just saying that this is how I would
do it if I had the time and the skill.


> A quick note about your SSD consideration: annotations are not the only things 
> we write to docdata/, we also write the viewport history (aka "restore last 
> seen page when you reopen a document") and filled form values.

I have seen this when analyzing the XML files.  I would suggest using a
temporary file and save from the temporary file to the persistent file
in regular intervals, configurable by preference.  If the preference is
0, keep the current behaviour (let the user make the trade-off between
work-preservation and SSD-preservation).  If the preference is -1, never
write to a persistent file (i.e. let the user write only when they go
for the Save function).  The assumption is that /tmp is in RAM disk
(where it should be for an SSD-only system).


>> Another functionality that would be helpful to me and I have not found a
>> way to get around:  I now have hundreds of marked up PDFs.  The
>> annotations are all in $(kde4-config
>> --localprefix)/share/apps/okular/docdata/ and I would like to export all
>> of my library of marked up PDFs into PDFs with embedded annotations.
>> Any idea how I could do this?
> 
> There's no automated way to do this. You have to open and "Save As" each file 
> separately.

Here is a user account in favour of a "migration path".  A month ago I
represented my school at a national mooting competition.  Mooting is
like going to court, except that the case being judged is not a real
life case.  As part of the preparation I had to write a legal analysis
that referenced 59 judgements.  These judgements I can download in PDF
form, e.g. from
<http://www.canlii.ca/en/ca/scc/doc/2011/2011scc52/2011scc52.html>.  Out
of that 49 pages document, I needed to cite only a few phrases in my
legal analysis, but I needed the whole document present, and annotated,
in case the judges ask questions.  I ended up with over 2000 pages,
annotated in Okular.  Some courts will require hard copies.  Imagine how
many trees I have to kill to print 2000 pages.  Not to talk about the
cost involved, and the logistics of air travel with the extra weight in
the luggage allowance.  I was lucky this competition allwed tablets.  I
bought a Samsung Galaxy Tab 2 (not the best tablet, but it was too late
to order a Nexus 10).  I had no time to look for Okular for Android, but
Adobe Reader is available and good enough (they seem to put more love
into the Android version than the Linux version).  It just ended up
being a lot of wasted time to load each single of the 59 documents and
saving them, paying attention each time to select the "Save As" and not
"Save Copy As".  This was for one month worth of work only.  I have
hundreds of commented PDFs and I will be wasting even more time, or
losing my annotations.  For most of these documents I had my own
summaries, written in LibreOffice.  It was very easy there:  soffice
--nologo --invisible --convert-to pdf *.odt

To complete the account: the use of annotated PDFs on a tablet proved to
be conclusively positive even if I made use of the sellers return policy
and returned the Galaxy Tab 2 back because it is in my view a bad
tablet.  I expect that at some point this will become a standard
delivery of documents to court litigation.

Why is the Galaxy Tab 2 a bad tablet (off-topic)?  Because of
proprietary connectors; because of the Samsung proprietary layer on top
of stock Android that is actually more an impediment than value added;
because of the glare in the screen (I had to fight against the
reflection of the neon lights from the ceiling); because the body itself
of the tablet glides when put on an inclined podium surface (it should
be made of non-slippery rubbery material, and it should not force me to
buy a case to achieve that objective); and last but not least because of
its pityful screen resolution.  The iPad with retina display, the Google
Nexus 10, or the Asus TF700T lead the way.  Sorry for the off-topic.


> This is bug 159601. I've started some work on it, check out the "configurable-
> review-tools" git branch. I've got a few more patches on my laptop waiting to 
> be polished and pushed :)
> You can find some (old) screenshots here:
>  - http://mail.kde.org/pipermail/okular-devel/2012-July/012047.html (look for
>    the attachments at the end)
>  - http://mail.kde.org/pipermail/okular-devel/2012-August/012054.html

Looks great!  Now that I built my own Okluar I would like to apply the
patches and play with them.  Unfortunately the mailing list archive
displays them inside the HTML which is rather inconvenient; and I do not
see the patches attached to the bug report.  Can you please forward a
zip file with the patches or share them in another convenient way? or do
I really have to copy&paste them individually from my web browser into
text files?

> Thank you for caring about us and Okular,
> Please let us know if you are willing to work on these issues, we always need 
> manpower :)

Thank you for being so welcoming.  I hope that I did not ruff any
feathers with my comments, especially in light of the sensitivities that
I start to understand based on what I saw in the threads for bugs 301774
and 151614.

At the risk of repeating myself, I believe there are no alternatives in
the free software world that are as good as Okular in matching or
improving on annotation functionality that is available on proprietary
desktops through Adobe's Reader.  It is vital to the survival of the
Linux desktop that Okular thrives.  I am willing to work on the issues
discussed above, or on reducing the bug tracker to a manageable
dimension.  I would suggest first a discussion amongst contributors
about Document-Format Universality vs User Universality.  I believe that
making clear which one of the two has higher priority will solve a lot
of the conflicts that manifest themselves in the less than harmonious
comments.  It is also necessary to prioritize bug reports.

In the short term, I need to finish writing an essay, prepare for and
write exams.  I hope to have more time available after exams, at the end
of April.  Thank you for welcoming me here, for encouraging me to
express my opinion, and for the time and consideration that you have
given to my writings so far.

Yuv

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 901 bytes
Desc: OpenPGP digital signature
URL: <http://mail.kde.org/pipermail/okular-devel/attachments/20130331/99698ec6/attachment.sig>