T13545: Resurrect the Web Archiver

[Jonathan Marten]

marten created this task.
marten added a project: Konqueror.

TASK DESCRIPTION
  This refers to the "Archive Web Page" plugin previously available in Konqueror.  It was removed in 2019 because it relied on KHTML DOM access and could not be ported.
  
  It was useful, though, so I've looked at bringing the web archiver function back without having to depend on KHTML or indeed whatever KPart is being used.  It does this by having a very small Konqueror plugin that simply passes the page URL to an external program.  After putting up a dialogue for configuration options, it then passes the URL to wget in its 'page and requisites' mode, which downloads the page and all images, stylesheets etc. that it requires.  The downloaded page is then saved in the original web archive format, or optionally a tar or zip archive or unpacked in a directory.  No access to the part's DOM is needed, and only the wget command is needed as a runtime dependency.
  
  I've also brought back the web archive thumbnailer, using either WebEngine or WebKit as the renderer.  WebEngine is the supported default option but is not 100% reliable (depends on a timeout) due to its asynchronous loading.  WebKit is unsupported but still works.  Neither of these depend on any KPart or other framework, they use QWebEngine/QWebPage only.  This can generate thumbnails for web archives and also HTML files.
  
  So the question:  is this worth putting back into Konqueror, and if so in what form?  At the moment all three of the major components (plugin, archiver program and thumbnailer) are where they were originally in the Konqueror plugins/webarchiver directory.  Only the first of these really needs to be in the Konqueror source tree, though, and the archiver could be useful as a standalone command (kcreatewebarchive <URL>).  So should everything stay within the Konqueror source, or could there be three components:
  
  - plugin - remains in Konqueror source
  - archiver - as a separate repository (in the 'network' category?)
  - thumbnailer - as a separate repository (in kdegraphics?)
  
  The desktop group and those interested in Konqueror are added as subscribers.  Any thoughts would be appreciated.

TASK DETAIL
  https://phabricator.kde.org/T13545

To: marten
Cc: dfaure, stefanocrocco, #konqueror, #plasma, marten, akilgus, rrosch, domson, demsking
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.kde.org/mailman/private/kfm-devel/attachments/20200828/e26babb7/attachment.htm>