[GSoC idea] QML/QGraphicsScene Video Painting

[Trever Fischer]

> Hello everyone,
>
> My name is Harald, I am student in Austria and I'd like to pick up on
> the Phonon idea "QML/QGraphicsScene Video Painting" [1] for GSoC 2011
> ;)
> I have been student for Ubuntu last year (also a KDE related project)
> and would like to work on KDE technology this year. I have a bit of
> experience with C++/Qt, GStreamer, libvlc and QML, didn't do any GL
> stuff yet.
>
> I think this is a very important project as QML is becoming more
> important and creating good looking video players is not very easy
> with QWidgets and the usually used overlay painting for videos.
> Additionally with work being done on the Plasma Media Center a whole
> new use case arises that deserves good support for the sake of user
> experience (unlike what QGraphicsProxyWidget provides).
>
> I have been looking into this topic quite a bit and noted the
> following requirements:
> a) must be as fast as possible, no one likes choppy video playback
> b) must use as little CPU as possible, since QML is a target on
Well don't leave us hanging, bro.

> c) must be maintainable ;)
> d) must work everywhere
>
> If you have any further thoughts, I really appreciate input :)
>
> From above requirements I concluded the following:
> Regarding a) and b): painting using opengl should be preferred as
> doing it using QImage and friends is rather heavy on the CPU.
> Regarding c): as QML (at least currently) is based on the
> QGraphicsScene magic an implementation should be done for a
> QGraphicsItem (or rather QGraphicsObject) and then just wrapped by a
> QDeclarativeItem.
> Regarding d): at least GL and raster painting should be implemented,
> most mobile devices will have half-way decent GL support, so we can
> compensate for the (usually) weaker CPU. On a desktop we have stronger
> CPU, so even without GL it should be possible to provide sensible user
> experience. Latter will require field tests though, as I am not
> entirely sure running a video at 1600xSomething fullscreen will still
> work all that well.
This more often than not has to do with the fact that the video needs
shrunk down to a screen that is 1/3 the size. If the video is a reasonable
size, it might be possible to leverage VDPAU in the backend somehow:
http://www.opengl.org/registry/specs/NV/vdpau_interop.txt
>
> Painting in a QGraphicsItem is done by overloading the pure virtual
> paint() function, which among other things takes a QPainter as
> parameter. This helps a great deal with supporting multiple painting
> methods (GL, raster, whatnot). Please have a look at [2] for the
> general idea.
> We'd have the QML element which is really based on the QGraphicsItem.
> The QGraphicsItem uses a (private) QPainter implementation that will
> use a well defined interface to get the actual painters to paint
> stuff. The painters would be implemented in the backend.
>
> An alternative approach would be to enhance the existing experimental
> VideoDataOutput class to work fast enough for live playback, in which
> case we would have the painters in libphonon and grab data via the
> enhanced interface for VideoDataOutput.
>
> Both options have advantages and disadvantages, while using
> VideoDataOutput would make supporting the new QGraphicsItem very cheap
> from a backend POV (as both GStreamer and the VLC backend have sort of
> working implementations already), it withholds the backend from
> further optimizing the painting in what ever ways the underlying
> libraries might support. For example I could imagine that in
> phonon-gstreamer we could have a gstreamer plugin directly in the
> gstreamer pipeline and have it act as painter.
> Having the painters in the backends might however lead to duplicated
> code (unless I find a way to do the general stuff really nifty in an
> abstract base class in libphonon).
>
> Of course there is no reason why we should not do both, surely
> VideoDataOutput could use some improvements anyway. Therefore I'd like
> to propose starting off with having the painters in libphonon (private
> of course) so we get a working solution ASAP and then move on to
> providing the backends with the ability to have the painters built in
> (if that should be necessary/desired at all).
>
> What do you think?
+1. Sounds like an interesting project, can't wait to meet you in IRC.
>
> [1]
> http://community.kde.org/GSoC/2011/Ideas#Project:_QML.2FQGraphicsScene_Video_Painting
> [2] http://i.imgur.com/L8vyr.png
>
> best regards,
> Harald
> _______________________________________________
> kde-multimedia mailing list
> kde-multimedia at kde.org
> https://mail.kde.org/mailman/listinfo/kde-multimedia
>


-- 
Trever Fischer (tdfischer)
Fedora Ambassador, KDE Hacker
http://wm161.net
GPG: C40F2998 hkp://wwwkeys.pgp.net