KDE Sound and Multimedia Plan

[George Kiagiadakis]

On Wed, Dec 1, 2010 at 2:11 AM, Alex Fiestas <alex at eyeos.org> wrote:
> This is my list of needs, what is yours?

Let's see...

I think we should take this to types of applications.

The first type of application here is the simple player (amarok,
dragon, knotify). We already have phonon for this type of application
and I think it's great.

Then, we have more complex playback needs, for example in games. Games
need to be able to track several channels with different sounds
playing in parallel with different patterns (one with some effect, the
other on repeat, the other just when the player moves, etc...). They
also need to be able to position audio in 2D or 3D space, in which
case they probably need OpenAL, and we've already seen kde games
taking that path, first  gluon, now granatier...

Then we come to capturing. Capturing audio is one possible case. I
don't know if we have any application doing that at the moment, but a
sound recorder would be nice to have. This however can be extended to
more complex usage than just simple capture and save. (i.e. capture
audio, transform it, add it some effects, encode it in some format,
stream it on the network, do speech recognition, etc...).

Capturing video is similar. There is the simple capture + show/save
and the more complex capture, add effects, show, take snapshots,
encode (perhaps adding audio too at this point), stream it on the
network, apply face/object recognition algorithms, etc... One complex
scenario is what kamoso is doing right now. Note that doing a
screencast (capturing video from the desktop) also falls into this
category imho (at least, in gstreamer it is the same thing, just swap
v4l2src with ximagesrc for example in kamoso and you make a
screencast!)

After that, we have the all-in-one use case the combines all of the
above: VoIP. A VoIP application needs to capture audio and video,
possibly add effects, show preview of the input video, transform,
encode, stream on the network and in parallel receive from the
network, decode, playback and in more complex cases also pipe all the
audio/video inputs/outputs into an encode bin to record your call.

Next come the audio/video editors. Complexity increases even more
here. I'm not sure exactly what their needs are, but I'm sure they are
even more demanding than voip.

Also, audio/video converters. These are simple enough, just open,
decode, encode, save.

Then, CD/DVD rippers. These are also quite simple (I use gst-launch on
the command line to rip audio CDs for example, which proves that a
simple pipeline can do it).

And... I think that's it. There are also the possible combinations of
the above and maybe something else that I can't think of right now.

Regards,
George