Some ideas for the aRts-replacement

[Dik Takken]

Hi All,

I have been following the discussions on this list about the future of
Arts in KDE, and have been thinking about what we could do.

Please read my idea about replacing the current sound API with something
new that is not Arts, not GStreamer or anything else.

This API has the following properties:

* Applications that just need a 'play this' command will get it.
* Applications that want to fiddle with equalisers, mixing and so on will
also find all they need.
* The API is a complete abstraction of the underlying technology. Using
Arts, GStreamer or just plain ALSA does not change anything for the
application programmer.
* The API and it's implementation can be made very flexible. When there is
no Arts or GStreamer available, it will still work.
* Give the user more control of what happens to sound output of
applications and what sound system is used by KDE
(Arts,GStreamer,whatever).


The API has two levels: one High Level interface and one Low
Level interface. The application may choose at what level it wishes to
talk to the sound system.

High Level API
---------------------------

The High Level API offers every KDE application a set of 'Sound Sockets'.
KDE applications can just plug in to a socket and output sound. What
happens to the sound depends entirely on the socket that the application
connects to. Sockets may apply filters to the output, record to compressed
files, stream the ouput over the internet, or just send it to the local
soundcard. There should be a central place in KDE where the user can
define different sound sockets for different goals/applications. Also,
every sound application should allow the user to choose which socket it
connects to. Possible socket definitions include:

App Output -> GStreamer Equaliser -> ALSA Output

App Output -> GStreamer OGG Encoder -> GStreamer ICECast Output

App Output -> Splitter -> ALSA Output
                       \-> LAME Encoder -> Write to harddisk

App Output -> Splitter -> ALSA Output to Front Speakers
                       \-> Reberb -> ALSA Output to Rear Speakers

What technology de socket depends on, depends entirely on the definition
of the socket. You can define a simple socket that plugs right into
/dev/dsp. It will work even if you have nothing else but OSS/ALSA
installed. You can also define a socket that uses GStreamer and it's
plugins. The underlying sound system is 100% decoupled from the Sound
Socket API and it's implementation.

The High Level API is the easiest to implement and can easily be
supported on any system, even if Arts or GStreamer does not support that
particular system. In that case, there might be only one socket for
applications to choose from, but sound will still work. On systems that
do have Arts/GStreamer/Whatever KDE applications will automatically be
able to use their features.

Low Level API
----------------------------

The low level API should allow applications to define their own sockets.
It will be the more complex part of the implementation. Applications
should be able to ask for a list of possible sound processing
capabilities, create a sound processing chain and register it as a sound
socket. Each processing element ('plugin'?) in the chain has properties
that can be controlled by the application. For instance, an application
could add an equaliser to the chain and create a GUI for the user to
adjust the frequency gains real-time. If this equaliser is done by
GStreamer or some other technology does not make any difference to the
application.

The low level API may have to report that there are very few plugins
available on systems that don't have Arts/GStreamer/etc. If a particular
application can live with that or not is entirely up to the application
itself. This is a disadvantage compared to the high level API, which will
never fail (unless there is no sound card, but even then you could have a
/dev/null socket :-) )

Input Sound Sockets
----------------------------

Besides output sockets, the API could also support input sockets. The
sources for the input sockets can be defined globally by the user. What an
applications listening for sound will 'hear' depends entirely on the input
socket that it connects to. Again, the user should be able to make that
choice in every application. Possible sources for input sockets may
include:

- The local soundcard
- The output of another Sound Socket
- A file from the harddisk
- A sound stream generated by a KDE application on another machine

This will allow an application (or the user!) to create a digital radio
station by defining the following chain (Assuming that all used
components are available in the underlying subsystem):

- Microphone Input        -> Volume A \
- Input from MP3 player A -> Volume B -> Down Mixer -> Shoutcast output
- Input from MP3 player B -> Volume C /

A few more notes:

Most sound processing can be done by software from external projects. We
only need to direct how this external software is used. No need to write
a full-blown in-house 'sound-server' again.

The idea of Sound Sockets can be extended to Video Sockets, especially
when GStreamer joins the party. It would create the possibility to easily
add video processing capabilities to KDE applications.

Is this what we want/need?

Cheers,

Dik