[Kde-hardware-devel] [GSoC] Preliminary UPnP support proposal

[Tuomo Penttinen]

Bonjour,

On Mon, April 26, 2010 12:11 pm, Friedrich W. H. Kossebau wrote:
> Terve,
>
> Dimanche, le 11 avril 2010, à 19:08, Tuomo Penttinen a écrit:
>> Hello all,
>>
>> On Fri, April 9, 2010 4:20 pm, Friedrich W. H. Kossebau wrote:
>> > Vendredi, le 9 avril 2010, à 03:15, Tuomo Penttinen a écrit:
>> >> On Wed, April 7, 2010 3:09 pm, Friedrich W. H. Kossebau wrote:
>> >> > Mardi, le 6 avril 2010, à 23:41, Kevin Ottens a écrit:
>> >> >> On Wednesday 31 March 2010 14:42:03 Tuomo Penttinen wrote:
>> >> >> > I agree. The UPnP discovery protocol is lightweight enough that
>> >> >> > I wouldn't worry about it in most environments.
>> >> >
>> >> > Emphasis on _most_ environments ;)
>> >>
>> >> Indeed. ;)
>> >
>> > And add "today" ;)
>>
>> Well, if it isn't a problem today, I'd say it's less likely to be a
>> problem tomorrow, right? ;)
>
> With more and more mobile devices and especially in the same local
> networks, I'd say, well, wrong :)

Are you suggesting it is an increasing trend that more and more devices
implement the IP stack in full, but yet they are unable to handle a
protocol such as SSDP and the situation is only getting worse? I sure hope
that is not the case. ;)

>> >> Second, in the worst case the registrar would have to inform every
>> >> registrant upon receiving a SSDP message via some IPC mechanism
>> >> anyway.
>> >
>> > Yes, but just the worst case (really, what kind of registrants would
>> > be interested in all devices besides device browsers?).
>>
>> And registrants who just don't care or want more control and want to
>> make the decisions by themselves.
>
> Registrants like what?

Well, it could be any software that for one reason or the other doesn't
follow the best-practises and guidelines set by the designer of the
system. Especially if the failure to follow the guidelines does not cause
noticeable problems.

>> Device sniffers and validators probably
>> belong to this category as well.
>
> Special cases, no?

Yes, but my point was that you probably shouldn't consider the "worst
case" in this matter as something highly unusual. Especially if you
consider how UPnP is usually used. At least to me it seems that in most
networks UPnP is used for A/V stuff and controlling routers. In these
types of networks someone interested of UPnP is actually interested of the
majority of the UPnP devices available.

>> >> Third, UPnP discovery is lightweight compared to the description
>> >> phase, action invocation and eventing, so I'm not sure caching the
>> >> results of discovery alone will have any meaningful impact on a large
>> >> scale. I say this because if an application is interested in SSDP
>> >> messages it is probably interested of UPnP in general, which includes
>> >> action invocation and eventing. You can't really cache these and
>> >> incidentally these two are potentially far worse resource consumers.
>> >
>> > This doesn't stop SSDP from being outsourced to a central process with
>> > some gain.
>>
>> No, but my point was that optimizations are best targeted to issues that
>> measurably matter. Inlining a call to a function that performs a
>> bubblesort doesn't change the fact that the algorithm is still
>> quadratic. ;)
>
> Sure, but here UPnP = bubblesort, and Cagibi approach = inlining, no?

Well yes, if Cagibi approach is about caching the SSDP messages only. Then
again, we haven't established yet the utility of SSDP caching in one way
or the other.

> So let's replace UPnP ;)

If you mean that let's remove UPnP altogether, then yes, that would work. ;)

> [snip content="some useful info about SSDP"]

[snip content="some discussion of Solid"]

> Disclaimer: I am not yet a real Solid expert.
> Solid as we discuss it here is basically one single hardware abstraction
> abstraction library (tm). It wraps the operatingsystem specific hardware
> access libraries with a common Qt-style API, to achieve the "code once,
> compile everywhere".
> So e.g. if you are interested in the status of the network (like KMail or
> Konqueror) you use and link to the Solid lib.
> Or if you want to list the attached storage devices (like indirectly all
> programs using a filedialog, so almost all), you query and link to Solid
> for these devices.

Okay, how does Solid wrap these hardware access libraries? Are the
libraries loaded in separate processes or does Solid link to them?
Regardless of the answer, shouldn't UPnP be handled as all the others no
questions asked? I ask this purely from the standpoint of system design.

In addition, what is Solid supposed to do with UPnP? What kind of "access"
/ UPnP features are needed / wanted? Is this Solid UPnP back-end supposed
to be generic enough that it can be used outside Solid too? If so, is it
your intention that it is used like that?

> So, unless Kevin or someone else implements loading of backends
> on-demand-only doing the UPnP backend for Solid with a full UPnP library
> means indeed almost all programs would link against that.
>
>> >> > Currently KDE software basically will be a client to services from
>> >> > UPnP devices (being control point in UPnP terms). So if there is a
>> >> > convenience lib one should be just for client stuff IMHO. Server
>> >> > stuff should be handled by a different lib. I suppose that code for
>> >> > server stuff is larger and would just be unneeded payload for most
>> >> > applications (also in disk size). (not sure how the P2P situation
>> >> > with mobile devices proves me wrong here)
>> >> >
>> >> > We also wouldn't put http server stuff into the http access lib,
>> >> > would we?
>> >>
>> >> Actually, that's not entirely true. UPnP eventing requires a control
>> >> point to listen for asynchronous events published by UPnP devices.
>> >> The protocol is called "GENA", which is layered above HTTP and it
>> >> requires minimal HTTP servers on both sides. In addition, I wouldn't
>> >> say that the server stuff is larger or more complex compared to a
>> >> proper control point. In many ways a UPnP device is actually more
>> >> straightforward to implement compared to a control point that
>> >> implements the UPnP stack in full and provides some type of an API
>> >> for users.
>> >
>> > Now, I would also put the eventing stuff into a dedicated proxy
>> > process, if only for firewall and security reasons. Or am I the only
>> > one to consider it a less good idea to have a full UI program with my
>> > user rights accessable from the network?
>> > Just that I have no real clue yet how this could be done best. Due to
>> > authorization stuff there possibly should be one central process per
>> > user, not globally.
>>
>> This is another very interesting topic for discussion. :) I'd say it is
>> about UPnP security (or lack of it) in general, not just about listening
>> sockets. I think it is fair to say that the UPnP base design is not
>> secure. There are the DeviceSecurity and SecurityConsole device
>> templates to address at least some of the more prevalent security
>> issues, but the base architecture is not secure. Because of that the use
>> of UPnP without additional security measures in a public network is an
>> inherent security risk no matter where you offload the socket code.
>>
>> So before getting too carried away with security issues related to UPnP,
>> I think it would be important to define the use-cases and requirements to
>> find out how you are really going to approach the UPnP world. These
>> should help in defining the security requirements and responsibilities
>> of the system. If the requirements define proper security as an important
>> system design attribute and that it is something *you* are responsible
>> for, you're going to need whole lot more than the aforementioned
>> offloading of socket code into a presumably safe process.
>
> So what have been the use cases you designed and developed HUPnP for so
> far?

It is a Qt library, a collection of classes, which purpose is to enable
UPnP connectivity according to the UDA v1.1. In that context the
"use-cases" are about the API and how it is supposed to be used to get
something done. For example, "what classes to instantiate and what methods
to call to discover a device of certain type and to subscribe to its
events". Everything beyond that level is for the users to decide and
hopefully HUPnP scales well for different purposes.

>> >> The HUPnP shared library is about 1.2 megabytes built on my 64 bit
>> >> kubuntu machine. I haven't optimized for the binary size yet, so
>> >> there could be some leeway. The object codes for control point and
>> >> upnp device functionality are roughly equal in size and they actually
>> >> share quite  a bit of the code base, including the "device model"
>> >> users code against on client and server side. Taking purely the
>> >> server stuff out would probably yield a save of 300-350 kilobytes.
>> >> Certainly, purely UPnP device (server) stuff isn't required at a
>> >> client that is purely a control point. On the other hand, there are
>> >> valid use-cases where a client application will need both the server
>> >> and client code.
>> >
>> > Poses the question if it is a good idea to implement both controlpoint
>> > and device in the same instance. Again for firewall and security
>> > reasons I would run the server stuff in a separate program/process.
>> > Don't you think this is a valid concern?
>>
>> As noted above, I don't think security in this case is this simple. But
>> since I find data/information security fascinating, I can't help but
>> bite and forget that. ;)
>>
>> So, I'm guessing you mean there's more attack surface when client and
>> server code are run in the same process? Basically you're indicating
>> that it increases the possibility for an attack, where the exploitable
>> bug is either in client or in server, but the input that triggers the
>> control flow leading to that would come in from the other? I admit, this
>> *could* be more prevalent when the two are in the same process, but
>> there are no guarantees that the malicious input can't reach the
>> exploitable code even if the two live in different processes.
>> Furthermore, separating the two as described definitely incurs a
>> performance + complexity overhead, which might matter.
>
> The latter is a price for more security. The idea with different processes
> would be that the server process would be run as a low-rights system user,
> so the impact of an exploit is reduced.

Yes, the common practise for any service that can be run with the said
privileges. However, in UPnP the "server" (UPnP device) often requires
much more than that to be operable. For instance, a MediaServer requires
at minimum read access to some type of storage medium where the media can
be found. But it could also require write access, since the specification
details actions that depend on it. Or if you consider MediaRenderer, which
has to display images and play video and audio. Or the
InternetGatewayDevice, which has to poke holes into a firewall.

However, I'm not saying you cannot or shouldn't come up with a design
where the devices, control points and user code are all somehow separated
at process-level. That kind of structure could certainly provide better
security in terms of preventing exploit code doing damage. But it doesn't
come cheap and you really have to wonder if it's worth it, because at the
end of the day you're still left with an architecture that by default
provides no means for authentication and authorization and uses plain-text
everywhere.

All in all, I'm trying to answer to this: "Poses the question if it is a
good idea to implement both controlpoint and device in the same instance"
that yes, sometimes, as I think there are valid use cases for that. The
combination of a MediaRenderer and a control point comes to mind. And to
this: "Again for firewall and security reasons I would run the server
stuff in a separate program/process. Don't you think this is a valid
concern?" that it could be a valid concern in some situations, but that
still does nothing to improve the non-existent security of the UPnP
architecture while it makes the design and implementation of the software
more difficult.

I'd avoid running UPnP in a possibly hostile network altogether, unless
I'm sure the UPnP devices available are for public use and communication
with them doesn't expose any information that could be used maliciously.

[snip content="some discussion of the size of a shared library"]

>> But this is where opinions start to fly. Some
>> want shared libraries small, some a bit bigger and some even very large.
>> Consider any Qt library for instance. When you link to QtGui you rarely
>> (if ever) make full use of it, but it still makes sense to have a single
>> shared library instead of ten, even if it is a bit bigger and contains
>> "bloat" to you in some case. I could throw in a joke or two regarding
>> one's "preferences" concerning size, but I probably should not. ;)
>
> I try not to imagine what you were thinking of ;)

Awww, your loss, you missed "a lot". ;)

[snip content="some discussion of the benefits of a central UPnP service"]

>> These benefits are noteworthy and I personally find the idea truly
>> interesting (and quite possibly worth implementing), but I'd like to
>> raise some of the disadvantages here as well, since they may very well
>> outweight the benefits depending of the desired use:
>>
>> * Possibly significant increase of complexity. Now, I don't consider
>> writing such a central service a problem. It's basically a really robust
>> control point that offers an IPC interface for clients use. I do
>> consider using such a service "problem". A problem in a sense that it
>> is much more difficult to use compared to a library loaded in-process,
>> which provides a type-safe, hopefully very usable object model for
>> interacting with UPnP  devices. Certainly you could write a helper
>> library for the clients that does exactly that: a decent object model
>> for interacting with the central service. It could even be written in Qt
>> to allow seamless integration with all the other Qt stuff.
>
> Sure, and I would expect a complete solution for us to contain this.
>
>> Regardless, all that definitely increases
>> complexity in various ways. I'd say the impact as a whole is notable.
>
> Numbers, please ;)

I'm sure you know that analyzing the complexity of yet unwritten code and
its ramifications to other systems is a very hard thing to do. ;) And I'm
sure you agree that a system that uses this central service approach is
more complex than a system that does not. I doubt anyone can give you
figures here, but on one hand you have a system in which you need only a
UPnP library to do UPnP. On the other hand you have a system that requires
an extra service, a UPnP library and a helper library that can communicate
with the extra service to do UPnP.

I'm trying to say that in my opinion designing, building and maintaining
this central service and the helper library for it on each supported
platform is a non-trivial thing to do and this is not required with the
other approach. Obviously the level of increased complexity depends of the
capabilities of the central service. If it truly is a full-fledged control
point that supports multiple simultaneous users I'd say (even without the
numbers) the increase is significant. I'm just thinking what it would
require to build an IPC interface for the functionality the HUPnP's
control point provides and make the control point usable by multiple
concurrent users. Then again, if the central service provides only SSDP
services we're back on square one where I still doubt the usefulness of
the service.

>> * Writing multiplatform software becomes harder. I'm not sure if this is
>> a valid concern or not, but having a dependency to such a central service
>> requires the service to be present on all supported platforms as well.
>> It is no longer a matter of writing an application and linking it to a
>> UPnP library.
>
> Well, the UPnP library ideally hides the complexity/difference of the
> supported platforms away in the API. I guess you also are targetting
> other platforms like Windows. Don't they already provide their own libs
> for that, which you ideally would wrap around with the same HUPnP API?
> So you would have different backends, choosen by compile flags. And for
> Linux/*BSD/... you would offer a compile flag to have the SSDP classes
> forward to Cagibi or a similar daemon.

I'm not sure I follow here. I thought your central service would be the
one using some UPnP library, not the other way around. In any case, at the
moment HUPnP doesn't wrap anything. It provides a native C++ UPnP stack
using Qt in the interface and besides Qt itself it depends only on QtSoap.
And yes, it is cross-platform. I'm currently targetting Linux, Mac OS X
and Windows, but I see no reason why the code wouldn't work on other
platforms where Qt works as well. That's because besides some very small
differences in UDP multicast code it's the same Qt code base on all
platforms.

To change this approach you'd have to convince me why would I introduce
dependencies to external systems when I already have a cross-platform
library that is perfectly capable of doing all that by itself. And to
qualify optimization as a reason to do such a change you're going to need
some really convincing numbers along with a procedure that I can use to
verify them. ;)

>> * Efficiency may be impaired in regard to action invocation. I don't
>> think this is a concern, but since the topic has been on the wall,
>> I thought I should mention this. I don't have any numbers to show
>> right now, but it is easy to assume (and be wrong ;)) the vast majority
>> of resources are spent on invoking actions and eventing. This is
>> because to do anything with a UPnP device you have to invoke an
>> action and it is SOAP all the way. Always going through a middle-man
>> incurs overhead even if it only relays the data.
>
> Which I do not advocate for. Instead I think SOAP talking should be done
> directly, like HTTP is often done (then perhaps it could be an idea to
> have a proxy process per remote UPnP device, similar to KIO slave
> processes).
>
>> * If the majority of the benefits of the system depend on everybody
>> using it, how can you enforce the use of it in favor of, say, using HUPnP
>> directly?
>
> Sorry, do not understand what you mean here?

This comment of yours and my previous comments together probably hint that
there might have been a slight misunderstanding on my part. ;)

I thought that you were aiming for a system, which provides this central
service concept, which is a type of a UPnP control point (but even more)
that should be used by any software wanting to interact with UPnP devices.
This central service in turn would use a UPnP library to provide the means
for that. But, I guess you were not thinking that far and that actually
explains some things in retrospect. :-)

However, the argument still applies; if the majority of the benefits of
this central service concept depend on everybody using it, how can you
enforce the use of it? For instance, I'm not too keen on changing HUPnP to
use the service without solid evidence of the usefulness of the change.

>> All in all, this is a very interesting field of discussion and I'm glad
>> to participate, but I must point out that I don't have such a good idea
>> yet what exactly are you planning to do with UPnP, who and what are
>> involved and so on. So, everything I just wrote could very well be
>> pointless, which of course makes me feel very good about spending a fair
>> amount of time and thought in writing this. ;-)
>
> It is of much value, at least to me, as I have to defend my idea with
> Cagibi and see if it stands.

That's good to know. :)

Regards,

Tuomo