[Kde-hardware-devel] [GSoC] Preliminary UPnP support proposal

[Friedrich W. H. Kossebau]

Vendredi, le 9 avril 2010, à 03:15, Tuomo Penttinen a écrit:
> Hello all,
> 
> On Wed, April 7, 2010 3:09 pm, Friedrich W. H. Kossebau wrote:
> > Hi all (and Tuomo),
> > 
> > Mardi, le 6 avril 2010, à 23:41, Kevin Ottens a écrit:
> >> On Wednesday 31 March 2010 14:42:03 Tuomo Penttinen wrote:
> >> > I agree. The UPnP discovery protocol is lightweight enough that I
> >> > wouldn't worry about it in most environments.
> > 
> > Emphasis on _most_ environments ;)
> 
> Indeed. ;)

And add "today" ;)

> >> Actually I like numbers. :-)
> >> 
> >> My worry lies in the fact that we're going for the full game of having
> >> it in libsolid, which means that such a discovery will be triggered by
> >> any apps which starts and ask for a StorageAccess (which means at least
> >> anything which spawns a file dialog).
> > 
> > And slightly worse. Any SSDP message coming in will wake up all of the
> > processes which listen for these, just to find out if it is interesting
> > to them or not. Minor case, but still burns ressources (think of mobile
> > uses) and should be avoided as a standard pattern.
> 
> So you would replace the aforementioned "wakeup of all" with some type of
> central registration mechanism? This system would enable processes to tell
> the registrar which messages / resources they are interested of and in
> turn the registrar would inform them upon reception of desired message
> types?

Exactly. I could also envision a trigger system to have the central system 
(like Cagibi) additionally start dedicated handlers if certain devices show up 
or disappear.

> Fair enough, but to me it would seem that maintaining and using such a
> registry comes up with a price, which justification still isn't so clear
> to me. Of course, it doesn't have to be, but I find this interesting
> enough to comment. :-)
> 
> First, it's not entirely straightforward to pull numbers in this matter to
> show that the registration mechanism is so much more efficient that more
> complex code and a dependency to an external service are warranted.

I don't think the code is that more complex. Old and known patterns, did it in 
a few days ;)
External service: yes. But the KDE platform is full with these, so one more is 
not really a problem per se.

> Second, in the worst case the registrar would have to inform every
> registrant upon receiving a SSDP message via some IPC mechanism anyway.

Yes, but just the worst case (really, what kind of registrants would be 
interested in all devices besides device browsers?).

> Third, UPnP discovery is lightweight compared to the description phase,
> action invocation and eventing, so I'm not sure caching the results of
> discovery alone will have any meaningful impact on a large scale. I say
> this because if an application is interested in SSDP messages it is
> probably interested of UPnP in general, which includes action invocation
> and eventing. You can't really cache these and incidentally these two are
> potentially far worse resource consumers.

This doesn't stop SSDP from being outsourced to a central process with some 
gain.

> > Though HAL/Solid isn't any better here, both also don't offer any kind of
> > active search/query (cmp. view in database), if I looked correctly
> > (DeviceNotifier reports simply about all devices), so wake up all of the
> > processes.
> > 
> > Still no numbers. And these might be small. But all those small numbers
> 
> add up :)
> 
> Indeed they do and I'm not saying that you shouldn't consider the little
> details. I'm just not sure if devising a cache mechanism for UPnP
> discovery will provide any true benefit.
> 
> In any case, here's some numbers and background information about the UPnP
> discovery for the interested.
> 
> To discover all UPnP devices on a network a control point usually sends
> 1-3 UDP messages to the 239.255.255.250 multicast address. A single
> message is enough, but more may be sent due to the unreliable nature of
> UDP. Each UPnP root device in turn should respond *directly* to the
> control point with 3+2d+k UDP messages, where 'd' is the number of
> embedded devices and 'k' is the number of distinct service types within a
> device tree. Often UPnP devices do not have any embedded devices and
> contain only a few services. But again, the datagrams may be sent more
> than once due to the unreliable nature of the protocol.

Does it really have to be 3+2d+k UDP messages? For Cagibi I just query for all 
"urn:schemas-upnp-org:device:upnp:rootdevice:1", listen to the 
answers/notifications, read the device description from the given location and 
should have a complete view of everything, no?

At least I remember someone once teached me that UPnP devices are static in 
their setup, so only whole UPnP (root) devices come and go, not also embedded 
devices and services individually. Is that wrong?

> On the other hand, a UPnP device has to announce itself to the network
> using the said multicast address when it becomes online. After that it has
> to repeat the announcements periodically before they expire until it goes
> offline, which it also has to announce. A root UPnP device announces
> itself, its embedded devices and distinct service types similar to the
> response it sends to a discovery issued by a control point. The interval
> between the repeated announcements seems to be quite often something
> around 5-15 minutes, although the UPnP device architecture specification
> instructs to use values greater than or equal to 30 minutes.

Caring for the (repeating) announcements/responces of the locally active UPnP 
device implementations is also a task I see for Cagibi (no need to wake up all 
processes offering UPnP devices for an alive-update or discover-answer). Bonus 
for still sending byebye for crashed device processes ;)
There could be also support for not running instances, so Cagibi would all the 
time report about the existance of the UPnP device, the program itself is only 
started if some controlpoint starts to talk to it. And then can suspend itself 
again after idling a while.

> A typical discovery request is around 125 bytes in size and a typical
> response to that or an "alive announcement" is between 300 and 400 bytes.

The response is done randomly in a time span of 0 to max (e.g. 3, as given in 
the request) seconds, right?

> It is important to note that control points should perform discovery only
> when they enter network, since devices are required by the UDA
> specification to announce their presence as outlined above.

With a central proxy/cache like Cagibi this also would improve, as discovery 
is only done once at the start of Cagibi, not on every start of every program 
with interest in UPnP (like there is by the Solid UPnP backend).

> >> > In my opinion, even the description
> >> > phase doesn't add that much of network load that I'd want to pay the
> >> > price of added complexity, which the inter-process caching solution
> >> > introduces.
> >> 
> >> Fair enough, let's avoid premature optimization. And it's all buried in
> >> the backend anyway so it's not like we're going to break BC later on if
> >> we introduce complexity for the optimizations.
> 
> My thoughts exactly.

Well, having done some work already in this area I don't think this is 
premature now. Remember Solid is linked to almost every KDE program. Which 
means the backend to integrate UPnP stuff is active in many processes, even 
those not interested in UPnP devices. And for the UPnP backend to work I have 
to have a complete cache of all UPnP devices there.  When I played with Cagibi 
as a lib for the Solid UPnP backend I had to link Solid also to QtNetwork. 
With Cagibid as a daemon I don't. And I also do not need to maintain a local 
cache, just like the HAL backend on demand relays requests to the HAL daemon 
the Cagibi backend relays requests to the Cagibi daemon. The code has become 
even smaller compared to before.
Also for the network:/ kio-slave this is all that is needed. It just cares for 
the description and the presentation url, never interacts with the devices 
itself. The overhead for UPnP listing this way is just around a hundred lines 
of code altogether.

> > Currently KDE software basically will be a client to services from UPnP
> > devices (being control point in UPnP terms). So if there is a convenience
> > lib one should be just for client stuff IMHO. Server stuff should be
> 
> handled
> 
> > by a different lib. I suppose that code for server stuff is larger and
> 
> would
> 
> > just be unneeded payload for most applications (also in disk size).
> > (not sure how the P2P situation with mobile devices proves me wrong here)
> > 
> > We also wouldn't put http server stuff into the http access lib, would
> > we?
> 
> Actually, that's not entirely true. UPnP eventing requires a control point
> to listen for asynchronous events published by UPnP devices. The protocol
> is called "GENA", which is layered above HTTP and it requires minimal HTTP
> servers on both sides. In addition, I wouldn't say that the server stuff
> is larger or more complex compared to a proper control point. In many ways
> a UPnP device is actually more straightforward to implement compared to a
> control point that implements the UPnP stack in full and provides some
> type of an API for users.

Now, I would also put the eventing stuff into a dedicated proxy process, if 
only for firewall and security reasons. Or am I the only one to consider it a 
less good idea to have a full UI program with my user rights accessable from 
the network?
Just that I have no real clue yet how this could be done best. Due to 
authorization stuff there possibly should be one central process per user, not 
globally. 

> The HUPnP shared library is about 1.2 megabytes built on my 64 bit kubuntu
> machine. I haven't optimized for the binary size yet, so there could be
> some leeway. The object codes for control point and upnp device
> functionality are roughly equal in size and they actually share quite a
> bit of the code base, including the "device model" users code against on
> client and server side. Taking purely the server stuff out would probably
> yield a save of 300-350 kilobytes. Certainly, purely UPnP device (server)
> stuff isn't required at a client that is purely a control point. On the
> other hand, there are valid use-cases where a client application will need
> both the server and client code. 

Poses the question if it is a good idea to implement both controlpoint and 
device in the same instance. Again for firewall and security reasons I would 
run the server stuff in a separate program/process. Don't you think this is a 
valid concern?

> For instance, there could be a device or
> an application that is both a media renderer (a UPnP device) and a control
> point. The control point fetches the content from a media server and uses
> the built-in media renderer to render the content.

The program implementing this kind of UPnP device simply links to both libs, 
control-point and device-impl. But this kind of device is an exception, not 
the rule, isn't it? I expect most UPnP usages in KDEs software to be 
controlpoint-only, so the server part in a single lib would be just bloat to 
them.

> > Doing the discovery in a proxy/cache not only avoids work duplication
> > (increase of resource needs per process), it has also the advantage to be
> > faster (cache). And it is done for other discovery systems, too: There is
> > Avahi for DNS-SD, as a central small proxy process (our code layer to it
> > is KDNSSD).
> 
> It could be faster if properly implemented, I agree, but are such
> (possibly very small) gains in speed in this particular area worth the
> trouble? Furthermore, if the work to be done is very small the benefits of
> avoiding it tend to be very small as well... ;) I'd still be inclined to
> properly profile and test the need for such a system before actually doing
> it. :)

Sure, if it's just about the pure resource usage I agree. But I hope you 
understand that I think there is more than just this to consider. 

> >> > By the way, if any of you have ideas or feature requests for HUPnP, or
> >> > feedback in general, this is a good chance to influence the
> >> 
> >> development
> >> 
> >> > before the API & ABI is locked for the first major release. I'd
> >> > appreciate your comments.
> > 
> > Would like to give you some more comments, but please accept my limited
> > time and find my ideas/thoughts in these emails :)
> 
> Of course, I did and I thank you. Some interesting thoughts there to which
> I enjoyed answering. :)

Thanks. Which I enjoyed to reply to again :)

Cheers
Friedrich
-- 
KDE Okteta - a simple hex editor - http://utils.kde.org/projects/okteta