id3, playlists and DB

[Ivan Sergio Borgonovo]

On Sun, 13 Jan 2008 13:59:48 -0500
Jeff Mitchell <kde-dev at emailgoeshere.com> wrote:

> On Sunday 13 January 2008, Ivan Sergio Borgonovo wrote:
> > Not that I've fallen in love with NFS and I'm aware of some
> > pitfalls... but actually I think it is still one of the most
> > diffused "nfs" around and a lot of stuff works properly on it and
> > I think there is no simple alternative to it.

> Nope.  There's no good alternative.  There's also no good network 
> filesystem  :-)

There are if you have enough time to put them in place, but that's
far from an aptitude [...] vi /etc/fstab up to my knowledge.

> taglib wouldn't be recommended.  It'd be required.  Amarok doesn't
> work without taglib, period.  You must have it somehow, somewhere.
> Might not be in sid.  In which case, knowing Debian (two years
> behind, full of already-fixed bugs), you may want to compile taglib
> from SVN.  The 1.4 release is too out-of-date to be considered
> reliable.

I never had the feeling that sid was out-of-date. In the past I had
the feeling it was like being on a roller coaster with some updates,
but generally stuff get fixed in days. And I'm not the sysadmin kind
of guy, so I'm not versed to fix boxes and I've a low tolerance to
admin problems. I wouldn't have chosen Debian if it didn't let me
have a reasonably quiet life.

Anyway I've taglib 1.4.

> > While great for "home users" and from a simplicity point of view
> > it seems it is not the right solution for centralised archives.
> > Furthermore if people can move files... they can edit tags and the
> > "write only" archive argument loses a bit of sense.

> But it should work just fine for centralized archives.  After all,
> the information is stored in a shared database; you only need one
> copy of Amarok to update it.

Unless the rows are locked... well you end up changing the UIDs on one
client and the tag on the other etc...
Otherwise I can't understand the behaviour I observe.
And still you wrote you don't have to change tags and file position.
I didn't outside amarok and I still observe these problems.
I could understand them if UIDs where computed even taking into
account path, but it seems not the case.

I'd be nice to have an amarok server, really use read-only mount and
let the server take care of the changes on the fs and DB.
So DB operations could always be coherent with fs operations.

> Linux distribution.  Debian's "stability" is overrated, and has
> nothing to do with how stable or bug-free individual packages are.)

sid stability is an oxymoron ;)

Anyway I'd bet on the reliability of the services involved (NFS and
Postgresql).
I use NFS everyday from sid, lenny and I used it from a forgotten
version of ubuntu (OK... all anarchists, distributions). The
Postgresql server is also my development server and it accept pretty
high loads.
That doesn't mean that there is no chance that a cosmic ray passed
through an HD platter turning my server in an evil gnome misbehaving
just with amarok.

> > So I can't think nfs is involved when I change the genre of 5
> > files and 4 get changed and 1 doesn't...

> It could be.  Who knows.

OK... some more info... mp3tag doesn't work. Uberprgramm kid3 showed
the changes. But still some files show updates just at the second try
in the playlist.
Sorry for misleading you.

> > or well I could understand it if 
> > the DB uses the "UIDs" to update files

> No.  Database side, it's a simple update statement.  File side,
> it's taglib calls.

Hypothesis... not updated UIDs between clients?

> > It could be NFS related (I'll check) but I wouldn't consider it an
> > NFS fault even if I've the feeling that switching that box to NFS4
> > may help, but LSI/Dell were so *kind* to kill support for CERT ATA
> > RAID and I'm stuck with sarge on that server for a while.

> 3ware FTW.  Dell's built-in cards are always crap.

Next time if I really have to save money I'll go for soft-raid.
SERIOUS, battery backed-up RAID hardware is going to cost me a kidney
anyway.

> Of course not.  The playlist isn't refetched from the database when
> you close and open Amarok -- there'd be no way to have all the
> customizations you may or may not have made to it.  When you close
> Amarok, the current playlist is saved -- along with all the
> metadata information -- to a file.  current.xml IIRC.  When you
> open Amarok, it's loaded back into the playlist.  So it's behaving
> exactly as expected.

Then how am I going to obtain a refresh of the info associated with a
playlist?

Updating the collection doesn't have any effect. I'm running "Rescan"
but it is still working. If I get it right it won't help anyway.
I think you're referring to the order of songs when you say
"customisation".
Up to my memory most (all??) the information are already stored in
the DB, duplicating them... well seems the cause of the limit I'm
experiencing.
Maybe BMP, Bitrate, SampleRate and Filesize aren't stored in the DB.
And keeping most of these information in more than one place is going
to cause no few sync problems.

On the other side keeping them all in the DB will make updating the
display much easier. You keep a copy of when you last updated your
"collection" view and ask to the DB all the songs that were changed
after that moment in spite of traversing the whole fs.

> > but deleting and repopulating the playlist is 
> > really time consuming

> It shouldn't be, especially not in 1.4.8.  How many tracks are we
> talking about?

Some thousands on the aforementioned CERC ATA RAID 1 through a 2x
1Gbit connection, P4 2.4GHz, 512Mb RAM.
The client is a AMD64x2 2Gb RAM.

> > and prone to crash/freez

> Again, it shouldn't be.

Unfortunately while I observed few crashes I saw a lot of freeze and
even when it doesn't freeze, with VERY large collection it becomes so
slow it is an annoying operation to populate a playlist.
eg. you select all the songs from the collection, you drag the mouse
on the playlist area and well if you're "too fast" it doesn't
understand you decided to paste them there. You do something else and
after it finished to see what it was going to move... since the mouse
is elsewhere it doesn't know anymore where to move them and abort the
operation (no crash... just don't get what it had to do).
I observed quite a lot of freeze when I use Music-brainz and when I
do long sessions of tag editing without closing amarok.

> > never changed tags outside amarok... just changed them on
> > different clients. I'd expect changes show up as soon as a
> > playlist is reloaded, not after some black magic.

> If you are changing them within Amarok, then, yes, they should.

But it is not happening. Or at least not as I expect.
Let me narrow down the problem.
I saw this behaviour when I'm editing a tag for several files at a
time and some don't get updated. You see amarok "Writing..." but some
songs may keep the old tag or take another one.

Another curious behaviour is... you change a tag in a client. It
doesn't show off in the play list. You "Edit track information", it
shows the new one but it doesn't update the playlist.

> > With this exception no direct action on files/tags has been taken
> > outside amarok but still I get the above mentioned behaviours.
> > Consider that eg. changes don't show off no matter which client I
> > use and in both clients collection was rescanned, playlist killed
> > and restored etc...

> It'd be interesting to see how things behaved if you used Samba
> instead. That's what I use, and I haven't seen any of the problems
> you describe.

As I said I'm not in love with NFS and I'm not in love with Debian
too. But my lan showed in many circumstances to be up to the task
even on comparatively much higher loads on pgsql and nfs.
NFS is not perfect but it was there long before Debian, samba, kde.
It is just a bit younger than NetBIOS.

> > Please don't take it wrong.

> Don't worry, we're not.  We tend not to take offense when people
> have weird, wacky isolated problems that no one else has.  We also
> tend to think it's an artifact of their setup, not an artifact of
> our program  :-)

It ain't that weird, maybe uncommon to have thousands songs on NFS.
But NFS is not the kind of thing I'd define weird.


thanks anyway for helping me in narrowing down the problem and get
more conscious about the role of playlists.
I've read that playlist management has taken a different direction in
amarok 2... but as a matter of interface having an interface place
where to edit/add tags with a DB back end in a multi-user env is
still a killer feature for me.

Maybe making my use case more explicit will put some light on why...
You may not like all the music you rip from CDs, there may be several
people in the same environment that don't share exactly the same
taste, you may have old music you forgot where it came from or didn't
rip properly.

Sharing meta-info like genre, rating and score, being able to edit
genre is a good basis on which build up dynamic lists, so that the
whole family can listen music.


-- 
Ivan Sergio Borgonovo
http://www.webthatworks.it