Discourse

Fri Nov 30 08:26:28 GMT 2018

On Thu, Nov 29, 2018 at 11:32 PM Harald Sitter <sitter at kde.org> wrote:
>
> On Thu, Nov 29, 2018 at 9:03 AM Ben Cooksley <bcooksley at kde.org> wrote:
> >
> > On Wed, Nov 28, 2018 at 5:25 AM Lays Rodrigues <lays.rodrigues at kde.org> wrote:
> > >
> > > Let's not go in that way.
> > > As a ~new person~ on KDE, 3 years only, we need to move to a modern web. At least in my point of view, I really think that using old stuff doesn't attract new people. In that I have a few ideas for some KDE websites go modern, but that is a project for the future.
> > > Discourse is a way to do that. I don't have much idea on how is the cost to maintain such an application, but in the field to setup it, I don't
> > > think that is hard since we just need docker and Postgres.
> >
> > Before we look into deployment, has an initial evaluation been done to
> > determine how individual communities around specific applications
> > would work with Discourse?
>
> How would one best go about evaluating? Do we know what communities do
> with the current forums? Do we have the original evaluation of how we
> ended up with phpbb somewhere?
>
> > Usually these communities only want to look at posts for their
> > specific application, and in some cases we have customisations to
> > support their specific usecases - which our existing forum supports
> > (see https://forum.kde.org/viewforum.php?f=276 for instance).
>
> Which customizations are there, what do they do, where is their code
> and who maintains them currently?

Based on a quick review of the git diff between the unmodified phpBB
we run and the currently deployed code we have:

- Anti Spam Protection mod
- Mark as solved mod
- Forum banner / Newsbar mod
- Guided posting wizard
- Profile Icons mod (shows OS/Distribution icons by people's name on
the topic views)
- Brainstorm / post voting system
- Identity integration
- Sphinx Search modification (needed to scale, database based search
simply doesn't work with the number of posts the forum has in it)
- RSS feed support (this actually leans on Sphinx Search to do
everything it does)
- Tagcloud mod

Some of these are simply because phpBB at the time didn't have the
necessary functionality. Others were because we ran into scalability
limitations at the time based on the hardware Sysadmin had available
to it (for a large portion of the Forum's early life, including when
Sphinx was implemented, it was running on a machine with just 1GB of
RAM) so they may not be as much of a problem now (especially as InnoDB
now supports text search, which it didn't back then)

>
> > Once we're satisfied that's all okay and can accommodate everything we
> > need, we then can look into deployment. At this point we'd:
> >
> > 1) Look into the actual technology stack they use (seems to be Rails
> > based in this case) to make sure there aren't any potential snags
> > there
> > 2) Evaluate what support it has for authentication options (Identity
> > requires LDAP at the moment, but will move to OAuth2 at some point
> > using a custom API)
>
> I've just had a look... Writing auth plugins is insanely easy. As in:
> I probably spent magnitudes more time on our jenkins oauth plugin than
> it would take to write any auth plugin for discourse ^^
>
> > 3) Determine what's needed to import existing data we have
>
> A full phpbb database dump from starters. Beyond that I am not sure
> what existing data actually entails?

Migrating over topics/posts into a new structure for the most part.

Unlike Reviewboard, this is usually pretty straight forward as the
syntax people can use in posts is limited to that supported by BBCode.
We migrated over MyBB posts into phpBB without issue, and over the
years also integrated in existing content from Amarok's SMF based
forums as well as others (who weren't using phpBB)

>
> In fact, I am also not sure why everyone seems to consider importing
> 10 year old forum posts a given requirement here. We migrated away
> from reviewboard with zero data migration, and the data that is in
> reviewboard is largely much more relevant than posts on why changing
> the wallpaper in Plasma 4.0 didn't work IMHO.
> Data migration is hugely expensive in both human and computer time;
> it's easy to say we want this, it's another thing to actually
> practically make it happen. In fact, I know of a semi-recent forum
> migration where a company that heavily relied on forums for user
> interaction migrated by having users manually replicate threads they
> wanted to keep into the new forum and after a couple of months threw
> the old forum away. Often times the cost-benefit is just super poor
> for data migration in general and most kinds of forums in particular.
>
> That being said, if data can be easily migrated I say hooray. Whether
> it deserves to be a requirement I am not so sure about.
>
> > 4) Ascertain how best to structure things to make it easy for
> > end-users to work with
> > 5) Investigate what anti-spam options are available and how
> > maintainable any customisations we need to support KDE specific
> > workflows will be
>
> Can you describe these workflows a bit?

As someone who hasn't had the chance to use the forums in a very long
time unfortunately, i'm not really in a good position to comment on
this.
I know several applications including Krita use it quite a bit though,
so the people who lead that on the forum are probably better placed to
comment on this.

>
> There is this https://www.discourse.org/plugins/akismet.html not sure
> it's sufficient though. One could always opt to write custom plugins
> if necessary, that really depends on what we need though. AFAIK
> discourse also has a fairly comprehensive REST API, so one could
> possibly also approach it from the other end and have a micro service
> hit spammers with a ban hammer after the fact.

There are two parts to this:

1) Stopping bad content from being posted in the first place
2) Allowing forum moderators to quickly cleanup spammers (ban + delete
all the posts they made) - who in many cases are bots and can thus
make dozens of posts very quickly.

>
> > My main one here is the lack of any options for installation other
> > than Docker which makes no sense for a Rails application. Looking into
> > their Docker image installation script I see that they build both
> > Nginx and Imagemagick themselves (and stepping outside of package
> > repositories is generally a bad idea). Imagemagick is of grave concern
> > as this project has had numerous security advisories in the past and I
> > see the version they're using isn't the latest one. I have further
> > concerns for Nginx as they include a third party compression module,
> > Brotli, whose codebase hasn't been touched in 2 years (plus it's a
> > compression method, so you have the risk of CRIME/BREACH attacks)
>
> https://github.com/discourse/discourse/blob/master/docs/SECURITY.md

Unfortunately that article doesn't really address issues like how they
monitor their dependencies for security issues, and in turn update the
application (i'm treating the Docker container as a monolithic
"application" in this instance). It addresses attacks where the
application is simply being used as a vector to attack the user, while
not really addressing the application being used to attack the
underlying server hosting it.

Because it runs in Docker they're responsible for the entire software
stack - so any time their base distribution makes a security release
of any of the packages they use, they need to refresh the image. They
also need to monitor advisories for Nginx and Imagemagick, and update
their image accordingly whenever either of them makes a release.

To make matters worse, unlike the distributions, they won't be
receiving any pre-release advance notice to prepare these updates, so
Discourse as an application would be insecure for a much longer period
of time (from the point it comes to their attention when the advisory
becomes public, to the point they publish an updated image) than
applications which are deployed outside of Docker.

>
> HS

Cheers,
Ben