Forums » About the forum itself »
Forum will be offline (for a short time) and moving soon
Added by Flole Systems 11 months ago
This is a heads-up that the forum will be offline for 24-48 hours later this week as we move from Redmine to a modern forum software and enable GitHub issues for bug tracking. This will be a deliberate 'hard' change with no migration of old (mostly very old) content and issues. Redmine will be read-only accessible on https://old.tvheadend.org for two months to allow manual recreation of must-have forum content or current issues in new locations. Reminders will be posted to the new forum as we count down to the final Redmine turn-off.
Issues have already been migrated to GitHub, so bug reports should be opened over there. New bug reports on Redmine have been disabled now, but old bug reports can still be accessed.
Redmine accounts will be pre-registered in the new forum to preserve usernames, email addresses, and registration dates. We cannot migrate user passwords so everyone will need to follow a "forgotten password" workflow to (re)set their new forum password. Your Redmine username (not full name) will be your identity in the new forum. Redmine accounts that we believe to be spam-bot registrations (approx. 55% of all accounts) will not be migrated. Redmine accounts with email address usernames will not be migrated to prevent the email from becoming visible. If you are forced to re-register but want your original forum registration date restored, please register with the same email address then ask admins to update your user profile (we can cross-reference your email against the old user DB to do this).
The new forum brings lots of improvements: dynamic thread/post tagging, media integration, mobile device support, user management and anti-spam controls. Online communities are defined by their users not their forum software, so it will look and feel different, but different will soon be the new normal. Moving issue tracking to GitHub will help to increase activity and engagement around the codebase, and in the long-term we hope this will encourage greater contribution as we move away from a "sole maintainer" mentality.
Replies (21)
RE: Forum will be offline (for a short time) and moving soon - Added by Jonas Lang 11 months ago
Great news. I’ll file this message for posterity sake though.
Pity you’re only keeping this (now old) forum online for two months. You’d be surprised how useful a tool it has been when referring new users to solutions for problems they’ve experienced. A lot of wise old heads contributions that have been reused on so many occasions.
As you’ve stated so often here not much new development on the TVH project over the last number of years so I’m not sure you could actually say the information contained here is somehow outdated.
Still I’ve managed to grab what I needed from this forum once the announcement was made. Not sure what others have managed to do.
Going forward it should be interesting to see what shape the new forum will take and looking forward to all these new contributors once the new forum goes live.
RE: Forum will be offline (for a short time) and moving soon - Added by Flole Systems 11 months ago
The decision on how long it will stay is not final, if the new forum is still empty in 2 months then it'll be longer.
RE: Forum will be offline (for a short time) and moving soon - Added by Dave H 11 months ago
Is there a reason not to copy the contents of the forum to the new one? Or at least to preserve the contents so it can be extracted or copied later?
RE: Forum will be offline (for a short time) and moving soon - Added by Flole Systems 11 months ago
There isn't an automated way to copy it over. I wanted to do that aswell, but it seems impossible.
The forum will be preserved internally, I wanted to make a browseable public copy of it somewhere aswell, but it's over 80GB and so far there isn't any way to host such a huge archive.
RE: Forum will be offline (for a short time) and moving soon - Added by Christian Hewitt 11 months ago
I'll ensure there's some form of static archive created as 80GB is no problem (disk is cheap). I'll also make a full dump of the Redmine DB and filesystem in case we ever need to resurrect the instance. However, anything good in old forum threads really needs to be extracted, simplified, and added to project documentation so users can find the needed info by reading 'a' page or 'a' chapter instead of needing to read their way through a multi-year thread. NB: Sorting out documentation is also something the project needs to do as it hasn't really been touched since 4.0. I'm wondering out loud how many forum threads could/would have been prevented by updated docs? - sorting that will need a broad community effort.
RE: Forum will be offline (for a short time) and moving soon - Added by Dave H 11 months ago
Thanks Christian. I'm not surprised there isn't an automatic way to copy the forum across - neither Redmine nor Flarum are mainstream forum software. But if the static copy is readable I can have a go at converting it.
I suspect one problem with docs is that TVH has separate groups of users. I just use it for Freeview recording (i.e. DVB-T(2) ). I know nothing about other forms of broadcast or about IPTV, streaming etc and I know nothing about how other countries manage their broadcasts. I know a little bit about the hardware I happen to use (TBS) and even less about Kodi. I expect there a lot of people in a similar position but who use different aspects of the system. I suspect finding people who have a broad understanding who are willing to lead a documentation effort may not be easy. But it is a very worthwhile goal.
RE: Forum will be offline (for a short time) and moving soon - Added by Flole Systems 11 months ago
I think what would be a more promising approach is converting/reading the database and somehow dumping that into Flarum. For obvious reasons the users need to be migrated/linked aswell, so that everyone has the right posts associated with their account.
RE: Forum will be offline (for a short time) and moving soon - Added by Jonas Lang 11 months ago
Might be worth announcing that the new forum is live and ready for posting (if that’s the case) simultaneously here and at the new forum. That could be the cutoff point with a redirect from here to the new forum. Any work on the old forum could be done at your leisure.
RE: Forum will be offline (for a short time) and moving soon - Added by Flole Systems 11 months ago
It's not ready (yet), as the user database needs to be exported/migrated. There will be a cutoff after that and it will no longer be possible to do anything here (likely at the end of this week). From that point onwards everything has to happen at the new forum. That ensures the user database is migrated properly and no new users can register here.
RE: Forum will be offline (for a short time) and moving soon - Added by Jonas Lang 11 months ago
I wonder if it’s necessary to migrate the user database. I think you mentioned that 50% or possibly more of the user database are spammers and looking at the activity up to about 5 years ago I imagine another 25% of those user accounts are dormant.
I wonder if a fresh start of the user accounts might be a better way forward by encouraging both currently active and potential new users to reregister/register on the new forum.
Maybe this could be achieved by sending out an an auto email to the current users advising them of your intentions and what they need to do to continue participating in the forum.
Might work as a little wake up to some too. Again just a suggestion to help avoid any additional work going forward.
RE: Forum will be offline (for a short time) and moving soon - Added by Flole Systems 11 months ago
I understand your idea, but we don't want to become "Freenode 2.0" by nuking all users. It should be clear that the old "Jonas Lang" is the new "Jonas Lang" aswell, and not someone who is a huge fan (or someone with bad intentions). After a year everyone who hasn't signed in will be deleted, so we will eventually clean the user list, just not now.
I think we could do what you proposed though, but in a slightly different way: Passwords won't be migrated, so we could send everyone a "reset password link", together with an explanation why it is necessary to do this and an invitation to have a look at the brand new forum. If they decide to not use it, fine, after a year their accounts will be gone. Maybe even include a link to delete the account immediately in case someone wants that.
RE: Forum will be offline (for a short time) and moving soon - Added by Christian Hewitt 11 months ago
It's not impossible to migrate Redmine to Flarum or any other new forum software. However no tools appear to exist so we would need to author something from scratch. In a more established project team we'd likely have a resident python genius itching to take on the task, but our 'team' is small and still in the early stages of finding its feet and there isn't one. The alternatives are finding and paying someone to create the tools, waiting an unknown time until we manage to recruit someone suitable to the team, or simply grandfathering the data; which requires the least effort, no money, and moves the project forwards quickly. IMHO it's not a hard choice.
Redmine is made for the corporate world and defaults to first/last name identities so there's a few people whose identity will change with the switch to username (Jonas will be one of them) but the main impact is aligning their Tvheadend identity with the one used publicly in other forums. In end-user forums nicknames are normal and look friendlier, so I expect a few complaints but see it as an overall positive move.
Forums do have hierarchies and little status symbols. Everyone will start with a zero post count, but those who post will quickly reassert themselves and we will migrate the registration dates so 'old hands' can still be seen as such. I've skipped creation of user tiers as Redmine doesn't have them but Flarum allows proper 'Moderators' (with real moderation tools) with some visual cues on roles, and anyone who donates through OpenCollective should be auto-tagged as a sponsor if they use the same email address on both sides. If we want to add forum things in the future we can.
Kodi attempted bulk password reset notifications after their forum breach last year and it's considerably harder than you think to one-time send a large volume of identical emails to a global userbase and have them reliably delivered to inboxes (most end up in spam). It will also cost us money with Mailgun (or any alternative service; we'll be over the limits). Some sticky posts in the forum news section will suffice.
If you start poking around in the Redmine user DB some of the spam-registration patterns are quite clear so we'll skip migration for ~55% of all current users. I don't see value in doing a further cull in 6-12 months time. In my experience users often take a multi-year hiatus from forum visits; the "if it works, don't fix it" principle is strong and users only visit when something breaks. If we do a good job people don't visit often. If we want to forcibly engage our audience, we can have an auto-update feature and push something buggy
I'm tenatively planing downtime for Monday. Partly because more people are likely to use the forum over the weekend. Mostly because I'm running a half-marathon tomorrow and will be tired after (never a good time to do something important).
RE: Forum will be offline (for a short time) and moving soon - Added by Dave H 11 months ago
Flole Systems wrote:
I think what would be a more promising approach is converting/reading the database and somehow dumping that into Flarum. For obvious reasons the users need to be migrated/linked aswell, so that everyone has the right posts associated with their account.
That's exactly what I'm proposing.
RE: Forum will be offline (for a short time) and moving soon - Added by Dave H 11 months ago
Christian Hewitt wrote:
It's not impossible to migrate Redmine to Flarum or any other new forum software. However no tools appear to exist so we would need to author something from scratch.
I just offered to have a go at it, and you have ignored the simple question I asked that is an obviously essential precursor. Will the saved database be readable?
RE: Forum will be offline (for a short time) and moving soon - Added by Christian Hewitt 11 months ago
Let's chat in Discord.
RE: Forum will be offline (for a short time) and moving soon - Added by Dave H 11 months ago
Why? Why not keep it open?
RE: Forum will be offline (for a short time) and moving soon - Added by Jonas Lang 11 months ago
Dave H wrote:
Why? Why not keep it open?
A question I’ve asked on a number of occasions too. If discussions are dragged to Discord what’s the point in a forum in the first place. Open source and open discussion is what we signed up for.
RE: Forum will be offline (for a short time) and moving soon - Added by Christian Hewitt 11 months ago
In my personal experience synchronous communication tools like Discord/Slack/IRC/etc. are simply more effective for admin and development collaboration and building the background trust and 'team' relationships that fuel a healthy project than asynchronous forum and email exchanges. Coming from a InfoSec background I'm also driven to have admin oriented conversations in private not public.
RE: Forum will be offline (for a short time) and moving soon - Added by Dave H 11 months ago
FWIW, Christian Hewitt sent me some private messages on Discord. I asked him to move the discussion to email since I don't like Discord but he hasn't responded since. I'm not posting what he sent, since he sent it privately, but I'd urge him to post it here.
In my experience email and simple forums work well, and being asynchronous is an essential feature. I'm with Jonas. Open source and open discussion is what we signed up for. Sure there are some things that need to be sent in private - keys, passwords etc, but there's no reason for most things to be hidden.
RE: Forum will be offline (for a short time) and moving soon - Added by Jonas Lang 11 months ago
Dave H wrote:
FWIW, Christian Hewitt sent me some private messages on Discord. I asked him to move the discussion to email since I don't like Discord but he hasn't responded since. I'm not posting what he sent, since he sent it privately, but I'd urge him to post it here.
In my experience email and simple forums work well, and being asynchronous is an essential feature. I'm with Jonas. Open source and open discussion is what we signed up for. Sure there are some things that need to be sent in private - keys, passwords etc, but there's no reason for most things to be hidden.
In an open forum setting going to a separate platform to continue a discussion is disrespectful to the members. It’s generally indicative of how the hierarchy view its membership. The same hierarchy who are hung up on badges and titles. It’s actually gets tiring and irritating after a while.
If you want to retain the members interest in this forum you’ll do everything you can to promote healthy discussion in an open and transparent way. No need for behind closed doors discussion to make your point.
The new look forum will stand or fall on this basic concept. While I do accept that the TVH project was on life support it’s important that current participants are not scared off contributing for fear that their contributions are somehow less than worthy for consideration.
RE: Forum will be offline (for a short time) and moving soon - Added by Christian Hewitt 11 months ago
The threat of a hard deadline prompted several efforts to migrate forum content from Redmine. One of them has progressed nicely and later today (5th Feb) we will be taking Redmine offline to run the live migration. This will be 'softer' migration than previously communicated, so we hope you like it.