There was no overhaul in fall. We made some adjustments in mid-June to balance vote decay but the algorithm itself was not changed - decay is still there as always, as is the rest of the ranking algorithm. One of the things the June adjustments did was to stabilize the number of "minimum votes" we keep for older guides, to address some ongoing complaints from authors whose ranks kept jumping around a bit sporadically. That number is still fairly low though.
Regarding Volibear, there is only one guide updated for season 7, so it is not ranked against anything else. The guide listings are "stacked" in to groups. The first group is "Season 7 guides", and for Volibear there is only one. The second group is "Season 7 builds", that is guides with little or no text in them. The third group is "Season 6 guides", then "Season 6 builds", etc... So that guide is not necessarily ranked above any of the other Volibear guides, it's just the only one in the first "Season 7 guides" group.
However I did just notice a bug, it's not showing as a "Season 7" guide on the browse guides page under the top rated tab. We'll take a look at that.
Regarding difficulty accumulating votes, we're open to suggestions. We've discussed some ways to incentivize voting but we don't want to end up with just a lot of random votes from people that didn't really read the guide. We've also discussed testing guest votes, where users can vote without logging in. It's not off the table, but we haven't dabbled with it yet.
Regarding Volibear, there is only one guide updated for season 7, so it is not ranked against anything else. The guide listings are "stacked" in to groups. The first group is "Season 7 guides", and for Volibear there is only one. The second group is "Season 7 builds", that is guides with little or no text in them. The third group is "Season 6 guides", then "Season 6 builds", etc... So that guide is not necessarily ranked above any of the other Volibear guides, it's just the only one in the first "Season 7 guides" group.
However I did just notice a bug, it's not showing as a "Season 7" guide on the browse guides page under the top rated tab. We'll take a look at that.
Regarding difficulty accumulating votes, we're open to suggestions. We've discussed some ways to incentivize voting but we don't want to end up with just a lot of random votes from people that didn't really read the guide. We've also discussed testing guest votes, where users can vote without logging in. It's not off the table, but we haven't dabbled with it yet.
Hi Matt, thanks for the considered response. I meant the June adjustments.
As you guys must be aware the problems with old votes are:
- the vote could have been given to a guide with very different content
- voters do not necessarily revisit guides to keep their vote relevant to the current guide
- guides that are out of date - and therefore maybe badly misleading, if there have been game changes - keep all their votes.
Based on the champion guides I have written myself and other champions which I follow closely, I still feel that the decay rate for old votes is too low, now - though I agree with you that the decay rate was perhaps too high, before June 2016. Someone who puts a lot of effort into making a new and better guide than the existing offerings has huge difficulty in having their guide climb to the top, even with consistently positive, recent votes. This is discouraging. The only real opportunity is at the start of a new season, thanks to the ordering system that you described. :)
Looking at Volibear, I see what you mean. I assumed that all three Volibear guides marked "FR" for fresh were for Season 7 but looking again you are correct, only one of them has been updated for Season 7. Therefore I guess it's possible this problem may "fix itself" over time as other Volibear guides get updated for Season 7. I'm still very unimpressed with how out of date that guide is, when you actually read its text - the Masteries and Jungle Routes sections are from Season 5 :( And with a 91% score from all its old votes, it's going to be a tough challenge for anyone to displace that guide from the top...
----
On voting, how about allowing guest votes for users who have spent more than 15 minutes reading a guide, or who have opened all the spoilers which probably means they have read the whole guide, or who have viewed a guide more than once on different days, or who have viewed more than one guide for the same champion.
Or to get more votes from logged in voters, a contest with voters randomly selected for a small prize.
Or icons or awards for users who have voted a lot? Or an automatic +rep for voting?
As you guys must be aware the problems with old votes are:
- the vote could have been given to a guide with very different content
- voters do not necessarily revisit guides to keep their vote relevant to the current guide
- guides that are out of date - and therefore maybe badly misleading, if there have been game changes - keep all their votes.
Based on the champion guides I have written myself and other champions which I follow closely, I still feel that the decay rate for old votes is too low, now - though I agree with you that the decay rate was perhaps too high, before June 2016. Someone who puts a lot of effort into making a new and better guide than the existing offerings has huge difficulty in having their guide climb to the top, even with consistently positive, recent votes. This is discouraging. The only real opportunity is at the start of a new season, thanks to the ordering system that you described. :)
Looking at Volibear, I see what you mean. I assumed that all three Volibear guides marked "FR" for fresh were for Season 7 but looking again you are correct, only one of them has been updated for Season 7. Therefore I guess it's possible this problem may "fix itself" over time as other Volibear guides get updated for Season 7. I'm still very unimpressed with how out of date that guide is, when you actually read its text - the Masteries and Jungle Routes sections are from Season 5 :( And with a 91% score from all its old votes, it's going to be a tough challenge for anyone to displace that guide from the top...
----
On voting, how about allowing guest votes for users who have spent more than 15 minutes reading a guide, or who have opened all the spoilers which probably means they have read the whole guide, or who have viewed a guide more than once on different days, or who have viewed more than one guide for the same champion.
Or to get more votes from logged in voters, a contest with voters randomly selected for a small prize.
Or icons or awards for users who have voted a lot? Or an automatic +rep for voting?
SpikeThePike wrote:
guides that are out of date - ... - keep all their votes.
SpikeThePike wrote:
And with a 91% score from all its old votes, ...
I get the impression that you don't entirely understand how vote decay works on MOBAFire. The way it is set up is that votes that have been cast too long ago - more than 30 days - don't diminish in value, they simply expire (users can vote again to refresh their vote).
However, to ensure that a guide's score can still be compared to others even if it happens to get (almost) no votes within that period of time (for whatever reason), there's a set minimum amount of votes that is taken into account at any time (it's this minimum that was adjusted in June).
Just to be clear: if a guide receives less votes within 30 days than this minimum, extra votes are taken from the "expired" pool to compensate. This is the only scenario in which votes that are considered old can count towards a guide's score, and also the only scenario in which "tested" guides have an edge over newer guides (since a new guide probably won't have older votes to fall back on). If both guides receive (more than) enough votes to be above the limit, the old votes won't play a part whatsoever.
I hope that clears a few things up, let me know if you have any questions :)
Finally, regarding the
Volibear guide you mentioned, I should point out that stating that a guide is "up to date" without having actually updated the guide is considered misleading and it's something we can take action against. Of course, should you decide to report a guide for this reason, it would be useful if you could include some clear examples of why you think the guide is outdated. You can also try to work it out with the author first of course, sometimes it is no more than a genuine mistake ;)
Regarding the vote incentives, the issue with any type of reward for voting is that you will have users that just go and vote on a bunch of random guides to quickly meet the qualifications for the reward. It will lead to "more votes" but it won't lead to "more accurate scores", which is what we actually want.
We discussed a system where you're presented with a series of random guides, and you have three options - up vote, down vote, or pass - and you get your reward even if you pass, to discourage blind voting. The main issue there is, most people will likely just click "pass" repeatedly until they get the reward, or even if they try to fairly assess the guide, it might be for a champion they are not very familiar or experienced with, and they likely won't take the time to actually go in game to test out the build, so it's still not a "high quality" vote. Ultimately I think they'll be voting mainly on the effort they feel the author invested. Which is certainly important, but a high effort guide can still be outdated or incorrect.
If we allowed guest voting, it would likely be protected only by the IP restrictions that we already have in place. We have a number of protections built in to the system to protect against vote abuse based on IP addresses, and most of those would apply the same to guest users as to logged in members.
I don't think requirements like "must be on the page for x minutes" or "must scroll to the bottom" or "must expand all spoilers" would do any good. The main risk with allowing guest voting is with people using it as a form of cheating or a form of personal attack. An author trying to cheat by up voting their own guide repeatedly, or by down voting competing guides repeatedly, or someone with a personal issue with an author trying to hurt them by down voting their guide repeatedly. To these users, such requirements would only be a minor inconvenience. They've already jumped through hoops to bypass our IP restrictions, they will certainly take the time to bypass those additional rules as well. Meanwhile it would add friction, discouraging legitimate votes from other guests.
Those are just my personal opinions though, maybe I'm wrong! :)
We discussed a system where you're presented with a series of random guides, and you have three options - up vote, down vote, or pass - and you get your reward even if you pass, to discourage blind voting. The main issue there is, most people will likely just click "pass" repeatedly until they get the reward, or even if they try to fairly assess the guide, it might be for a champion they are not very familiar or experienced with, and they likely won't take the time to actually go in game to test out the build, so it's still not a "high quality" vote. Ultimately I think they'll be voting mainly on the effort they feel the author invested. Which is certainly important, but a high effort guide can still be outdated or incorrect.
If we allowed guest voting, it would likely be protected only by the IP restrictions that we already have in place. We have a number of protections built in to the system to protect against vote abuse based on IP addresses, and most of those would apply the same to guest users as to logged in members.
I don't think requirements like "must be on the page for x minutes" or "must scroll to the bottom" or "must expand all spoilers" would do any good. The main risk with allowing guest voting is with people using it as a form of cheating or a form of personal attack. An author trying to cheat by up voting their own guide repeatedly, or by down voting competing guides repeatedly, or someone with a personal issue with an author trying to hurt them by down voting their guide repeatedly. To these users, such requirements would only be a minor inconvenience. They've already jumped through hoops to bypass our IP restrictions, they will certainly take the time to bypass those additional rules as well. Meanwhile it would add friction, discouraging legitimate votes from other guests.
Those are just my personal opinions though, maybe I'm wrong! :)
Again thank you for considered and detailed responses.
@Wayne3100: Yes, you are right I did not understand vote decay before, I assumed that all votes were counted but the weighting of each vote reduced as time passed.
But ... the system as you describe it sounds broken!
* I would estimate most guides receive fewer than 4 votes in each 30 day period on average. This is easy to see by dividing the number of votes by the age of a guide in months. (Maybe rare exceptions like a new guide by PsiGuard, or a featured guide...)
* Guides outside the top handful receive way fewer votes, probably less than 1 vote on average in each 30 day period.
* The "minimum" vote level therefore is almost always going to be triggered. This inevitably locks in and gives equal weighting to old votes which are outdated.
* Without a good number of critical voters who are willing to actively downvote guides which have become outdated, those outdated guides are going to stay locked in as "highly rated" guides for a long time
I can see that a system like that might work better with a larger active userbase of voters. But we come back to the problem that Mobafire just does not have sufficient voter numbers, currently. Therefore currently every individual vote is precious.
@Matt I see what you mean re. more accurate votes. So what about incentives to make users more likely to register on the site? (And also minor incentives to make it more likely people will actually login when browsing, if not logged in automatically?)
Currently the only/main reason to register and login is if you want to create your own guide, or participate in a discussion. Probably the vast majority of users therefore never have any reason to register.
Personally if I'm logged in to the site I will generally vote on a champion guide if I have a strong opinion on the guide. (But if it's a champion I have never played, I probably won't vote because I don't feel qualified to have an opinion on the guide yet.) But if looking at the site casually, e.g. on my phone, I am not logged in.
Of course haters and personal attacks are a problem with any democratic site. But low numbers of voters actually gives a similar problem, giving disproportionate weight to the haters and personal attacks, because 1 or 2 downvotes can have a significant effect on ranking. And a hater will be more strongly motivated to vote than an average user.
@Wayne3100: Yes, you are right I did not understand vote decay before, I assumed that all votes were counted but the weighting of each vote reduced as time passed.
But ... the system as you describe it sounds broken!
* I would estimate most guides receive fewer than 4 votes in each 30 day period on average. This is easy to see by dividing the number of votes by the age of a guide in months. (Maybe rare exceptions like a new guide by PsiGuard, or a featured guide...)
* Guides outside the top handful receive way fewer votes, probably less than 1 vote on average in each 30 day period.
* The "minimum" vote level therefore is almost always going to be triggered. This inevitably locks in and gives equal weighting to old votes which are outdated.
* Without a good number of critical voters who are willing to actively downvote guides which have become outdated, those outdated guides are going to stay locked in as "highly rated" guides for a long time
I can see that a system like that might work better with a larger active userbase of voters. But we come back to the problem that Mobafire just does not have sufficient voter numbers, currently. Therefore currently every individual vote is precious.
@Matt I see what you mean re. more accurate votes. So what about incentives to make users more likely to register on the site? (And also minor incentives to make it more likely people will actually login when browsing, if not logged in automatically?)
Currently the only/main reason to register and login is if you want to create your own guide, or participate in a discussion. Probably the vast majority of users therefore never have any reason to register.
Personally if I'm logged in to the site I will generally vote on a champion guide if I have a strong opinion on the guide. (But if it's a champion I have never played, I probably won't vote because I don't feel qualified to have an opinion on the guide yet.) But if looking at the site casually, e.g. on my phone, I am not logged in.
Of course haters and personal attacks are a problem with any democratic site. But low numbers of voters actually gives a similar problem, giving disproportionate weight to the haters and personal attacks, because 1 or 2 downvotes can have a significant effect on ranking. And a hater will be more strongly motivated to vote than an average user.
You need to log in before commenting.
This system had a major overhaul in Fall 2016, causing older "established" guides with lots of old votes to stay at the top of the rankings. I believe the purpose was specifically to prevent new guides with - for example - 3 positive votes and therefore a 100% score - being suddenly ranked top above an older guide with let's say 50 votes and a 90% score.
I think the overhaul has gone too far. There are now some badly out of date, and therefore poor quality, older guides which have very high or top rankings because they've been around since Season 1 and have accumulated a lot of votes over those years.
A specific example is
If Mobafire wants to be seen as the site for the best guides, it really needs to work on rewarding the good guides better.
(I think another problem is the small number of votes compared with the number of page views. Therefore even one or two votes can make a big difference to a guide's ranking. I guess this is mainly because users have to log in to Mobafire to vote and most casual users do not bother to log in. Also there is no real incentive for users to vote.)