Click to open network menu
Join or Log In
Mobafire logo

Join the leading League of Legends community. Create and share Champion Guides and Builds.

Create an MFN Account






Or

MOBAFire's first Mini Guide Contest of Season 14 is here! Create or update guides for the 30 featured champions and compete for up to $200 in prizes! šŸ†
's Forum Avatar

Reworking The Scout System

Creator: Maintained February 25, 2019 5:31pm
1 2
Maintained
<Stalker>
Maintained's Forum Avatar
Show more awards
Posts:
1102
Joined:
Dec 8th, 2013
Permalink | Quote | PM | +Rep February 25, 2019 5:31pm | Report
As of currently, the Mobafire scout is more or less useless. My vision of what the scout system should be is a tool that helps filter out the good guides from the bad ones, regardless of user vote count. That being said, the scout system should not have a monopoly over deciding the positional ranking of a guide either. Let's talk about suggestions for reworking the scout system. Here's what I have in mind:

  • Scout votes would show to normal mobafire members too.
  • Scout votes separate from user votes.
  • Needs a minimum amount of votes to show (maybe 5).
  • Scouts can vote as users or as scouts.
  • If they vote as scouts, they must comment as to why they think the guide is good/bad, it must be genuine (Minimum 100(?) words, enough to express your thoughts in detail).
  • Give scouts an incentive to vote, but if they abuse the system they can be suspended from scout role, maybe voting on featured guides of the week grant points, so that scouts can vote on the same guides so that said guides could get ratings, or have an official thread for people to request scout reviews.
  • Getting suspended from the scout role revokes said scout's votes and points.
  • Inactive scouts have their votes revoked, as just like guides, votes can become outdated too.
  • Members can request a scout re-review if their guide has been downvoted previously and improved.
  • A scout's vote will not be more influential than a user's vote in determining the ranking of a guide.

Here's what it could look like to users (excuse the bad photoshop):




With the introduction of such a system, a scout needs to be held to high standards. That means one could no longer be a scout automatically through the +Rep system. Instead, they are manually selected. How would a scout get in then? And how would a potentially good scout be filtered from a potentially bad one?

Someone looking to join the scouts could have to review 3 custom-made/chosen guides as a test, and then the scouts would then review the application to determine their authenticity, but that's just an example.

A common concern is that there aren't enough active people/scouts/whatever to uphold and sustain such a system, but if a solution for such an issue is found, what are your thoughts on such a scout system? Is there anything you would like to add to it? Or perhaps you would just like to have a different system in place? To put it simply, I would like to hear your thoughts.

Special thanks to iPulsefire for the signature!
PsiGuard
<Retired Admin>
PsiGuard's Forum Avatar
Show more awards
Posts:
10290
Joined:
Jun 26th, 2011
Permalink | Quote | PM | +Rep February 25, 2019 10:09pm | Report
Sent you my thoughts in a PM. I'd be interested to see other people's opinions on these ideas as well (and I don't want to dictate the conversation). If you see anything you agree or disagree with in this post, please leave a comment with your thoughts as I'd love to hear them as well.
Thanks to Jovy for the sig!
Chromuro
<Brawler>
Chromuro's Forum Avatar
Show more awards
Posts:
291
Joined:
Mar 6th, 2018
Permalink | Quote | PM | +Rep February 26, 2019 4:02am | Report
I already gave my feedback regarding the system, but I was still thinking about three points:
  • the "comment to vote as scout": we already know that the "comment to vote" system is annoying as hell and I think nearly no one uses that system as intended (I mean, wait to get to 20 votes, then activate the CtV to see future comments' votes), this doesn't change for scouts. They would be annoyed to do 50+ comments per week (50+ if we count only the possible featured champions and if we count only 3 updated guides per champion), even if this would help the author to improve his guide. Maybe something like a "random CtV" where if you have to comment or not is random (like a 50/50 maybe?), this way they should get less work to do probabilistically. Obv that they can comment by their own initiative, but at least they aren't forced to do so.
  • the incentive to vote: obviously we should talk about what kind of reward a scout could obtain, but for now I don't have any suggestion (I already discarded my idea of a challenger reward-like system even if personally I like the idea. But, you know, it's my idea so obviously I like it lol). What I wanted to suggest instead is the system that could lead to that reward+a system for the scouts to choose the featured guides of the week. The idea is pretty simple: every scout receives a guide queue where there are all the updated guides of the champion rotation (maybe you could put it near the messages button, so they can receive it like a notification) and they can vote, comment, change their vote, whatever it comes in mind for the guides they receive. There should be a limited time window for this (like two days) and in that time window the authors can still update the guides so that scouts can evaluate them. After two days, the queue gets stopped, no more guides can enter it, and from that the featured guides of the week get chosen. Maybe in the meantime change the featured guide list so it reflects the actual rotation instead of being a... top featured guides? I don't really know how the guides are chosen. Returning to the queue idea, the reward system would be something like the present ranked system, where you unlock your reward the more you play. Think about it as a year-long raffle "giveaway" only for scouts: for every action they do as a scout they receive a certain quantity of points. After a certain threshold, they get their reward. Simple as that. Again, for the rewards I don't really know what to suggest, sorry :<
  • the scouts' score: the more I thought about it, the more I thought there was something off, then I understood what it was. Little explanation (of my point of view): the current votes/score system works because there isn't a limit to how many votes a guide can receive, so the score changes accordingly, and it goes on until the guide gets forgotten or archived. The scouts' score wouldn't really work (at least for me) because the number of scouts is limited (to 33 rn); we have to see it from the pov of a random user: ok, I see that your guide has 10k votes and has that cool green bar, but why there are only 33 scouts votes (even if the orange bar is full)? does this mean that the scouts don't think this guide is good? maybe I'll try something else. In the end, the general rule of thumb is: if your guide has a lot of votes and its score isn't disastrous (in the limits of the yellow bar), people generally don't care about the score. The same goes for scouts' score and it's even worse when there is that kind of votes discrepancy; people don't know that there are only 33 scouts, they see only 33 votes. So it could be done this way: introducing to you the magnificent SCOUTS GRADE! tldr in the guide listing, instead of having the scouts score+votes there would be a scouts' grade that is decided by the scouts' score (like between 75-85 is B, 85-95 A, 95+ S and so on). Then, inside the guide, you can put all the informations like the number of votes, the score ecc., but at least the first thing that the random users see it's a grade that represents the scout's decisions, so it wouldn't be any more "why there is this discrepancy?", but a "wow, it has a lot of votes and the scouts say it's an S guide, nice let's read it"

Then just a little thought after this wall of text: this refers both to the eighth point and to the conclusion of maintained's post, obviously there is the need to rework both the system AND the role (so a new regulation for them that dictates want they should do, what they shouldn't do and things like that), but I leave that to you, I'm a simple user so I don't really know what is best for them.

Credits to Jovy!
Jovy
<Admin>
Jovy's Forum Avatar
Posts:
9609
Joined:
Nov 18th, 2011
Permalink | Quote | PM | +Rep February 26, 2019 5:01am | Report
perhaps instead of forcing scouts to write ~100 words per scout comment (which I personally don't think is a big deal anyhow) they would get something like this pop-up instead:



Another suggestion: have the scout comments stickied to the top of guide comments. And make it 2-3 scout points to feature a guide rather than 5, we will never have 5 active scouts imo.

Finally there should be a way for users to request scout reviews. In order to requst a scout review, they must have an in-depth tag so as to avoid spam requests from guides that are not worth a scout point for sure. Also the indepth tag should be changed so that the guide actually does need to be indepth in order to get the tag (the threshold is too low right now) It can be a tool like what the editors and moderators have:




Needs a few more brackets like role and rating (if we want to include that) but that's the gist of what I was thinking.



My biggest gripe is that we won't be able to accumulate enough active scouts to get a system like this going. Who should be a scout anyhow? It makes sense that they would need to be a certain elo (gold is considered "high" elo but I wouldn't agree) which narrows down the list of potential scouts even more. Morever, even if you're a high elo player that doesn't mean you are automatically qualified. Last season I was plat 2 (not high but would be considered as such on mobafire) but I know I am not qualified to judge a top or mid lane guide per se (though I could use pro builds to make sure the build is good or whatever). Additionally, I prefer to Q max Lulu and buy sorc boots every so often even though that isn't meta; so I could refuse to +scout a Lulu guide that doesn't do the same, because that's now how I play or what I think is best. Just an example.

Point being, it'd be very hard to find a good quality scout that will be completely objective and knowledgeable about a large variety of champions/roles. Let alone several.

Another issue is that scout points are permanent whereas the quality of a guide fluctuates. Scout points should be refunded every year or something. I think it's better to do it every few patches in theory, because a guide can be scout-approved and not updated for several months otherwise. But again that'd be a lot of work for potentially just 2-3 scouts who are doing this stuff for free.

That's my $1.50 on the issue.
Vapora Dark
<Perfectionist>
Vapora Dark's Forum Avatar
Show more awards
Posts:
10491
Joined:
Oct 16th, 2011
Permalink | Quote | PM | +Rep February 26, 2019 6:49am | Report
My initial thoughts were that there wouldn't be enough active scouts, but reading over your points I had some thoughts about the way the scout system could work that would actually personally motivate me to be an active scout again.


Quoted:
That being said, the scout system should not have a monopoly over deciding the positional ranking of a guide either.

If it doesn't influence the guide rankings then it remains useless, but likewise you don't want a system that overrides the vote scores either. So what if a scout point simply counted as ~5 votes (random number subject to change)? It would work within the vote system rather than above or below it, and as a scout I could actually feel myself having an impact even if I'm the only scout reviewing each guide. There's been tons of cases in the past where I see awful guides at #1 with a much better guide at #2, so I upvote the good one and upvote the bad one but it doesn't change anything. Likewise if I +scout or -scout right now it's more than useless even when the scout system was at its peak in terms of activity, because -scouts don't really do anything if the guide was too bad to get +scouts anyway, and the +scout on the good guide is useless if no other scouts review the same guide.

I feel like one of the biggest flaws of the Scout system in the past was that it relied too much on scouts basically teaming up to double scout a guide since the odds of 2 scouts running into the same guide and thinking to give it a +scout were fairly low, except for the "celebrity" status users of the site. If you look at the scouted guides list and see the scout points that remain from the era when people still actually gave scout points and see how many have a minimum of 2 scout points which was the minimum to be featured, only 5 out of 31 guides were written by authors who have no reason to be known to the wider community beyond writing their guide. The rest are all admins, mods, veterans, guide contest winners and long-time users/unofficial veterans who for a time were active on the forums. And even worse is that pretty much all of those guides were #1 already anyway. Rather than the scout system actually being of any help, it was just a bunch of celebrity authors using it to pat each other on the back for their good work.

But who could blame them? With that system any individual scout giving a +scout had little to no impact. There are 60 other guides on the scouted guides list who only have 1 scout point, every single one of them is a case where a scout tried to fulfill their role but would have more productively spent their time teaching a rock how to speak. This demotivates them and causes them to become more inactive, and with each inactive scout the system becomes more and more useless until now where I literally forget that I even have the option of +scouting guides that I'm upvoting. The scout system in my opinion would work much better if scouts had an impact as individuals.

Also I'm not even a huge fan of the entire concept of having to give feedback. For starters, if I'm going on a holy mission to cleanse MOBAFire of bad #1 guides, I'm going to get a lot of hate from those authors for it, so at the very least I would want scout reviews to have the option of being anonymous.

But aside from that, I also hate the idea that we should be fixing people's bad guides for them. Let's say the main flaw of this Soraka guide is the author recommending really bad first items on Soraka like Locket of the Iron Solari or Warmog's Armor, considering the meta is to rush either Redemption, Ardent Censer or Athene's Unholy Grail. So I tell the author to fix that and they do and great, now it's for the most part a good guide! So I give it a scout point. But now what happens when the meta changes?

If that guy wasn't capable of following the meta on his own before, what happens after he has to figure out what items to build on his own again? What if he doesn't realize the meta's changed and he changes nothing? Or what if nothing changes about the meta but he tests Spirit Visage rush Soraka and thinks it's broken and adds it to the guide after I've already reviewed it as a good guide and given it a scout vote?

I feel like good guides should be written by good authors, not band-aid fixed into good guides by bad authors, because only good authors will be capable of ensuring their guide stays good throughout any meta changes. The scout's role, at least with my version of what a scout vote would do, should be to help the voting system and more importantly the guide rankings be a more accurate representation of which guides are the best MOBAFire has to offer.

I have no problem with anyone that chooses to offer feedback of their own accord whether they're a scout or not, but I would hate for the expectancy of fixing other people's guides for them to be an intrinsic part of my role as a scout. All I want to do is to help allow good guides to shine, and actually feel like I have an impact in bringing down a guide when it's very clearly awful even just compared to its competition. Fixing bad guides is only a temporary solution to the fact that certain people just should not be in a position of authority for their champion on MOBAFire, which is the underlying problem that I would want the scout system to help tackle.




Those are my thoughts on how the scout system would most optimally work. Anything working aside from the vote system like featuring guides in the past worked will always be either too influential or not influential enough, anything requiring more than one scout to use their powers on the same guide will inevitably lead to demotivation with the system and a rapid decline of scout activity.
KuuHaKu_OtgmZ
<Member>
KuuHaKu_OtgmZ's Forum Avatar
Show more awards
Posts:
419
Joined:
Sep 23rd, 2018
Permalink | Quote | PM | +Rep February 26, 2019 8:58am | Report
I'm always here reviewing guides, so I think it would indeed be nice if there was a better system for this.

Quoted:
So I tell the author to fix that and they do and great, now it's for the most part a good guide! So I give it a scout point. But now what happens when the meta changes?
I think that this could be fixed by adding a value to scout votes based on elapsed time.
Let's say a scout voted on a guide on day X
  • His vote in that day has a value of 1.
  • His vote will lose value by a Y rate every day, until it becomes 0.
  • If the season changes then the vote will lose its value, if the season has a major update (changes first decimal) then the vote will lose half of its value, if it's a minor update (changes second decimal) it loses 25% of its value.

Obviously this is just an example (I'm not sure if this is how Riot defines major and minor updates (it's usually done like this)).
Anyway, I'd just like to leave my upvote for this idea, I think it could improve A LOT MOBAFire's guides since people will refrain from making "troll" guides (unless at the right category) just for the sake of doing it without even reviewing its content.
Make sure to check my guides!
Latest Legend
<Member>
Latest Legend's Forum Avatar
Show more awards
Posts:
3039
Joined:
Dec 7th, 2012
Permalink | Quote | PM | +Rep February 26, 2019 10:53am | Report
I think Chromuro's ranking suggestion would be a great way to quickly identify safe guides, a rough estimate in letters (E,D,C,B,A, maybe S) would be fastest and doesn't place too great a weight of scout votes on guides (good guides wouldn't be overly differentiated by scout score, they would all get an A). I wouldn't offer any more details about the scout votes to anyone. (each scout would vote on a letter, or maybe from 1 to 10, all votes would be anonymous, moderators / admins would be the only ones on the lookout for rogue scouts)

Finally, I would let the guides be ordered by vote count primarily (or however the system currently works, "active" votes?), but after that make sure a guide with scout grade X would not be below a guide with scout rate Y < X. This way, guides that don't have a scout grade yet don't get penalized for it, but guides with bad scout grades will sink below those with high scout grades.

For this to have any meaning, scouts would have to vote sparingly. Maybe get to vote once a week. If more than five Vi guides get scout ranking, then the scout ranking slowly starts eclipsing the normal ranking completely, and that would be frustrating for everyone. (of course scout votes would have to decay as well)

Guide authors can always create a topic in the forums (say build and guide discussion) if they would like to discuss their scout grade, but I think it would be bad if the authors could see which scouts gave them which grade, and I think it would be a bit silly if scouts would continuously have to justify themselves with comments or whatever. All the users (read: me during loading screen) care about is that they can quickly see what guides are safe picks.

I don't think scouts are still promoted by rep count, but if so that would be bad in this system. Scout grades would carry a lot of weight, so manual recruitment would be essential I think. I also wonder about the performance of the current scouts. I don't think they would all have to be challenger level players, but I think the worst fears in an impactful scout system are technically incompetent scouts and circlejerking guide author scouts.
********'s a pretty good fertilizer
KuuHaKu_OtgmZ
<Member>
KuuHaKu_OtgmZ's Forum Avatar
Show more awards
Posts:
419
Joined:
Sep 23rd, 2018
Permalink | Quote | PM | +Rep February 26, 2019 11:29am | Report
Quoted:
For this to have any meaning, scouts would have to vote sparingly.
This is absolutely true, maybe add a system where scouts can "remove" weekly votes in case they change their mind after the vote has already been submitted.
Make sure to check my guides!
Maintained
<Stalker>
Maintained's Forum Avatar
Show more awards
Posts:
1102
Joined:
Dec 8th, 2013
Permalink | Quote | PM | +Rep February 26, 2019 12:30pm | Report
Chromuro wrote:
we already know that the "comment to vote" system is annoying as hell and I think nearly no one uses that system as intended (I mean, wait to get to 20 votes, then activate the CtV to see future comments' votes), this doesn't change for scouts.


While I do agree that C2V is annoying, I'd argue that it should be important for a few reasons. Favoritism; in the past, some scouts used to -Scout a guide because of the author, rather than the quality of the guide. The opposite happened too. It's still possible to do so with C2V, but it would be easier to detect if someone were abusing the system for their own cause. Transparency; if the scout system were to be public, you wouldn't want a subpar guide writer wondering why their guides have such low scout scores and coming to the wrong conclusions, then making a forum post about how scouts have it out for them.

Chromuro wrote:
introducing to you the magnificent SCOUTS GRADE!
Well I'm on board with this idea! Forget mine :p I agree with the rest of the paragraph too but I'm not going to quote the entire thing to save some space.


So what if a scout point simply counted as ~5 votes (random number subject to change)?


The would give a single scout way too much power. For the overwhelming majority of guides out there, you'd be able to knock out the top guide out on your own by -Scouting their guide and +Scouting another guide unless you're well over 100s of votes.

But aside from that, I also hate the idea that we should be fixing people's bad guides for them. Let's say the main flaw of this Soraka guide is the author recommending really bad first items on Soraka like Locket of the Iron Solari or Warmog's Armor, considering the meta is to rush either Redemption, Ardent Censer or Athene's Unholy Grail. So I tell the author to fix that and they do and great, now it's for the most part a good guide! So I give it a scout point. But now what happens when the meta changes?


I think most top ranked guides tend to at the very least have the correct cheat sheet (items, spells etc. to an extent), or at least it used to be and they stopped updating the cheat sheet, whether out of laziness or any other reason. If their guide has a bad cheat sheet from start of its publication, its unlikely to reach the top and it wouldn't need a -Scout. If the guide was good at the start, but is now outdated, it would be -Scouted, and if the author does end up updating it, the guide under it would have surpassed it by then, although this part is more theoretical. A scout could also choose to +Scout a better guide instead, if one exists, to help it get to the top, rather than -Scouting the top guide.

Now if an author has a good cheat sheet and understands the meta build, but the rest of the guide ends up falling flat with no content, and they end up fixing it after getting a few -Scouts, then its unlikely that the guide content will be horrible if, for example, the items were to change. What I'm trying to say is, its unlikely for a first item Warmog's Armor Soraka guide would get rank 1 in the first place to warrant a -Scout.

I have no problem with anyone that chooses to offer feedback of their own accord whether they're a scout or not, but I would hate for the expectancy of fixing other people's guides for them to be an intrinsic part of my role as a scout.


A scout's job is not to fix bad guides, but to make sure bad guides don't shine, that I agree with. When you're commenting on a guide, you don't have to explain how fix a guide's flaws, just explain why this guide was warranted a -Scout.

I hard agree with everyone else you've said.

Jovy wrote:
In order to requst a scout review, they must have an in-depth tag so as to avoid spam requests from guides that are not worth a scout point for sure. Also the indepth tag should be changed


Yes please, I see way too many short guides with that tag :(

I love the idea of the pop-up, it acts as a checklist and makes the scouts' lives easier if done correctly. Also, F is for Flash. Looking at your scout requests idea makes me wonder why it wasn't implemented in the first place, it should definitely be implemented imo.

Jovy wrote:
Who should be a scout anyhow?

That's what the recruitment system is supposed to determine, rather than automatically choosing a scout based on their elo :p

Jovy wrote:
Scout points should be refunded every year or something. I think it's better to do it every few patches in theory, because a guide can be scout-approved and not updated for several months otherwise.


As KuuHaKu_OtgmZ mentioned, points should either decay or be made null after some time. How much time that should be is too much for my smol brain to think about.

Special thanks to Jovy for the signature! Check out her shop
FatelBlade
<Member>
FatelBlade's Forum Avatar
Show more awards
Posts:
606
Joined:
Jan 5th, 2012
Permalink | Quote | PM | +Rep February 26, 2019 2:47pm | Report
Any system would have to "force" scouts to remain active to be sustainable. The people on this site are lazy and let various "roles" or "duties" fall dormant since in the past roles were given out according to rep. You need to get people willing to do the job (application process) that are qualified and they have to maintain certain standards. A "good" guide should have a set of objectives it needs to fulfill, such as an accurate cheat sheet, a pros and cons section, how to lane/jungle with that champion, combos etc. This "standard" would be codified and scouts would be able to refer to it and remain objective when looking over new guides.

It may be overzealous, but there should also be an automatic system that assigns different scouts new guides to go over. It may be new guides with an in depth tag (I agree with Jovy that it needs to be changed, the current one is ****) as well as a backlog of current Top 10 guides with the tag, or maybe Top 5. What matters is that scouts would have the obligation to go over these guides, scout them out, and give a rating based on the standards previously mentioned.

In terms of not wanting to fix other people's guides, there are ways to help a guide author without quickly bandaging their problems. Instead of telling someone what items to build, you would inform them that their items aren't meta, and explain why meta items are good (could be linked to a catch-all FAQ with common suggestions). Your role wouldn't be to fix their specific guide but to help them improve the overall quality of their guide writing skills.

Also, there needs to be some oversight regarding established guide writers would gain such a role. A clear conflict of interest is being able to scout or do any work on a champion who a scout has already written a guide for, and I'm not sure if any more restrictions would have to be put in place.

Also, a form of C2V needs to be implemented for anyone performing as a scout. The current system is ****, with an emoji being enough of a comment to vote, and a scout should have the responsibility to actually write out their thoughts on the guide, and why it received the score it did. Random guests and users don't have to justify their votes (which leads to **** guides rising to the top, similar to older sites such as Champion Select) since you're allowing the ****ty majority of the LoL playbase to determine what a good guide is and isn't. Requiring accountability not only helps to make sure all the guides get their fair shake, but also shows to guests and new users of the site that there's a logic behind why certain guides/builds are better than others.
Thank you Jovy for this bomb *** sig!
1 2

You need to log in before commenting.

League of Legends Champions:

Teamfight Tactics Guide