In celebration of launching No Man’s Sky on PS4 this past Tuesday and the upcoming release for PC on Friday, one of Wikipedia’s resident editors over the gaming section, Masem, did an interview with Wikimedia talking about the importance of Wikipedia’s sourcing, their neutral points of view and their ever-growing video game project that collects, catalogs and details information about all sorts of interactive entertainment.

The article details how Wikipedia’s ranked list limits the referencing of false rumors and “problematic content” by utilizing a stringent sourcing rule set governed by the editors of Wikipedia.

Within the Wikimedia article, Masem explains…

“Wikipedia generally excels at covering contemporary topics, like video games, that are featured in online sources—it makes anyone that can access the web a good researcher in the area,” […] “the coverage of these pop topics tends to be some of the best up-to-date comprehensive coverage on the web.”

So how do they get the most “comprehensive coverage on the web”? Sources… and lots of them.

There’s actually an article on Wikipedia dedicated to the video game section’s sourcing methodology. It’s under the WikiProject Video Games section.

The section details that due to the nascence of video game journalism, reliability can be difficult to determine and therefore sourcing needs to be taken seriously. The lede helps establish what Wikipedia looks for when covering video game topics referenced from video game websites, stating…

“Because the fields of video game journalism, research, criticism, and commentary are relatively new compared to similar coverage of traditional media, traditional means of sourcing can be somewhat rare. In addition, the simultaneous development and expansion of Internet-based sources alongside the modern video-game scene has led to a much higher degree of exclusive online coverage than is the case with other media. These factors make the determination of reliable video-game sources a complex issue”

There are nine sections covering what to look for how to look for it when it comes to sourcing information from gaming websites. They also offer a comprehensive list of websites that are good for sourcing, discretionary, and those that are basically banned from being sourced due to being unreliable. The last one is the most interesting one.

Over the last two years we’ve really seen who can and cannot be trusted within the realm of video game media to provide readers with reliable, fact-based information. However, Wikipedia relies on a different form of scrutiny for determining who is and is not reliable.

For instance, websites with user-submitted content are labeled as being unreliable, this includes forums like GameFAQs and even NeoGAF, both of which are considered “Unreliable” on the grounds of “user-submitted” content.

News aggregators like N4G and VG Releases are also labeled as “Unreliable” due to user curated content.

Major corporate websites like Gamespot have exclamation points by them, indicating that their reliability is “situational” and may not be used in every circumstance. The reason for this is because sometimes Gamespot has freelance individuals providing content for the site. There’s a healthy debate that continues around Gamespot and its reliability due to the sometimes flimsy structure of its staffing.

Surprisingly, IGN is not in the same boat as Gamespot despite having a lot of user-generated content and also using freelancers frequently to cover big stories. IGN is considered to be a verified reliable source.

However, in one case a factually ambiguous statement was made in an IGN article regarding the Atari 5200 and there was no other fact-based statements to verify or contradict the statement, leading to an argument about the reliability of IGN’s claim. Masem chimed in stating that basically if a site is known for being reliable then the information is probably good…

“We need to be aware just because a source, known to be an [Reliable Source], printed something, does not make it a true fact (the only fact is, the [Reliable Source] printed that statement). Common sense, other sources, consensus, and the like can all say when a published statement is wrong, inappropriate, or misrepresents the non-opinion-based truth, and ergo this is a case where we would simply ignore one possibly-mistaken statement from a normally reliable source in favor of consistency with all other sources out there.”

Worse yet is that in one instance the biography information on IGN was called into question when one editor realized users could edit the biographies. This led a few other editors to chime in noting to treat the biographies no different than an IMDB source, which can also be edited by the community.

So why doesn’t IGN get put into the same boat with Gamespot if they’re not always reliable? Well, because as Masem pointed out… they’re “normally reliable”. So they get a pass.

And just for reference, IMDB is labeled as an unreliable source by Wikipedia.

Another interesting entry is VG 24/7. They’re considered reliable by Wikipedia, with a page from various editors supporting them due to Patrick Garrett’s pedigree and the rest of the upper staff at the site, with Teancum writing…

“The upper staff seem to be strong, and though there seem to be some unknowns, their articles look to be scrutinized by editorial staff – or at least I didn’t see any issue with it.”

For reference, VG 24/7 was the same site that wrote a preview for Uncharted 4 based on Uncharted 2’s gameplay, as reported by Game Revolution. VG 24/7’s publisher Patrick Garrett also published a hit-piece on Mark Kern on February 17th, 2015 and denied Mark Kern a right to respond.

However, credentials from VG 24/7’s staff seem to outweigh misinformation and hit-pieces; credentials and accreditation. The latter of which is how some sites in Wikipedia’s database make it in as reliable sources whether they have reliable information or not. For instance, Polygon is cited numerous times even though they parroted Kotaku’s misreport about Yooka-Laylee’s budget, but they’re still considered reliable, as reported by TheGG.

Additionally, some sites like legendary gaming outlet Blues News listed due to references from just about every major gaming website from Wikipedia’s reliable source list, thus it’s included as a reliable source. Just for reference, there’s nothing inherently wrong with that.

To Masem’s credit, however, it is mentioned that just because a bunch of reliable sources cite a site does not mean that the information is always reliable, writing…

“As a caution, just being mentioned by other RS’s doesn’t necessarily mean its always a RS. It helps however, and certainly for [Blues New’]s case, there’s no question about it.”

Yet this same logic does not apply to DualShockers, who is not considered a reliable source by Wikipedia. The talk pages refer to DualShockers as “bloggy” and that “it seems to be” spam. In a separate page GameRanx and DualShockers are brought up again as recent as July, 2015, this time a few more editors chime in stating that GameRanx “looks unreliable”, and not that they actually found any legitimate complaints about the content other than that there’s no mention of “formal editorial oversight”.

For DualShockers they mention that sites like CNET, Time, VG 24/7 and Gamespot have cited them for some of their breaking news coverage, but editor CZAR shoots down DualShockers’ reliability, stating…

“It factors in, but in this case I’d say that DualShockers is being cited the same way they would cite a third party blog or critic. Professional credibility is usually a factor alongside (not in replacement of) editorial oversight. But I’ve said enough on this so others should chime in”

For as far as we know DualShockers has yet to lambast a game based on preview footage of a game released seven years ago. Nevertheless, in the eyes of Wikipedia credentials outweigh massive errors of misrepresentation.

But if citations aren’t enough to cover for credentials, why is The Jimquisition considered a situational source? According to CZAR, there’s no explanation given why Jim Sterling is more reliable than DualShockers, but he does say that he’s less reliable than sites with editorial oversight, writing on July 4th, 2015…

“ As a one-man outlet, the Jimquisition is feasibly never the best source for statements of fact. Even then, I’d only use his opinion when it is notable, as in referenced by other outlets, but even then I’d cite that secondary source instead of Jimquisition directly… I’d also be hesitant to give him “self-published expert source” carte blanche and I think the guideline’s advice on that mirrors what I just said. So, situational.”

Other editors were definitely more willing to give The Jimquisition the “Reliable Source” badge, based on his work at Destructoid and The Escapist.

So then clearly a site like TechRaptor, with a dedicated staff, editorial oversight and an actual ethics policy would be eligible for being considered a reliable source? Right? They’ve even been cited by some of Wikipedia’s reliable source alums, so that warrants a badge of honor, yes? Apparently not.

According to Wikipedia editor Thibbs, back in October of 2014 he explains why TechRaptor isn’t eligible for the “Reliable Source” moniker, writing…

“Well I looked over the staff roll but I don’t see a lot in the way of credentials. There are some college students and some graduates and of course there are quite a few video game fans, but have any of them working within the industry or written for other RSes in the past? Just having a staff roll and an editorial policy are good signs, but they aren’t conclusive. Similarly, interviews with big-name interviewees is a good sign, but it’s far from determinative. One of the key questions is whether a putative RS has a “reputation for fact-checking an accuracy”. To look for a reputation we look to see what the other estalished RSes are saying about the source. On my own I see VG247 citing them here. Apart from this I find little or nothing.”

So having editorial policies, having college graduates, having oversight, and even having citations still isn’t enough for some sites, but if you’re Jim Sterling with no oversight whatsoever you’re a situational source. Is that correct?

In the case of Extra Credits, despite not having editorial oversight, despite being a YouTube outlet focused mainly on opinions, and despite not having an ethics policy, Wikipedia editor Thibbs actually confessed that he was on the fence for giving them the “Reliable Source” green check, writing…

“Regarding the reliability of this source for factual matters, I’m on the fence. Personally I rather like the show, but a lot of it is opinion-heavy and I’m not finding a lot of info on editorial policies or author credentials. It seems that there was some degree of industry connection from the outset in the form of James Portnow, and they’ve been associated with Edge, Escapist, Penny Arcade, and Screw Attack in the past, but beyond these groups I see few citations to them.

 

“So if we just look at the source devoid of context then I’d be dubious of its usability as an [Reliable Source], but since it has been used by several of our listed [Reliable Sources] in the 6 years since it was started, the question is whether they have gained sufficient reliability to be listed as an [Reliable Source]. I do see coverage from rockpapershotgun, bit-tech.net, polygon, gry-online, and several other listed RSes, which clearly shows they’re notable. So again I remain on teh fence about its factual reliability.”

Extra Credits is not listed as an “unreliable source” the way TechRaptor is, despite the fact that there’s no legitimate reasons given why they should be considered a reliable source other than the fact that some of the other sites that Wikipedia lists as “reliable” have cited them. In a way, it almost feels like that “citogenesis” meme that was floating around after Slate writer David Auerbach brought up how you can essentially turn a fabrication into a truth on Wikipedia by circulating citations of false information through Wikipedia’s reliable sources.

The citation conundrum reached fever pitch when Wikipedia’s editors directly began to brigade David Auerbach’s personal life, attempting to get him fired after he exposed a bureaucratic circlejerk of ideological aggrandizement by specific editors, as reported by Breitbart.

So where does that leave readers? Well, more than anything it’s imperative to check the sources. Wikipedia picking and choosing which sites are reliable is entirely dependent on the ideological preferences of the editors.

While niche sites Gematsu and Siliconera are considered reliable, Niche Gamer is not. While The Mary Sue is considered reliable on a situational basis, OnRPG is not. While Vox-owned SBNation is considered reliable, the home of EVO, Shoryuken, is not. While Polygon is considered reliable, Eventhubs is not. And while Kotaku is considered situational for its blog-tier content, Blistered Thumbs was considered unreliable simply because it was a sub-division of ThatGuyWithTheGlasses.

Ultimately, if you want to be considered a “Reliable Source” for your gaming coverage by Wikipedia, you have to meet the following criteria:

• You need to have an established staff of writers who have contributed to Wikipedia’s already established pool of “Reliable Sources”.

• You need to have editorial oversight unless you’re really, really popular, like Jim Sterling.

• You need to have an ethics policy, unless you’re already producing content cited by Wikipedia’s “Reliable Sources”.

• You can’t be a solo act unless you’re also really, really popular (like Jim Sterling), or the Wikipedia editors like your content.

• You can’t rely on user-submitted content unless you have enough reliable citations from Wikipedia’s “Reliable Sources” to outweigh user-submitted content.

• You need to be cited frequently, but only by sources recognized as “Reliable Sources” by Wikipedia’s editors.

• You can’t be a news aggregator.

• You can be a review aggregator.

• You can’t be a blogger, unless you write about things that Wikipedia editors agree with, then you can be used as a reliable source under situational circumstances.

• Most importantly, if there is “un-discussed determinations” on your site or “Wikipedia Silence” from editors, then it’s likely that your site is “unreliable”.

(Main image courtesy of Ashion)