social.outsourcedmath.com

User Verified Fake News Moderation System Idea

I suggest (for the tech gurus out there) developing and implementing the following open source user run fake news moderation system (as a module or addon):

Add a user verified fact check flag (not the same as the reporting option) with three states: opinion, submitted for fact check, user verified. The default state is opinion on all posts.

Any user can publicly flag a post to be fact checked but that user must also provide the fact check as a comment. (Simply flagging is insufficient since one must be a part of the solution to not be a part of the problem.)

The original post is then automatically moved to a fact check team moderator account so it is seen by all users that follow it as members of a "fact check team". (Comments are disabled when the flag is set for fact check and the original post cannot be liked, disliked, or shared while in this account being "moderated" over the course of ten business days.)

Fact check team members can like or dislike the fact check comment to "vote" in support of the fact check or not.

The original poster is notified their post has been moved to the moderator account as a result of being flagged with a fact check submitted. This triggers two possible actions:

The original poster has the opportunity to provide a single response countering the fact check.

Fact check team members can see the counter response and "vote" on it separately from the fact check.

Over the course of the ten business day moderation period, the "votes" are automatically tallied.

If the response from the original poster has more votes than the fact check submitted, the post is restored to the original poster's account and the fact check flag is set to user verified. The fact check and the counter response become historical (those comments cannot be reshared). This does not prevent the original post from being flagged again for fact checking because sometimes new information becomes available.

If the response from the original poster has fewer votes than the fact check submitted or does not exist, It is marked as fake news and becomes historical in the moderation account (removed from public view and not returned to the original poster's account).

Any critiques, ideas, or takers?
This entry was edited (5 years ago)
This is a user run verification system. The person submitting a post for fact check is providing the information. I could even submit my own post to be fact checked to get user votes on it so it can be user verified. Others could do the same initially or subsequently if new information is available countering the original user verification. If my original post provides a link to factual information, it is not as likely to be flagged. Will people attempt to game the system? Sure they will. But the ability to have subsequent fact check requests puts the power back into the hands of users to counter the gaming.
This entry was edited (5 years ago)
ivan zlax hubzilla (AP) (via DFRN)
@Shelenn Ayres i do not understand how will the fact check be conducted.
How is the "fact check team" different from ordinary experts? Why their checking could be trusted?
Example: let's say someone who is an anti-vaxxer makes a post citing pseudoscience or simply stating something that has been debunked by peer reviewed research or scientific fact. Rather than leaving it up to a single moderator or fact checker to read and moderate every post, someone who has read the post that believes the original post is fake news can submit it to be fact checked. The submission requires the person requesting the fact check provide the factual information showing why they believe it is fake news.

The "fact check team" consists of moderators who own the moderation account and all users who follow that account so they can vote on posts that have been flagged for fact check.

Second example: a user posts inaccurate information about a political candidate's record. Another user flags it to be fact checked and provides accurate information from the candidate's congressional track record. The "fact check team" reads the new information provided checking it themselves then votes.

During the moderation period, the post is not visible to the public - only the "fact check team". This temporarily removes potential fake news from public view and prevents resharing of it until it is found to be factual or permanently removed from public view as fake news.

In the case the post is restored, the fact check information is historical so we all learn. Any official sources can be used by the person flagging a post to be fact checked as appropriate. UK law does not apply to US law, US law does not apply to EU law, and so on. There are places where reciprocity between allied countries exist however but that becomes points of discussion in general that we all learn from. But with science, there are global sources based on accepted scientific fact backed by peer reviewed research.

Any ideas for improvement in the idea are welcome!
ivan zlax hubzilla (AP) (via DFRN)
@Shelenn Ayres
Example: let's say someone who is an anti-vaxxer
Sounds great. Can i ask you for an experiment to do fact-check for all the comments from this post? This is just the case.
makes a post citing pseudoscience
But what about fringe science? Is it permissible?
or simply stating something that has been debunked by peer reviewed research or scientific fact
Sorry, but it looks like neoliberal censorship method.
Please look to this article: https://www.insidehighered.com/news/2019/01/08/author-recent-academic-hoax-faces-disciplinary-action-portland-state
Western peer reviewed sources this is not a reliable source of information for me after this case.
Also, somewhere in China, India or Russia, something may be called scientific that will not be allowed to publication in Western academies. If i understand correctly, are Western Academies a priority for you?
The "fact check team" consists of moderators
If you wrote it before, then i would have no questions. This is not about checking the facts, but about moderation. This is not the same thing. No need to replace the concepts.
Any ideas for improvement in the idea are welcome!
My idea for improvement is simple, do not use "fact check" concept instead a "moderation"
Shelenn Ayres friendica (via ActivityPub)
@zlax Clearly you misunderstand the original post. Perhaps someone else can help explain it. Your reference to neoliberal is a clue to your perspective. This idea is entirely apolitical with full transparency and fairness on all sides.

Science requires peer review and repeatable experiments to validate results. Pseudo- or fringe science (they are the same thing) is defined as science that is not valid so if that is challenged by peer reviewed science and accepted fact, the fact check team decides which is fact and which is fiction. Otherwise, expert services have to be paid for, admin salaries have to be funded because the time and labor is excessive interfering with their ability to pay bills, and users will not be able to use services for free without ads or other monetization (which is antithetical to the free social network concept).
This entry was edited (5 years ago)
I love this idea and one of my delusions of grandeur is to make an argument forum that can resolve all but intractable arguments. However, please be advised...

In all human history a cycle exists:
  • A Society or Group forms for a common good to protect itself from some greater harm to its members.
  • This group attains some modicum of power and/or influence to do good for its members.
  • Corrupt nefarious entities recognize and desire that power.
  • Said corrupt entities usurp control of the system or cause chaos within the system to redirect its effectiveness for selfish antisocial reasons.
  • goto 1
Shelenn Ayres friendica (via ActivityPub)
Yes we know human nature. The idea is to be transparent putting the power in the hands of the users. My idea is general enough to address potential gaming of the system which in turn is addressed by subsequent flagging. Something is better than nothing at this point. Currently, admins have the power to control users without public redress. Users have the power to move or create their own node. However, admins have the power to block entire nodes as well. This is a private space but most admins like myself would like it to be monitored by the community and not a single admin. Policies have to be enforced if they exist and that requires a lot of labor to read every post as a node grows (in addition to administrative labor and costs to provide the service for others to use for free). It is not feasible to expect any admin to have time to review and address every single post even on a small node. My idea is to crowdsource in a fully transparent manner that would be spelled out in the terms.
ivan zlax hubzilla (AP) (via DFRN)
@Shelenn Ayres
Clearly you misunderstand the original post.
I ask you to clarify to me what i did not understand.
Perhaps someone else can help explain it.
Your last sentence is an invitation to discuss and clarify. I appeal to you, Shelenn.
Your reference to neoliberal is a clue to your perspective. This idea is entirely apolitical with full transparency and fairness on all sides.
If this is so, then i ask you not to be irresponsible, and directly answer the questions that i asked you in the previous message (about fringe science and Western Academies).
And i also ask you to fulfill my request - to carry out a test "check the facts" of the link - which i left to you about vaccination. Using your method.

I am fulfilling your request for "Any critiques, ideas". Please reciprocate me and fulfill my request and answer the questions. Please do not be irresponsible. Thank you in advance for this.
Shelenn Ayres friendica (via ActivityPub)
@ralphy Those are valid points but the proliferation of fake news in social media has shown us the far reaching negative impact on society - everything from discrimination to murder. This idea directly allows the "fact check team" (crowd sourcing) to settle disputes when a post is challenged as not factual yet removes it from public view while in dispute. Anyone can be involved on any side allowing direct participation and transparency.

Some companies and organizations have attempted to supply expert fact checking, flags on posts, flags on images, flags on videos, etc. Some of these experts have been shown to have bias in their determinations. Some companies are relying on more than one fact check source they pay to review content for them. But in all cases, these are private spaces where the owners of those spaces have the legal right to restrict content.

What we've learned as a society over the past few years (due to technology exploitation) is we must come up with a system that: puts the power regarding user generated content into the hands of the users rather than corporate or government entities, allows as much freedom as possible without harming others, facilitates learning and the spread of truths among the userbase, helps foster cultural awareness in a global society, promotes democratic community ideals, and prevents the spread of fake news regarding public concerns like climate change, health, and politics. Realize this idea is focused on global community run social media - not corporate or government owned social media.

This idea like any idea needs to be vetted and all concerns addressed. But, I have yet to see a concern raised that is not addressed by the design of the idea (even though it is clear there are some language barriers preventing clarification). A single moderator or company authorized team, government source of information, medical research source, etc is not making the determination. A group of community users, unlimited in size, votes on each challenge on a case by case basis. In that process, arguments from both sides are considered and vetted. In the end, users have learned from each other, engaged in civil discourse, and helped prevent the spread of potentially fake news. You are correct, no one can be trusted 100% and new information comes available all the time as we learn things. Subsequent posts and fact check requests can be made which helps fine tune the system, helps combat abuse within the team, and helps combat those attempting to game the system. Crowd sourcing in this manner in a transparent way also addresses costs making it more feasible than a donation model to fund salaries of paid moderators.
This entry was edited (5 years ago)
How about levels of prestige for the fact-check people, like that ring of trust thing does. Members who can be contacted in real life get more weight than anonymous types. Members of the domain are higher than non-members.

This whole problem is hard because truths are at most countably infinite, but fiction is at minimum uncountably infinite. Fake stories and supporting arguments for fake claims can grow exponentially fast whereas fact checking is complicated and moves slowly and linearly.

The idea of flagging with a quote of the disputed claim is great because it makes a hard problem of mere flagging with no specifics into a solvable problem of verifying a single specific statement. Thus a fact check flag should be only permitted with accompanying of a single statement in the content that needs checking. One user would possibly place many flags on a single post. Other users would agree with the challenge, or disagree and add a mandatory link to some evidence or a post outlining why they think the original claim is defendable.

The whole challenge and counter should have the standard fallacies of logic as checkbox add-ons to the item. Perhaps the logic is solid but the premises are wrong, so there would also be checkboxes for the premises being wrong.
Shelenn Ayres friendica (via ActivityPub)
"Participation Badging" is certainly viable and a proven positive approach in general for social media or games. This is a great idea!

However, rewarding submissions for fact checking would reward a user for submitting a fact check on a false post as well as someone submitting a fact check on a true post. So I am not sure about that reward.

If we reward users every time one of their original posts get a user verified status (considered NOT fake news by the fact check team), we may generate more work than the team can handle within the moderation period BUT it would lead to a lot more user verified content. So this might be a good approach and is positive reinforcement. This wouldn't necessarily need to be a badge reward but simply information on a users profile showing how many user verified top posts they have.

Regarding the challenge itself, as a rule the fact check team would presume the original post is true then consider the information provided by the user requesting the fact check. Some will be clear cut while others will be a matter of opinion and not clear cut proof. The information submitted must provide a factual basis the fact check team can vet. In some cases, information learned later can result in a subsequent fact check submission.

My original thought was more along the lines of preventing the proliferation of fake news. If an article being shared has been debunked already by non-profit fact check organizations, one might simply provide that information as part of a fact check submission. If someone is sharing a political article that misstates facts about a candidate's record, one might submit the record as part of a fact check submission. If someone is sharing pseudoscience or fringe science that has been disproven by peer reviewed science, one would include that information as part of a fact check submission. If someone shares an article where a politician tells a lie in support of that politician, one might submit the proof of the lie as part of the fact check submission.

I do agree we should allow some discussion on the posts being vetted between the fact check team members for transparency. So we would need to expand the idea to include that.

However, I get what you are describing in further detail with the check boxes etc but the devil's advocate in me sees some concerns:

We know that one cannot get full meaning from a single sentence within a paragraph or a single paragraph within an essay and so on. So I don't think we can vet in that manner because that would result in far too much out of context interpretation.

An approach similar to Politifact, Punditfact, Factcheck.org, and other non-profit public news fact checking is what I had in mind for the fact check team. So you prompt some ideas for a user verified rating scale of sorts. How do we ensure personal beliefs are not going to influence scaled ratings?
ivan zlax hubzilla (AP) (via DFRN)
@Shelenn Ayres
Asking to carry out a test makes it clear you do not understand the original post
Not this way. A request for a test is an attempt to demonstrate that you approach this issue in advance from the biased positions. ("anti-vaxx is fakenews and needed for checking, but pro-vaxx is not")
My attempts to clarify have failed.
You have not even tried. I asked you specific questions, you ignored it. I'll repeat:

How will the fact check be conducted? Is it enough to attach a link to the "official" source?
and
> makes a post citing pseudoscience
But what about fringe science? Is it permissible?


This is an important questions. The answer to which can show the motivation of your initiative. I ask you again: please do not be irresponsible, answer it. Thank you in advance.
It does seem there may be a language barrier no offense intended.
No, it is not. Please do not build this barrier. I understand you well, please do not be fenced off from me.
Ivan your questions have been answered
No, my questions was ignored. I repeat it again in this message, hope you will not ignore it again.
Crowd sourcing also helps the crowd learn facts and prevents the spread of fake news.
So far, the system proposed by you is no different from ordinary collective moderation. I don’t understand reason for why you to replace concepts "fact check” and “fact checking team” using instead of "moderation" and "moderators".
ivan zlax hubzilla (AP) (via DFRN)
@Shelenn Ayres you again ignored my questions, very sorry.

You invite to criticism and ideas, i fulfilled your request.

So in result i offended you with this. Sorry for that.

But this is not a constructive discussion, it is some kind of emotion.
I don't really have time for this, but Ivan, I believe initially you were asking Shelenn to perform a service with a system that does not yet exist.
Can i ask you for an experiment to do fact-check for...
And to perform the non-existent service on content that has been widely debunked for a long time now.

Your dialogue on this thread doesn't seem to have an attitude of productive collaboration. I could be wrong.

People can produce their own proof of Russell's Teapot very rapidly in this modern age of Photoshop. Unverifiable claims are fictional on their face. e.g. you can open any physics book and reproduce the experiments yourself to verify them, but claims in holy books cannot be reproduced so they are unverifiable.
I agree with Korn a bit on his (Just on the science topic) but not as a rule. The non-perfect system of peer-reviewed journals does have some disinformation slip through. It doesn't mean we toss the baby with the bath water. If our society didn't have so much "Publish or Perish" in academia, a lot less disinformation might exist in the journals. If the pride of existing experts in a field didn't blind them to new evidence refining their model, we would be far ahead of where we are today. If there were laws against industry paying for research to make them look good, disinformation from academia may not exist.

I diverge mostly on the idea of letting people investigate and come up with their own conclusions. First, we are evolutionarily modified to look for confirming evidence not disproving evidence, so if something isn't flagged as unverified beforehand the likers will run with it and never look back. Second, were mostly stretched to our limits, so don't have time to check things; typical people stop at the headline and run with that. I've shown many people that the content of an article link they sent me disproves the claim they were trying to verify with the link. Third, look at where we are today, most people aren't good at this stuff. The end of the latest "This American Life" (Beware the Jabberwock) bears it out towards the end of "Act 2 - Alex in Wonderland when Jon Ronson says he noticed "Something very strange kept happening" in the last 6 minutes. Our desire to not pull cards out of our potential cardcastle of knowledge is the self-preservation instinct to avoid an existential crisis. As somebody who's been through a few, I like them, but it's an acquired taste.
Shelenn Ayres friendica (via ActivityPub)
@wnymathguy I agree these are valid points of concern. But I also have a little faith in the user population and let's not forget the moderators that own the account used for the fact check team are there to address concerns. The idea is not asking the fact check team to become fact checkers. It is asking the team to review both sides of an argument presented regarding whether a top post is true or not.

For example, as a team member I am going to read what is posted and compare that to what is submitted.

If what was submitted is opinion without factual sources, I would vote dislike on the submission (no) because it does not prove the original post is not true. (Sort of like innocent until proven guilty.)

If what is submitted is a direct factual debunk of the original post and I believe that source is legitimate (like a congressional record, NOAA data, etc), I am going to like (vote yes) the submission provided proof the original post was fake news.

So let's look at a science topic: I totally agree that we have far too much money influence in academia and research. But I also believe these things would be considered and facilitated by the moderators for the fact check team on controversial topics.

In the case of climate change, we know 97% of scientists around the world believe climate change is not only happening but is caused by human behavior that could be changed. There are lots of articles on this available. It is up to the fact check submitter to provide that information on a post that is supporting climate change as a hoax. The same is true for anti-vaxxers whose views have not only been widely debunked by scientists and medical professionals but whose views have led to needless deaths and harm to others. Allowing the proliferation of either of these fake news topics has direct impacts on human life.

Any other ideas on a system of checks and balances are welcome 😀 The larger the team that more valid the results. The vast majority of social media posts do not need any form of moderation - picture sharing, support, convos, and opinion sharing. When a post is made that is fake news and no one reports it, the fault is on the poster and the community not reporting it. If it didn't get reported because the poster didn't have a lot of followers, the negative impact of that missed fake news is far less than for one that was reported. Again, the idea puts the power into the hands of the community to avoid users having to pay for expert moderation services. The people who work for high volume fact check organizations are not dong it for free - because those are full time jobs. My idea is to hopefully address the issues in the free social network while not adding labor costs.
This entry was edited (5 years ago)
You made me think of a work queue with that last post. Also, are you using the word posted as the top node of the thread, and submitted as comments linked to the top node?

So a work queue of posts to check would be formed by any member of any status flagging a Post or Submission and anybody with fact-check status can put their time in on evaluating that queue. Within the queue, any member that's a potential "carrier" (human virus), i.e. a person with many followers, would have higher priority and so be worked on first.
Shelenn Ayres friendica (via ActivityPub)
OH that sounds like a good idea setting priority by potential views!

On the other question, I originally hadn't thought of comment posts - only top posts. But I suppose comment posts should be subject to fact check submissions as well if we are to be thorough.

This website uses cookies to recognize revisiting and logged in users. You accept the usage of these cookies by continue browsing this website.