recaps of the top 'ask me anything' interviews from reddit and more...
I built a platform for journalism with “open source” fact checking. In the age of information (and misinformation) overload, the goal is to help the best journalists stand out by making their fact checking process fully transparent and reviewable. AmA!

There are a lot of projects out there trying to detect and flag misinformation. Even though I know these projects have the best of intentions, I believe they’ll never be able to keep up. The energy it takes to refute bullshit is much greater than the energy to produce it. \1])

Instead, I’m trying a different approach. I recently launched this platform for journalism with “open source” fact checking. Journalists can easily annotate their articles with their fact checking so that the readers can verify the evidence and reasoning for themselves, whenever they like. The hope is that by making it easy for the best journalists to “show their work”, in a way that’s easily reviewable by readers, they'll stand out from those who have no work to show.  You can check it out here

My name is Yaz Sinan, a programmer out of Toronto who’s been experimenting with building fact checking tools over the last 3 years. In that time I’ve also personally participated in over 500 fact checks. Ask me anything!

Proof

P.S (worth mentioning):

–  this approach only works for journalism covering information based on publicly reviewable evidence. This includes legislation, public government initiatives, whistle blower documents, and scientific data. This isn’t a good fit for journalism based on undocumented sources.

- This approach doesn’t eliminate bias. One can provide completely accurate facts and still introduce bias by omitting facts that don’t agree with their views. I do think however that helping the accurate provable facts stand out from everything else would still be a meaningful improvement to what we have today.

- I don’t expect the average reader to click into and explore the evidence for every claim. Just like the average consumer of open source code rarely reads the code. The point though is that it’s out there for anyone who wants to check it, so whoever wants to double check can do so anytime.

- Our next big project is fact checking the commitments and track records of the Democratic Presidential candidates. DM me if you’re interested in participating!

Footnotes:

[1] Paraphrasing Alberto Brandolini

Edit 1: Wow! Thank you all for this incredible response! If you like the idea of journalism with completely reviewable evidence and and reviewable reasoning - subscribe to SourcedFact articles! Everything we research is under the Creative Commons License, so completely free for anyone to read and completely free for any journalist to use in their own work. https://sourcedfact.com/subscribe/

Will be signing off for tonight, but will continue to answer questions tomorrow. Thanks again for all of the support!


27
questions
11,754
score
August 22nd 2019
interview date
PigBenis3

I like the fact that the platform is strict about primary source evidence. Videos are sometimes considered good primary source evidence though, how do you handle video deep fakes, or edited videos?


yazIam

Now that video deepfake technology exists, most videos are not acceptable evidence on SourcedFact. There are exceptions. One example is a video of someone speaking validated by that same person. Say Trump tweets a video of himself at a campaign rally. In that video, he says X. That video would be acceptable evidence for claims that "Trump said X".

LTT82

journalism with “open source” fact checking.

I don't quite understand this. Where do the articles come from? Who is doing the fact checking? Are you hiring journalists to write for your publication?

Which part is open sourced or is all of it open sourced?


yazIam

By "open source" fact checking I mean that every claim comes with a reviewable fact checking page that includes both the primary source evidence and the reasoning behind why that evidence proves that claim. Really I'm stealing the term "open source" from software here and defining it to mean reviewable fact checking. Check out any of the articles here to see what this reviewable fact checking looks like!

> Where do the articles come from?

Currently I work with a team of fact checkers to produce the content. Any journalist or concerned citizen is welcome to submit content if they like, as long as they always back it with reviewable fact checking. Anyone is free to use this concept of "open source" fact checking on their own sites, and I'd happily help journalists who would like to adopt it.

Cassi-Chaos

I don't see any advertising on your site, how are you funding this?


yazIam

Self funded so far. At first it was just me coding. I eventually recruited a small team of paid fact checkers and got them using the platform. The SourcedFact that you see now is the result of the learnings from working with these fact checkers.

I deeply believe that making it as easy for journalists to share every step of their fact checking process with their readers will help the best journalists and the best information stand out in this sea of noise. So even though I may start looking for sources of funding soon, worst case scenario I’ll keep funding SourcedFact to the best of my abilities. 

Edit: Added second paragraph

itsacalamity

I'll preface and say that the site isn't working for me, clicking the buttons makes nothing happen.

That said-- as a journalist getting paid peanuts and having a dozen stories on deadline, what's my motivation for taking the extra time to do this? How do you actually see this being adopted by people in the field?


yazIam

I'll preface and say that the site isn't working for me, clicking the buttons makes nothing happen.

Apologies, there seems to be a bug on tablets with bigger screens. Will fix.

That said-- as a journalist getting paid peanuts and having a dozen stories on deadline, what's my motivation for taking the extra time to do this? How do you actually see this being adopted by people in the field?

The idea is that by making their fact checking completely reviewable by readers, can earn the trust of more readers. I'm sure if we prove over time that readers are more likely to engage with and trust content with reviewable fact checking, then I imagine more journalists will be inclined to put in the time.

Will this work to earn the trust of every reader? Definitely not. Some people don't care about evidence. But this will be the best way to earn the trust of those who do care about primary source evidence and sound reasoning.

We're also aiming to make the lives of journalists easier, as I agree, they're getting paid peanuts and are working on crazy deadlines. Every fact check that we've authored on SourcedFact is freely available for journalists to use under the Creative Commons License. As our repository of fact checks grows bigger, we hope to make their lives easier and easier.

All-TimeGringo

Independent fact checking sites like Politifact and Snopes are more popular than ever, yet it doesn't seem like consumers of news are any more well-informed than before they existed. Why do you think Sourced Fact will be any more impactful than these other sites?


yazIam

Sites like Politifact and Snopes are ‘post-publication’ fact checking. They aim to fact check statements that have already been spreading. SourcedFact is about capturing the fact checking process that good journalist go through while researching their articles and then helping them share that with the readers.

Post-publication fact checkers alone won’t fix the misinformation problem because they just can’t keep up. The energy it takes to refute bullshit is much greater than the energy to produce it. If we make it easy for journalists to “show their work” - that is, to share their evidence and reasoning with their readers, whenever possible, in a format easily consumable by readers, maybe more and more people will start expecting journalists to "show their work" as a starting point. In that kind of world, instead of trying to disprove every false statement, more and more people would require that something comes with primary source evidence attached before they even consider it.

Now you may ask whether or not building a societal expectation like that is an achievable goal. Who knows. First I’d like to build a resource for open source journalism - journalism where the evidence and reasoning is always reviewable by the reader - for anyone who wants it, however big or small that group of people may be.

I should say though, there are precedents for that type of expectation. Imagine going to the grocery store tomorrow, and all of a sudden all the food items no longer have ingredients listed... we'd all freak out. We expect to be told what's in our food. Maybe one day it will be an expectation that journalism always comes with the work that went into it - reviewable evidence and reviewable reasoning.

Edit: Added last 2 paragraphs

Rav3na3l

I think this is super cool and incredibly useful. I hope the project goes well.

I do have one question though. We live in an age of journalism where some journalists will cherry pick facts to suit their story. Do you have any plans to maybe publish a list alongside the project with a list of journalists who do the most accurate and researched reporting?


yazIam

Thank you u/Rav3na3l :)

You're right that one provide a list of completely accurate facts and still introduce major bias by which facts they omit. The plan to tackle that is to enable anyone who wants to contribute to SourcedFact, regardless of political leanings, to contribute - as long as they support their claims with primary source evidence. Not a prefect solution, but hopefully it will help. Also I think that just helping the accurate provable facts stand out from everything else would still be a meaningful improvement to what we have today.

I don't have any plan to create a journalist ranking site or anything. I think post-publication fact checking is a losing battle, because there's no way to keep up. As mentioned above, the energy it takes to refute bullshit is greater than the energy to refute it. Beyond that, sites that rank journalists have to earn the trust of their readers just like anyone else, so unless they provide all of their evidence and reasoning, I don't know why anyone would trust them any more than the journalists they're ranking.

ColinHalter

How is this any different than inline citations with a rollover action to show a blurb about the source? It doesn't really seem like you're doing anything new here.


yazIam

The verification pages on SourcedFact include a much more thorough discussion about the authenticity of the evidence and the reasoning behind it than what you'd have in a rollover blurb. Here's an example of fact check that just wouldn't work in a blurb, from this net neutrality piece.

Further, some of our fact checks require multiple documents to prove, and their fact checking pages get quite sophisticated, here's one that required over 20 documents, from this article.

WeakEmu8

Why only Democratic candidates?


yazIam

Just because they're selecting a Presidential candidate right now. You're welcome to start a project on a different topic if you like of course, as long as the claims are backed with primary source evidence. DM if you're interested.

Though, this is just the current topic we're working on, as you can see on the site, we've done many others, and will do many more in the future.

LiberateJohnDoe

How will you deal with anonymous sources -- informants and others who require anonymity? Sometimes it's not only leaking their name or personal info that puts them in danger, but revealing secondary information (like who would be the few prior who had access to a certain bit of information).

Traditionally, these points would be vetted between journalist and editor or board of editors (and possibly lawyers). There are certain functions of an editor that may be difficult or impossible to reproduce.


yazIam

Just going to paste this excerpt from my post description above, apologies for the repetition: "this approach only works for journalism covering information based on publicly reviewable evidence. This includes legislation, public government initiatives, whistle blower documents, and scientific data. This isn’t a good fit for journalism based on undocumented sources."

Anonymous sources are an important part of journalism, but that's just not SourcedFact's focus. I think we can add a lot of value by just creating a place where people can go to verify claims proven by public evidence.

jpastore

If facts are omitted to create bias, wouldn't allowing people to supply those facts disarm the bias?


yazIam

Indeed! One of the upcoming features I'm working on is a 'facts comment section'. Where readers can attach facts to an article they think provide missing context, as long as they provide reviewable primary source evidence and reasoning in the exact same way the journalists are required to on SourcedFact.

The_Wanderer2077

So the facts are open sourced, but the platform isn't?


yazIam

I intend to open source SourcedFact but because I would currently need to separate out the anti-spam and anti-gaming code, doing that work just falls under some higher priority items right now. One higher priority item would be to move user contributions to a distributed ledger so that there’s a new layer of transparency regarding which user fact checks get rejected. My hope is that making that completely transparent while keeping the anti-spam and the anti-gaming code closed will be the best of both worlds.

I will however quote a HN commenter who commented on the same question about SourcedFact being open source in the past, as he worded it way better than I did: "That's fine with me. The two concepts of "open-source" at hand are drastically different. The lack of ability to verify the website's source code doesn't prevent me in any way from seeing its content which helps me verify journalist claims." https://news.ycombinator.com/item?id=19576320

LadySalvatore

I've seen some projects in the news that are trying to use AI to detect 'fake news'. Does SourcedFact use any artificial intelligence?


yazIam

Currently no, the priority is to make journalists' evidence and reasoning as easy to share with their readers as possible.

I think artificial-intelligence-based solutions to misinformation have to work to earn the trust of their readers just like anything else. I think the only real way to do that is to always show your evidence and reasoning. So, AI based solutions will have to make their conclusions as to what’s true and what’s false easily auditable by their readers. If they don’t do that, they’re just expecting people to trust them because they have an AI, and that’s just nuts.

Having said that, I think AI will be a very important aid in journalism in the future, but again, only if the conclusions are made to be easily reviewable by readers.

Randomundesirable

What happens when you have a very large number of biased people doing the fact checking ? I'm specifically worried about China and India. India moreso because if relatively open internet access and English speaking population. For example If you go to r/worldnews any news article about the internt/communication lockdown in kashmir is immediately downvoted to oblivion .


yazIam

Just going to plagiarize myself from a previous comment:

We manually vet fact checkers before making their contributions public. Now, why should you trust that we're vetting people well?

The point of “open source” fact checking, is all the primary source evidence and reasoning is completely reviewable by you, the reader. So if we ever screw up, a simple click by a subset of readers into the fact checking will show them that our evidence or reasoning is bogus. So by design “open source” journalism would reveal manipulation if it ever happens.

No outlet or platform should ever be given carte blanche on accuracy, no matter how good a job it’s doing today. Say you click into some of SourcedFact’s fact checks today and find that they check out. Doesn’t mean you should never check to see if we’re still doing a good job again. This applies to any outlet, no matter how old it is. Human organizations, like newspapers and news sites, are subject to change, and eventually decay. SourcedFact is no exception. The point of “open source” journalism is that we always present our evidence and reasoning, so whenever you want to check if we’re still doing a good job, we make it as easy as possible for you to do that.

okapidaddy

Nice project. Do you think there should be a certification system for journalists? Maybe even a licensimg system? It's been kicked around here and there in j-schools for decades, but always rejected. Perhaps it's time? Thoughts?


yazIam

Interesting question. My thoughts are past history is no substitute for evidence. Why rely on the fact that a journalist passed some test a while ago when they can just provide their evidence and reasoning in a way that's easily reviewable by readers every time. At least, whenever that's possible, ie: whenever the evidence is publicly accessible primary sources.

Sometimes we have to decide who we trust by past performance, I just don't think that'll be necessary for journalism backed by public document evidence.

The_Oddity_

I'm late to this party, but just wanted to say how innovative and incredible this idea is. It's a very inspiring project in this age of misinformation, and way to go for spending so much time on creating this.

What gives me hope that this project has legs is your admission that some stories could still have some inherent bias, by cherry picking facts, that as it stands it isn't a be all end all, but an effective tool to weed out untruths. But that also doesn't bother me, because after checking out the website, I currently see it as a place where those articles will provide facts on one side of a story, and then other articles could amend that, or include the opposing factual views, so the community together could build the big picture. Is that the right idea/something that could happen?


yazIam

Indeed! Anyone can submit content, even and especially if it presents an opposing view to content we already have, as long as they back their facts with primary source evidence.

2andrea

Far too much of what passes for journalism is based on people identified to consumers only as unnamed and inside sources. How will those facts be crowd checked?


yazIam

To reference my post description above:

this approach only works for journalism covering information based on publicly reviewable evidence. This includes legislation, public government initiatives, whistle blower documents, and scientific data. This isn’t a good fit for journalism based on undocumented sources.

So this doesn't work for unnamed inside sources at all. I think though focusing on creating a place that makes it as easy as possible for readers to review publicly accessible primary sources is still adding a lot of value.

Sigmachi789

Obviously not a Reddit employee ??


yazIam

I will happily take a salary from anyone who wants to give me one. As long as I can do my own thing and just keep working on SourcedFact ;)

(In case my joke gets misunderstood, I am not a reddit employee)

ivanbaracus

Have you heard of Barrett Brown and his "Pursuance" project that's also doing something similar with investigative journalism. What are your thoughts?


yazIam

I hadn't, looking at it now! Thanks for sharing this!

boyscout_07

Are there any plans to try and get major news media outlets to jump in and start using this for themselves and to check others (like creating a formal/informal network of news media), or just to operate it as voluntarily submitted only?


yazIam

I'd love to help as many outlets and journalists as possible adopt "open source" fact checking into their content. That is, make it as easy as possible for their readers to review their evidence and reasoning.

yesofcouseitdid

How do you propose to avoid/mitigate/combat the following scenarios?

The people who need to care, don't, and won't. Trump lies by the sentence quite often, and by the paragraph almost always, and there are plenty of people doing very thorough and well-documented debunks of his shit - and his base do not care.

Best case, your site gains traction and becomes popular, a major story breaks and you get referenced in the news cycle, he utters one dismissive labelling you as fake news, and that's all you are to his base. They will not care. They already think the "librul media" is willing to lie through its teeth to them, and they'll just believe you're a part of that machine as soon as you're critical of their god-king - because make no mistake, your site will have the appearance of "liberal bias" to any imbecilic onlooker, what with conservative lies being so prevalant and so debunkable.

And, with the general news-consuming public being so averse to even fact checking from well-documented actual organisations with proven track records of being decent, the expectation that any significant mass of them will have the time or care to do their own fact checking from nested layers of primary sources, each requiring its own further deep-dives to assess the reliability of... it's hopeful, let's say.

And. As soon as you do become significant, you'll be gamed into noise. Astroturfing is serious business these days and state-level outfits are good at doing it on a grand scale. Sure it'll be tricky to astroturf you at first, but content farms have existed for well over a decade, and that's all it takes to defeat you.

Something like this will have value, but only to those of us who already care enough to seek out truth already. It'll make our lives easier. It isn't going to be the panacea to better informing the masses.

Call me old and jaded and cynical, and I'm sorry for raining on your parade, but, well. As Tupac once crooned, that's just the way it is.


yazIam

> Call me old and jaded and cynical, and I'm sorry for raining on your parade, but, well. As Tupac once crooned, that's just the way it is.

I would never speak ill of someone who quotes Tupac.

> Something like this will have value, but only to those of us who already care enough to seek out truth already. It'll make our lives easier. It isn't going to be the panacea to better informing the masses.

That's just it though, I want to build a resource that makes it as easy as possible for those who care for primary source evidence, to review that evidence. I have no idea what precentage of news consumers care about that, how big or small. Doesn't really matter to me, first I want to focus on helping those who care.

KDobias

How does it have to scientific studies that disagree? Is it a "majority wins"? At what point do you chock it up to being unclear factually?


yazIam

The type of open fact checking you see on SourcedFact would be used to prove a statement like "This study concluded X". It wouldn't be used to prove a statement like "X is true". That may not seem like a lot, but it's surprising how often people (and some journalists) misquote studies just to get a headline. If you're always forced to attach the original study a long with detailed reasoning as to where the study corroborates your claim - that kind of misquoting will hopefully happen a lot less, or at least it will stick out like a sore thumb.

forgotmylastuser

Hey, great job with this. One thing missing is...url's on stories? Verification has urls, but if I want to share an article with someone, looks like there is no way to do so?


yazIam

Sorry that's just a flaw with the homepage demo, will fix. They have unique URLs though, here are the 3 you see on the homepage:

https://sourcedfact.com/a/net-neutrality-how-we-got-here

https://sourcedfact.com/a/cell-location-data-without-warrant

https://sourcedfact.com/a/government-measures-against-israel-boycotts

Aofwa

Any public API? I’d love to create a mobile app or something.


yazIam

Happy to make the API public, I'll message you as soon as the AmA dies down and we'll discuss :)

nightrss

Have you considered using something like steem as a blockchain backup?

This project seems like a perfect fit since a blockchain record creates an un-alterable log of changes.


yazIam

Indeed! I'll quote one of my responses above:

Having said that, I intend to move user contributions to a distributed system (kind of how crypto currencies work). Meaning that even though we may decide a user fact check is inaccurate and flag it as such, it would be impossible for us to censor it or delete it. So you could always check for yourself if we’re screwing up.

aquatermain

It sounds like a fascinating and useful idea.

My question is, do you have plans to expand your project to include journalism in other languages?


yazIam

Absolutely!

good4y0u

This is awesome. Is there a way to build an add-on that will automatically run news sites through your algorithm? I assume you have some sort of Machine learning behind it to make your checks. Otherwise it would be hard to keep up with a site like this.


yazIam

Not yet, but as the database of fact checks grows, we'd be looking at building a tool that automatically checks an article to see if the SourcedFact has applicable fact checks!

cracksilog

How are you dealing with publications that use anonymous sources? What’s your basis for fact-checking them since they’re anonymous?


yazIam

Good question. I'll quote my original post: "this approach only works for journalism covering information based on publicly reviewable evidence. This includes legislation, public government initiatives, whistle blower documents, and scientific data. This isn’t a good fit for journalism based on undocumented sources."

Card image cap
yazIam