Source score: YouTube comment template to encourage referencing
I started following Bret and Heather in the early days of the pandemic, and thought they were a great source of insight into what was actually going on on various fronts: the lab leak, vaccine safety & efficacy, ivermectin, etc.
As I watched more of them, though—and possibly as I watched them less—I started to notice them say things without backing them up.
This is an issue on a lot of podcasts, and it got me thinking of a way to constructively encourage better behaviour.
The idea is this:
Under each video you watch, you can post a Source Score comment indicating how many of the interesting claims made in the video are backed up.
The goal isn’t to provide a tool for people to snarkily post a “0” on videos they don’t agree with; it’s to incentivise good-faith podcasters to go to the extra effort of making sure all their claims are referenced. A good source score comment – voted to the top, of course – should be worn as a badge of honour and an indication that both audience and creator care about the truth. Consistently bad scores should be a heads up that your viewers want more rigour.
A great example of a website where people can find all the claims you make along with references would be Jamie Metzl’s pandemic origins page. I don’t know if Jamie does podcast appearances, but if he did, a link to that page would do nicely for all relevant claims.
The template
The source score comment template would look something like this—I chose a B&H video to try it out on:
Source score [01:23:45 - 01:34:56]: 0/8 (0%)
6:27 Differential risk between age groups is to do with the density of ACE2 receptors: 0/1
7:09 Mode of death is micro-bloodclots: 0/1
8:34 Young, healthy children are at effectively zero risk from Covid: 0/1 (suggest https://www.medrxiv.org/content/10.1101/2021.11.30.21267048v1, https://adc.bmj.com/content/early/2021/11/01/archdischild-2021-323040)
8:40 There is a general recognition that around 40% of children have already had Covid: 0/1
9:22 The CDC has acknowledged that they don't have a record of a single case of someone that has recovered from Covid getting it a second time and passing it to someone else: 0/1
9:35 The CDC estimates that 148 million Americans are immune: 0/1 (Brownstone Institute mentioned but specific reference not obviously reachable from homepage or Google search)
9:58 The idea that reinfection is possible is a result of confusion caused by false positive PCR tests and early tests' inability to distinguish between flu and Covid, and in fact you can only get Covid once: 0/2
Rules: sources can be mentioned directly in the video or made available via a link in the description. Links may be via an intermediate website and need not be listed explicitly per statement. As long as there is a clear way for viewers to find out what the statements are based on, and it is reachable from the video, it is up to the viewer to navigate to the information they're interested in.
Scores are out of 1 if a link can be given to a source that clearly supports the statement and no credible sources can be found by a reasonable web search supporting the opposing view. If credible sources supporting the opposing view can be found then scores are out of 2, with the second point given if the video, description, or linked page addresses at least one such opposing source.
Source score template by Gus Hogg-Blake: https://gushogg-blake.com/p/source-scoreThe format is:
A header line consisting of
Source score, optional time range covered (if you didn’t do the whole video),points/totaland percentage score.Claim lines consisting of: the timestamp; the claim; the score out of 1 or 2; and optional justification, further explanation, notes, or suggestions for references in brackets.
Rules text and link to this page.
Useful?
I don’t know if this is too subjective to be applied reliably and honestly enough to become a useful signal, for either viewers or creators. I’m optimistic for the following reasons:
It takes effort to write a detailed, researched, and well-formatted comment. It should be fairly obvious at a glance whether the author has put this effort in in good faith, and comments should get up- or down-voted accordingly.
It takes much less effort to click the timestamps in the comment and verify the scores, facilitating the up- and down-voting.
There are wildly different YouTube audiences per channel/topic, as measured by the type of comments that get upvoted on different videos. I bet that there are at least a few microclimates where the audience would generally appreciate, and enjoy writing, these comments enough for them to be useful.
Opposing sources
Possibly more frustrating than not mentioning any supporting sources is when a podcaster doesn’t acknowledge the best known or most accepted counter to their claim. This seems to be especially true for fringe theorists, but more mainstream people do it as well. That’s why you can get an extra point (or, I guess, not lose extra percentage points) if you mention these. I think it’s important to reward this highly, but it also brings in another dimension of subjectivity (whether the opposing source “can be found” and “is credible”).