Publishing in a Pandemic: The Preprint Revolution

Part 1: Publishing in a Pandemic: The Preprint Revolution

00:00:16:04 Most researchers have never seen
00:00:18:16 anything like this before.
00:00:22:06 A pressing, urgent need|for the dissemination
00:00:26:04 of research results in a rapid fashion.
00:00:42:03 Even though scientists have|been communicating primarily
00:00:44:08 through journal articles|for many, many decades,
00:00:47:28 the peer review process,|as we know it today
00:00:50:06 is actually relatively new.
00:00:52:25 The famous Watson and|Crick paper that reported
00:00:56:24 the discovery of the structure|of DNA was not externally
00:00:59:13 peer reviewed when it was published.
00:01:01:28 Albert Einstein retracted|one of his papers
00:01:04:11 from a journal when they|attempted to send it out
00:01:08:10 for external peer review,|which he viewed as an insult.
00:01:26:26 Preprints are usually early|drafts of scientific papers
00:01:31:15 that are shared online
00:01:33:09 so that information can be|disseminated between researchers
00:01:36:29 prior to the completion of the|journal peer review process.
00:01:40:28 During the COVID-19|pandemic, researchers are
00:01:44:17 posting preprints related to coronavirus
00:01:48:00 at a much higher rate suggesting
00:01:50:07 that the research community|is becoming more familiar
00:01:52:22 in the biomedical|sciences with the concept
00:01:54:29 and benefits of sharing their work early.
00:01:59:15 Many funders and|institutions have recognized
00:02:02:20 the value of preprints particularly
00:02:04:28 when there is a public health emergency,
00:02:07:02 such as an outbreak like|SARS, Zika or Ebola.
00:02:13:10 In the SARS outbreak, the|majority of papers were published
00:02:18:02 in traditional journals only|after the outbreak had ended.
00:02:30:13 And as a result they did not have
00:02:32:07 the same impact they could|have on public health.
00:02:39:14 The process of journal|publishing is slow relative
00:02:42:17 to the immediacy of an outbreak.
00:02:48:11 So this is the first time,
00:02:50:12 this COVID-19 outbreak|is really the first time
00:02:52:22 that we're seeing the|widespread use of preprints
00:02:55:19 during the public health emergency.
00:03:01:16 Sharing research results benefits,
00:03:04:23 not only the individual researcher
00:03:07:06 but the scientific community
00:03:08:20 and the general public at large.
00:03:10:27 By putting results out there,
00:03:12:25 where others can build off of them
00:03:15:17 and with appropriate credit
00:03:18:13 and start investigating|new areas of research
00:03:21:18 is something that makes discovery
00:03:23:12 in science happen faster.
00:03:25:11 For example, studies that|look at the persistence
00:03:28:20 or transmission of droplets
00:03:30:27 that occur in breathe
00:03:32:29 can be helpful for designing|personal protective equipment
00:03:37:05 or understanding how healthcare workers
00:03:40:12 and others can protect themselves.
00:03:46:00 Many researchers use social media
00:03:48:09 and in particular Twitter to share
00:03:51:03 and discuss research|findings, especially preprints
00:03:55:11 and other information that's distributed
00:03:58:00 in a less formal way.
00:03:59:20 On Twitter conversations on preprints
00:04:01:22 can happen very rapidly
00:04:03:14 and involve a flexible|group of researchers
00:04:06:14 who all bring different|perspectives to their conversation.
00:04:09:27 There is a kind of crowdsource|community attention
00:04:13:22 on a paper and that's important
00:04:15:28 because by communicating|publicly researchers broaden
00:04:19:29 the audience that they're speaking with,
00:04:22:10 they're not just putting something out
00:04:24:25 to a private Slack channel or|discussing with their own labs
00:04:27:19 or speaking to the people who can afford
00:04:30:26 to be in the room at a conference,
00:04:32:21 they're sharing something|openly with the whole world.
00:04:35:13 And there's also third-party sites
00:04:37:21 for preprints that provide commentary
00:04:40:12 and organized review or rapid review.
00:04:48:09 The concerns around preprints|in a pandemic are amplified
00:04:51:29 by the intense media attention.
00:04:54:13 In this environment, any|new findings are newsworthy
00:04:57:17 and so there's the potential|for incorrect information
00:05:00:27 or information that turns|out to be not robust
00:05:03:25 to be amplified and|disseminated to the public.
00:05:07:08 Journalists or perhaps the|readers might not be aware
00:05:10:22 of the provisional nature of preprints.
00:05:13:26 It's very important|that we have peer review
00:05:18:04 to evaluate research that might|be methodologically flawed
00:05:23:19 or to identify problematic|science as much as possible.
00:05:28:14 The benefit of peer review is
00:05:30:07 that it is a very thorough process.
00:05:32:20 So there is always a danger|that by trying to speed it up
00:05:36:14 there might be some|compromises to the quality.
00:05:39:20 For example, a paper claiming|uncanny sequence similarity
00:05:43:26 between coronavirus and HIV was posted
00:05:47:15 on the preprint server bio archive.
00:05:55:02 It garnered over 100 comments|over the course of a few days
00:05:59:18 before being withdrawn by|the authors voluntarily
00:06:02:19 within 48 hours.
00:06:04:15 So while this is an example
00:06:06:14 of potentially damaging|use of a preprint server
00:06:10:07 to spread potentially harmful information,
00:06:13:07 it's also an example of the|power of community commentary
00:06:17:02 to help correct the scientific record.
00:06:23:10 Some journals are changing their policies
00:06:25:18 and practices in response|to the COVID-19 pandemic.
00:06:29:18 Because there's a demand
00:06:31:14 for peer reviewed information faster,
00:06:33:14 some journals are accelerating|their peer review process
00:06:36:12 so that they can publish|papers much faster
00:06:39:01 than they could normally.
00:06:40:00 Other journals that are|traditionally publishing their work
00:06:43:18 behind paywalls have|removed those restrictions
00:06:46:06 to enable anyone to read|the research results.
00:06:51:00 It's especially important to be familiar
00:06:53:13 with what a preprint is,
00:06:54:25 how it's different from a journal article,
00:06:57:09 and to look for the opinion of experts
00:07:00:21 who are really familiar|with the techniques
00:07:03:01 and the field of research|in terms of interpreting
00:07:06:04 what it means for scientific discovery.
00:07:09:14 Researchers when talking to|journalists need to be clear
00:07:13:00 about what a preprint is.
00:07:15:10 Journalists, I think also must convey
00:07:18:25 in their article information|about what a preprint is.
00:07:26:11 Science progresses only as|quickly as it can be shared
00:07:29:12 between researchers|enabling those researchers
00:07:32:21 to communicate with one another
00:07:34:04 in more efficient ways can increase
00:07:36:10 the process of discovery.
00:07:38:05 Opening up the peer review|process to incorporate comments
00:07:41:04 from readers that are outside
00:07:43:04 of the traditional reviewer|pool can also diversify
00:07:46:25 the input into that peer review process.
00:07:49:18 So perhaps enabling a community comment
00:07:52:15 and encouraging community|comment on preprint servers
00:07:54:20 can compliment general|organized peer review
00:07:56:28 and make it even more robust.
00:07:59:16 My hope is that a silver|lining of this pandemic is
00:08:03:03 that researchers will|see new opportunities
00:08:06:16 to change the way that they communicate
00:08:08:20 to make it faster, more|open and more efficient.

Part 2: Early Career Scientists and Preprints Q&A

00:00:10:02 - I am excited to have our three panelists joining us
00:00:14:17 and I'll just introduce everybody quickly.
00:00:18:10 I think we'll have a good discussion.
00:00:20:00 Where we're gonna focus our discussion today
00:00:22:19 on preprints in general
00:00:24:14 obviously that the pandemic was a big theme
00:00:27:07 within this video in particular,
00:00:29:17 but preprints in general
00:00:31:08 and how that relates to early career scientists.
00:00:37:17 And I think all three of our panelists
00:00:39:23 are very well poised to talk about that.
00:00:42:16 So first up we have Jessica Polka,
00:00:44:24 who you all just just saw in the video.
00:00:48:14 Jessica is a biochemist
00:00:49:23 and executive director of ASAPbio, which stands for
00:00:52:18 Accelerating Science and Publication in biology.
00:00:56:05 It's a non-profit initiative promoting innovative
00:00:58:08 and transparency via preprints and open peer review.
00:01:06:14 Also a very fun fact in 2016,
00:01:09:06 Polka was described in the journal Nature
00:01:12:12 as an agent of change
00:01:14:00 for explaining how junior researchers
00:01:17:01 can increase the impact of their work.
00:01:21:07 And let's see, next up, we have Liz Silva
00:01:24:00 from the University of California, San Francisco.
00:01:27:06 Liz is Associate Dean Graduate Program
00:01:30:06 and an adjunct assistant professor
00:01:32:04 in the Department of Social and Behavioral Sciences
00:01:35:17 Liz is a trained by biomedical research scientists
00:01:38:12 with experience in science policy
00:01:41:03 particularly relating to publication and research ethics
00:01:44:19 as well as an early career training.
00:01:48:16 Prior to her current role, she was a senior editor
00:01:50:23 at Plus One and served as director
00:01:53:09 of UCSF's Motivating INformed Decisions initiative
00:01:57:04 which re-imagined the career development process
00:01:59:23 for the future of the biomedical workforce.
00:02:03:13 And finally we have Kate Carbone.
00:02:07:07 Kate is an industry postdoc
00:02:09:07 and studies immune cell biology across scales
00:02:12:12 from a biochemical reconstitution to in vivo tumor models.
00:02:17:13 She's currently focusing on cancer immunology.
00:02:20:05 She earned her PhD from UCSF in the lab of Ron Vale
00:02:24:05 and published preprint papers as a grad student,
00:02:30:03 important to get that in there.
00:02:32:06 And at her company now
00:02:34:20 Kate leads a preprint journal club
00:02:37:05 where she presents and summarizes different findings
00:02:40:15 and discusses those results.
00:02:43:03 So welcome all three of you.
00:02:45:21 I'm excited to have you all here.
00:02:50:01 I guess maybe before we get started,
00:02:53:21 I wanted to sort of gauge what some of your reactions were
00:03:01:17 after watching the video.
00:03:04:03 We're a year into some of the things
00:03:08:00 that were kind of brought up about preprints
00:03:15:03 and about publishing during the pandemic.
00:03:18:15 I wonder if each of you could maybe comment a little bit
00:03:23:04 on what this past year has made you think about
00:03:28:16 in terms of the publication process.
00:03:31:11 Whether that has to do with the pandemic specifically,
00:03:34:20 or just anything that this year has brought up for you.
00:03:39:21 So I guess maybe Jessica,
00:03:43:07 we'll just start with you real quick.
00:03:46:06 - I feel like I've already occupied the stage.
00:03:49:19 I thank you so much, Kevin for the invitation
00:03:51:22 am grateful for that
00:03:53:02 and also Kate and Liz,
00:03:54:23 I'm excited to have this conversation with you
00:03:56:15 and listen to you.
00:03:58:18 I think it's interesting to look back on that video
00:04:04:06 to some extent many of the themes
00:04:07:13 are the same that they're all true.
00:04:10:07 That there is a real opportunity to use preprints
00:04:14:04 to increase the speed of dissemination
00:04:16:22 but there's also brand new concerns
00:04:18:22 about how preprints can be used or possibly misused.
00:04:23:06 So for example later in the year, after recording that video
00:04:28:22 we ran a survey about concerns
00:04:31:10 and perceived benefits of preprints.
00:04:33:11 And among people who had both post-preprints
00:04:37:14 and those who hadn't.
00:04:39:07 Premature media coverage of preprints
00:04:41:01 actually came out as the top concern
00:04:43:17 ahead of preprints being scooped by others and so forth.
00:04:48:22 So I do think that there's a certain level of consciousness
00:04:55:05 and a sense of responsibility among researchers.
00:04:58:22 That we go with primary participants in that survey.
00:05:04:23 I think that the concept of media coverage
00:05:08:08 we've seen preprint servers adapt
00:05:09:24 by changing their screening policies.
00:05:13:08 And there have been even additional review initiatives
00:05:16:22 springing up during this time as well.
00:05:19:15 So I definitely think the system has somewhat adapted
00:05:22:16 to some of these challenges.
00:05:24:16 But I'm really curious to hear perspectives of others.
00:05:29:19 - Let's see maybe Liz,
00:05:30:18 do you wanna speak to that a little bit?
00:05:35:15 - Sure, I completely agree with Jessica
00:05:37:24 that preprints have just gotten this spotlight
00:05:39:20 because of the urgency of publication.
00:05:42:07 And I think it's a combination of that urgency
00:05:45:24 but it's also the fact that
00:05:47:08 the public is become a very acutely aware this year
00:05:51:05 of the full process of scientific discovery.
00:05:54:14 They're following it in real time
00:05:56:06 which is something that just never happens.
00:05:58:23 And I think what's really interesting
00:06:01:19 is that there are individuals who are pointing to preprints
00:06:04:07 as being a problem, but it's not really preprint its COVID,
00:06:07:14 its COVID that's elevating the discourse
00:06:11:21 in a way that the public and the media
00:06:13:16 would not usually see.
00:06:15:15 And I don't think it's a preprint problem.
00:06:17:03 It's a problem across all the peer review
00:06:20:01 and I already tried of scientific publication
00:06:22:10 whether it's peer reviewed or not peer reviewed.
00:06:26:04 I think that's the biggest take home that I've seen in COVID
00:06:29:22 I've got lots of thoughts that are pre COVID on preprints,
00:06:32:24 but yeah I'll leave that there.
00:06:36:19 - Great and Kate, what are some of your thoughts?
00:06:43:21 - Yeah, I think Jessica and Liz really said it all
00:06:46:11 but this year has been quite a stress test
00:06:49:21 for preprints it seems.
00:06:51:08 And myself, when I first posted preprints was in 2016.
00:06:56:11 Bio-archive was really just first taking off
00:06:58:21 and putting your work out into the world
00:07:01:05 is always a little bit scary.
00:07:03:15 Especially as the younger stages
00:07:05:17 when you're trying to establish your name
00:07:07:05 establish your credibility.
00:07:09:12 The thought of putting out work before it's ready
00:07:11:13 is a bit intimidating.
00:07:14:10 And so seeing in COVID that there's now this hotly watched,
00:07:18:12 intensely heated debate over these preprints.
00:07:22:02 I think is sparking this really interesting discussion
00:07:26:12 about how we share science and how we talk about science.
00:07:31:03 I'm excited to see where it continues going.
00:07:34:22 - Actually Kate, no sorry go ahead.
00:07:37:13 - So I would love to add something
00:07:39:08 to what Kate was describing, which is really interesting.
00:07:42:10 Something that I had heard over and over again,
00:07:44:16 that one of the critiques of preprint
00:07:46:00 is that it's not vetted
00:07:48:07 but over and over again, we hear from researchers
00:07:50:09 and they are actually doing a lot more vetting
00:07:53:10 before they put it onto bio-archives
00:07:55:08 than they ever did before submitting to a journal.
00:07:58:04 So they're subjecting themselves
00:07:59:13 to a much higher level of rigor.
00:08:01:09 And I've heard that from students and faculty
00:08:03:22 and postdocs alike.
00:08:08:04 - Kate I'm curious about as a graduate student,
00:08:12:18 you did publish some preprints and this is,
00:08:18:16 I don't know, a non weird way to ask this question
00:08:22:00 but I was wondering how did you know to do that?
00:08:24:22 I don't think as a grad student myself
00:08:27:05 when I was in school, I don't think I even knew
00:08:32:10 that that could be part of the process.
00:08:35:23 What went into your decision to go that route?
00:08:41:17 - I think in a way, I was lucky to be surrounded
00:08:46:17 by this conversation as it was happening.
00:08:48:14 So, as you said, I did my PhD in Ron Vale's lab.
00:08:54:06 I joined in 2013, I think ASAPbio
00:08:57:02 that featured to meeting was in 2016.
00:09:00:12 And I posted both of my preprints in 2016.
00:09:05:15 So it was really, that conversation
00:09:07:19 about how advantageous preprints can be for your career
00:09:11:10 in terms of communicating science quickly,
00:09:15:00 and Ron's paper with the data
00:09:18:19 aggregated by Courtney and Walter
00:09:20:20 came out around that time.
00:09:22:05 Also showing that this pressure on trainees
00:09:25:19 to produce these magnum opus style works
00:09:28:23 was actually really disadvantageous for graduate training.
00:09:32:22 So for those papers, it was a total no-brainer.
00:09:37:12 It was obvious to me that
00:09:38:20 I wanted to show that work on bio-archive.
00:09:42:08 I think it's been interesting now to be involved
00:09:44:10 in some more collaborative projects
00:09:46:09 where I'm not the primary author,
00:09:48:18 maybe it's not entirely up to me
00:09:50:11 whether work is published on a preprint server or not.
00:09:54:19 And I'm kind of learning in that environment
00:09:57:18 where it's not taken for granted
00:09:59:04 how to advocate for preprint servers.
00:10:00:24 And it kind of puts the pressure on you
00:10:03:08 to understand the whole conversation
00:10:05:22 and explain to your colleagues
00:10:07:18 why this is a worthwhile step.
00:10:13:18 - Now that you're out of school and in industry,
00:10:19:14 what is that process like
00:10:20:22 in terms of making those decisions?
00:10:25:05 - I'm just learning, I'm a co-author on a paper
00:10:30:12 that was not posted on a preprint server
00:10:33:04 and that decision was ultimately
00:10:36:14 made by the the corresponding author, the first author.
00:10:40:17 I'm preparing my own manuscript now.
00:10:43:09 And I would like to see that published on a preprint server
00:10:48:02 but it felt appropriate to wait until submission.
00:10:52:09 And when that paper is really completely finished,
00:10:55:08 then I think I feel confident bringing it up
00:10:57:21 to my coauthors and explaining that.
00:11:02:12 I think this is a really great way to communicate science.
00:11:05:13 And as Liz mentioned,
00:11:07:10 we have our own internal review processes.
00:11:10:11 First of all, all of the authors on the paper
00:11:12:18 are reading it with their most critical eyes.
00:11:16:19 We have other people in the department
00:11:18:04 who are looking at it.
00:11:20:02 Genentech also has a legal review process
00:11:22:11 before work can be disclosed.
00:11:25:08 So, by the time we would be ready
00:11:27:22 to put that work on bio-archive,
00:11:29:16 I'm going to feel quite confident in it.
00:11:37:11 - One of the things that I was curious about as well was,
00:11:43:24 in the video Jessica you talked a little bit about
00:11:49:17 what some of the benefits are to publishing preprint.
00:11:56:16 I'm curious, focusing on that early career stage.
00:12:01:23 There's some unique things about
00:12:05:00 whether you decide to do that or not do that.
00:12:07:15 And I'm sure, especially when you're earlier on,
00:12:10:09 you're very heavily swayed by whoever your advisor is,
00:12:13:13 and what their opinion might be.
00:12:15:21 I'm just wondering if you have anything to add about that.
00:12:21:15 - Yeah definitely.
00:12:22:22 I think that Kate really hit the nail on the head
00:12:26:12 with this point about the grad student.
00:12:29:13 The timeline of graduate education
00:12:32:03 compared to the timeline of papers.
00:12:35:08 And especially the timeline of the review process.
00:12:37:22 So, I think it's really interesting to,
00:12:41:09 Kate referenced this paper that Ron published in 2015.
00:12:46:21 To some extent,
00:12:47:17 I think the reason that it's taking graduate students
00:12:50:01 such a long time to publish a paper as Kate mentioned,
00:12:53:15 is because these papers now have a multiplier
00:12:58:23 of the number of figures and they did 30 years ago.
00:13:01:08 There's just expectation of tons and tons of work
00:13:04:01 to go into a single paper.
00:13:05:21 So when you your preprints are posted
00:13:08:22 at the time of submission,
00:13:10:14 they definitely help with that to some extent,
00:13:13:02 because you are making the work public
00:13:15:07 head of this journal review process.
00:13:18:12 But I do think that there still are fundamental challenges
00:13:22:06 in the way that we communicate research
00:13:25:05 that are not being addressed by this practice.
00:13:28:08 That's it, I'm curious really to hear
00:13:32:22 Kate's strategies and what's worked in the past.
00:13:36:20 But I think that as a postdoc,
00:13:42:08 my postdoc advisor Pam Silver
00:13:44:01 was actual advisory with bioarchives.
00:13:46:04 She was very supportive at all times as well
00:13:49:20 but in my own experience talking to people who perhaps
00:13:53:23 are not yet preprint supporters or advocates,
00:13:57:19 I think is really compelling to show support
00:14:00:12 from funders support from colleagues
00:14:03:17 who are also posting preprints
00:14:05:07 to have one-on-one conversations
00:14:07:04 to understand what the benefits are.
00:14:11:10 So I would love to hear from Kate as well
00:14:15:08 teaching this,
00:14:16:04 But I certainly think that for early career researchers,
00:14:18:24 there's a huge benefit but there's also this challenge
00:14:21:05 of being in a position where there might be power dynamics
00:14:25:05 where you're not the only one calling the shots
00:14:27:10 about what happens with your paper.
00:14:30:04 So I think that arming yourself
00:14:33:05 with information about policy surrounding preprints
00:14:37:20 about the usage of preprints the potential benefits.
00:14:39:22 I'll drop a link into the chat with some resources
00:14:43:18 but I think that bringing that data can only be helpful.
00:14:50:17 - Liz, I was curious from your perspective as well
00:14:53:24 as someone that helps help so many early career scientists.
00:15:02:06 your own decisions about about preprints
00:15:04:05 as well as what kinds of advising
00:15:06:20 you give to folks that are earlier on in their career?
00:15:09:14 surrounding that as well.
00:15:12:10 - Absolutely this honestly falls in a bigger bucket
00:15:15:23 of conversations that trainees have to have
00:15:18:23 with their advisors on what serves their own best interests
00:15:23:04 versus what the PIs believe
00:15:26:06 that the interests of their trainees
00:15:28:18 are in the interest of the lab
00:15:29:18 or the interest in their funding sources.
00:15:31:24 And this really moves honestly
00:15:34:06 into a phase of a bit of a negotiation.
00:15:36:08 If you're dealing with somebody
00:15:37:05 who's particularly calcitrant, there's the policy aspect
00:15:44:18 which I think speaks to the trainee.
00:15:50:04 Sharing their what they value as a researcher,
00:15:53:16 as part of their identity as a researcher
00:15:55:10 what they value in the work that they do.
00:15:57:10 And the public responsibility,
00:16:00:24 if that's really important to you,
00:16:02:02 then communicating that is a value
00:16:04:01 that you have as a researcher is definitely important.
00:16:07:16 In addition, I think that there is a process
00:16:10:01 where you have a conversation
00:16:11:18 about the pros and cons of preprint
00:16:15:18 for the PI and for the student.
00:16:17:21 And so it's going in with
00:16:20:08 what are your professional objectives?
00:16:22:13 What are your career objectives?
00:16:23:24 What will the preprint do for you?
00:16:25:18 That's important to you.
00:16:27:04 What will the preprint do for the PI?
00:16:30:04 That will meet their objectives
00:16:31:12 in increasing the profile,
00:16:34:00 One small tidbit that I think is really helpful
00:16:36:15 for people to bear in mind in all of these conversations
00:16:39:07 is again there's a perception that putting
00:16:41:06 out your pre-print will subject you to scooping.
00:16:43:23 But there's a lot of indications
00:16:45:05 that it's actually the reverse
00:16:46:04 because you've put your name out there first.
00:16:47:17 And so a paper will get submitted to a peer reviewed journal
00:16:52:19 with increased profile of bio archive,
00:16:55:24 there's a very, very good chance that an editor or reviewer
00:16:58:10 will know that that work has already been published
00:17:00:00 in the preprint server.
00:17:00:21 And they'll want to know why that wasn't cited
00:17:02:05 or how did you come about it.
00:17:03:08 So that's one little tip for the negotiation process.
00:17:07:01 And I think it's really a negotiation
00:17:08:13 of what's important to you and for your objectives and why.
00:17:13:15 - And just as a follow-up, I wonder do you know how,
00:17:20:04 for someone that's looking
00:17:22:09 for their next opportunity in science,
00:17:25:05 which that can be many different types of things.
00:17:28:01 Do you think how are preprints viewed on like a CV
00:17:31:21 of a potential candidate or from a selection committee
00:17:35:12 or something like that?
00:17:36:20 I'm sure that's something that may be changing as well.
00:17:41:00 But I wonder if Liz,
00:17:42:15 maybe you could comment on that a little bit.
00:17:46:06 - It's very complicated and I agree.
00:17:48:21 You hit it on the head there Kevin.
00:17:50:16 It is changing absolutely
00:17:52:00 as preprints gain traction and profile
00:17:54:12 I think it is changing.
00:17:56:23 It depends on the type of career that you are going for.
00:17:59:11 There are still very traditional agencies,
00:18:01:12 very traditional funders
00:18:02:18 that want to see peer reviewed papers.
00:18:05:15 There is an enormous misconception
00:18:08:14 that the coin of the realm
00:18:10:13 is a peer review published paper
00:18:12:10 regardless of the career that you're going into,
00:18:14:08 any faculty member who tells you that, is incorrect.
00:18:17:23 I guarantee you, you did not get into patent law
00:18:20:04 because of your nature paper I promise.
00:18:25:24 And I'm seeing much more frequently
00:18:28:22 when students or post-docs advocate for
00:18:32:23 getting their work out there in preprints
00:18:34:24 is because they have a better understanding
00:18:36:22 of what it is that the employers are looking for.
00:18:40:18 So part of that negotiation becomes a conversation
00:18:43:22 about what their career objectives are.
00:18:46:11 I want to move into science policy.
00:18:49:00 I want to move into industry.
00:18:50:11 And having that research in your back pocket
00:18:53:12 about what those future employees
00:18:55:06 or those future career opportunities
00:18:57:08 are actually looking for, becomes part of that negotiation
00:19:00:08 and part of that ammunition you have
00:19:01:13 when having a conversation.
00:19:03:06 The best way to determine that
00:19:04:23 would be to do informational interviews with,
00:19:07:21 as you're thinking about what you want to do
00:19:09:05 and what your career objectives are
00:19:10:09 regardless of trajectory.
00:19:11:19 If you want to do a post-doc
00:19:13:05 don't assume you know what people are looking in a postdoc.
00:19:17:00 Engage with people who are in the labs
00:19:20:20 that you want to work in the industry or in the companies
00:19:24:02 that you want to work with.
00:19:25:06 And find out what they are looking fof
00:19:28:01 in hiring somebody.
00:19:31:01 - Maybe just add on to that
00:19:32:15 which I think is excellent points.
00:19:36:13 I think that people are posting,
00:19:42:15 well preprints are for example a bio-archive by definition
00:19:46:02 prior to journal acceptance.
00:19:47:23 So I definitely think that even for these careers
00:19:51:16 that Liz is mentioning
00:19:52:16 where of course peer review is valuable.
00:19:55:13 I think that the peer review process
00:19:59:00 I would argue is valuable
00:20:00:14 regard outside of the process of looking for jobs as well.
00:20:05:12 But I think this is not an theatre or dichotomy
00:20:09:17 that you can post a preprint and to my knowledge,
00:20:16:24 many of the journals that were previously hesitant
00:20:19:21 about preprints or had reservations about preprints
00:20:22:03 have changed their policies in the last few years.
00:20:24:15 So it preprints are really something that you can add on
00:20:27:21 to the journal publishing process.
00:20:29:08 And while some people may want to leave their work
00:20:32:18 as a preprint and not take it forward, there is no reason
00:20:36:02 that you have to decide between those two options.
00:20:41:03 - I can totally agree.
00:20:46:09 - I can add as well terms of the job application process.
00:20:50:04 I recently had the opportunity to help review applications
00:20:53:08 for the genetic postdoc program.
00:20:55:15 And I think it is really helpful
00:20:58:09 especially as a graduate student
00:20:59:21 if your papers is a revision,
00:21:01:15 if you write in press or in revision on a CV,
00:21:05:14 it's hard to know what that means.
00:21:07:04 Whereas if you put a bio-archive link,
00:21:08:24 like for one that means I can take a look at that research
00:21:11:08 and understand what you're doing.
00:21:13:00 And given that I am a big fan of preprints myself,
00:21:17:13 it's also a little bit of value signaling that,
00:21:19:24 Liz mentioned that look at that
00:21:21:04 and said like, "Oh, wow this person is really passionate
00:21:24:06 about open science about collaboration,
00:21:27:00 about being a part of the community."
00:21:29:05 And if you get the right person, I think that really
00:21:33:14 can elicit a very positive response.
00:21:38:19 Myself I've never seen anyone upset
00:21:41:12 about a preprint being on at CV.
00:21:43:22 You're never gonna get a negative response
00:21:45:22 because you're including that
00:21:48:00 but you have this potential really significant upside
00:21:50:11 of finding an ally in whatever job you're applying for.
00:21:57:17 - One of the things that I wanted to make sure we hit on
00:22:00:24 as well was just about the strategy of reading preprints
00:22:06:06 I think even learning how to read scientific literature
00:22:09:24 with a critical eye
00:22:11:01 is such a major part of scientific training.
00:22:14:02 And doing it with with preprints
00:22:16:16 requires even a little bit more,
00:22:18:06 like another level of nuance.
00:22:19:21 And I think we've talked about it in the context of
00:22:22:16 journalism covering preprints, stuff like that.
00:22:25:04 But I think for, for someone who is learning as a scientist
00:22:30:09 I'm wondering if any of you have have any thoughts about
00:22:34:16 strategies for thinking about how you read a preprint
00:22:38:16 or training others on in terms, how to do that?
00:22:47:11 - Well, just briefly I'll say that this is one thing
00:22:49:23 that is available for preprints
00:22:52:07 that is not necessarily available
00:22:53:16 for peer reviewed journal articles
00:22:56:17 is that people are much more willing to discuss
00:23:00:13 and comment on preprints.
00:23:02:08 So something like 5% of bio-archive preprints have comments
00:23:05:20 in the commenting section.
00:23:06:23 And more have tweets that reference them.
00:23:09:14 Oftentimes they say things like cool paper but
00:23:12:02 there's often substantive comments there.
00:23:16:15 Whereas when an article is finally published in a paper,
00:23:20:02 there's less of a opportunity and need
00:23:23:03 for people to comment publicly.
00:23:24:23 Perhaps that's one reason why in general,
00:23:28:13 the 5% of bio-archive is actually quite high
00:23:31:03 compared to existing common counts on journal websites
00:23:35:00 or public comments when the experiment was running.
00:23:39:17 So I think that it's one thing that could be helpful
00:23:42:20 is to look for other comments
00:23:46:10 and reactions on other peer review sites on Twitter
00:23:51:17 in the comments section bio archive.
00:23:53:09 Of course, it's important to read with a grain of salt
00:23:56:14 but it could be that some people
00:23:58:00 bring expertise that is unique
00:23:59:23 and provides a different lens on the preprint
00:24:02:18 So that's certainly one difference
00:24:04:17 aside from as you mentioned.
00:24:06:20 Perhaps this increased skepticism.
00:24:11:11 - That's post-publication peer review right?
00:24:17:00 And one thing that's really interesting is
00:24:19:07 that I'm increasingly seeing
00:24:20:17 preprints being used as a training tool for postdocs
00:24:24:16 and students to learn how to critically evaluate
00:24:28:07 how to be reviewers in the peer review system.
00:24:31:04 So a lot of programs,
00:24:32:10 a lot of T32 programs in particular are PhD programs.
00:24:37:18 Generally they're incorporating preprints
00:24:40:01 into their training of students
00:24:42:14 on how to read the scientific literature.
00:24:44:23 And how to evaluate those critically.
00:24:46:24 I think they, they do some tricks
00:24:48:03 like maybe hide the comments until afterwards
00:24:50:05 or they don't tell them where the final paper was published
00:24:52:05 and then they can reveal the final changes.
00:24:54:14 But yeah, it's been a really interesting tool
00:24:57:03 for actually learning how to be a peer reviewer.
00:25:03:04 - Yeah. I think I've also learned a lot
00:25:05:14 about how to be a better scientist by reading preprints.
00:25:08:19 So I do a monthly review of preprints with my lab
00:25:12:19 and it's hard.
00:25:14:22 So there's a ton of content.
00:25:17:07 I start with having keyword alerts that come into my inbox.
00:25:21:18 And there's just an absolute flood of papers,
00:25:24:08 which can be a little intimidating.
00:25:26:23 And then I try to narrow things down by titles
00:25:30:13 that are really in my discipline.
00:25:32:17 I do find these preprints I tend not to read
00:25:35:02 outside of my immediate discipline as much as I do
00:25:38:08 maybe looking at journal articles.
00:25:41:01 And then from there something I'm really grappling with
00:25:44:03 is that I lean very heavily on who the authors are.
00:25:47:06 Do I know them?
00:25:48:19 Where are they from?
00:25:49:20 Have I heard them speak at conferences?
00:25:52:16 Which is not the best way to triage good science
00:25:57:19 but because there's so much content on bio-archive,
00:26:00:19 I find that that's hard to avoid.
00:26:03:05 And then we talk this in a group.
00:26:05:10 And so I think once I've gone through those steps
00:26:08:19 of applying these filters, we talk about the work.
00:26:15:01 We talk about whether we think it's robust
00:26:16:21 we talk about what it means for our science
00:26:18:14 and that I think is really the most exciting
00:26:21:23 and that you see where paper comes out.
00:26:23:23 At the end, after we've looked at it
00:26:26:07 bio-archive is also really interesting to see
00:26:29:18 what the rest of the field thought
00:26:31:05 compared to what we concluded.
00:26:35:08 - Actually, that certainly gets at
00:26:37:08 another thing I was curious about with
00:26:40:24 just the sheer volume of information that's out there.
00:26:43:20 I usually think about so when I was in grad school
00:26:46:02 my advisor told me when he was in grad school,
00:26:48:12 he would go to the library every Sunday
00:26:50:19 and he would grab the journals from his field
00:26:54:05 and he would get caught up on the literature
00:26:56:08 and that's how he did it.
00:26:58:14 Which is obviously not how I have to do it
00:27:02:00 and not how anyone is able to do it anymore.
00:27:04:04 There's just so many journals
00:27:05:15 and also so many places where where work can end up.
00:27:15:15 One thing I'm wondering about is
00:27:26:07 there's so there's so much work that's out there.
00:27:29:00 And do you think more or fewer people read preprints?
00:27:32:20 Or is that not relevant or is it the who is reading it
00:27:39:02 that's more important?
00:27:40:14 I wonder maybe Kate and
00:27:43:02 I see Jessica nodding your head as well,
00:27:45:20 maybe both of you could comment as well.
00:27:49:07 - I have maybe two different perspectives.
00:27:52:14 The first is that I came from cell biology at UCF
00:27:55:16 where I think they
00:27:56:15 were extremely early adopters of preprints.
00:27:58:23 And I think my peers in cell biology, check bio-archive
00:28:04:14 are more likely to be on Twitter.
00:28:06:23 Now I'm working in cancer immunology.
00:28:09:14 And I think that field is maybe a little bit slower
00:28:12:16 to pick up preprints.
00:28:15:03 Again, I haven't encountered any hostility towards preprints
00:28:17:23 but it's not something that's on people's minds
00:28:21:11 the way I found it was in cell biology.
00:28:23:09 So when I'm presenting papers, I think it's really impactful
00:28:30:09 because people see their peers who are
00:28:33:00 maybe big name labs that are presenting
00:28:35:15 or that are publishing preprints
00:28:37:16 and have kind of normalizes the whole process.
00:28:39:19 But I don't think yet, the majority of cancer immunologists
00:28:45:19 are reading bio archive on a regular basis
00:28:49:03 the way that might be true in cell biology.
00:28:51:16 Again, this is just my own personal impression.
00:28:53:21 So I can't say this is true for sure, but it's hard.
00:28:57:09 And it feels like with reading journal articles as well,
00:29:00:20 there's some level of randomness
00:29:02:13 for what work you're going to see
00:29:03:23 and what work you're not going to see.
00:29:05:10 And leaning on your colleagues
00:29:06:22 to help keep you up-to-date and in the loop
00:29:10:23 it's has to be a group effort.
00:29:14:13 - I definitely agree about the challenge of Kate.
00:29:16:23 I think to build on what you said as well, Kevin
00:29:20:08 There is a paper from Rich Abdill
00:29:22:16 looking at usage metrics of bio archive papers.
00:29:28:06 And interestingly, in the first month of posting
00:29:31:19 there was actually higher per article views in 2016
00:29:37:09 than there are now, even though of course yeah the research,
00:29:40:23 it's not a dramatic decrease.
00:29:42:14 But I do think that what is happening
00:29:44:09 is there's a little bit of dilution
00:29:45:22 of the fact that it used to be,
00:29:47:11 if you go to a bio-archive
00:29:48:13 open up one of the subject collections
00:29:50:20 and browse through all the papers in a couple of minutes.
00:29:53:18 Now, the preprints are becoming more widespread
00:29:56:11 that the rate of prepint posting is approximately 8%
00:29:59:10 of the total literature on PubMed.
00:30:03:08 That we need to adapt away
00:30:06:06 from this kind of journal table of contents style approach
00:30:09:19 which frankly it sounds like still works for Physics.
00:30:12:16 Because of physicists we'll talk
00:30:14:14 about proofing archive over the morning cup of coffee.
00:30:18:13 But I think that for us or for researchers
00:30:21:21 who are more kind of focused in the Biology space,
00:30:26:21 that keywords kind of alerts, are going to be the way
00:30:30:20 that people learn about pre preprints moving forward.
00:30:33:13 So certainly preprints will show up in Google scholar alerts
00:30:37:11 there are tools like Metta Semantic scholar
00:30:40:24 that can help find and alert you to preprints
00:30:45:01 based on various different algorithms or AI.
00:30:48:17 The fact that preprints are now being indexed
00:30:50:24 on PubMed in a trial
00:30:53:21 of COVID-19 relevant preprints by NIH authors.
00:30:58:03 And Europe PMC, which is a European mirror of PubMed.
00:31:03:20 Has been actually capturing the full text of preprints
00:31:07:15 for about two years now.
00:31:08:18 And so they are really a fantastic resource.
00:31:11:15 If you want to set up alerts that include preprints in a way
00:31:15:12 that you could digest along with other journal articles.
00:31:19:07 But I think this is a huge problem.
00:31:21:13 And as the number of preprint servers increases
00:31:26:20 it only heightens the need for better discovery strategies.
00:31:32:05 - Well, let's see one question that's come in about
00:31:35:23 reading outside your discipline.
00:31:37:20 Is that why would or wouldn't you read preprints
00:31:42:23 outside of your discipline?
00:31:43:22 Is it a volume thing or expertise thing?
00:31:46:21 I think maybe Kate, you had mentioned it
00:31:50:23 and Jessica you were also nodding about it as well.
00:31:57:06 - So again, because of this volume problem,
00:31:59:15 I think that's really why I just don't have the bandwidth
00:32:03:07 to be searching outside my discipline terribly often.
00:32:08:22 Whereas in a journal when there's only a few articles
00:32:11:23 it's just easier to see that research right in front of you.
00:32:15:23 So predominantly my reading outside my discipline
00:32:18:22 on preprint servers is because it's work done by my friends.
00:32:24:04 And I rely on other tools.
00:32:28:16 It's not necessarily that
00:32:30:17 there's that additional level of peer review.
00:32:32:16 It's just simply a volume of content issue.
00:32:39:02 - I definitely agree I think that for me
00:32:43:07 peer review is not perfect by any stretch of the imagination
00:32:47:19 but I do find myself reading a journal article
00:32:53:23 outside of my field, there is some sense that it has passed
00:32:57:04 more sense checks than a pre-print.
00:33:01:07 It's also I think important to know
00:33:03:08 that preprint servers have different screening strategies.
00:33:06:12 So for example Meta Archives
00:33:08:05 has pretty involved screening process,
00:33:10:06 they're checking for conflict of interest, et cetera.
00:33:15:14 There are a preprint servers that don't have any,
00:33:19:23 or I should say places you can find preprints
00:33:22:07 maybe it's even more product category
00:33:24:07 where there is no pre-screening
00:33:26:14 except for apps like a spam check.
00:33:29:13 And I think that this is,
00:33:32:03 let's just say that the content
00:33:33:21 posted on Meta Archive has been through peer review,
00:33:37:09 but just as a little sidebar of that
00:33:41:21 there's a whole range of different levels of scrutiny
00:33:47:15 that we can undergo both within journals
00:33:50:03 and occupant servers
00:33:54:00 - Liz, there's another question that has come in
00:33:56:22 that maybe you might be well-poised to talk about.
00:34:00:12 Which is you had mentioned how your values and your goals
00:34:07:14 and everything in in your career might dictate
00:34:10:02 where you want to publish things
00:34:13:24 or how you want to publish things.
00:34:15:16 I'm wondering if you have any advice
00:34:17:13 or if you've offered thoughts to others about
00:34:21:06 when those values don't align with who your supervisor is,
00:34:26:16 strategies of getting around that or working through that.
00:34:31:23 If you have any thoughts about that.
00:34:36:16 - I think the real question there is,
00:34:38:20 what do you do if your PI is willing to ignore your values?
00:34:42:10 Which is a much more difficult question.
00:34:45:05 Certainly, it's very common for it to not be aligned,
00:34:48:17 but hopefully when you're working
00:34:50:05 with somebody who respects you as a person
00:34:52:04 and wants to see how you can come into alignment
00:34:54:16 which is why I referred to it as a negotiation.
00:34:57:21 It's a lot more difficult
00:34:58:22 if somebody wants to ignore those values.
00:35:03:23 With a student there are better support structures in place
00:35:10:13 because you have the dissertation committee.
00:35:13:23 The hope is that you've got a group of faculty
00:35:16:20 on your dissertation committee.
00:35:18:16 That includes at least one person
00:35:20:10 who is willing to be an advocate and support
00:35:22:14 for you as a professional without,
00:35:25:01 there's an inherent conflict of interest
00:35:26:13 between the student and a PI.
00:35:28:00 They've got their own objectives.
00:35:30:10 They've got their own needs too right?
00:35:31:22 They're trying to keep their own small company running.
00:35:35:06 And those needs don't necessarily align
00:35:37:20 with what the individual's career objectives are
00:35:40:03 but hopefully on your dissertation committee,
00:35:41:24 there's somebody who is there to advocate for you.
00:35:45:15 So relying on those other, other mentors
00:35:48:10 other advisors and other advocates
00:35:51:12 I think it's probably the best way to go
00:35:53:23 but when somebody is not willing to listen, it's tough.
00:36:02:05 - Let's see, sorry I'm returning to my question list now,
00:36:12:22 Oh yeah actually Kate and Jessica have either of you had,
00:36:17:15 it sounds like Kate you had a very good
00:36:22:06 and supportive experience,
00:36:24:23 as a grad student at least I wonder
00:36:26:17 what the process has been like
00:36:28:14 to sort of advocate for preprints
00:36:30:14 or advocate just for your values or surrounding preprints
00:36:34:10 outside of your grad experience.
00:36:39:06 - To be honest, I'm a bit nervous to do it.
00:36:43:01 So I think I have established
00:36:46:10 that I care a lot about preprints,
00:36:48:18 my mentors know that I'm doing this journal club,
00:36:51:24 they know that I'm here today.
00:36:54:12 And so I don't imagine it will come as a surprise
00:36:57:08 that for my first or co-first author work
00:37:00:10 that I would like to see that printed
00:37:02:07 or published on a conference server.
00:37:03:19 But I do feel like I'm in a position where
00:37:07:15 I need to be a knowledgeable advocate
00:37:09:16 for preprint publication.
00:37:11:21 And so that's a bit intimidating.
00:37:14:16 I'll probably go check out the ASAPbio website
00:37:17:07 to refresh myself on statistics.
00:37:21:01 And we're having a conversation within the postdoc program
00:37:23:08 to try and have some more institutional recommendations
00:37:26:00 to encourage frequent publications.
00:37:27:22 So that individual postdocs
00:37:29:20 can be armed with that endorsement.
00:37:34:03 I don't know how it goes.
00:37:35:01 I don't particularly know now working in a space
00:37:38:07 where we're dealing with some human clinical trial data
00:37:40:22 if there'll be additional conversations
00:37:44:13 that come up because of that.
00:37:46:11 There's just a lot of different stakeholders that play now.
00:37:49:02 And I'll have to see how that goes.
00:37:56:22 - I definitely hear you Kate there.
00:38:00:17 I think anytime that you are proposing,
00:38:05:00 changing the status quo
00:38:07:11 or making any sort of innovations,
00:38:12:23 I think people begin to,
00:38:15:08 there's a lot of reaction for good reason
00:38:18:11 and valid concerns.
00:38:19:18 That I think Liz mentioned this idea of
00:38:21:18 let's like sit together get on the same side of the table
00:38:26:00 and really talk about what the potential pros and cons are.
00:38:29:09 But I do believe that for preprints, there are some fears
00:38:34:22 that are certainly brought up.
00:38:36:22 And I do think that it's possible
00:38:38:22 that preprints are not the right solution
00:38:41:03 for every team, for every project.
00:38:44:02 For example, I know that bio-archive has
00:38:48:24 not basically temporarily halted the posting
00:38:51:24 of certain types of research for public health concerns.
00:38:54:10 So I think that it is really important to bring this
00:38:58:00 sensitivity to concerns.
00:39:02:17 But I do feel that ultimately
00:39:04:12 if we think about what researchers are trying to do
00:39:07:15 we're trying to build knowledge.
00:39:09:16 And the only way to do that is by sharing it.
00:39:13:11 And by getting feedback that ultimately
00:39:15:22 will strengthen our own work.
00:39:19:02 I think that from my own experience they continue to
00:39:23:02 post these non-biology preprints,
00:39:28:00 that's the last like Biology preprint I posted.
00:39:32:00 I think that it was a really valuable experience.
00:39:34:24 We've actually posted ahead of submitting to a journal
00:39:39:01 by about a month or two and got really helpful feedback
00:39:44:10 on the preprint from a colleague that we had known
00:39:47:04 although certainly it's not unheard of to have
00:39:50:06 a completely unsolicited feedback as well.
00:39:53:18 And we also got an invitation to submit the article
00:39:57:06 to a journal.
00:39:58:02 And so, I think that while that has not happened
00:40:01:18 for every single preprint,
00:40:03:10 I think there's not a lot of downside,
00:40:09:06 but a lot of potential for lots of good to come from it.
00:40:12:20 And that I find is a message that can be helpful to convey.
00:40:21:04 - Another question that's come in about that process
00:40:25:14 is the review of preprints is obviously more public
00:40:30:07 than peer review.
00:40:31:23 You get to just look at your own review or comments
00:40:34:22 and feel good or bad about yourself
00:40:37:02 in your own private way.
00:40:38:00 But if there's comments on social media
00:40:41:12 about preprints in that way,
00:40:44:05 things become a lot more public in that way.
00:40:47:22 I wonder if any of you have had any experience
00:40:50:13 with having to deal with that more public layer of scrutiny
00:40:55:06 it's worth work it's in a preprint.
00:41:01:15 - I can just say that in this case the comments
00:41:05:23 I think that there's actually been a survey by bio-archive
00:41:09:08 conducted I think in 2018 where they found that
00:41:12:22 something like 40% of preprint authors
00:41:14:18 have received private feedback by email
00:41:17:14 on their preprint.
00:41:18:15 Which is almost a factor in order of magnitude higher than
00:41:21:20 the number that are getting comments on their papers.
00:41:25:06 Actually I think that for this study,
00:41:27:08 the authors posted multiple preprints.
00:41:29:01 I would point being there's a lot more private feedback
00:41:31:11 than public feedback.
00:41:33:01 Ideally, we would live in a universe
00:41:35:10 where it would be kind of normal
00:41:37:23 to have civil public critique of papers
00:41:41:20 because ultimately this critique is really necessary
00:41:45:12 in cases where there's other people,
00:41:47:17 as we've been talking about who are reading that preprint
00:41:50:04 who need to have that context of understanding
00:41:52:18 which parts of this deserve more attention and more inquiry,
00:41:57:10 what parts are critical and so forth.
00:42:01:05 I think that there's definitely
00:42:06:02 one thing that I think can alleviate these concerns
00:42:08:09 is the fact that there are these sort of organized
00:42:11:12 peer review initiatives,
00:42:12:20 for example at Johns Hopkins, at Oxford, at Mount Sinai
00:42:18:18 and also groups like Pure Community
00:42:21:00 and who are organizing peer review process.
00:42:23:15 So there is this idea of having this editorial layer
00:42:28:15 to ensure that the comments are civil and constructive
00:42:33:09 as they're being posted publicly.
00:42:37:22 - I actually have an extra question for Jessica
00:42:39:15 in case you might know the answer to this
00:42:41:18 cause it strikes me there's two things here
00:42:43:12 there's public versus private.
00:42:44:19 And then there's also the anonymous versus identifiable.
00:42:48:00 And so the reviewers have different motivations
00:42:50:24 depending on all of those things as well.
00:42:53:01 And I know that people have been looking for a long time
00:42:56:00 at signed peer review versus anonymous peer review.
00:42:59:16 Do you have any insight into how that affects the tone
00:43:04:00 and the content of the reviewers?
00:43:08:21 - I'm not actually aware off the top of my head of studies
00:43:16:20 that compare these two things.
00:43:18:08 Although I do know the study of publishing peer review
00:43:23:12 and the effect of that.
00:43:24:19 I'd just love to hear your perspective
00:43:26:03 on this and also Kate too.
00:43:27:23 I mean, I think that the key tension
00:43:29:15 with requiring science peer review
00:43:31:13 is whether people feel comfortable
00:43:33:20 possibly critiquing someone who is senior to them.
00:43:37:20 Yeah. I think that there are some potential,
00:43:41:04 it would definitely change the dynamic
00:43:42:24 between reviewers and authors.
00:43:44:18 People might feel more responsible
00:43:46:19 for what they're saying and less likely to be critical,
00:43:48:23 but at the same time, you know, it does raise issues
00:43:54:08 about trying to garner favor from an individual
00:43:59:06 and people might fear retribution as well.
00:44:02:16 So I had to xx it's a very complicated issue.
00:44:06:24 And again, it's one where I wish that in this utopia
00:44:09:16 we would all feel comfortable to attach our names.
00:44:11:16 But I think that moving to a system of that nature
00:44:15:05 prematurely would disadvantage
00:44:16:21 certain groups of people disproportionately.
00:44:23:19 - And Jessica would know the statistics better than me,
00:44:26:12 but I think at this point
00:44:27:11 if you're getting peer review on your preprint
00:44:29:03 you're one of the lucky ones.
00:44:31:13 And so I think it's a good tool to share with colleagues.
00:44:36:24 One of my preferences is actually cited.
00:44:39:03 And I remember when I was a grad student and that happened
00:44:41:16 I was like, "Oh my God, this person is reading my paper?"
00:44:45:09 And so that felt really cool.
00:44:49:00 You know, I also want to share that
00:44:50:08 there was an interpretation of one of the results
00:44:52:11 in my blueprint that actually changed
00:44:54:20 because of peer review because of additional experiments
00:44:58:23 that's on the internet and that happens.
00:45:01:20 And that's real, I would imagine
00:45:06:23 that those kinds of things are much more common
00:45:09:22 than having someone, one of your colleagues
00:45:12:09 tear your work apart.
00:45:14:06 Or even to be lucky enough to get
00:45:16:12 really detailed commentary
00:45:18:16 that you might get from a journal or from review comments
00:45:22:02 or from a formal review process.
00:45:25:05 I think most people aren't taking the time
00:45:28:12 to provide that kind of feedback
00:45:30:10 to move very many preprints at all.
00:45:35:04 - Well, we're winding down on our hour.
00:45:39:07 I do want to thank all three of you for joining us.
00:45:42:24 This has been, been really helpful conversation
00:45:47:23 and yeah, we were really looking forward
00:45:50:09 to having more events like this into the future
00:45:53:17 and more videos like this into the future as well.
00:45:56:13 So yeah, Jessica, Liz and Kate,
00:45:59:07 I really want to thank all three of you for joining us and
00:46:01:16 for really expanding the impact of this video.
00:46:09:17 - Thank you so much
00:46:11:03 - Thank you.
00:46:11:24 - Thank you

Videos in this Talk

Part 1: Publishing in a Pandemic: The Preprint Revolution
Audience:
- Student
- Researcher
- Educators
- Educators of H. School / Intro Undergrad
- Educators of Adv. Undergrad / Grad
Part 2: Early Career Scientists and Preprints Q&A
Audience:
- Student
- Researcher
- Educators
- Educators of H. School / Intro Undergrad
- Educators of Adv. Undergrad / Grad

Speakers: Kate Carbone, Elizabeth Silva, Jessica Polka

Total Duration: 0:55:05

Recorded: April 2020

All Talks in Science and Society

Talk Overview

As the COVID-19 pandemic escalated in the beginning of 2020, there was a need for the rapid dissemination of scientific information to understand everything from how the virus spreads, to how it affects the human body, to how the disease can be treated and prevented. Jessica Polka, Executive Director of ASAPbio, explains that, for this reason, the pandemic has inspired an increasing number of scientists to post their research papers on preprint servers. Publishing papers in traditional journals with formal peer review can take a lot of time. Posting manuscripts on preprint servers, on the other hand, can happen quite fast. But, considering that preprints do not undergo formal peer review before they are posted, is this prudent? Dr. Polka talks about how preprints experience their own crowd-sourced, informal peer review process through public commentary on social media and other venues by scientists and other experts, and this discourse can actually improve the quality of the paper over time. A major caveat of preprints is the potential for the media and the public, who may not be aware of the provisional nature of preprints, to disseminate erroneous or inaccurate information, and so, this requires proper education and training on what preprints are and how to interpret them.

In the second video, we have an online Q&A (hosted on January 6th, 2021) discussing the benefits and limitations of preprints from the perspective of early-career scientists. Panelists included Jessica Polka, Elizabeth Silva (Associate Dean of Graduate Programs at UCSF and former editor at PLoS), and Kate Carbone (an industry postdoc).

Speaker Bio

Kate Carbone

Kate Carbone is an industry postdoc and studies immune cell biology across scales, from biochemical reconstitution to in vivo tumor models. She’s currently focusing on cancer immunology. She earned her PhD from UCSF in the lab of Ron Vale and published preprint papers as a grad student. At the company where she currently works, Kate… Continue Reading

Elizabeth Silva

Elizabeth Silva is the Associate Dean of Graduate Program and an adjunct assistant professor in the Department of Social and Behavioral Sciences at the University of California, San Francisco. Elizabeth is a trained by biomedical research scientists with experience in science policy, particularly relating to publication and research ethics as well as an early career… Continue Reading

Jessica Polka

Dr. Jessica Polka is the Executive Director of ASAPbio, a non-profit organization that works to address the efficiency by which scientists share their results by promoting innovation and transparency in life sciences communication. As part of her advocacy work, Polka served as president of the board of directors of Future of Research, and is a… Continue Reading

Credits

Elliot Kirschner (Wonder Collaborative): Executive Producer
Shannon Behrman (iBiology): Executive Producer
Rosa Veguilla (iBiology): Producer
Rebecca Ellsworth: Editor
Eric Kornblum (iBiology): Editor, Videographer
Chris George: Design and Graphics
Maggie Hubbard: Design and Graphics

Comments

ankeet says

December 22, 2020 at 9:40 am

i just want to thank iBiology, such a nice talk

Reply

Part 1: Publishing in a Pandemic: The Preprint Revolution

Part 2: Early Career Scientists and Preprints Q&A

Talk Overview