As we also talk about in the journal clubs, there are a lot of depth levels one can reach when reading papers. One need not read every paper to the same depth. In fact, it's a disaster if one does, because then one is often over-learning a few works at the expense of learning from many (or alternatively, not reading a few papers to great enough depth). Instead of reading a paper from beginning to end and only then assessing it, here is my suggested approach to reading scientific literature. First, (1) look at the title and authors. Then, (2) read the abstract. Then, (3) flip through looking at section titles and figures/figure captions and read the conclusions. Then, (4) skim the results and discussion. Only then, (5) do an assistive reading through the entire paper. Lastly, (6) give the paper a resistive reading. Recall that assistive reading is fast---knowing that there are holes in one's understanding, but with faith that either the author will explain what you've read or knowing that you can always go back if that lack of understanding turns out to be important. You should not make any annotations when assistive reading. The goal is building a mental map of the concepts within the work as you move through it. Resistive reading is slow, questioning, and note-making. Critically, after each of the steps 1-5 there should be a moment of reflection and metacognition. The reflection should be aimed at answering three questions: What is the takeaway message? What evidence is being shown to support that message? Is it convincing? The metacognition is aimed at answering: Is continuing to learn more about this work worth my valuable time?
This is a reading flow chart that describes the 6 steps to reading a paper that are described in the accompanying text.
This last point regarding metacognition is critical. If one applies the program above to literature surveys, then one is able to triage through dozens of articles (stopping somewhere between steps 1 and 5) to find the handful that require an assistive or even a resistive reading. My prescription is to use ADS and Google Scholar to find many articles based on steps 1 and 2 that could be valuable, keeping articles that make it through step 2 as a tab in my internet browser. Then I proceed back through those tabs, downloading each paper, then moving through step 3 and, possibly, step 4 in the downloaded pdf. I make a notation either on scrap paper, in a draft, or in a txt file that I want to move to step 5 for a given paper. I usually then do the full readings (steps 5 and 6) at a separate time.
By step 4, I have confidence that my reflections (takeaway message, evidence, believability) are at a level that if asked those three questions in journal club, I could provide answers to them and point to the primary figures or locations in the text where the evidence for the main claims are. At the same time, I likely couldn't provide answers to many possible follow-up questions that require detailed knowledge of the paper's context, methods, or results. So when presenting a paper at journal club, I do try to at least do step 5, an assistive reading.
To put some numbers to the difference in speed between assistive reading (step 5) and resistive reading (step 6). A typical two-column scientific journal article has about 750 words per figure-free page. Assistive reading should be able to proceed at about 250 words per minute (wpm), and so one should be able to read a figure-free page in about 3 minutes. Typical articles contain about 10k words, so that would mean an assistive reading should take about 40 mins (letter length articles 2.5k words should take about 10 mins). Resistive readings can take substantially longer. It's plausible to progress at well under 100 wpm, so a 10k work article could take many hours. This is why it's not good practice to plunge straight into a resistive reading: it's very time consuming. Furthermore, the assistive reading will often make an eventual resistive reading faster since (1) one will know what's actually important in the paper as opposed to an aside, (2) one has context clues that appear after as well as before a section of text, and (3) one is typically reading with a purpose, i.e., one has already identified from the assistive reading which concepts are the tricky ones that will require interrogation/time to understand.
To re-iterate, I put this practice into action by going through steps 1 and possibly 2 as I encounter papers (e.g., from listservs, social media, from other papers, etc.). If the paper is worth more time, I send it to a folder on my laptop called triage. I then set aside some time each week to do a triage process, where I move through steps 2 through 4, as necessary. If after step 2 (or 3 or 4), I think that I’ve taken what I can from the paper, I move it into my paper organization system (i.e., Bookends). If I decide that I really want to read the paper (step 5), I then find time in my calendar and read it then. The few papers that require step 6 will need additional time allotted.