Evaluating information
Evaluating information is an important aspect of critical thinking, and one that you should demonstrate in your assignments. Evaluation includes both selecting appropriate sources and understanding the strengths and limitations of the sources you use.
This can be challenging at first, as there isn’t a simple checklist which can be used for all sources of information. However, it is a skill you can develop. This tutorial presents guidelines and examples which will help you learn to:
Selecting sources
Realistically, you don’t have time for in-depth analysis of every source you find. Therefore you should start by narrowing down your sources to those that are most likely to be useful and reliable by:
Use appropriate search methods
Using an effective search strategy can reduce the time you spend evaluating sources that aren’t useful. For example, if you need to find peer reviewed articles, you can limit your results to articles from peer reviewed journals when searching.
For advice on how to search effectively, see Where to search and How to search.
Assess relevance based on title or abstract
Look at the source’s title, abstract or summary, or briefly skim the main text, and think about how it could help you address your assessment task. For example, the source might:
- explain an important theory;
- provide evidence; or
- serve as an example.
Activity
Relevant or not?
Some sources appear superficially related to your topic, but don’t help you address the task.
Imagine you’re working on an essay with the topic below:
In 2010, Mark Zuckerberg claimed that privacy was no longer a ‘social norm’ (Johnson, 2010). Do you believe that attitudes to privacy have been fundamentally changed by the popularisation of social media?
Based on their title, do you think the following articles would be relevant?
Check if the information is from a reputable organisation
Sometimes it can be hard to judge a source just by looking at it. This is mostly an issue when searching public websites, for example with Google or Baidu, as the quality of sources you find can vary a lot. If you’re not sure about a source, try looking up the publisher or organisation that produced it to see if they are reputable.
For example, the “Alliance of Australian Retailers” (AAR) sounds like an organisation for shop owners. However, if you do a web search for it you’ll see that many sources, including mainstream news media, say AAR was funded by the tobacco industry to lobby against cigarette packaging laws (e.g. Castle, 2015; Davies, 2010; Macey, 2010; Scollo & Greenhalgh, 2018).
Check for a reference list
Academic sources typically have a substantial reference list or bibliography. If a source doesn’t include references, or refers to low-quality sources, it may be less reliable.
However, there are some quality sources which don’t include a reference list, especially primary sources where the author has collected information directly. For example, a report from the Australian Bureau of Statistics may rely only on data the Bureau has gathered. For a more detailed description of primary sources see Understanding what information you need.
Detailed appraisal
Once you’ve decided a source is worth reading in detail, you should perform a more nuanced evaluation. Even good, reliable sources often have limitations or context that is important when analysing the information they provide.
Some things you may like to consider are:
How does the source treat the topic?
There are many ways a source can approach a topic, e.g.:
- In depth, or a broad overview;
- Theoretical, or applied;
- With numbers and statistics, or with non-numeric information.
These differences affect what you can get out of a source. For example, statistics about homelessness might tell you how many people are homeless, but may not explain why, or what being homeless feels like. Think about what your sources can and can’t tell you, and how they will help you address the assignment task.
How applicable is the information to your topic?
Even if a source is relevant to your topic, it may not match up perfectly. The source may focus on a slightly different question, or use different definitions or assumptions. It’s okay to use information from these sources, but you need to keep the differences in mind.
Activity
Images: Hans Braxmeier, StartupStockPhotos, and Andrew Martin. Images sourced from Pixabay and used under Pixabay License.
How was the information obtained?
Some sources present information the authors have collected themselves, for example a scientific paper which presents the results of an experiment, or a company report which includes sales numbers.
For these sources you should consider how the information was collected, as this can affect how it should be interpreted.
Activity
Images: Diego Fabian Parra Pabon, Alexas_Fotos, David Schwarzenberg and Juraj Varga. Images sourced from Pixabay and used under Pixabay License.
How does context influence the source?
When evaluating information, you should consider not only what a source says but also the circumstances surrounding it, including when, where and why it was created.
Activity
Images: Steve Buissinne, Benita Welter, and Darko Stojanovic. Images sourced from Pixabay and used under Pixabay License.
Sometimes you may want to use a source as an example, rather than because you believe what it says. In this case, you need to examine the context to be confident that what you’re citing is an authentic example.
For instance, if you used a folk story as evidence that people used to believe in fairies, you would need to check that the story is actually from the time and place you’re studying, and that people believed it was literally true.
How strong is the reasoning?
Most sources present analysis and draw conclusions based on the information they present, rather than just listing raw data. This can help you make sense of the information, but weaknesses in the analysis can make the conclusions less convincing.
Some things to look out for are:
- Assumptions which might not be true. E.g. An economic analysis assumes the price of steel will remain stable, but other information suggests it will increase.
- Relevant factors that are not addressed. E.g. An article blames schools for childhood obesity, but doesn’t consider the role of activities outside school.
- Conclusions that go further than the evidence supports. E.g. A study suggests iron supplements benefit people diagnosed with anemia, but the authors conclude everyone should take supplements.
If you want to build your reasoning skills, further information on argumentation and reasoning can be found in this critical thinking tutorial.
Summary
- Evaluating information is an important part of critical thinking.
- Evaluating information includes both:
- When selecting sources, you should:
- Once you’ve selected sources, you should evaluate your sources in more detail. This evaluation will form part of your analysis.
- Some things you might consider are:
References
Castle, J. (2015, October 7). Front groups and ‘astroturfing’: The ‘grassroots’ movements campaigning on behalf of industry. Choice. https://www.choice.com.au/shopping/packaging-labelling-and-advertising/advertising/articles/front-groups-and-astroturfing
Davies, A. (2010, September 11). Big Tobacco hired public relations firm to lobby government. The Sydney Morning Herald. https://www.smh.com.au/national/big-tobacco-hired-public-relations-firm-to-lobby-government-20100910-154yg.html
Johnson, B. (2010, January 11). Privacy no longer a social norm, says Facebook founder. The Guardian. https://www.theguardian.com/technology/2010/jan/11/facebook-privacy
Macey, J. (2010, August 4). Big tobacco bankrolls anti-Labor ad campaign. ABC News. https://www.abc.net.au/news/2010-08-04/big-tobacco-bankrolls-anti-labor-ad-campaign/931280
Rossiter, M. W. (1993). The Matthew Matilda effect in science. Social Studies of Science, 23(2), 325–341. https://doi.org/10.1177/030631293023002004
Scollo, M. M. & Greenhalgh, E. M. (2018, November). InDepth 11A.2: Australian announcement of plain packaging legislation. In M. M. Scollo & and M. H. Winstanly (Eds.), Tobacco in Australia: Facts and issues. Cancer Council Victoria. https://www.tobaccoinaustralia.org.au/