(Is This Really What I Need?)
This article is intended to walk you through the process of analyzing a research resource — a book, article or website — and show you the questions to ask in order to judge if it will be useful for your research.
This article will show you how to:
— analyze a resource
— evaluate its information
— decide on its usefulness
The outline we will cover gives you
— an explicit framework to practice and
— the ability to decide if a resource will help your research.
The outline we will be covering gives you an explicit framework for an evaluation — a framework that, after you practice it a few times, you can make implicit and quickly run through so that you can decide if a resource will help your research.
How do You Know What You Need?
Evaluating a research resource actually involves two separate processes:
- First, you need to decide what do you want the resource to tell you. What problem are you trying to solve? What question do you want it to answer?
- Then, you need to examine the resource, and decide how well it accomplishes the desired task. Does it have the information you want for your research? All of it? Or just part of it? Does it have enough to make it worth using for your research?
Why is all of this work beforehand important for your research? I think Allison Hosier, an Information Literacy Librarian at Coastal Carolina University and co-author of a book on Information Literacy (which is a larger umbrella term over what we are doing here — learning how to analyze information), said in a blog post:
Communicating the value of your research starts with identifying the problem you are trying to solve:
“I think the reason thinking about what problem you’re trying to solve is useful is because it creates a clear path for creating an argument around why your research is important or even why it exists in the first place.”(Quote from Allison Hosier’s Blog, Studying Research JANUARY 13, 2022)
So the question of what information you want from a resource is a key part of the larger question, “Why am I doing this research in the first place?” (You’re going to have to decide that for yourself.)
Some of the possible different research contexts you might want to consider: academic, creative, personal, professional, scholarly, or scientific.
Okay, now to get to work:
What do you want this resource to do for you?
- Do you want it to familiarize you with a new subject? Are you looking to it for background information?
- Is it going to serve as evidence and “building blocks” for an argument you are making in your write-up?
- Are you looking for some fresh approaches to a topic you’re already familiar with?
- Are you looking for the latest developments in a field you already know fairly well
That’s probably a good range of expectations for the different levels of comfort you might have with any topic.
How much do you already know about this topic?
As for actual knowledge of the field:
- What general idea do you have of the subject? Vague? Or can you expound upon it for 10-15 minutes before you have to stop for a breath?
- Do you know the names of the prominent researchers and the current hot topics in the field?
- Do you have definite opinions about those hot topics? Are you deeply enough involved in the research to see a right or wrong in the field?
Now you should know what (and probably how much) you want the resource to tell you.
Become familiar with a published work or website
Now it’s time to take a look at the resource you are evaluating.
First, note the basic information for your resource.
- Then, skim through the work and answer these questions.
- The answers will help you decide whether to read it in depth.
What is the full title and year of publication?
Is the information going to be current and timely? Or is it older and possibly superseded?
For that matter, since this will vary based on how familiar you are with the research field, do you actually need current information? Or would an older (or more basic) work be more useful to you? Would it give you more background in the field, so that you understand what has been going on, later, when you get to looking for current research in articles?
Do you need actually need a “current events” perspective from a particular historical period? Does this fill that need?
What type of publication is this?
Is it an Encyclopedia article – where you can get a background summary of the field?
Is it a Book – that will cover a single subject in some depth?
Is it a chapter in a collection – so that you can get different perspectives on a broader topic?
Is it a research journal article – that covers a single narrow topic, and may provide some new insight?
Is it a review article — that helps pull together lots of information on a fairly specific topic? (If you can find a review article on your topic of interest, you have hit a goldmine! Read it thoroughly, and start mining its bibliography for more in-depth information!)
Is it an editorial or opinion piece? If it’s in your field, this can be useful for providing you with a perspective that you might not have gotten before. Read it and think about what it says. Does it change your mind? Does it convince you?
Or is it a news report – with factual information about a research topic that interests you.
What journal is it published in?
If you’re looking at a journal article, what journal is it published in?
Is it A Scholarly Journal, with Refereed Original Research articles?
Is it A Trade Journal, with News for Professionals in the Field? (These can be useful, depending on what you need.)
Is it A Popular Magazine, with Stories intended for the General Public?
Who are the Author and Publisher?
Who is the Author of the article (or book)? How much do you know about them?
What is their occupation, their position, their educational background and experience? (Research articles will generally give you the basics of this — their research laboratory or university affiliation, at least — and they will do it for all of the co-authors, too. Where are the various co-authors working? All at one lab? Or are they in labs scattered around the country or the world?)
From what you know, or have found out by their affiliation, is the author qualified to write on this topic?
Who is the publisher of the work? Are they a major publisher in the field, or just a small press with only a few publications to their name? Are they possibly a non-profit, society or university press?
What type of material does the publisher usually produce? Are you familiar with other publications from this press?
And, knowing the publisher, what can you say about the likely quality of this work? Do they have a reputation for publishing well-researched quality books and journals? Or are they known for publishing works that few people read, and fewer people cite?
What features does this resource have that might help you?
Now, look a little closer, and ask:
What’s listed in the Table of Contents or the Headings and subheadings of this work?
Does the author supply key words for the work that fit your subject or intrigue you?
Is there an index, a bibliography, a glossary, or illustrations included? These can be useful additions to the written word, or can make it easier for you to find the sections of the work that will be most useful for you.
By now, You should know:
Enough information to cite this work in your bibliography if you decide to use it, along with Information about the author and the publisher of the work.
So you know the work. Now you need to find out enough to decide if you want to use it.
This is where you become familiar with the actual *Purpose* of a published work or website
What is the work or website going to tell us?
What does the title, the introduction, or the abstract tell us about the scope and audience of the research?
Why did the author put for the effort to write the article or do the research?
Who did the Author write this for?
Did they write for:
- The General Public
- Policy Makers
How does the Author’s language or writing style show this?
Was the piece or website reviewed by others? (Did it go through the peer-review process?)
Analyzing the Viewpoint of a published work or website
What are the author’s frame of reference and intentions? Is this included in the thesis statement of the article or website?
(The thesis statement is usually given near the end of the abstract or introduction of an article, and sums up what the author intends to tell us.)
What viewpoints or biases are evident in the writing?
Does the author have a bias or make assumptions upon which the rationale of the article or the research rests?
— What are these assumptions or biases?
Is the author associated with a particular group of researchers? (Are you familiar with their work and viewpoints?)
Does This Work Answer My Questions?
Okay, time for an Intermediate question for your decision-making. By now, you should know the areas covered by this resource or website.
Is this work likely to be able to answer at least some of the questions you are asking of it?
Now we will look at How the Author Approaches his Topic — *HOW* he is answering your questions.
The Author’s Approach
Looking more deeply at what has been written, what is the author’s main argument in this piece?
Does the author clearly state his argument in the abstract or first paragraph of the piece? Or does he take a while to get to the point of his work?
What findings does the author present to support his argument? Do the findings he presents support this viewpoint?
— How does this viewpoint compare with others in the field?
Is the writing style of the work appropriate to its purpose?
Is the language easily read by the intended audience? (Or is it too erudite? Or too simple?)
Are the structure, parts and transitions clear?
Is the writing of the work coherent?
Do the arguments made make logical sense?
Does the author lead the reader from accepted fact to new theory in a logical manner?
Are the sources supporting the argument current and reliable?
Has the author cited the sources accurately? Or is something quoted out of context?
What conclusions are made from the findings?
Do the conclusions make sense in context?
Does the author fulfill the stated goals?
Challenging the Author’s Approach
Now to see how well the viewpoints presented match what you were expecting (or wanting).
Do you feel confident in challenging points in this work?
— Does the author defend any new conclusions in his piece?
How do the conclusions compare with other material in the field?
— Are the points consistent with, or do they differ, from the conclusions of other researchers? How well does the author support any radical changes?)
Are you comfortable with the author’s arguments and conclusions?
Evaluating the Author’s Approach
Does this work make a contribution to the literature?
Have other works referred to this work?
Is this work in tune with or in opposition to works by other researchers?
Are there specific works with which this one agrees or disagrees? Are you aware of them? Should you be?
Evaluating the Work Overall
Okay, now for the main question:
Will this piece make a contribution to your research?
Does it have information that will benefit your research?
Does it support or refute the argument you want to make?
(Can you defend your research against an argument being made in this work?)
By now, you should be able to make a decision about the work and its usefulness to your research.
If you are interested in reading more about the critical evaluation of sources, you may want to look at these sites:
If you are interested in finding out about learning a topic by “finding out for yourself”, take a look at the FOSIL Inquiry Cycle.
(Yes, this caught my eye because, at heart, I will always be a paleontologist.)
(Note that much of this presentation is based on the work I was doing while I was working at Carlson Library at the University of Toledo.)