Evaluating Sources: CARRDSS

Students in 9th grade at HBW learn that resources vary in their quality, detail, scope, bias and reliability.  They learn that a reference source may provide a general overview at the beginning of a research project, but increasing detail (perhaps from full-length books) is needed as the process of research goes on. Academic journal articles can provide a new insight or unusual perspective for deep analysis. As they move into high school, students transition from relying on resources freely available on the internet to scholarly, professionally edited sources that may be copyrighted and only  available in subscription databases. Students in high school work to discriminate between sources and understand their strengths and weaknesses according to the CARRDSS criteria:

  • credibility/ authority
  • accuracy
  • reliability
  • relevance
  • date
  • sources behind the text
  • scope and purpose.

Library Media Specialist Joyce Valenza recommends that you apply the criteria CARRDSS to evaluate Web-based sources.  Her approach evolved over many years of teaching information literacy. The table below introduces questions and tools to use CARRDSS to thoroughly evaluate sources for research. This content presented below was developed to expand upon the CARRDSS criteria with valuable input from Ms. Emily Orser, Ms. Anne Applin, and the library at the University of California at Berkeley.)

 Criteria:

 The Questions we ask:

 C  Credibility/ Authority  Who wrote this?  How authoritative/educated are they?  What are their degrees/ what credentials do they hold?  From where? Can you verify with an internet search, or by contacting their institution of learning?  What position do they hold? Is there a “contact us” section… or a biography section?  Is the author “unknown?” Does the e-mail address indicate if the person is working for a reputable institution or instead, are they contacted via G-mail, Yahoo or Hotmail, for example? Can you write to the author and get their CV?Who sponsors the site? Is it a government site?  A university site? People within institutions might be given their own personal pages linked to a domain within a large site (often indicated by a tilde ~name) but the content is not being verified or checked by others before it is posted. What is the mission of the sponsors? If the site lacks a sponsoring organization, it might be a “passionate individual” who may or may not be very knowledgeable. On the other hand, if it’s a sponsored site or a site that can afford its own domain/server (that you see at the beginning of the URL), can we infer that the author has financial resources, libraries and/or a research staff to support his/her work?  Note that the URL can indicate whether the site is hosted by an institution of merit, or just a free hosting site like Geocities or Google Sites or Ning on which ANYONE can publish. Has the site won awards… indicating that it is credible and helpful? Can you verify (through an independent source) that this site REALLY earned that award? Do many other web pages from trusted sources link to this site, demonstrating that it’s an important and authoritative source of information?  Tool:  learn about sites that link to this web site if you perform a search using Alexa.com.  Just put the site URL in the search box.
 A  Accuracy  Is this accurate or sloppy with the stats and details?  Do the facts stand up against other credible sources you are consulting?
 R  Reliability  Can I rely on this information? If I compared the claims made in this source, does similar information appear in other sources leading me to trust it as valid information?
 R  Relevance  Is this site helpful, given MY research focus?  Is the content tightly focused to answer my question, or is it way off track? Is it at my reading level? Given MY information need, is the site designed and constructed well enough for me to use it efficiently?Is this site really worth the time it will take to extract the information that relates to my research question? Did the search engine deliver this site to me because it’s a sponsoring link (i.e. the authors pay Google to put it at the top of the search results?)  Or did I get here because it was “pushed at me” by sneaky means?
 D  Date  How important is the “age” of the source I am using?  It really depends.  Is “current” information essential to my needs, or is it possible that older resources (primary sources) might improve the quality of my research?  After considering those questions, ask…How old is the site?  When was the source last updated– is there a “last updated” indicator?  Is the information currently valid? Do all the links on the page still work?  Is it possible that the information presented is out-of-date and lacking the insight of more recently published materials?
 S  Sources behind the Text  Does the author cite her sources to support her claims?  If the site links its publications, are they well researched and documented?  Do the references appear to be trustworthy in the same way I expect my web sites to be?  Do the referenced/linked articles indicate a balanced approach to a topic, or are they all supporting just one perspective on the issue?  (This could be indicative of a biased presentation.)Tool:  if you think this site might be a hoax site, you can investigate it using tools described in the blue box at bottom of article.
 S  Scope and Purpose of the Site  Scope:  How broadly and deeply does this site explore my topic?  Are claims supported with rich details, examples, long discussion of case studies, etc.?  Or is this source skimpy and undetailed?Web sites are messages with a purpose.  What’s the purpose of this site?  Is it to persuade me?  Inform me? Turn me into an advocate? Sell something to me? Get me to join an interest group of some kind? Support me in the development of a passion?  Do any of these sections appear that enable me to learn about the site’s mission: Mission Statement or About this Site or Join us! or Membership? Are there clues in the URL’s extension that indicate whether the site exists as part of a business venture (.biz, .com), to serve an educational purpose (.edu, .ac), a non-profit organization or advocacy group (.gov .net)?Is the information objective?  Fair? Information almost ALWAYS has a bias. What bias is demonstrated if this appears to be a source with a mission to persuade me rather than present all perspectives on a matter?  Am I getting a complete picture with fair treatment to all sides of an issue? If there is a links section, does it take me to biased sources?  Do any of these sections appear that would enable me investigate the site’s perspective or bias: Issues, Action, Advocacy, Publications, Topics— presented in biased language? Regarding any ads on this site, could they indicate that a particular segment of the population (sympathetic to a spirited cause) is being targeted for marketing? The internet offers you such a variety of types of sites as this lesson drives one to analyze, so learn how to recognize the type of web site you’ve encountered, and consider how and why the site was constructed, and if the purpose of the site can adequately address YOUR information when you use…

  • Blogs
  • Sites that sell products
  • Wikis
  • Question and Answer sites
  • Discussion lists (or forums or groups)
  • Scholarly works
  • Search engines
  • News/article sites
  • Databases/archives
  • Reference sources
  • Documents (primary sources)
  • Informational pages by amateurs as well as experts

 

Exercise 1:

What is your reaction to the following web sites?  Would you rely on them for your research?  What strategy do you use to evaluate how reliable they are for your research?  To what extent does applying the CARRDSS criteria help you to include or exclude any of the following sites for your research?

 Site 1 Credibility?  Sources?  Site 3 Credibility? Sources?  Site 5 Credibility?  Site 7 Scope/Purpose? Reliability?
 Site 2  Reliability? Scope/Purpose?  Site 4 Credibility?  Site 6 Credibility?  Site 8 Sources?

Interesting Facts:

  • Pitfalls of Internet Searching: Search Engines like Google are “free” to use.  But most of them survive by selling advertising in the form of “search results placement” at the top of the list of results, or with links at the side menu.  Those sites might be called “Sponsoring Sites.”  But they might not be the best sites for your research.
    • Some search engines only search through the first paragraph of a web site or the “properties” listed for the web site.  Key terms that are deeper into the web site are often “hidden” in what we call the “Deep Web” or the “Invisible Web” and the search engine does not access that deep.  No search engine is powerful enough to search the ENTIRE internet for the best sources.  At one point, the best that could be searched was estimated to be only 17% of web sites.
    • Many search engines list the most POPULAR web site at the top, determined by what web site has had the most visits. That’s why Wikipedia is usually the first web site listed in your search results.  It is widely visited. It is not necessarily the most authoritative web resource available, however.
    • Research has shown that most researchers don’t look past the first 3-4 web sites in a results list.
    • An advocacy group, seller or hate group is trying to persuade its audience; it is not concerned with being balanced or fair to another side.
  • Print or Internet? It depends upon the topic. Many information specialists feel they can better trust printed material, in most cases.  Here’s why.  And yet, the internet is able to be more current and inclusive of a variety of opinions, as Wikipedia shows us. History, pure science, biographies and many other scholarly topics might be most reliably covered in print. Yet, other subjects such as the newest technology uses, entertainment, celebrities, current events and very specialized hobbies are all topics whose web-based resources might be more detailed and specialized as well as more current than what one’s library offers in print.
  • News Sources and Political Fact Checking:  People who study the trends in news reporting note that journalism shows these trends that indicate that the consumer of news must be much more careful about “what they can believe,” how information may be biased, may lack detail. or be unsupported by fact, examples, statistics, etc.  A great source on this topic is a book titled, Blur: How to know what’s true in the age of information overload by Bill Kovach and Tom Rosenstiel (New York: Bloomsbury, 2010.)  We have a copy in TJ’s library.
    • As more people turn to the internet, our traditional sources of news are losing their advertising dollars. The “old trusted” network news corporations and big newspapers have less money to spend on research and sending journalists around the world… at the same time that they are having to cover more of the world.  Many news organizations rely on second-hand information.
    • Info-tainment… reliable?  In order to compete with many news outlets for a market, the news appears to concentrate more and more on celebrities and salacious stories and not matters of the most significance or meaning. People often complain that the news is “dumbed down” these days. What do you think?
    • The appearance of 24-7 news stations means that more programming is loaded with “filler” and “pundits”– people providing commentary rather than fact-based reporting. Newspeople have found that by hyping up the way they describe news events, they can “hook” their audience with a constant never-ending sense of URGENCY.  (It keeps people glued through the advertisements when they are constantly listing items as “breaking news.”)  Furthermore, the format of news shows has increasingly staged 2 “opponents” against one another for an argument or debate, and often times these “experts” are paid representatives of special interest groups.  These programs are much easier to provide than an in-depth documentary that collects information and video coverage from various sources.  But the anchorman must really be on his or her toes to challenge any misinformation provided by someone being interviewed.  These sites can help “fact check” the claims of the talking heads:
  • Domain Extensions (such as .com or .edu) are helpful only to a certain extent:

– anyone can register .com, .net, .org domain names.  So it’s not a great way to tell whether a source is “credible”– .edu and .gov can only be used by educational institutions and governmental institutions… but still not necessarily reliable

Sneaky Sneaky!

  • Former Alaska Governor and US Presidential candidate Sarah Palin’s supporters (and the world) were concerned when she made some statements that were historically inaccurate about the midnight ride of Paul Revere. Her statements made her look unintelligent and the press went to town.  Embarrassed and concerned about her reputation, her supporters went to the Wikipedia pages about Paul Revere’s midnight ride and they changed the details there to reflect her false statements so that if people were checking Wikipedia, it made Palin’s inaccurate version look “correct.”
  • People in politics also sometimes alter the facts about their enemies in online wikis (such as Wikipedia.)
  • Anyone with an agenda can put up a “factual looking” web site and steer people wrong.
  • Some people make hoax sites to entertain others.  But they get so many hits that they rise to the tops of results lists generated by search engines. Museum of Hoax Sites