Skip to Main Content

SOCW 5395 Integrative Seminar: Intervention & Evaluation

Intervention and Evaluation

While you will likely have done a bit of searching in the databases for your previous assignment(s), this is the one where you will really want to dig into the evidence. The databases in the left column are where you will search for studies that have been conducted on your population and intervention. 

When it comes to evaluation, there are certain assessment tools that you can use, but you'll also need to think critically about the elements of the study and/or intervention. The center column provides strategies for analyzing the literature.

Databases

Assessing Interventions with Tests & Measures

Authors sometimes make their instruments accessible to others. When shared, they are often appended to journal articles and dissertations, but they might also be found in books and on web sites.

There are two primaries ways to search for these measures & tests using UTA Libraries resources. The most straight-forward way is to look in the Mental Measurements Yearbook database.

The second way requires some strategic searching in the PsycINFO database. 

PsycINFO has a Tests & Measures search field. In order to search for articles that have used a measure on a specific topic, enter a very broad key term in the search field, and use the dropdown to select Tests & Measures.

The image above shows what a search for measures related to suicide risk and suicidality might look like. In the search bar, the suicid* is our search term, which will pick up any variations on the word suicide. The TM Tests & Measures selection from the dropdown tells the database that our key term must appear in the Tests & Measures field.

However, this does not guarantee that the measure that was used in the study will appear in the article. Sometimes measures are appended to the article, in which case, you might want to try a search that includes a second search line with the key term append* and the dropdown Tests & Measures. See image below.

Within the results of the above search, you should find one or more of the measures used in the study attached as an appendix or within the text of the article.

CRAAMP Test

The CRAAMP test is a valuable tool for helping you determine if an article you are reading is credible and a good fit to use as a source for your research paper. As you read, pay attention to the 6 following criteria.

  1. Currency: When was the information published or last updated? Is it current enough for your topic?
  2. Relevance: Is this information that you are looking for? Is it related to your topic? Is it detailed enough to help you answer questions on your topic.
  3. Authority: Who is the author or creator of the information (can be an individual or an organization)? Are they an expert on your topic? Has the source been peer reviewed? Who is the publisher? Are they reputable?
  4. Accuracy: Is the information true? What information does the author cite or refer to? Can you find this information anywhere else? Can you find evidence to back it up from another source? Are studies mentioned but not cited? Can you locate those studies?
  5. Methodology: What type of study did they conduct? Is it an appropriate type of study to answer their research question? How many people were involved in the study? Is the sample size large and diverse enough to give trustworthy results?
  6. Purpose/perspective: What is the purpose of the information? Was it written to sell something or to convince you of something? Is this fact or opinion based? Is it unfairly biased?

Use the template below to chart each of these criteria in the articles you use for your research paper.

Qualitative vs Quantitative Research

 
QUALITATIVE QUANTITATIVE
Methods include focus groups, unstructured or in-depth interviews, and reviews of documents for types of themes Surveys, structured interviews, measurements & observations, and reviews of records or documents for numeric or quantifiable information
A primarily inductive process used to formulate theory or hypotheses A primarily deductive process used to test pre-specified concepts, constructs, and hypotheses that make up a theory
More subjective: describes a problem or condition from the point of view of those experiencing it More objective: provides observed effects (interpreted by researchers) of a program on a problem or condition
Text-based Number-based
More in-depth information on a few cases Less in-depth but more breadth of information across a large number of cases
Unstructured or semi-structured response options Fixed response options, measurements, or observations
No statistical tests Statistical tests are used for analysis
Less generalizable More generalizable

Adapted from https://www.orau.gov/cdcynergy/soc2web/Content/phase05/phase05_step03_deeper_qualitative_and_quantitative.htm

Recommended Readings

When setting or establishing a new goal, consider using SMART goals. By using the acronym S.M.A.R.T. you provide structure to help ensure that a goal is specific, measurable, achievable, relevant and time bound.