Students:
UT Arlington Library's Libguide usability study indicated problems with both concrete design elements and how we conceptualize subject guides. Many of the issues identified by the study such as the need for larger headings and font sizes; uniform locations for librarians' contact information; and misuse of the Libguides' search box were easily addressable.
The conceptual problems are much more difficult to rectify. Whether testing UT Arlington's original guide design or the homepage design based on the University of Washington's recommendations, students were overwhelmed by the amount of content presented, had difficulty determining where to start their research, used only the language found in the prompt when searching, and were confused by how librarians categorized resources. Among others, the content category titled "Reference Resources" was particularly problematic.
Research about students' information-seeking behavior indicates that students conceptualize the research process differently than librarians. Rather than thinking of it as a series of steps, students instead struggle to establish four contexts: the Big Picture, Language, Information Gathering, and Situational (Project Information Literacy, 2009). Typical subject guides, including our Libguides, assume that students have figured out these contexts for their research project and simply need to identify specific search tools/databases/sources that will retrieve the desired information. The disconnect between what students need and what we provide results in confusion and dissatisfaction.
Kuhlthau's (2012) research indicates that instructional interventions when students are most confused/frustrated lead to greater satisfaction with the research process and the end result. Reconceptualizing our subject guides to provide instructional content, mapped to the contexts that students establish as part of their natural information seeking process, may lead to greater satisfaction. UT Arlington Library's First Year English guide is one potential model for meshing context, instructional content, and resources.
Transforming subject guides to a model similar to the First Year English guide requires that librarians shift their thinking. Rather than providing links to large quantities of resources potentially relevant to a broad subject area, we may better serve students by selecting significantly fewer resources for the subject guide. Specific topic and course guides, designed to direct researchers to resources with narrow scope or to address a specific information need, would be developed to supplement the broad subject guides.
It is also important to note that only one of the contexts, identified by Project Information Literacy's research, that students establish during their research process is related to information gathering, and yet this is the main focus of our subject guides to the exclusion of almost everything else. Supporting students in the other three contexts (Big Picture, Language, and Situational), will require librarians to develop significant amounts of instructional content. Exploring the best means for asynchronously presenting this instructional content will be the focus of additional usability research and assessments of learning.
For the second round of usability testing, the history subject guide was redesigned around the findings from the first usability study. Time on task and number of errors decreased while the completion score improved. Round two of the usability study used four of the five same tasks as round one. For those four tasks, participants on average completed them 3.84 minutes quicker than during the first usability test. Participants on average had 4.72 less errors during round 2. Errors coded include deviation from expected path, inappropriate use of search bar, inappropriate use of resource, and language errors. When comparing scores, participants using redesign 2 scored an average score of 1.66 up from an average score of 1.28 during the first round. Every task saw an increase in score except task 2, which many participants found the language to be confusing. If the scores for task 2 are left out, participants using redesign 2 scored an average score of 2.14 up from an average of 1.3.
Because of the page structure used in the redesign in round two, participants were more likely to use help content. During the first round, participants used help content on 16% of tasks, which increased to 29% during round two. The pages in round two all used the structure recommended by the previously mentioned University of Washington study. The larger main column contains the resources while the narrower left column contains help content. This design clearly distinguishes the purpose of each column.
The web page hits for the "Reference Resources" page increased significantly with each change in the guide. The "Reference Resources" page for the Pre-Usability Education Guide had 176 hits in a seven month period, or 5.6% of visitors who came to the home page. The same page within the Current Education Guide had 410 hits in a five month period, or 15.6% of visitors who came to the home page. The change in use is statistically significant with a p value = .03. While that page currently has the tab heading "Encyclopedias & More," it was labeled "Reference Resources" during the aforementioned five month period. This increase could be due to the added description on the new homepage or the smaller number of total tabs. The usability study results indicate that students were confused by the phrase reference resources. With this in mind, we suspect that adding a more descriptive label was a large factor in the increased usage of this content.
For the First Year English Research Guide, most of the reference materials were reorganized into a "Getting Started" page. This page also included instructional content about finding a topic, gathering background information, and narrowing a topic. In two and a half months, the "Getting Started" page had 1101 hits, or 24.9% of visitors who came to the homepage. Of course, classroom instruction can influence these web page hits and the population of users is different for the Education and First Year English guides.
Web Page Hits
The change to the current education guide increased the number of hits across many of the guide's individual web pages.
| Page | Hits for Pre-Usability Education Guide | Hits for Current Education Guide |
| Books & More | 192* | 292 |
| Statistics & Data | 36 | 119 |
| Web Resources | 40* | 137 |
| Instructional Videos | 234 | 96 |
| Reference Resources | 176 | 410 |
| Citing APA | 140 | 232 |
*these statistics include any hits for pages added. For example, The Pre-Usability guide had a separate page for books, thesis, and test prep materials. All of that content was added to the books and more page for the Current Education Guide.
Except for "Instructional Videos," the web page hits increased for all of the individual pages. The decrease in hits for "Instructional Videos" can be attributed to tutorials from that page being added to relevant pages throughout the guide. For example, the tutorial on plagiarism was added to the "Citing APA" page.
We concluded that the decreased number of tabs, and the homepage redesign, provided greater focus to the content throughout the guide instead of targeting students' attention only to recommended databases, as in the Pre-Usability guide. This conclusion is also supported by the University of Washington's LibGuide Usability study findings.
Except where otherwise noted, this work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. For details and exceptions, see the Library Copyright Statement.
© 2016-23 The University of Texas at Arlington.
University of Texas Arlington Libraries
702 Planetarium Place · Arlington, TX 76019 · 817-272-3000
