Issues in Science and Technology Librarianship
Science and Engineering Research Librarian
Assistant Head, Academic Outreach and Business Research Librarian
The research guide is a common tool librarians use to communicate with and instruct their audiences. Despite their prevalence and value to users, research guides remain underused. There are many examples of efforts encouraging guide use, but few studies have measured the effectiveness of that promotion. Academic Outreach librarians at Virginia Commonwealth University (VCU) promote research guides through inexpensive and small keytags--similar to retail customer loyalty keychain tags. Findings of a quantitative analysis suggest the keytags have positively influenced guide use and support anecdotal reports of increased interaction with intended audiences.
Virginia Commonwealth University (VCU) librarians routinely produce subject-based research guides for target audiences for use in instruction, orientation sessions, and outreach. In the past, handouts were the typical takeaway from these types of interactions; many were discarded or left behind by students. We asked ourselves what could we distribute that would be "cool" enough to appeal to students and that would encourage them to hold on to it long enough to get back to their own computers?
For the past two years, subject liaison librarians from VCU's Academic Outreach department have used keytags, similar to retail customer loyalty keychain tags, to promote research guides. Popular among librarians and students alike, the keytags have become an important subject-based promotion strategy. The purpose of this research is to measure their impact, particularly the impact on guide use.
The long and illustrious history of the research guide is well documented (Dunsmore 2002; Vileno 2007; Smith 2008). What started as bibliographies and booklists evolved into pathfinders with an instructional purpose to serve as "a kind of map to the resources of the library...a step-by-step instructional tool that will, if followed, place before the user those items that the most skilled reference librarian would suggest as basic to an initial investigation to the topic" (Stevens et al. 1973, 41). These pathfinders, now known as research guides, resource guides, subject guides, and even LibGuides, after the popular Springshare platform, continue to thrive in an online world. Studies have consistently found over 80 percent of academic libraries have web-based research guides (Cohen & Still 1999; Morris & Grimes 1999; Grimes & Morris 2000; Jackson & Pellack 2004), and now "research guides are as commonplace as books in libraries" (Ghaphery & White 2012, 23).
Given the ubiquitous status of the guide, it is not surprising they are highly valued by librarians. Jackson and Pellack (2004) surveyed heads of reference services at ARL libraries and asked, "Do you think creating and maintaining these pages is worth the time and effort they require?"; 75 percent of librarians said yes. Studies have also found students consider research guides useful or helpful (Staley 2007; Murphy & Black 2013). Courtois et al. (2005) asked students, "Was this guide useful?"; 52 percent rated guides as very helpful or somewhat helpful. Dalton and Pan (2014, 518) also reported positive feedback from students but highlighted another common finding: "...awareness of the guides was extremely low."
Reeb and Gibbons (2004) documented low guide usage citing unpublished data with a majority of guides across several institutions receiving under 300 hits a month. When Ouellette (2011) interviewed students, she found that guides were used as a "last resort" and cites one of the reasons was a lack of knowledge about their existence. That suggests a problem that Tchangalova and Feigley (2008) articulate well, "All of the time and effort put into the creation of a subject guide seems rather pointless if librarians remain indifferent as to whether or not potential users are even aware of the existence of subject guides."
Dalton and Pan (2014) updated this list with additional areas of concern in today's library including social media, promotion on the library web site, and digital screens located in the library.
The research on successful research guide promotion focuses primarily on the very first bullet point, use in instructional settings. The opportunity to collaborate with faculty on guides has been found to enhance use (Adebonojo 2010; Gonzalez & Westbrock 2010). Murphy and Black (2013) looked at factors affecting high usage for guides embedded a learning management system. They found course-related engagement and customization positively influenced usage but also noted that even passive use of guides in instructional settings could lead to high use. Foster et al. (2010) found evidence of increased usage with online marketing strategies but also noted a strong relationship between instruction and research guide visits. Castro Gessner et al. (2015) contend that the face-to-face interaction from instruction or broader reference work is an adequate marketing tool. At best, there is little understanding of how promotion strategies that extend beyond instruction affect guide use.
This lack of understanding is especially true for print promotion strategies. Despite the lack of information on effectiveness, print promotion still fulfills an important function as a physical takeaway from tabled events, orientations, and instructional sessions. Handouts, flyers, and business cards have long filled this purpose but have not garnered the desired interest at VCU. To address the need for an attention grabbing giveaway, Academic Outreach librarians partnered with the VCU Libraries Director of Communications and Public Relations to design a keytag.
Each keytag is a 3.5" by 1" (or the size of a business card cut in half lengthwise) cardstock tag with a URL for a research guide printed on one side and a subject-related image on the other side (Figure 1). To encourage students to keep the keytags, they were designed to be small, unique, and fun. They can be given away individually or strung together to make a customized keychain (Figure 2).
Figure 1. Example of an individual keytag, front and back.
Figure 2. Example of a custom set of tags on a keychain.
In Spring 2013, one keytag was created for the math guide as a proof of concept. The librarian supporting math was new and was establishing a relationship with the department, so she hung a poster advertising her services and included a pocket containing 45 tags. Over a two month period, 33 tags were taken, and feedback was positive. A statistics faculty member from the same building even requested a tag for the statistics guide.
Based on the success of the pilot, the promotion expanded to an additional ten tags for Fall 2013. The new tags were featured at the library's annual Library Fest, an event held during Welcome Week on campus. Subject librarians distributed tags at a table where they created custom research guide keychains for new students based on students' interests and intended majors. During that event, two librarians had almost 400 interactions that resulted in distribution of keychains, and once again, the feedback was positive from event attendees.
The promotion was popular enough to expand even further, allowing librarians to request tags based on their specific outreach needs. There are now 45 tags available with more in production. The keytags are one of the primary ways research guides are actively promoted beyond being heavily referenced during instruction sessions and reference transactions. The tags currently are used as giveaways for a wide range of outreach activities including instruction, consultations, orientations, and tabled events.
The response has been overwhelmingly positive. Over the course of two years, at least 38,000 keytags have been distributed. This scale of distribution for a promotional item is unprecedented for the department. Typically, tags are not given away without some mediation, so this number also represents a staggering number of interactions. Note that the customized keychain composed of multiple tags is common, so the distribution number does not provide an exact number of interactions. However, the very act of customizing a keychain requires that librarians directly interact with or otherwise determine the research needs of their audience.
The keytags have proven to be particularly useful during orientations and tabled events. They prompt a discussion of the range of library services and resources useful to a particular group or discipline and are a meaningful takeaway for students who attend orientation but receive little library instruction. To date, eight of the nine subject liaisons in Academic Outreach have used the keytags for subject specific orientations.
There was concern and some skepticism at the very beginning that students might not understand or want the keytags, but students quickly grasped the purpose. They were witnessed slipping the tags onto their own keychains and backpacks without prompting. Faculty have also remarked on the innovative nature of the idea and have continued to request tags for their guides. Librarians also have embraced the format, and all of the Academic Outreach librarians have requested tags for guides in their area. In addition, the program has extended to other departments and beyond research guides.
The positive response has been encouraging and provides strong evidence that the promotion accomplished one of the set goals, to create a giveaway that students are interested in taking. We also wanted to consider the impact on research guide use by comparing the performance of guides with tags to those without tags.
VCU Libraries maintains over 500 guides, and librarians continually remove obsolete guides and create new guides. Because of this constant change, it was not possible to include all guides for a quantitative analysis.
From our initial set of all research guides, we removed those which were inactive for all or part of the study time period, August 2012 to July 2015. Guides were deemed inactive if they had zero hits for both August and September 2012, both June and July 2015, or for any period greater or equal to three months. Profile, template, and holding guides were also removed, as these guides are for staff use only. Course guides and summer reading guides, with two exceptions, were removed as use varies greatly depending on whether the course is in session, making them unsuitable for our study. The two exceptions were course guides for UNIV 111 and UNIV 200. These courses are required courses for first and second year students at VCU and are in session every semester.
The research guide home page and our "How to Write a Literature Review" guide were both excluded because their usage, averaging over 7,000 and over 5,000 hits per month respectively for the study time period, overwhelmed data from the remaining guides, which had an average of approximately 140 hits per month. The Engineering guide was excluded because five new engineering departmental guides were created during the time period of our study, likely affecting use of the overarching guide. Lastly, guides with tags delivered after October 2013 were excluded from the study, as they did not have tags for long enough to be studied nor do they qualify for the control group of guides without tags.
After this selection process, we were left with a sample of 107 guides without keytags and 16 guides with tags.
As a rule, guide usage varies greatly based on the academic calendar. At VCU, this means December, May, June, and July typically see lower use, coinciding with winter and summer breaks, while October and February typically see the highest use. Because of this annual fluctuation, it is necessary to compare usage year to year, that is, comparing a time period with the same period in another year. For this analysis, we looked at two measures, the average monthly use per guide and the percent change, year-over-year, for each month.
We calculated the average use per guide for each month from August 2012 to July 2015. The keytags arrived in either August or October of 2013. Thus, we had approximately one year of data prior to the arrival of the tags and two years of data after.
We also looked at the percent change, year-over-year, for the average hits per guide for each month for both guides with tags and guides without. The advantage of looking at percent change year-over-year is that it compares each month to that same month the previous year, thus making it easy to determine if there was an increase or decrease in use.
We considered looking at individual guides one-by-one before and after their keytags were first distributed. We discovered, however, that this approach would not work. Individual guide use is affected by many things, including the university course offerings, library instruction and orientations, current events, and even student projects. Because of these other impacts on guide use, we found it was impossible to determine the effect of the keytags when looking at guides individually. To reduce this noise in the data, we looked only at the sample of guides as a whole.
We calculated the average monthly hits per guide for guides without tags (Figure 3) and guides with tags (Figure 4). As seen in Figure 3, overall use of guides has dropped. The average yearly hits per guide from August 2012-July 2013 for guides without tags was approximately 1,221, followed by approximately 1,163 for the following year and 836 for the last year of our study. This decrease can also be seen in Figure 4 for guides with tags, although it is less prominent. For guides with tags, the average yearly hits per guide the first year was approximately 2,400, followed by 2,012 and 2,092.
We are unsure what caused this decrease. During the study time period, the faculty and student body have been steady. However, VCU Libraries began using the Primo discovery tool in late fall 2011, started a push to increase undergraduate instruction in the 2012-2013 academic year which has resulted in instruction for almost 100 percent of VCU's sophomore writing classes, and launched a major web site redesign in January 2014. All of these things may have decreased the use of subject guides by making library resources easier for our users to access. In addition, many course guides, which were not included in this study, have been created during this time period. With their more targeted information, it is possible users of course guides have less need for the broader subject guides. We do not have any evidence to support any of these ideas, but as this decrease was apparent for all guides in this study, it appears to be unrelated to the keytags.
Figure 3. Average monthly hits per guide for guides without tags
Figure 4. Average monthly hits per guide for guides with tags
It is also evident from looking at Figures 3 and 4 that the guides with tags typically had much higher use than those without and had higher use even before the advent of the tags. Because guides were not randomly selected for tags, this difference is not surprising. Librarians selected guides based on outreach needs, which generally favored guides that were broadly applicable and popular. This difference in overall magnitude makes it more difficult to compare the performance of the guides with tags to those without.
To better understand the effect of the keytags, we also calculated the percent change, year over year, starting with August 2013, the month we received the first order of tags. As can be seen in Figure 5, the guides without tags had a slight increase in use, compared to the previous year, in August and September of 2013 but generally decreased in use. Guides with tags, on the other hand, performed worse, compared to the previous year, from August through December in 2013, but generally saw increases in use since January 2014.
Figure 5. Percent change, year-over-year, in hits per guide for each month from August 2013 through July 2015
Despite the decreased use from August through December 2013, the guides with tags generally outperformed the guides without for the two years following the advent of the tags. Guides with tags saw an increase in use for 14 of the 24 months studied. In contrast, guides without tags saw an increase in use for only three months, and actually decreased in use for 21 of the 24 studied.
There is still much to be learned about what drives research guide traffic, but this study does offer promising evidence that print promotion efforts, like the keytags, can have positive impact on guide usage. There are many other factors that could have affected usage numbers, but the keytag promotion was the only major, department-wide initiative that focused on research guides during the time period considered. The mere fact that resources were dedicated to more active guide promotion was a big change and may have been the greatest factor in the success of the project.
What is particularly promising is the opportunity that the keytags provide to interact with our audience, show interest in their work, and ultimately customize promotional materials for them. Students and faculty have responded to this low-tech approach, and the popularity of the tags far exceeded expectations. Librarians, too, have embraced the ability to share their research guides in a new way and to target audiences with a simple but interesting giveaway.
For these reasons, we consider the keytags a success and will continue to produce them as an important element of subject-based outreach. However, response data still need to be monitored to make future decisions about our approach to promotion. In general, our work in this area calls attention to the deficit that previous research has repeatedly identified. Librarians need to spend more time attracting audiences to the research guides that are useful for our users and take the time to assess promotional efforts. Exploring new options to increase awareness should become integral to the research guide creation process.
Castro Gessner, G., Chandler, A., & Wilcox, W.S. 2015. Are you reaching your audience? The intersection between LibGuide authors and LibGuide users. Reference Services Review 43(3): 491-508. DOI: 10.1108/RSR-02-2015-0010
Cohen, L.B. & Still, J.M. 1999. A comparison of research university and two-year college library Web sites: content, functionality, and form. College & Research Libraries 60(3): 275-289. DOI: 10.5860/crl.60.3.275
Dunsmore, C. 2002. A qualitative study of web-mounted pathfinders created by academic business libraries. Libri: International Journal of Libraries & Information Services 52(3): 137-156. DOI: 10.1515/LIBR.2002.137
Foster, M., Wilson, H., Allensworth, N. & Sands, D.T. 2010. Marketing research guides: An online experiment with LibGuides. Journal of Library Administration 50(5/6): 602-616. DOI: 10.1080/01930826.2010.488922
Jackson, R. & Pellack, L.J. 2004. Internet subject guides in academic libraries: An analysis of contents, practices, and opinions. Reference & User Services Quarterly 43(4): 319-327.
Moll, J.W. 2003. Web-based subject guides: An exploration of best practices. Available: http://web.archive.org/web/20090227100452/http://joy.mollprojects.com/myprojects/school/subject_guides/best_practices.htm
Morris, S.E. & Grimes, M. 1999. A great deal of time and effort: an overview of creating and maintaining Internet-based subject guides. Library Computing: Internet & Software Applications for Information Professionals 18(3): 213-216.
Murphy, S.A. & Black, E.L. 2013. Embedding guides where students learn: Do design choices and librarian behavior make a difference? The Journal of Academic Librarianship 39(6), 528-534. DOI: 10.1016/j.acalib.2013.06.007
Tchangalova, N. & Feigley, A. 2008. Subject guides: Putting a new spin on an old concept. Electronic Journal of Academic and Special Librarianship 9(3): 1. Available: http://southernlibrarianship.icaap.org/content/v09n03/tchangalova_n01.html
This work is licensed under a Creative Commons Attribution 4.0 International License.