MedlinePlus Mobile Evaluation

Mobile Gov Experiences are agency stories about creating anytime, anywhere, any device government services and info. This entry is a story shared by the National Library of Medicine._ _

The National Library of Medicine (NLM) launched MedlinePlus Mobile (m.medlineplus.gov) in January 2010 to provide authoritative consumer-level health information to a growing audience of mobile Internet users. MedlinePlus Mobile is a mobile-optimized Web site that contains a subset of the content that users can find on the full MedlinePlus site (http://medlineplus.gov), which has been online since 1998. In November 2011, NLM started a 12 month formal evaluation of MedlinePlus Mobile.

Why We Did It

We launched a formal evaluation of MedlinePlus Mobile because we want to make sure that we’re providing the best service we can to get MedlinePlus content into the hands of mobile users, so they can find reliable health on their mobile devices.

What We Did

Our study is centered around 3 questions:

• Who is using MedlinePlus Mobile?
• How are they using MedlinePlus Mobile?
• Are they satisfied with their visit to MedlinePlus Mobile?

We are also looking at overview data for use of the mobile site and mobile use of the full site.

For each of these questions, we have several data points and the data points come from a variety of sources. We are doing in-depth analysis of Web analytics data, from WebTrends and Google Analytics, user satisfaction survey responses from mobile-optimized version of the ACSI survey in both English and Spanish, and a usability lab test, which we conducted in February through GSA’s First Fridays program, which provides usability lab tests for federal government websites.

Here are the questions we’re asking in our survey:

1. Please rate the visual appeal of this mobile site.
2. Please rate the readability of the pages on this mobile site
3. Please rate how well the mobile site is organized.
4. Please rate the options available for navigating this mobile site.
5. Please rate the usefulness of the features provided on this mobile site.
6. Please rate the variety of features on this mobile site.
7. Please rate the accuracy of information on this mobile site.
8. Please rate the quality of information on this mobile site.
9. What is your overall satisfaction with this mobile site?
10. How well does this mobile site meet your expectations?
11. How does this site compare to your idea of an ideal mobile website?
12. How likely are you to return to this mobile site?
13. How likely are you to recommend this mobile site to someone else?
14. What was your primary reason for visiting this mobile site?
15. Were you able to accomplish your primary task? (with follow up “If you were unable to accomplish your task please tell us what you were trying to accomplish.”)
16. Please provide one suggestion to improve the MedlinePlus mobile site.
17. What is your gender?
18. Please select the category that includes your age.
19. What best describes your role in visiting MedlinePlus Mobile during this session? (If user selects “Other, please specify” a text box is triggered for open-end response)

How It Worked

We presented some preliminary observations from the first few months of our evaluation to the NIH MedlinePlus advisory group on June 4, 2012. The slides from that presentation are attached.

What We Learned

See MedlinePlus Mobile Evaluation Preliminary Report June 2012 (PDF, 2.8 MB, 27 pages).

In addition:

• Mobile use of the full site is HUGE compared to use of the mobile site overall. So, we’re currently reaching WAY more mobile users from our non-optimized site, and we haven’t seen any negative feedback or drop in satisfaction scores.
• Most of the use of the mobile site and the mobile use of the full site is from high end devices — this is important for us to know, it will significantly impact our strategy moving forward.
• The ACSI survey responses have been very interesting. We are still working with Foresee on some challenges related to the technology of serving the survey to mobile users though.
• We have huge amounts of data that we comb through each month. It was VERY overwhelming when we first started. But, we’ve used our own internal NLM wiki to manage the project, which has been a good tool. It’s also helped us to keep our work focused around the core evaluation questions (see above in “what we did” section).

What’s Next

We will continue with the evaluation through October 2012. We’re also working on some UI changes that we hope to release this summer in response to the recommendations from the First Fridays usability tests.