This post was originally published on the 18F blog. At 18F, we have employees across the U.S. Over time, we’ve cultivated our best practices for distributed teams and design methods. Yet, doing research as a remote team is still really hard. Here are some things that we’ve found make it easier. Six icons showing different types of video conferencing. Use tools like you would in real life Being a remote team doesn’t mean you should forgo any of your research rituals.
User Testing & Research
After spending 22 years in the U.S. Army, including 3 years as a recruiter, Julie Jackson realized that not only was she qualified to work in usability, but had a knack for it—especially because of her ability to strike up a conversation with nearly anyone, anywhere. Julie shares how her training in the Army has helped in her approach to usability testing, and gives a peek inside how usability testing works for USAJOBS.
At the beginning of 2017, the ITIF (Information Technology and Innovation Foundation) released a report that benchmarked 300 federal websites in four areas: page-load speed, mobile friendliness, security and accessibility. Some sites fared better than others, but the report highlighted that our federal sites have a ways to go (DigitalGov included) in these areas. Looking at these four metrics is important as they directly impact our customers’ first perceptions of the quality of our government’s digital services.
This post was originally published by Code.gov on Medium. It’s been a year since the federal government published the Federal Source Code Policy, which created the foundation for Code.gov. In honor of the policy’s anniversary, we checked in with our users to learn more about them, their needs, and the challenges they face. Our users were responsive, proving insights we translated into our newly released metadata schema that powers the Code.
This is part of an ongoing series highlighting the innovations and research happening at Walter Reed National Military Medical Center in Bethesda, Maryland. Wounded warriors who dream of returning to playing hockey, climbing mountains or simply brushing their teeth with ease can look to 3-D printing innovations at Walter Reed National Military Medical Center to help them return to daily living. The five-person team at the 3-D Medical Applications Center can print just about anything, from prosthetic attachments to surgical simulation models and custom cranial plates.
How user interviews helped spotlight the needs of a previously forgotten group. We may not like to admit it, but, most web services or sites have users that (for whatever reason) just aren’t well understood—and in turn, not well served. Conducting user interviews and making sure you get good participation from those groups can help you accomplish several things: you get a better understanding of a once mysterious user group, you show members of that group that you are trying to understand them, and you raise awareness among management that this user group is worthy of your attention.
Amidst the chaos of an active shooter event, preparedness is key to a seamless, swift and effective response—and a new video game funded by the Department of Homeland Security (DHS) Science and Technology (S&T) Directorate and the U.S. Army Research Laboratory just might do the trick. Enhanced Dynamic Geo-Social Environment, or EDGE, is a virtual training platform, available now to all response agencies nationwide. Built on the Unreal Engine, it allows responders of all disciplines to assume discipline-based avatars and simultaneously role-play complex response scenarios.
Whenever I hear someone complain about the process of a design critique, I’m always a bit surprised. Blame it on the fact that I’m a design school graduate, where critique is a mandatory part of the educational experience. I consider learning to give and receive feedback as one of the most relevant and useful pieces of my education. But translating the rules and reasons for critique from a classroom to the workplace can take a bit of practice.
When people think of government software, they often think of COBOL and PowerBuilder 5, with manual software deploys every three to six months on a fixed number of machines in a government-run data center. This perception is sometimes justified, but sometimes entirely wrong. Regardless, the perception makes many developers reluctant to work for the government because they worry about the frustrations of getting stuck in the bureaucracy instead of being able to iterate rapidly, ship products, and deliver value.
The Office of Information Policy (OIP) is pleased to announce its collaboration with GSA’s 18F team on the development of a National FOIA Portal. This is the next step in a long line of OIP initiatives working towards a National FOIA Portal going back to 2010 with the launch of FOIA.gov. Most recently, the FOIA Improvement Act of 2016 required the creation of a National FOIA Portal that is interoperable with agencies’ current systems and allows the public to submit a request for records to any agency from a single website.
On visiting The Great Inka Road: Engineering an Empire exhibition at the Smithsonian National Museum of the American Indian, it is impossible not be taken by the sheer scale of the Inka Road. Qhapaq Ñan, or the Road of the Inka, is a 25,000-mile long road system that fed the rapid expansion of the Inka Empire in the 14th and 15th centuries. It connected distant towns and settlements in the Andes, snaking up and down mountains, bridging impossible valleys, and traversing lush agricultural fields and terraces.
We’re excited to launch a complete redesign of USDA.gov featuring stronger visual storytelling components, a more modern user-experience with easy to find services and resources, and to top it off, a completely mobile-friendly design. Through careful planning, thoughtful design, and a primary focus on user experience and usability, we’ve taken the best of government and industry expertise and put it into creating our new website. This has been a year-long project, but to do this right, we wanted to make sure we tapped into every possible resource.
The Information Technology & Innovation Foundation (ITIF) recently published a report, Benchmarking U.S. Government Websites, that looks at the performance, security, and accessibility of the top 297 government websites. ITIF is a think tank in Washington, D.C. whose mission is to formulate, evaluate, and promote policy solutions that accelerate innovation in technology and public policy. Over the past 90 days, government websites were visited over 2.55 billion times. According to the Analytics Dashboard, 43.
Presenting data online that will serve a wide range of users can be challenging. It requires an understanding of the target users’ needs, interests, and familiarity with the use of data handling tools. This challenge can be especially daunting for government websites that present data for use by the general public. The audience for such data can range in extremes—from scientists to school children. Clearly, a single data tool would not adequately meet the needs of such a wide range of users.
We at DigitalGov want to hear more about you – your job, your role, the challenges you face — all of it — as you work to deliver more secure, effective, and reliable digital services for the public. We are going to start holding user-research sessions with our readers who work in the federal government. This will be a big part of how we listen to and learn about those who are providing the public with better services and what their core needs are.
DigitalGov University (DGU), the events platform for DigitalGov, provides programming to build and accelerate digital capacity by providing webinars and in-person events highlighting innovations, case studies, tools, and resources. Thanks to your participation, DGU hosted over 90 events with 6,648 attendees from over 100 agencies across federal, tribal, state, and local governments. DGU strives to provide training throughout the year that is useful and relevant to you. One of the most resounding comments from digital managers last year was people wanted to be able to attend all of our classes virtually.
If you were to perform research on the value proposition of training videos, you would notice that opinions are split on their efficacy. Despite all the tools that are out there that can help you evaluate video quality, views, and drop-off, there are some things that should be considered in the analysis of your organization’s videos. As a member of the Service Design practice at the Consumer Financial Protection Bureau (CFPB), I was tasked with a research project evaluating how non-consumers interact with the CFPB in regards to complaint data.
Summary: How to leverage your resources to reach Spanish-dominant Hispanics online. A recent DigitalGov University (DGU) webinar provided an introduction to the intersection of two teams with different audiences reaching consensus on goals to maximize insight and outreach effectiveness. Social Media Outreach Goals What does social media outreach success look like? Success is when agencies and stakeholders have developed relationships that support each other’s social media and digital campaigns.
6,000 feet deep, 18 miles wide, 5,000 people per day: The Reality of the Tribal Beat How can a place be remote and virtually unpopulated, yet constantly full of thousands of people and teeming with activity? It certainly seems impossible, but that is exactly the situation at Grand Canyon West (GCW), home of the Hualapai indigenous Indian Tribe and the famous Skywalk. Although well over an hour from the closest town, more than one million people visit each year — arriving mostly by helicopter and tour bus.
We all do it. Whether on Twitter, Facebook, or the comment section on a news article, it’s easy to get our writing on the internet. Many of us have personal websites or contribute to blogs. We work at organizations with content management systems that allow us to publish pages with a single button click. The fact that it’s so easy to publish content can trick us into thinking it’s equally easy to write useful content.
This past summer, 18F held an agile workshop for the National Technical Information Service (NTIS), part of the U.S. Department of Commerce. An agency with roots going back to World War II, NTIS is facing a future that requires a strategic realignment towards open data and services. This strategic alignment will also require that NTIS operate in a more nimble, proactive fashion when working with partners in the public and private sectors.
As you know, over the last few years DigitalGov has surfaced the innovative advancements many are making across the government space while providing a platform for learning best practices and coming together as a community. Over the course of the next few weeks, a small team from 18F and Office of Products and Programs are working on reimagining a future DigitalGov and DigitalGov University. We are looking to talk to a few readers of DigitalGov.
I recently sat down with Michelle Earley, Program Manager, to discuss the new changes for the 20th anniversary of USAJOBS. 1) What are the three big lessons learned from 20 years of building and managing USAJOBS? I think one of the greatest benefits of being an Agile program is that we are constantly learning. In 2013, our team implemented the first phase of the data warehouse which provided agencies with data that could be leveraged to improve recruiting efforts.
Come out and join us on Saturday, October 22, 2016 from 10:00 am – 5:00 pm for a Wikipedia Edit-a-thon on Gender Equality in the Innovation Hub at the National Archives Building in Washington, DC. Register for this event today! Help us improve Wikipedia entries related to gender equality with the National Archives and Records Administration. You do not need to have prior experience editing Wikipedia. During the event we will have an introduction to editing Wikipedia and a discussion of World War I Nurses and Red Cross records in the National Archives.
The Law Library acquired a large collection from William S. Hein & Co., Inc. to make all volumes of several collections (like the Federal Register) available in open access to researchers. Preparing these files by adding metadata for easy searching takes a lot of work, so this summer we asked law students and library students from across the country to help become our “crowd” in order to crowdsource metadata for a collection of 542 volumes of U.
As any experienced retailer will tell you, the customer experience begins at the store entrance. Note the friendly Walmart greeter, the approachable minimalism of an Apple Store, and the calculated whimsy of Anthropologie. Store designers understand that a customer’s decision to make a purchase is often made within seconds of entering. The same holds true for visitors entering a museum. And while most museums are not expert peddlers of merchandise (though some museum stores certainly are), the savvy ones value the entrance experience and work to iterate and improve.
One year ago this week, we launched vote.gov (also known as vote.usa.gov). It’s a concise and simple site with a single mission: direct citizens through the voter registration process as quickly as possible. It was created by a joint team of USA.gov staffers and Presidential Innovation Fellows, all of whom work within the General Services Administration (GSA). Did it work? Yes. In fact, it worked so well that Facebook made it the destination for their 2016 voter registration drive.
Many content managers in the digital world understand the irrepressible desire to improve, fix, edit, add, and move things around. It’s our job, after all, to nurture the ongoing process of creating, updating, and testing. But, there are those sites or pages that never seem to make it to the high-priority list. For our Web team, this was our Center’s staff Intranet site. Our Web team recognized that the Intranet was in need of attention.
USAJOBS’ Analytics Success: using analytics to create accurate testing strategies. Accurate testing strategies are crucial to ensure quality products. Hi-fidelity approaches ensure QA efforts are testing in a true-to-life manner, similar to real-world users. Inaccurate, lo-fidelity testing can miss situational bugs that become showstoppers in production. USAJOBS is leveraging the Digital Analytics Program (DAP) to form high-fidelity, accurate testing strategies that mimic production site-usage in the most accurate way possible.
No Longer an Idea of the Future, Artificial Intelligence Is Here and You Are Probably Already Using It
It might surprise some of you to know that artificial intelligence (AI) is already in use and a routine part of our daily lives, but we leverage this technology when we use our smartphones or other devices to ask Apple’s Siri, Microsoft’s Cortana, Google Now, or Amazon’s Alexa a question to get the facts or data we are looking for. Using your voice, you can say, “Where’s the nearest gas station?
Our goal for a more veteran-centered and innovative VA is shared. Our approach to innovation is collaborative. Our approach to innovation is driven by listening, understanding and responding to the experiences and stories of the Veterans we serve. We were huddled on squeaky chairs in the social room of a transitional housing facility in Los Angeles. It was early fall of 2014, when Chris gently picked up his trumpet, raised it to his lips, and began playing.
When you want to do a usability test, sometimes you have to step out of your comfort zone and get creative to get the job done. That’s just what happened to us. We’re well practiced at usability testing at USAGov—in person, remote, hallway tests, first-click tests—all of these things we manage without blinking an eye. But this spring, we tried something new. Our office was planning to make some changes to our IVR script.
The Smithsonian’s mission statement is wonderfully simple: “The increase and diffusion of knowledge.” The “increasing” is arguably the straightforward part – the Smithsonian has amassed a collection of over 138 million objects and specimens, and the Institution’s curators and scientists obsessively add to the world’s knowledge base, publishing papers, creating exhibitions, and sharing their expertise. But how can all this informational goodness get passed along to teachers, our nation’s most powerful “diffusers” of knowledge?
For the first time ever, air traffic researchers can view and analyze archived flight data collected and merged from all air traffic facilities across the U.S., with fast update rates ranging from one second to 12 seconds for every flight’s position. Previously, researchers only had access to national flight data that was similar to internet flight tracking, with one-minute flight updates and no information about flights on the ground at airports.
The CALC team is an agile team of four — six if you count the Scrummaster and the Product Owner — building a simple means to load price data into the original CALC tool. They’re an Agile team, which means everybody pitches in on everything to some degree, and here, in their own words, is some reflection on what happened when they all scrubbed in on the Discovery phase. How have you been conducting the Discovery phase?
A few weeks ago, the State Department held its first conference dedicated to user experience design, UX Exponential. The conference organizers invited me to speak, and in this two-part series I hope to summarize (as best as possible) the presentation I gave, “Foster The People: Building Empathy with Stakeholder Interviews.” In the first post of this series, I covered what stakeholder interviews are, why they’re valuable, and how to prepare for them.
Summary: Clinicians using electronic health record (EHR) systems to make requests for patients need an intuitive, but safe, method of confirming that they want to cancel a started function or form. Recently, Veterans Health Administration (VHA) developers asked Human Factors Engineering (HFE) to assess a concern that a confirmation dialog in the EHR contained unclear button labeling that might easily confuse or slow down clinicians who encountered it, and created inconsistent messaging across the application.
A few weeks ago, the State Department held its first conference dedicated to user experience design, UX Exponential. The conference organizers invited me to speak, and in this two-part series, I’d like to summarize (as best as possible) the presentation I gave, “Foster The People: Building Empathy with Stakeholder Interviews.” In this post, I’ll cover what stakeholder interviews are, why they’re valuable, and how to prepare for them. In the second post, I’ll cover how to actually run the interviews as well as some tips for synthesizing and integrating the results into the team’s shared understanding.
I first came across the redesigned IdentityTheft.gov on Reddit, of all places. Someone had posted a link to the Federal Trade Commission’s (FTC) newly redesigned site and wrote: I hope this never happens to any of you as the entire thing can be really stressful. The identitytheft.gov website is a true breath of fresh air…You can talk to an actual person. They also have this extremely easy wizard to click through your situation and it will auto-generate a “Recovery Plan” including dispute letters, steps to contact law enforcement, putting credit freezes, and basically protecting yourself.
At USAGov, we always put our customers first. In the wake of our rebranding efforts, our desire to create a positive user experience across the organization has pushed us to turn a scrutinous eye toward Kids.gov — a site focused on providing information and resources to parents, teachers, and kids. In a cross-organizational effort, individuals from the marketing, user experience, and performance measurement teams have joined forces to “reenvision” the site’s content and presentation to better suit the public’s needs.
Summary: Improving the way you engage with the White House through our online petitions platform In July 2015, we announced a big change in the way we would answer petitions on We the People. We committed to responding to you within a 60-day timeframe, whenever possible. We assembled a team of people dedicated to getting your policy questions and requests to the right people so you get the most informed response.
Enrolling veterans in retirement plans. Helping small farmers access credit. Surveying employees about their workspace. These projects might seem widely different from one another: they span different agencies and diverse audiences. But all three projects have been addressed by a new team in government that is helping agencies build things better, based on behavioral science. The Social and Behavioral Sciences Team (SBST) uses theories, research, and methods from the social and behavioral sciences to address and solve challenges faced by the public.
We hear a lot about agile software development being used in work with customers and end users. User stories are developed, coders and programmers turn them into prototypes, then testing is done to make sure the features work and do what is expected. But, agile is more than a way to develop software; it’s a mindset that favors iteration over knowing everything up front. So how can you have an agile mindset inside YOUR agency?
At GSA’s Office of Citizen Services and Innovative Technologies (OCSIT), we offer technology services and tools to make government work better. To help us gauge the effectiveness of the programs we offer to other government agencies, in 2013 we launched our first Government Customer Experience Index (GCXi) survey. This annual email survey consistently measures customer satisfaction, loyalty and ease of use for various OCSIT programs. A previous post about the GCXi (OCSIT’s 2015 Customer Survey—What We Learned) generated lots of questions from readers about the back-end processes we use to conduct the survey and turn customer data into action.
Fresh from last week’s article about workflows and their importance in the content creation process, I stumbled upon a new twist in content production known as pair writing. Many of you familiar with agile methodologies or software programming in general should know the term pair programming. Pair writing hopes to take some of the same efficiencies found in pair programming and apply them to content creation. Two Heads Are Better Than One Pair programming gained prominence in the early 2000s as a method to improve the quality of software by having two programmers work together while coding.
Government product managers sit at the intersection of three circles—business, design and technology. We play a key role in user experience (UX), because we are tasked with understanding users to build a product that is desirable and viable. This product could be a paper or online form, a website or a mobile app. Product management is different from project management. Product managers are the defenders and voice of the product’s customers, while a project manager is more concerned with balancing costs, scope and schedule issues.
With 14 test cycles under our belt, the Federal CrowdSource Mobile Testing Program has heard one recurring theme from our testers—”there’s too much information!” While both desktop monitor and smartphone screen sizes are growing, there is still no comparison. At our desks, many of us are using a 24 inch (or even bigger) monitor. How big is your smart phone? Way smaller than a desktop monitor. The user will have a radically different experience on a desktop, and they are usually expecting a different experience.
The rise in mobile device usage has created a rise in expectations: the public wants new and innovative interactions with all organizations, including government. Incorporating social media in mobile websites and native apps is one way federal agencies have increased public interaction. Six agencies have leveraged native app functionality for crowdsourcing purposes. The National Oceanic and Atmospheric Administration (NOAA) leads the way with three public-facing applications that transform ordinary citizens into citizen scientists: Dolphin and Whale 911, Release Mako and CrowdMag.
Government agencies have created a variety of apps to meet the needs of the public. As you join in on the mobile first trend and begin developing your shiny new mobile application, you will need to test it. There are a basic set of questions that must be answered: Does it function properly? Does it function properly on the different mobile devices your customers are using? Do all developers and testers need a collection of devices to physically test the application with?
Good customer service includes user-centered design. For one digital team at the Department of Veterans Affairs, creating a veterans-centered experience started with one word: explore. The ExploreVA website provides a single location for veterans and their families to research the benefits that they may be entitled to receive. Benefits include health care, education, employment, and many more services. VA’s Megan Moloney, Director of Digital Media Engagement, and Josh Tuscher, New Media Technologist, spoke about ExploreVA and the process it took to develop this user-centered, interactive platform.
The short answer is: it depends on your goals. If you Google “focus group,” you will have a host of positive and negative feedback, but the truth is that it depends on what your needs are. What Is a Focus Group? Focus groups are an inexpensive way to identify people’s preferences, motivations, thoughts, feelings and attitude towards a product or service. In a typical focus group, approximately 6 to 10 people spend 60 to 90 minutes voicing their opinions about your website or application.
Personas are fictional characters that describe an organization’s customer behaviors, emotions, attributes, motivations, and goals. They are an important tool to share customer insights and understanding across an organization. Personas also serve as a check to make sure your organization’s actions meet the needs of the majority of customers, including visitors to your website, contact center, in-person visits, and interactive voice response (IVR) self service customers. Why We Updated our Personas Personas aren’t new to USA.
When the Employment and Training Administration’s CareerOneStop team embarked on a redesign of the site’s online career, training, and job resources, they didn’t dive right into the technical work. Instead, they embraced a user-centered approach that focused on the user experience (UX). Focusing on UX means taking a step back to learn about users’ core needs and preferences. The team asked real users several questions about the site.
I used to teach 8th grade science in inner city Denver in the 1990s. After that, I supported special education students and their teachers in North Carolina. Around that time (mid-late 1990s), the Internet wasn’t really designed for kids –most of the electronic materials I came across for the classroom were on CDs and such. After learning more about design, Information Architecture, and now user experience, I began to realize that while digital services for kids looked really good on the outside, on the inside they were awful.
At the U.S. Agency for International Development (USAID), our new open data policy will begin making more Agency-funded data broadly accessible to the public. It completely changes the way we do business, and it also means that in the coming years, the amount of data we host on our open data website (known as the Development Data Library) will dramatically increase. So the question is: when we’re done overhauling our website, how will the user make sense of all that information to find exactly what they’re looking for?
To improve your digital systems with user experience (UX), you need people. And to get people in government, you need position descriptions. While DigitalGov has collected a wide variety of position descriptions, I thought I would create a post specifically on UX positions, and explain the difference between these jobs. Yes, there is overlap. But this is still an excellent place to get started. I am indebted to the helpful heroes at USAJOBS for scouring through their vast job database to find these examples.
How do you define user experience (UX)? That was the question posed to more than 100 people at the GoodGovUX event at the Artisphere in Arlington, Virginia, on February 24th. Attendees learned how government can improve the user experience of digital products, from intranets to forms to good ol’ fashioned websites. GoodGovUX co-founder Keith Deaven collected responses from the crowd, which was a diverse mix of people working in private industry, federal, and local governments.
User Experience (UX) is the comprehensive experience a person has when using a product or application, and usability is the ease of use (or lack thereof) when using it. Many of us have discovered the vast advantages of evaluating usability on our own; however, getting others to jump on board is often a different story. The most difficult part of integrating an effective UX program in your organization is getting the initial buy-in from developers and stakeholders.
Government websites need to address the needs of diverse audiences. Although translations are a first step towards engaging non-English speaking audiences, the intended audience may be alienated if information is not presented in a culturally relevant way. The National Cancer Institute (NCI) engaged in user experience research in order to better serve the U.S. Latino population. The research eventually led to the creation of Spanish language personas that NCI uses to design programs, products, and services that are culturally and linguistically appropriate.
Being able to design a website that users love is not too far away from being able to read their minds. While designers can’t read minds, that doesn’t stop them from using their website’s top tasks to make it seem like they can. A website’s top tasks include 5-10 tasks (depending on the scope of the site) that the majority of the website’s users want or need to do on the site.
So you’ve done a couple of usability studies, and a few people are starting to “see the light.” Now you’d like to take it to the next level and help your organization embrace user-centered design (UCD) as the philosophy that drives all your digital projects. But what is best way to do this? How can you change your organizational culture so the UCD seed you’re planting will take root and flourish?
Usability testing has provided our organization many important insights to improve our Web presence. Since the early 2000s, the National Library of Medicine (NLM)’s Web teams have actively sought and used usability testing tools; we have run “full service” usability testing almost yearly for various Web properties for sites such as NIHSeniorHealth.gov and MedlinePlus.gov. In recent years we gained new insights about mobile device usability through GSA’s First Fridays usability testing program (now called the DigitalGov User Experience Program), and through testing responsive Web designs with the help of a usability firm.
There’s what you expect your audience to think, and then there’s what your audience is actually thinking. Sometimes, these can be entirely different. But, you won’t know unless you test it. For the release of the 2014 Consumer Action Handbook (CAH), the Federal Citizen Information Center’s marketing team piloted a series of videos. The videos intended to showcase the expertise of the CAH’s editor-in-chief, Marietta Jelks. Using letters received from the public asking consumer questions, Marietta gave her advice, trying to help not only the writer, but other members of the public who may have similar problems.
They say that necessity is the mother of invention. For me, the necessity resulted from long product development cycles paired with short windows for user testing and little room for iteration. The “invention” was the discovery of a powerful set of tools for prototyping that are available on just about every office computer. I found that you can use “Developer Tools” in Microsoft Office’s Excel, Powerpoint and Word to not only draw the basic outlines of a wireframe but also build a functioning prototype that simulates many of the features you want in your final product.
In one sense, almost any type of user research is crowdsourced—you’re talking to people and using that information to improve your system. But in a true sense, crowdsourcing is more than just collecting information, it’s collaborating on it. We want to have real conversations, not one-time emailed suggestions without followups. So here’s a few tidbits on crowdsourcing User Experience (UX) for your site, mobile app, API or whatever else you’ve got cooking:
The cream of the crop of the top of the mountain of ALL of the surveys I run has to be the Federal User Experience (UX) Survey. It’s the second time I’ve had the privilege of running it with Jean Fox, research psychologist extraordinaire from the Bureau of Labor Statistics. When I start thinking about learning what all of my UX colleagues are doing, and designing solutions for them based on real data, I start clasping my fingers together like Mr.
Bob goes to a popular federal government site, using his assistive technology, and starts reading a teaser for an article. Just below the teaser, there’s an embedded video on the page. He presses the tab key, trying to navigate to a link for the full article, but suddenly he’s trapped—he can’t tab past the video. He’s stuck, and he can’t access the content. Frustrated, Bob leaves the site.
There are many buzzwords thrown around in the digital government universe, but the most impactful ideas are rooted in one action: engagement. Whether it is a tweet, a mobile app, or a community of practitioners, every digital program or service requires interaction between an organization and its customer. Engagement is also the foundation of all user experience initiatives and is this year’s theme for World Usability Day. In light of today’s global celebration of UX, the DigitalGov team is highlighting five important facts about UX work that is done in the U.
Whether they pop up while perusing an e-commerce site or land in your inbox after your bumpy flight in from Chicago, surveys are used in many different industries to gauge customer satisfaction and glean insight into user motivations. They are a useful tool in the kit of a user experience designer or anyone who is involved with improving the usability of a product. Surveys seem deceptively easy to create, but the reality is that there is an entire industry and an academic field based on survey design.
Uncovering meaningful analytics from months or years of Web metrics is daunting, at best. So how do you make great Web improvements using metrics? Whether you’re just getting started in Web analytics or you want to take your program to the next level, you should focus on accurate data, customer service, and concrete goals, said Sam Bronson, Web Analytics Program Manager at the Environmental Protection Agency (EPA), in a Sept.
Editor’s note: Building off the great discussion started around Customer Experience, we’re looking at the difference between User Acceptance Testing and Usability Testing. If you develop software, you’ve probably heard of User Acceptance Testing. You may also have heard the term Usability Testing. Same thing, right? Nope. And confusion here can cause big problems. Last year I was developing a mobile game for Android—think Whack-A-Mole meets mutant veggies. Eight months into the project we decided to do some user acceptance testing to find some bugs before launch.
Meet Hannah Rubin, who works in the Congressional Research Service (CRS) at the Library of Congress. She’s the focus of this month’s Member Spotlight. In addition to her “real” job, she’s also currently working as a “20%-er” with the DigitalGov User Experience Program via our Open Opportunities program. What do you love most about your current job/position? CRS has a unique mission: to provide objective, nonpartisan, confidential, and authoritative research and analysis for Congress throughout the legislative process.
How do you find participants for your usability studies? I spoke recently with the User Experience Community of Practice about how we recruit participants for usability and cognitive studies at the U.S. Bureau of Labor Statistics (BLS). Hopefully I can give you some new ideas about recruiting volunteers to fuel your user research. At BLS, we need different types of participants for different studies. Very often, we are looking for members of the general public.
Why does a Cancer institute need a User Experience lab? Simply put: To learn about their customers—people living with cancer and those who care about them—and build the best possible products with them in mind. “Cancer has a journey and we wanted to create a lab to capture the substance of that journey, understand what is needed and help design technologies to support people affected by cancer,” said Silvia Inéz Salazar, an Informatics Research Laboratory Manager at the National Cancer Institute (NCI).
Trying to measure usability can be a head scratcher. How easy something is to use depends on where you are, who you are, and a number of other factors. Luckily in the world of usability, there exists a post-test survey known as the System Usability Scale, introduced in 1986 by an engineer named John Brooke, who was trying to solve this very dilemma. The SUS is no stranger to federal agencies.
Last March, the openFDA team shared their still-in-progress API to potential users as part of the U.S. General Services Administration (GSA)’s API Usability Program. FDA created openFDA to allow researchers and developers to search their vast trove of public data, including information about adverse events (reports of undesirable experiences associated with the use of a medical product in a patient) submitted to the agency. The API Usability Program brings together developers from agency APIs and the private sector to evaluate how the API can be improved to be more user friendly.
After an agency-wide redesign of program websites that targeted the public and prioritized a common “look and feel,” the federal Office of Child Support Enforcement at the Administration for Children and Families had a visually appealing website. The problem: Key stakeholders—state and tribal child support agencies, employers, and other partners who deliver program services and access the site daily—complained they could no longer easily find needed information. Their feedback prompted us to facilitate a UX-minded focus group to recommend improvements that met both users’ business needs and the redesign goals.
The good news: Your boss is interested in User Experience! The news: She wants you to do something about it… NOW. Well, don’t be alarmed; you can start by figuring out two simple things: Identify the Stage you’re in in the development cycle. Write it down. Choose a User Experience (UX) Technique that makes sense in your development stage. While choosing the technique you should first know what the technique offers and how long it takes, so you can make an informed decision.
You might recognize them by the user controls, if provided, that allow you to move from one newsy item to the next. They go by various names, including: carousel, slider, slideshow, banner, and gallery. Many government homepages have them. In a recent email exchange on the Web Content Managers listserv, the consensus was carousels met the internal, official need to share information. However, most agreed carousels were a necessary evil, but in general preference, were an annoyance.
Most people relate the term “heat map” with something they see during the weather forecast on the nightly news, those colorful maps that vividly illustrate how hot it’s going to be during an impending heat wave. The word “heat map” may not usually however, conjure up images of a widely used Web usability tool; but for those who manage Environmental Protection Agency’s (EPA) website, that is exactly what the phrase brings to mind.
Over the years, the staff intranet at the National Archives and Records Administration (NARA) had become increasingly difficult to use. Old, irrelevant content routinely bubbled to the top of search results, and essential employee tools were hard to find. NARA staff agreed that the site was due for an upgrade: fixing NARA@work was voted a top priority for 2013 in the annual Employee Viewpoint Survey. NARA managers, from the Archivist of the United States on down, supported the effort and helped recruit staff to participate.
In 2012, the Federal Reserve Board used the Top-task methodology to redesign our intranet, called Inside the Board, which had not been significantly updated since it was launched in 1995. After determining the top tasks the audience needs to accomplish on a website, you can run usability tests to gain knowledge and improve the site. The project was wildly successful. Task completion ratings rose to more than 90% after the redesign, from 58% on the legacy site—drastically increasing the productivity of the Board’s employees.
While many people tout the death of the home page, it’s still an important piece of the user experience on USA.gov. In 2013, 30% of all sessions on USA.gov included the home page—that’s 8.67 million sessions. The numbers for GobiernoUSA.gov are even higher—79% of all sessions included the home page. According to Jakob Nielsen, “A homepage has two main goals: to give users information, and to provide top-level navigation to additional information inside the site.
The Bureau of Labor Statistics (BLS) recently created a new Web page made especially for students, so who better to give it a test run than children attending “Take Your Daughters and Sons to Work Day”? We took advantage of this event held on April 24, 2014, at the Department of Labor to hear what students had to say about the new website. The K-12 page is made for kids from kindergarten through grade 12.
The annual Consumer Action Handbook, from GSA, is a guide to making smarter decisions with your money. In both its print and online formats, it includes a compilation of buying tips from across government agencies, updates on the latest scams, and a robust consumer contact directory. But the most popular part of the book is the sample consumer complaint letter. The letter template is printed in every edition of the Handbook.
What if a single piece of paper could make your mobile app work 20% better? It’s hard to imagine something as unimpressive as paper influencing our 21st century smartphones, but it’s true. Well before we get into the design and coding phases, we can show customers a mockup of an idea of what our product might look like. It’s called a prototype (or a wireframe)—it’s a model of a design that’s still in development.
The road to more user-friendly government websites does not have to be long and scary. In fact, there is a growing network of people and resources to guide you along the way. My office in the National Oceanic and Atmospheric Administration (NOAA) has been fortunate enough to benefit from some of this support, most recently in the form of a “usability walkthrough.” Where the Road Begins We were coming off the heels of having completely redesigned and relaunched our website, response.
Smaller doesn’t mean more popular when it comes to smartphone screen size. According to mobile analyst Canalys, shipments for phones with screens larger than 5″ represented a third of total shipments worldwide in Q1 this year. Devices with a screen size larger than 5″ are more popularly known as “phablets” (not quite tablets, not quite phones). Government agencies have been implementing responsive design so their Web properties adjust to screen size.
Over the past year, a GSA collaboration has seen a project that offers API Usability Testing to federal agencies go from the pilot stage to a regular, robust series. Already, 13 agencies and programs have participated, and several more participate with every monthly session that passes. The best examples from across the government have made clear that one of the most important tasks of API producers is to regularly engage their developer community and listen to what they have to say.
APIs and User Experience go together like gummi bears and ice cream. An API is a product just like a car, a website or a ballpoint pen. It’s designed to help someone do something. Products are either designed well—they meet expectations and deliver value—or they are designed poorly and create frustration and confusion. Inevitably, bad products are abandoned without a thought, like an old T-shirt with holes in it.
Incorporating usability testing throughout the entire design process, especially before launch, allows you catch glitches and/or make design changes prior to anyone seeing it live. When more than minor adjustments need to be made to your site, it’s much better to have completed them before the public sees it. For Christina Mullins, a Contracting Officer at the Public Building Service in the General Services Administration (GSA)’s Region 3 based in Philadelphia, usability testing was a new frontier, and one that quickly proved valuable.
For a small shop with a small staff, limited time, and a small budget, redesigning a website (and testing that redesign for usability) can be daunting. At least it seemed so to us when we redesigned the National Oceanic and Atmospheric Administration (NOAA)’s National Ocean Service website in November of 2013. We met the challenge by keeping things simple. One solution was to adopt the popular, open-source Twitter Bootstrapframework, which is very flexible and well documented.
User testing isn’t just for websites—it’s for any product that has an audience. Which is everything, really. And that includes print materials, signage and infographics as well. Focusing on the User Experience is especially vital for the U.S. Food and Drug Administration (FDA), which is committed to effectively communicating about products that affect the public on a daily basis. Brian Lappin works for the Risk Communication Staff at FDA. His team supports the agency in making sure that all types of communications—video, graphic and Web—are easily understood.
Most analytics tools can tell you how many times a link on your page is clicked on, but they can’t help you draw conclusions about a page with just a mere list of top links. A tool called a heatmap turns data into a data visualization, so you can more easily see how people are interacting with the design. With it, you can find out some really important stuff: if the page design plays a part in clickthroughs, where on the page your users are moving, and what on your page might be worth featuring/not featuring.
With a calculated process, the right tools, and a staff willing to make it work, you can measure user experience (UX) on your websites and implement usability changes that show results. In a recent DigitalGov University webinar entitled “Measuring User Experience”, UX supporters and practitioners heard from Achaia Walton, Senior Digital Analyst at the Department of Health and Human Services, about finding what critical things to measure to make websites more user-friendly.
Building quality mobile products is the greatest challenge for succeeding in the mobile space according to an infographic by SmartBear. One key to developing quality mobile products is testing, as “nearly 50% of consumers will delete an app if they encounter just a single bug.” As a result the following processes are used to ensure a quality mobile app: Manual Testing – 27.96% Automated Testing – 18.16% API Testing – 16.
Collecting visitor, engagement, and traffic data for your digital channels is nothing new. By this time, you have a lot of data about your website. How do you use and present that data to make meaningful recommendations? How do you use the data to tell a story and grab your stakeholders’ attention? Sarah Kaczmarek, from GAO, has worked with GSA on a series of webinars to answer these questions.
There’s no way to sugar-coat it. It’s that time again. Tax time. And just in (tax)time, the Internal Revenue Service has updated its handy IRS2Go Mobile App for iPhone/iTouch and Android phones. The 2014 version of the app offers a more elegant and streamlined visual experience — and a few convenient new features including: The Refund Status tab now has a “status tracker” for users to see the status of their tax returns.
Hallway testing is a usability test set-up in a high foot traffic area, utilizing bystanders to test your product. Your participants will be people who happen to be walking down the hall and are able to afford 5-10 minutes of their day. We on the USAJOBS team have found hallway tests successful for multiple reasons, most notably the number and variety of test participants available. In the five hallway test sessions we have conducted recently, we averaged better than thirteen participants per test.
We were hoping for 30, but we got more than 100 user experience professionals and novices on Jan 28, 2014, for our User Experience (UX) Summit at the General Services Administration. The event was sponsored by the User Experience Community of Practice and the DigitalGov User Experience Program. Here’s what we discussed: The Paperwork Reduction Act (PRA) Three speakers shed some light on the vitally important PRA process: Bridget C.
What if Thomas Edison Didn’t Use Test Cases? “I have not failed. I’ve just found 10,000 ways that won’t work.” ~Thomas A. Edison Software testing is like a science experiment. The Tester must plan: methods, steps, and paths through the Application Under Test (AUT) to completely exercise the application and uncover undesirable issues before its release. If Thomas Edison did not document his 10,000 test cases, he may have wasted time by executing some tests multiple times, or he may have missed some combinations, thus making the development of the light bulb take even longer or not at all possible.
As covered in the Mobile Product Testing Guidelines article, there are various approaches to mobile testing. This article is a resource of the Federal CrowdSource Mobile Testing Program and focuses on two common test types are compatibility testing and functional testing. Compatibility Testing The Wikipedia article on compatibility testing states the “Compatibility testing, part of software non-functional tests, is testing conducted on the application to evaluate the application’s compatibility with the computing environment.
Like a kid in a candy store… Every time we go to the mall, the kids pull us to the candy store with the floor-to-ceiling tubes of colored candy. The kids quickly grab their bags and scavenge from each of the bins until their bags fill up. They usually don’t even get to the second aisle of candy. As testers, we do the same thing with new technology, gadgets, and new devices as kids in a candy store.
Setting measurable usability goals will help your team to assess the performance of your site throughout development. Whether your assessment is at the beginning of the process, throughout iterative wireframe testing, after release, or all of the above, bench marking and improving on task performance can only improve the usability of your site. Measuring Top Task Completion The most effective usability goals measure the ability for users to complete top tasks when visiting your site.
A report by the PEW Research Center, 12 trends for shaping digital news, looks at how the internet and digital devices are changing news consumption habits. While half of all Americans still prefer to get their news from television and print, younger Americans cite the Internet as their main source for national and international news. Findings from the report include: 50% of the public now cites the internet as a main source for national and international news 71% of those 18-29 cite the internet as a main news source 19% of Americans saw news on a social network “yesterday” in 2012 64% of tablet owners and 62% of smartphone owners said they got news on their devices in 2012 31% of tablet news users said that they spent more time with news since getting their device, and 34% of the Twitter discourse about Hurricane Sandy was news and information The report also cites ‘grazing’ the news has become more popular with younger adults and online readers who get their news when they want on mobile devices compared to older adults who get their news at regular times.
I’m the kind of guy who loves tests. Not SATs, or BMI tests, but usability tests: connecting target customers with a government website and watching how they interact with it. Our DigitalGov User Experience Program (formerly known as First Fridays)has taught dozens of agencies how to conduct these vital tests, and we even created a monthly group to talk about how to run these tests, and make them better. We call it the User Experience Community, and all federal employees who are interested in usability testing are welcome to join.
Guest post by Brenda Wensil, Chief Customer Experience Officer for Federal Student Aid. Established in late 2010, FSA’s Customer Experience Office is responsible for identifying, measuring and reporting customer expectations and satisfaction with the financial aid services and products offered at Federal Student Aid. In my last post, I shared about last year’s launch of StudentAid.gov by the U.S. Department of Education’s Office of Federal Student Aid (FSA). The new site consolidates and combines content and interactive tools from multiple web sites and features instructional videos and infographics to help answer frequent questions about federal financial aid.
Usability and accessibility are slightly different lenses to assess user experience. It is possible to be strong in one area and weak in the other. Using either approach alone could result in an inaccurate view of your site’s user experience. Evaluating your website with both usability and accessibility in mind gives all users the best possible user experience. What is Usability? Usability relates to the how easy things are to use.
You have started developing your mobile product, but you may be wondering what and how to test. As with any form of software development, mobile testing should be done intermittently throughout all development stages. This article was developed as part of the Mobile Application Development Program to provide agencies with some general testing strategies, types, tools and testing scripts. The information on these testing pages has been pulled from the Mobile Gov Community of Practice and private sector resources.
Guest post by Ellen Langhans, healthfinder.gov Program Manager in the Office of Disease Prevention and Health Promotion at the Department of Health and Human Services, with contribution from Tim Hudak, Web Analytics Specialist in the Office of Communications at the Department of Agriculture. How can we get more of our subscribers to open the emails we send? This is a question that anyone who creates email campaigns for newsletters or other communications is always wondering.
Security testing is used to ensure that a mobile product does not pose a threat to agency IT systems and databases. In addition, privacy testing ensures that an app does not put the user’s personally identifiable information into a compromisable position. This article was developed as part of the Mobile Application Development Program. See our general guidelines to testing article for more resources on mobile product testing. Government Guidance Please coordinate with your ISSO when creating mobile or digital products.
Performance testing is used to verify that an app or web page will display quickly to the user and will continue to function as the number of users increases to peak loads. Performance is an important consideration for mobile applications because the connection speed of users is often slower and more variable for mobile users than desktop users. Surveys have shown that users will often stop using applications or web sites that load slowly.
Accessibility testing is a subset of usability testing and is the inclusive practice of making websites and mobile applications usable by people of all abilities and disabilities. You do this by testing mobile websites and/or applications on all leading browsers, mobile devices and screen readers. This article was developed as part of the Mobile Application Development Program. See our general guidelines to testing article for more resources on mobile product testing.
When redesigning a site, it’s easy to place menu items, text and other content wherever you can make them fit. It’s harder to take a step back and ask the strategic question: Is this the best place for this? A good rule of thumb is to never make any changes randomly—base your decisions on user data. The DigitalGov User Experience Program team evaluated Business.USA.gov on June 1, 2012, and their usability recommendations were adopted by the Business.
Many technical websites have a hard time explaining information to the general public. This happens because users don’t understand the industry-specific or scientific terms. Fortunately, solutions to these problems are fairly easy—changing menu and navigation item text, or adding a line of explanatory text on key pages or complex graphics. The DigitalGov User Experience Program helped conduct a usability test on the Department of Energy’s fueleconomy.gov mobile site in February 2013 that resulted in three top usability problems and solutions.
You’ve seen videos, podcasts, and audio files on your favorite sites—whether they’re government, private sector, or entertainment sites. These are often viral media: media clips that are wildly popular, are shared through blogs or e–mail, produce chatter on the web, and increase traffic to websites. Some government agencies are using this phenomenon, by participating in the “social web,” to further their missions and support the President’s mandate for government agencies to be transparent and collaborative with U.
If you want to make a website more efficient and user friendly, then it’s not enough just to have your most valuable information on the site. People are busy—they want to find what they’re looking for, and they want it fast. You don’t always need to redesign an entire site to make things easier to find. Sometimes, a few small changes can do the trick. The DigitalGov User Experience team looked at Army.
Functionality testing verifies that the functions of a product or service is working as intended. Each function is tested by providing appropriate input, verifying the output and comparing the actual results with the expected results. Usability testing measures the ease of use and intuitiveness of a product or service by asking users to perform a task and observing what they do, where they succeed and where they have difficulties.
The Bureau of Alcohol, Tobacco, Firearms, and Explosives has some really valuable information for the public that a lot of people search for on ATF.gov. It’s important that the information is easily and quickly accessible. Government agencies reach a wide audience with their information, so making sure everyone can understand your content is important. The DigitalGov User Experience Program performed an expert usability evaluation of ATF.gov in December 2012. The team identified the following three major issues that could quickly be fixed to make the site more usable.
Good APIs can transform intimidating data sets into something that people can use. Good government APIs can create a better connection between government and citizens. The Digital Government Strategy has spurred Federal adoption of APIs, and the Digital Services Innovation Center and DATA.GOV teams are supporting these efforts by releasing a swath of guidance and resources, hosting online education series and launching a number of hosted tools at Labs.Data.gov.
What do kids know about Web design? As we found out, quite a lot. Recently our DigitalGov User Experience Program teamed up with the Kids.gov team to get some big time feedback from some pint-sized testers in a hallway test. We tested with almost 20 kids ages 6 to 14 at our GSA office, made possible by “Take Your Child to Work Day.” We also tweeted some results under the hashtag #kidsgovtest
More and more people use search as their primary means of finding what they are looking for. When users get confused by the search results, or can’t immediately find what they are looking for, they’re going to get frustrated. They may even leave the site for good. The DigitalGov User Experience Program helped test Regulations.gov on October 5, 2012, to find three high–priority, fixable problems that could make the user experience much easier and more pleasant.
When designing a site, remember that your terms and icons are like signposts that show people where your links and pages lead. Make sure that you use words and pictures that are easily understood or people will have trouble using your site. Small changes like underlining links or adding arrows to indicate expandable information can vastly improve the usability of your site. The DigitalGov User Experience Program helped test SaferBus, the U.
When users interact with a website to find information, it is important that we help them find their way by using plain language, clear terminology and visible help text. On December 7, 2012, the DigitalGov User Experience Program helped test the U.S. General Services Administration’s Contract Vehicle Navigator website. This Navigator site helps contracting officers find contracts that best meet their needs. Through usability testing, three key problems were identified.
A website with too much information on the homepage, or any page, will overwhelm users in less than a second. They will be unable to find a starting point to accomplish what they came there to do. If users are not able to locate the information they need and/or are unable to get past the homepage, they will go to another website to look for the information they are seeking.
One of the most vital parts of any website is its starting point. When a visitor arrives on the main page of your site, they should be able to quickly tell what the main tasks are and how to perform them. Visual cues and plain language are the best ways to accomplish this. The SAM.gov site was created to consolidate several acquisition and bidding systems in one central location. It’s a large site, and with so many potential tasks available, it’s important that visitors are able to quickly figure out where they need to go.
Acronyms and jargon are fine when you want to communicate quickly to an internal audiences or to like-minded readers. Once the scope of your audience widens, however, these elements can make your pages harder to understand. The IRS recognized that its pages about tax planning for retirement were reaching an audience beyond tax professionals, and asked the DigitalGov User Experience Program to help test for usability and user experience.
After conducting a usability test and listening to customer feedback, the Weather.gov team and the DigitalGov User Experience Program identified these three issues as both important and quickly solvable. Problem 1: Terminology and Labels Confusing The terminology and labels used were either too technical or too abstract for users to understand—a far cry from the plain language style required in government. On the homepage, users encountered map tabs for “Graphical Forecasts” and “National Maps”.
Many government websites are informational in nature – you don’t sign up for things or buy anything. Instead, you look for something – a name, a ruling, some contact information. Informational sites – and scientific sites in particular – can be a challenge to design. With so much information, how do you make the important content stand out? The National Science Foundation’s NSF.gov site conducted a usability test with some help from the DigitalGov User Experience Program.
Not all usability changes are dramatic. Sometimes a few small tweaks can make a site significantly easier to navigate, or make important but hidden content pop off the page. The DigitalGov User Experience Program helped test Insite, GSA’s intranet, on September 21, 2011. GSA took the feedback from their usability test and made some changes to the existing design. While seemingly small, the changes made a huge difference in the usability of the site for GSA employees.
Websites allow newer government programs to establish a visual identity that introduces them to users and conveys the importance of their work. On April 18, 2012, the DigitalGov User Experience Program helped test GSA’s Federal Risk and Authorization Management Program (FedRAMP) site, which at that point was less than six months old. Three immediate needs were identified. Problem 1: Purpose of Program Not Clear The homepage text was filled with jargon and acronyms, and provided no clear guidance for the user to understand why they should engage with FedRAMP.
Any government product – whether used by millions or a very specific audience group – need to be as easy to use as possible. The Office of Natural Resources Revenue (ONRR) collects and dispenses revenue related to energy production on leased federal and American Indian lands. As a result, their audience has very definite information needs that need to be met quickly. The DigitalGov User Experience Program tested the ONRR site in August 2011.
_ Mobile Gov Experiences are agency stories about creating anytime, anywhere, any device government services and info. This entry is a story shared by AIDS.gov._ _ _ AIDS.gov implemented an innovative model for responsive design by combining the former AIDS.gov and m.aids.gov to a fluid site accessible on computers, smartphones and tablets. View the webinar on AIDs.gov’s responsive design. Why We Did It Testing showed that more and more people were trying to access the website via mobile device but not all mobile devices were receiving the m.