From the Doc Side : usability

Documentation Survey Results and Responses

Justin Ward — Wed, 29 Apr 2009 14:44:00 GMT

The 2009 Q1 results for the documentation survey have been compiled. Here is a quick summary of our key findings for Q1:

We received 153 responses in Q1 2009. That, my friends, is a new record. And with all those responses comes a ton of great feedback (more on that in a minute).
For our five overall rating categories - Accuracy, Completeness, Easy to Understand, Accessibility, and Usefulness - the most frequent rating was a 4 or 5 (out of 5).
Our highest overall rating category was Accuracy. (Mean = 4.26)
Our lowest overall rating category was Easy to Understand. (Mean = 3.82)
72.4% of survey respondents were able to find the answer to their question in the documentation the last time they accessed it.
The last time our respondents had a question, we were able to help 62.5% resolve the issue on their own with documentation instead of them having to contact Customer Support.

I like compiling the statistical data and comparing trends from quarter to quarter. Those metrics are important and help us understand our weaknesses and strengths. But my favorite thing about reviewing survey results is reading all the user feedback. You can really get a feel for the struggles and pains users experience when they can't complete tasks. For example, when a user says he can’t run an import because the guide doesn't explain which fields are required, that's great information. We can work with that. It requires no trend analysis. And when we call the user, even those who sound upset or angry in the survey, they are typically appreciative that we responded, and usually have even more input for us. It's a great example of how this survey helps us improve our documentation.

Likewise, you can sense the relief when a user is able to find information he needs and perform his job better. Knowing that the documentation is helping users and doing what it's supposed to be doing is great to hear. It validates a lot of hard work and effort.

Here are some of the responses collected this quarter (names withheld to protect privacy):

"PLEASE expand the documentation for Dashboards! There is not enough information about functionality with Share Dashboards and other descriptions are not detailed enough. I've had to do a lot of trial and error and testing on my own."

"Create a guide to explain Actions and Action Track creation."

"I would recommend that when the user guides are updated that an email be sent out to inform us."

“I really appreciate having the user guides. I go there first. Yes, I am pleased, too, with the Knowledgebase, and the telephone support. However, I will almost always first go to the user guides, because I want to know as much as possible about the topic with which I'm dealing, and the user guides point out features I may either not understand, or features that our organization is not yet using. So, typically do I not only find an answer to my question, but I learn something else I can apply.”

That’s just a sample of the responses we got last quarter. I look forward to reading the survey responses every quarter because I know I’ll learn different things about our users and how they use our documentation. So if you've ever wanted to know if users really do read our documentation, the answer is a resounding yes. We hear from them every day.

To be entered in our 2009 second quarter drawing, complete our online survey. It takes only a few minutes. To send us feedback about our user guides or help file documentation at any time, email documentationcomments@blackbaud.com.

What Do Users Need?

Lindsey Robbins — Thu, 06 Nov 2008 19:09:00 GMT

A simple question. What do users need? My mind starts thinking about it and I realize there are just too many answers to mentally fathom. Recently, someone on my team shared a great link to discussion results from a recent CIDM conference.

http://www.infomanagementcenter.com/enewsletter/200811/third.htm

I thought the "What we learned" section contained a lot of great answers. Seems thorough but of course it can never be a complete list. Users' needs are ever evolving and ever changing.

However, one particular statement I've thought about before came up in the discussion.

Users need answers to their questions.

Obvious - Yes. Easy to execute - No. How can you ever provide all the answers to each individual's questions? The answer is you probably can't but you try to answer as many as you can and provide ways to answer the rest by other means.

In Documentation, we recognize that while we aim to help every single user, there are times where your particular situation calls for Knowledgebase or Customer Support. However, that doesn't stop us from trying. Your questions always provide us opportunities to improve our Help.

Here are the main lessons learned from the article...

Answer the users' questions and help them do their job.
Make it easy for users to find the answer.
Provide users access or links to related information.
Give users an easy forum for reporting issues and collaborating.
Give users information in a format that suits their work environment.
Technology is changing user expectations.

What do you think, does this sound accurate?

Documentation Survey 3rd Quarter Results

Lindsey Robbins — Mon, 27 Oct 2008 14:37:00 GMT

On October 9th, Justin announced our third quarter documentation survey winner. I thought it might be nice to discuss some of our results from the survey with our blog readers.

A link to our survey appears on the Blackbaud User Guides pages (accessed from the Support menu), this blog, EE/FE/SIS Help files (a test run), and in the support newsletters once each quarter. We received 73 responses in Q3 2008.

Overall Key Findings:

For our five overall rating categories – Accuracy, Completeness, Easy to Understand, Accessibility, and Usefulness – the most frequent rating was a 4 or 5 (out of 5).
Our best overall rating category was Accuracy. Out of 73 responses, 62 rated us 4 or 5.
72.6% of survey respondents were able to find the answer to their question in the documentation the last time they accessed it.

Question: Please give an overall rating for the following aspects of documentation (user guides and help files only) on a scale of 1 (least favorable) to 5 (most favorable)?

Discussion:
While we are pleased our numbers are mostly 4’s and 5’s we are actively looking for opportunities to improve and we even received some great suggestions from the open ended responses on the survey.

Question: Which Blackbaud product were you using when you accessed the documentation (user guides and help files)?

Product	Responses	Percentage
The Raiser's Edge	51	69.9%
The Financial Edge	11	15.1%
The Education Edge	4	5.5%
BSIS	1	1.4%
BBNC	5	6.8%
The Patron Edge	1	1.4%
The Information Edge	0	0.0%
BBDM	0	0.0%
ResearchPoint	0	0.0%

Discussion: While we are always pleased with the number of responses for The Raiser’s Edge, we are looking for ways to improve the amount of feedback we receive for our other products. While we know we don’t have as many users on The Patron Edge as The Raiser's Edge, we value each user’s feedback equally.

Question: The last time you had a question, did you find the answer in the documentation (user guides and help files)?

Discussion: We are happy to have so many “yes” answers to this question. We hope it means you find your answer quickly and get back to completing your tasks. However, we also see that 27.4% of you did not find your answer. This question is one of our critical ones. We’d love for to you always find your answer. Our numbers have been improving in this category but there is always more room for improvement.

Here are some sample user comments and suggestions pulled from the survey (only initials to protect the privacy of our clients):

User guides are enormous. Small, more streamlined versions, maybe for more discrete tasks would be helpful. I frequently find myself selectively printing only the relevant sections of a guide.
TH

The user guides are easy to understand and are helpful. I like being able to find an answer without having to call customer support.
JH

Love the support offered by Blackbaud, especially easy access to the User Guides - don't have to buy a hard copy, and don't have to deal with free documentation that is purposely incomplete in effort to get me to buy the manual! Good job Blackbaud. Always happy to recommend your company.
AB

Your User Guides are very extensive and frequently help me answer my questions. However,
sometimes when searching I get results that are not dealing with the program I specified.
JF

I especially like the guides when I want an initial overview of the module. They are well organized and explain the technical aspects very clearly.
MT

Discussion: I think the open ended comments are my favorite part of the survey. It's not easy to quantify the information but hearing exactly what you think is critical. And, we take what you say to heart. If you allow us too, we try to follow up with as many people as possible especially when they offer up suggestions, areas for improvements, or mention problems. For example, to the user who mentioned the guides are enormous we would try to see if it would have been better for the user to search for the answer in the help file versus the user guides. Sometimes it can be a matter of choosing the right resource in the situation. And, we do realize that we are unable to help every user every time because there can be situations when contacting the help desk is the right decision.

Thanks again to our Q3 survey respondents! And remember, if you take our survey, every quarter we enter the respondents into a drawing for a $25 Amazon.com gift card.

Trends in User Research Review

Lindsey Robbins — Mon, 04 Aug 2008 14:59:00 GMT

Conference: STC Summit 2008
Session: Trends in User Research
Panel: Jean Anderson, Todd Zaki Warfel, Ginny Redish, Whitney Quesenbery, Karen Bachmann (moderator)

I heard four main questions during the Q&A panel session:

1. How do I show value?
Answer – Back up your desire to do research with examples of past success. Fresh in the field with no past successes (yet), find examples of success from your colleagues in the field.

2. People in company use usability terms but don't know what they mean.
Answer – Don’t use your usability terms. Instead, use your process terms. Explain what you will be doing. For example, explain what an ethnography is and what you hope to gain by doing it. Then you won’t be discussing terms and wondering if the people in your company really understand what you will be doing.

3. How long do your usability projects take?
Answer – Doesn’t necessarily need to take a long time. Panel responded that they are able to get to what client needs and complete research very quickly. In order to do that, they go quickly to the questions:

Why?
What Do They Hope to Change?
What are the issues?
What are customer support, professional services, etc hearing?

4. What's New or BIG in the field?
Answer – Agile (scrum teams), Prototypes, Ethnographic based methods (but not strict), One-on-One interviews (because surveys tell you “what” but one-on-one tell you “why”), be flexible & combine techniques, use user forums, and remote usability tools.

Impressions of the Session:

Go and do usability testing, don’t worry about the details.
The value of any Q&A depends on the questions being asked. I can tell people are still relatively new to usability based on the first half of the questions in the session.
Don’t waste time. Go straight to the “why?” and then figure out what you need to do user research on.
One of the great values of listening to a panel is hearing some of their success stories. It’s then not as much about takeaways but rather hearing how other user research is successful. It’s encouraging.
It’s the same for user research as it is for fundraising. Just ask. Ask people if they want to participate in your research. Don’t make it more complicated.
Formal is not necessary. I think I could have a whole discussion on whether user research needs to be formal but I like the idea that you uncover the problem, figure out the best combination of user research required, and then go complete it. There isn’t any wasted time or dragging feet.

Also, they shared a great resource they found helpful on slideshare.net. It’s a presentation on How to Be a UX Team of One by Leah Buley at the 2008 IA Summit in Miami, Florida.
http://www.slideshare.net/ugleah/how-to-be-a-ux-team-of-one