Documentation Survey 3rd Quarter Results

Lindsey Robbins — Mon, 27 Oct 2008 14:37:00 GMT

On October 9th, Justin announced our third quarter documentation survey winner. I thought it might be nice to discuss some of our results from the survey with our blog readers.

A link to our survey appears on the Blackbaud User Guides pages (accessed from the Support menu), this blog, EE/FE/SIS Help files (a test run), and in the support newsletters once each quarter. We received 73 responses in Q3 2008.

Overall Key Findings:

For our five overall rating categories – Accuracy, Completeness, Easy to Understand, Accessibility, and Usefulness – the most frequent rating was a 4 or 5 (out of 5).
Our best overall rating category was Accuracy. Out of 73 responses, 62 rated us 4 or 5.
72.6% of survey respondents were able to find the answer to their question in the documentation the last time they accessed it.

Question: Please give an overall rating for the following aspects of documentation (user guides and help files only) on a scale of 1 (least favorable) to 5 (most favorable)?

Discussion:
While we are pleased our numbers are mostly 4’s and 5’s we are actively looking for opportunities to improve and we even received some great suggestions from the open ended responses on the survey.

Question: Which Blackbaud product were you using when you accessed the documentation (user guides and help files)?

Product	Responses	Percentage
The Raiser's Edge	51	69.9%
The Financial Edge	11	15.1%
The Education Edge	4	5.5%
BSIS	1	1.4%
BBNC	5	6.8%
The Patron Edge	1	1.4%
The Information Edge	0	0.0%
BBDM	0	0.0%
ResearchPoint	0	0.0%

Discussion: While we are always pleased with the number of responses for The Raiser’s Edge, we are looking for ways to improve the amount of feedback we receive for our other products. While we know we don’t have as many users on The Patron Edge as The Raiser's Edge, we value each user’s feedback equally.

Question: The last time you had a question, did you find the answer in the documentation (user guides and help files)?

Discussion: We are happy to have so many “yes” answers to this question. We hope it means you find your answer quickly and get back to completing your tasks. However, we also see that 27.4% of you did not find your answer. This question is one of our critical ones. We’d love for to you always find your answer. Our numbers have been improving in this category but there is always more room for improvement.

Here are some sample user comments and suggestions pulled from the survey (only initials to protect the privacy of our clients):

User guides are enormous. Small, more streamlined versions, maybe for more discrete tasks would be helpful. I frequently find myself selectively printing only the relevant sections of a guide.
TH

The user guides are easy to understand and are helpful. I like being able to find an answer without having to call customer support.
JH

Love the support offered by Blackbaud, especially easy access to the User Guides - don't have to buy a hard copy, and don't have to deal with free documentation that is purposely incomplete in effort to get me to buy the manual! Good job Blackbaud. Always happy to recommend your company.
AB

Your User Guides are very extensive and frequently help me answer my questions. However,
sometimes when searching I get results that are not dealing with the program I specified.
JF

I especially like the guides when I want an initial overview of the module. They are well organized and explain the technical aspects very clearly.
MT

Discussion: I think the open ended comments are my favorite part of the survey. It's not easy to quantify the information but hearing exactly what you think is critical. And, we take what you say to heart. If you allow us too, we try to follow up with as many people as possible especially when they offer up suggestions, areas for improvements, or mention problems. For example, to the user who mentioned the guides are enormous we would try to see if it would have been better for the user to search for the answer in the help file versus the user guides. Sometimes it can be a matter of choosing the right resource in the situation. And, we do realize that we are unable to help every user every time because there can be situations when contacting the help desk is the right decision.

Thanks again to our Q3 survey respondents! And remember, if you take our survey, every quarter we enter the respondents into a drawing for a $25 Amazon.com gift card.

Documentation’s Online Survey

Justin Ward — Tue, 13 May 2008 18:36:00 GMT

Last year, the documentation team created an online survey to capture client feedback about our help content. We’ve done surveys in the past, and while we typically came away with valuable information, we always wished we’d gotten more responses. So this time around, we tried something a little different. Instead of the survey being a one-hit-wonder, we decided to leave the survey up indefinitely -- or as long as our users keep responding.

So far, this approach has worked well. The survey went live last year, and to date, we’ve received over 300 responses. Now, when we evaluate the data, we have more information to base our decisions on, and we can compare response trends from quarter to quarter.

What do we do with the information?
The whole purpose of this project is to improve our help content, both in the help files and user guides. We review survey responses on a weekly basis. When we need more clarification about something a user said, or when users have problems with our documentation, we contact them (just the users who give us permission). All this feedback is then compiled and shared with the team. When we analyze the data, we look for trends in responses so that we can focus our discussions.

For example, we recently focused our improvement efforts on five specific Raiser’s Edge user guides. By reviewing the survey data from the past year, we knew where the problem areas were in these guides. I don’t think we could have honed in on these guides and made meaningful changes without having such a high volume of client feedback over an extended period of time.

We've restructured and reorganized guides for other products as well, improved indexes, and added additional cross references to make our content easier to use. We found that in many cases, we had the information users needed, but they couldn't find it!

Right now, we are averaging 50 to 75 responses per quarter, and I hope this trend continues. If you haven’t taken our survey and would like to, click here. And don’t forget, each quarter we randomly select one of our survey respondents for a $25 Amazon gift card.

From the Doc Side : online survey

Documentation Survey 3rd Quarter Results

Documentation’s Online Survey