February 2009 - Posts
These are the words that some Blackbaud DBAs first hear when starting their position. You've never heard of Blackbaud, don't know what a "BMC" is, and haven't worked with a database program like this before. "Where do I begin" is the usual first question that gets asked. I'm going to help you answer that question and a few more. We will go over the basic things you need to know and where to find answers to questions you didn't know you had. This will be broken into several segments, as there is a lot to go over.
First off, welcome to Blackbaud. Your job is going to be easier because of the way Blackbaud built the products it sells to users. Second, we need to define some terms. These terms will be used by Blackbaud representatives and staff, so it will make it easier to communicate if we have these terms in common.
BMC or Blackbaud Management Console - The SQL Server User Interface created by Blackbaud where we can attach databases, create backups, and run maintenance on the databases. The Blackbaud Management Console is located on the application server.
Server (Application Server) - This is normally a dedicated machine (computer) that is more powerful that the standard machine used by an employee. When Blackbaud representatives speak about the application server they mean the machine where the Blackbaud Management Console is installed.
Workstation - This is a machine that has a Blackbaud product on it, but does not have the Blackbaud Management Console. This machine accesses a database that is stored on the server. Workstations are normally desktop computers where users do day to day work.
SQL or SQL Server - SQL Server is a Microsoft product used for database management. All current Blackbaud products have a SQL Server version that runs the database. This is the most common database management program for Blackbaud products. It can either be pronounced "S Q L" like the letters, or "sequel" like a movie sequel. Both pronunciations are correct, and most reprentatives familiar with SQL will understand it either way.
Oracle - This is another database management program. Oracle is less commonly used by Blackbaud products, but is still used for some Raiser's Edge databases.
Login - This one seems simple, but is one of the most confusing without context. There are typically four Logins that are used with Blackbaud products.
- Windows Login - this is a user's login to Windows and the network - this is done before any Blackbaud products are even touched.
- <Product> Login - this is the login and password to gain access to a Blackbaud product - like Raiser's Edge or Financial Edge.
- Blackbaud Website Login - this is the login a user has to access the Blackbaud website - it is typically not the same as a Product login - it grants access to Knowledgebase, Downloads, Case Central, and FTP.
- SQL Login - this one is rarely mentioned, but is also important. This is a SQL account set up to grant users' rights to the SQL Server databases. It is most commonly associated with full versions of SQL running in Windows Only authentication - and only users familiar with SQL are going to make any sense of that last line
DBA or Database Administrator - This is you, the person in charge of maintaining the database and securing backups of the databases. Some DBAs do not actually work in the program and do not have a login for it. I recommend getting a login for the database to test with in case you ever have problems with users logging in and need to have an account to test with.
Admin or Local Admin - This is a Windows defined role that means this Windows User has full control over everything on this computer. If a user is unable to perform a function, like running a report or logging in, Blackbaud will often ask if the user is a Local Admin or have you tested the same issue as a Local Admin to determine if this is a permissions issue. Admin is not a Blackbaud account in a database - but a Windows role.
Domain Admin - This is a Windows defined role that has access to all machines in your Network. This account has Local Admin rights to all machines. Domain Admin is not a Blackbaud account - it is a Windows account on your Network.
Supervisor - A Blackbaud "Super User" account and the default account that comes with all Blackbaud databases. The Supervisor account has full control over the database and can add and remove new users in the database. Typically not a Windows account, but an account used for Blackbaud products.
That completes our list of typical terms you will hear when working with Blackbaud. Next time we will go over how to find your SQL Server Instance, how to find out what Blackbaud products are installed, and how to tell the versions of each product.
This past week support received a call in which one of our client’s servers shut down unexpectedly. Shortly after this shutdown users started to receive error messages in the application and couldn’t work anymore. When they went to restore their database through their backup application, they learned that the backup program wasn’t actually backing up their SQL Server database because it was locked by the SQL Server database engine (As long as SQL Server is running it will lock the files so that they cannot be moved or copied). This is not an uncommon scenario, we have received numerous calls from clients in which they are using a third party application, such as Symantec’s Backup Exec, and their database is not being backed up. Sometimes the client catches this early and corrects the issue before the need to restore. In cases such as this one, the client doesn’t catch it until the need to restore their database.
In the above scenario, I see two main issues:
- The client is not using Blackbaud’s supported backup method
- More importantly, the client was not testing their restore procedures.
As I mentioned, this isn’t the first time this has occurred and I doubt it will be the last. In an effort to help alleviate the frequency of lost databases, I will be posting a series on Backups and Blackbaud Products. In this post, we will discuss our basic backup procedures. In later posts, we will discuss more advanced backup solutions.
In our scenario above, the client was not using a supported backup method. By “supported” I mean, the client was not using the Blackbaud Management Console (BMC) to create their backups. There are alternative backup solutions that will work with our products, such as SQL Server Transaction Log backups or Symantec's Backup Exec. However, the BMC is the only supported method of making backups. The one caveat to this is our higher level products such as NetCommunity or products on the Infinity Platform. We would certainly do our best to help restore a database through a third party application if need be (we never want to see any one lose their data) but we may not have the experience needed with the third party application to help restore properly. This is why we highly recommend using the BMC for backups, especially at organizations that do not have their own IT departments.
Now let’s discuss actually backing up the database. The BMC offers the ability to manually create backups or to schedule backups. A manual backup would be run before you plan on making a big change to the database, such as updating to a new version, running a Global Delete or bulk importing data. Scheduled backups should be run every night through the BMC to protect your data against the unexpected. This can include, your server shutting down unexpectedly (which can cause database corruption), hard drive failures, natural disasters or even theft. These unexpected events can severely impact your organization's productivity and bottom line if you not able to recover from them in a timely fashion.
When scheduling backups in the BMC the backups must be created on a local hard drive. If you were to experience a server crash or a hard drive failure a local backup is not going to help you restore your database since it may be deleted or corrupted during the failure. For this reason, we recommend that you store your backups on an external media such as a tape drive or a network storage solution. To move the files externally you will need to either move the files manually or choose a third party Backup solution. When using a third party backup solution you would set the application to create a backup of the folder where the BMC backup files are created. Make sure the scheduled time for the third party backup provides sufficient time for the BMC to finish backing up all databases. For example, if you have the BMC scheduled to backup at 10:00 PM, start the third party backup at 11:00 PM. For most databases 30 minutes to 1 hour should be sufficient but this can vary depending on the size of your database. Using this method you will successfully backup your Blackbaud database to an external device without running into issues of locked files, as in the scenario above.
Once you have moved these files to the external media the next step to having a complete backup policy is to move the database files to a secure offsite location as often as possible. In the case of a natural disaster such as a flood or fire, it is possible for your external devices to be destroyed. Having an offsite backup reduces the risk of losing your entire database in a case such as this. Since natural disasters are much less common than a hard drive crash or an operating system failure, you don’t need to move the files offsite every day. How often you move these offsite depends on your business needs. Ask yourself, "How much work are you willing to lose in such a disaster?" For example, if the cost of moving your backups offsite daily exceeds the cost of reentering a week of data then you should be okay moving a weekly copy of your backup offsite. Once you have moved your database offsite you almost have a complete backup policy.
Testing your backups regularly is just as important has creating your backups (What good is a backup if you cannot restore it?). This will help accomplish two things. First, it will verify that your backups are being created successfully. Second, it will verify that you know how to restore your backups in a timely fashion. In my next post we will discuss the importance of testing backups and different methods to setup a testing environment.
In the meantime, if you are interested in learning more about backups here is some recommended reading:
Blackbaud Backup Procedures
Microsoft Backup Overview
Importance of Database Backups and Recovery Plan
Hello and welcome to Inside the Edge, the blog geared to IT professionals and Database Administrators (DBAs) that work with Blackbaud Products. My name is David and I will be one of the contributing authors to this blog. There will also be other contributors from our Advanced Support Teams, who work with you everyday to help you troubleshoot complex installation or technical issues. In this blog, we will cover a wide range of topics to help you better you understand the technical workings of Blackbaud Products. This will help you to efficiently troubleshoot and manage your Blackbaud systems.
A few of the items we plan to discuss are:
- Troubleshooting techniques and Utilities
- Tips and Tricks for the Blackbaud DBA
- Installation Tips and Tricks
- Interesting Tech News
- Tips for the "Accidental DBA" (we will discuss this term in a later post)
- Virtual Machines
This is not a comprehensive list by any means, if there is anything you would like to see feel free to contact me or post it in the comments.
To get started I would like to point you to one of my favorite sites for troubleshooting utilities, SysInternals. SysInternals is a site created by Mark Russinovich that initially was independent of Microsoft. However, Mark knew as much or more about the Microsoft Operating System than did Microsoft. So they decided to bring him on board and bought his web site. Mark has a great blog that discusses his techniques for troubleshooting various issues in Windows. All the utilities from SysInternals are free to use and very helpful.
If I could suggest one utility for you to familiarize yourself with, it would be Process Monitor. It is an easy to use utility that will display what resources are being accessed during any process. We will go over how to use it in a later post.
I would also like to encourage you to leave comments so that we can make this blog as interactive as possible. I want you to become as much a part of this blog as me and my fellow contributors.
With that I welcome you again to our blog! I hope that you enjoy the posting and learn a little something new.