I've just had it explained to me how our organisation currently merges duplicate records, and it's long-winded and manual to say the least, but more concerning for me is that by using "Constituent->Merge->Merge .. into" from the file menus, there's no way of automatically determining whether I'm about to overwrite new data with old. I'm convinced there must be a method which allows me to take advantage of the 'last altered' dates a lot of the tables use for their records. Can anybody shed some light?

Furthermore, can I extend this to bulk-merging a large list of constituents? I have identified a shamefully large number of duplicates, largely relating to degree, MSc and PhD constituents being added new rather than updated, and to do by hand would take hundreds of man hours.

Many thanks,
Pete
 

Anybody got a clue on this? It's one of my more labour-intensive tasks if it's not possible to automate, and I don't really fancy dropping hundreds of man hours on it..

Thanks,
Pete 

 I know that Blackbaud sell a duplicate merge plugin that allows you to merge many dupes in one go. I cannot seem to find any information about it on their site though.

 I have done similar types of customisations for various clients and would be happy to discuss what could be done otherwise.

Thanks

David

At my previous employment we purchased the plug-in for a one time fee of around $5,000.  At that time, you could use a query as a base to search for dupes instead of doing the whole db at once.  You could merge records one by one or let the program run and do the merging without verifying each record.  Even when doing the merging on a one by one basis, it's a much faster process.

Hope this info helps.
Helen

hejones@rider.edu

My issue is really one of two parts, and you've both answered the first part (which is "Can it be done automagically"), but I have a lingering concern...

Does said plugin allow for the process to understand that there's a potential (let's assume two records to be merged) for record 1 to have some new data, and record 2 to have some new data. Can this plugin determine which data is newer and merge accordingly? Or will it simply attribute one record as being a master and discard data from the other record?

 Thanks,
 

Peter

I seem to remember (it has been 18 months since I last used) that the merge would not duplicate same attributes and constituency codes. I think all other info will merge over to the new record, giving you duplicate info in the master record. I do recall that addresses are merged to the master record.  Clean up still needs to be done after the merging is complete.

When you are "setting' up the process, you have a few options to determine which record you want to be the master record, such as oldest/newest record, record with the most gifts.  There may be a few other options but I cannot be sure.

 Helen

Sorry Helen, I think perhaps I wasn't fully clear. If Record A has newer Address information, and Record B has newer Education information (just as an example) , then will this plugin know to merge together both pieces of newer information, or would I end up with something along the lines of new address information and old education information (or vice versa)?

 
Perhaps you're able to clear this up for me David? And if it isn't able to then perhaps send me a message outlining how much it might cost to make such a customisation?

Cheers,
Pete
 

 I only saw the merge utility in brief but from what I recall it is not so sophisticated that it will take the latest information from one and the latest from another to merge but rather it takes all information and merges it into one record (as Helen said with the exception of constituency codes and attributes where there will not be duplicates)

 David

If you select record A as the master record, it retains all the primary info..such as address, business and education.

The info from record B is merged in as additional info....additional addresses, relationships ..incuding spouse/businesses/schools.

So, your final "master" record could end up with the spouse in as primary and again as an additonal relationships...same with business/schools...

The merge process does not take into consideration which address has the most recent "from date" or was entered most recently.

By you selecting which record you want to retain as the "master", that tells the system that all that info will remain as preferred/primary.

This is how I remember the plug-in working.  Any chance it has been updated in 18 months?

 Helen

We just recently purchased this, and the answer is no.  You cannot select which pieces get merged into the new record.

 However, perhaps this helps - In the screen where you look at 'groups' of records to merge, you can see some pertinent data (id, address, name, etc), and you can go directly to the individual records from that screen, so you could move information as needed.  But it still works the same way where one record is primary and that's where all the information goes.

 One warning we found.  We were testing the utility on 7.82.1554.0 and we found that there were problems with merging the boolean (checkbox) values on Bio1.  On a number of them, it took the records from the 'merge from' record rather than the 'merge to'.  (This is not a problem with the utility, but how that version managed merges).

As of the latest version of RE, we found that 'requests no email' still does not merge properly.

 Otherwise, the utility works great for us, and has allowed us to get rid of about 1% of the database by identifing different sets of duplicates.

One more word of advice to any groups out there doing this: we had a third party identify dupes for us (they had a sophisticated system that could check Bobs versus Roberts versus Bobbys, etc.)  We got merge-purge numbers for each set of records.  We tried to find places to import that number so that we could make the dedupe utility identify those as the groups.

We had to use birthplace for individuals since we don't use that field.  Organizations we had a heck of a time finding a good field to use - we tried both Target (and added the table entries), as well as No. of Subsidiaries since we do not use.  Both of those fields, while they are on the list of fields to use to idenfity dupes in the Business Rules, do NOT work as of the latest version of RE to merge against.  We ended up importing the list into No. of Subsidiaries, exporting postal codes based on that, imported the merge-purge number into postal code, ran the de-dupe utility (you can run it to identify the dupes without doing the merges and it will keep the list - like a static query), and re-imported the postal codes.

I have asked BB to add a user-defined column on the records table that could be used for anything, that could also be used to identify dupes, so that one could import this type of thing instead of using a field for what it wasn't designed for.

Just a note of caution.