Nope, you're not alone!  This contact issue has been one of my biggest RE headaches for 8 years now.  There's no easy way about it.  I am, however, trying something new for this year's Ball.  I created an individual relationship attribute called "Contact Type Detail" (for lack of a better name), and it's there that I'm going to tag who my Ball contacts are by using the description "Swan House Ball Solictiations."  Once I become comfortable with how this works, I'll probably also use it for Annual Fund solicitations, Sponsorship mailings, etc.

We took the gift contact attribute idea one step further. We have two gift attributes for the gift contact 1) Gift Contact Addressee and 2) Gift Contact Salutation. We generate our tax acknowledgments through a conditional export and included code to use the Gift Contact attributes if they exist for the addressee and salutation. If not attribute exists, we use the regular logic to address the letter to the primary contact.