Mike Hall wrote about friends, a personal CRM for the command line. It’s a ruby gem that does most of the things I was trying to do using Emacs.
It seems worth a try, so I installed the gem and created a few shell aliases to make things faster:
alias faa 'friends add activity '
alias faf 'friends add friend '
alias fla 'friends list activities'