Ok, seems like I found something at least:
In fckconfig.js are the following 2 lines:
FCKConfig.SmileyPath = FCKConfig.BasePath + 'images/smiley/msn/' ;
FCKConfig.SmileyImages = ['regular_smile.gif', 'sad_smile.gif', 'wink_smile.gif', 'teeth_smile.gif', 'confused_smile.gif', 'tounge_smile.gif', 'embaressed_smile.gif', 'omg_smile.gif', 'whatchutalkingabout_smile.gif', 'angry_smile.gif', 'angel_smile.gif', 'shades_smile.gif', 'devil_smile.gif', 'cry_smile.gif', 'lightbulb.gif', 'thumbs_down.gif', 'thumbs_up.gif', 'heart.gif', 'broken_heart.gif', 'kiss.gif', 'envelope.gif'] ;
Changing those doesn't yet give the desired result (namely an extra smiley), but it seems like the way to go. Ideas still very much appreciated.