/message/lib.php line 609
link_to_popup_window("/message/history.php?user1=$userid1&user2=$userid2$keywords$position", "message_history_$userid1", $fulllink, 500, 500, $strmessagehistory, 'menubar=0,location=0,status,scrollbars,resizable,width=500,height=500', false);

should be

link_to_popup_window("/message/history.php?user1=$userid1&user2=$userid2$keywords$position", "message_history_$userid1", $fulllink, 500, 500, $strmessagehistory, 'menubar=0,location=0,status,scrollbars,resizable,width=500,height=500', true);

so that the function returns rather than echos.