This short guide will tell you how to configure irssi so that it will keep showing ISO-8859-15 (or whatever you select) charset letters in terminal no matter what the other users on the channel are using. This way you won't need to worry anymore about UTF-8 junk messing with your terminal that uses ISO-8859-15 or the other way around. Here are the needed irssi commands:

/set term_charset ISO-8859-15
/set recode_fallback ISO-8859-15
/set recode_out_default_charset ISO-8859-15
/set recode_transliterate on
/set recode_autodetect_utf8 on
/set recode on
/save

That should do it. There's no need to restart irssi assuming the terminal is really using the same charset as set in the term_charset variable. Further explanations about those commands can be obtained with

/help recode