Please use the forums on asp.net/forums to post general dotnetnuke questions/problems.  There are more specific per module and probably don't get as good of a response.

This is a host settings item which is cached.  I am surprised that clicking the Update button does not clear cache.  This also depends on how the module uses the SMTP settings.  The Forum module, for example, has its from address as a module setting.