Performance can be affected by many factors. The best thing to do is eliminate as many bottlenecks as possible:

Skins: Lean code, small file sizes and fewer http requests.

Ping: Ping your site every couple minutes to keep it alive. This will reduce the lagtime the first time someone visits. There are several utilities for this, but I use SmarterPing

Performance Settings: See Mitch Sellers blog on best practices

DB Cleanup: I have a regular database cleaning routine: http://mndnn.com/Blog/tabid/63/ID/18/...

Other than that, you have to look at the hosting provider. On a shared server at GoDaddy, you could be one of thousands of sites on that server, all competing for the same resources. You do get what you pay for in hosting.

It has nothing to do with DNN actually. It has to do with the hosting server and the setup and tuning.

To be honest about it, no NET cms is exactly a speed daemon. For many reasons related to .. Microsoft.

One problem you could have is that very low traffic websites go to sleep after a while. And the next visitor get a delay, sometimes even 20-30 seconds in various functions. This can be disabled in IIS/pool.

Other you could try is to change the caching to FILE if you memory pool is low (do you have a dedicated pool) ? If you do, try Memory and max size of cache. And see how it goes.