More information on this - we have been experiencing this problem since the date of our original post. Since then we have moved to a new server - Win2008/IIS7/16GB memory/Quad Core and the problem actually has become worse.
Here is the situation - on the old server (2 cpus), certain DNN sites would "go rogue". If only one site had this condition it would take 50% +/- 2% of cpu for several hours. If more than one site had the condition the cpu would peg at 100%.
On the new server if one site has the "condition" it takes 25% of the cpu, each additional DNN site that "goes rogue" takes an additional 25% of the cpu until the server is pegged at 100%.
I guess it makes sense that on a dual core machine a single rogue instance of DNN takes up 50% of cpu and on a quad core machine a single rogue instance takes up 25%.
Killing the W3WP process eliminates the problem, but eventually the sites chime back in and start up again on their quest to kill the server performance.
We host many DNN sites for many customers - so we are running into this situation every day - each site uses its own app pool. Since moving to IIS7, with just a few exceptions, the server continues to respond to all web requests on all web sites, however performance is very poor - and does not meet our SLA to our customers.
Many of the sites that "go rogue" are very low use - and we have verified that during these periods of high cpu activity there is no http activity going on - nobody and nothing is accessing the site. We have verified this through IIS web logs and also using third party software like IIS trace.
We have used SQL Server Profiler to check SQL activity and find nothing out of the ordinary. There are scheduled tasks running for these rogue sites to purge logs etc, but these same tasks are running for other DNN sites that are not consuming CPU. We see that the tasks for the rogue and non-rogue sites take just milliseconds to complete.
We have taken dumps of the sites that are consuming cpu, but they provide no useful information because the site is actually doing something - it is not hung, so taking a dump is just a crap shoot.
We attempted to use Microsoft Visual Studio 2010 remote debugging to determine what was being executed, but that failed as well because we are connecting to the server on another domain and we are unable to get remote debugging to load symbols when connecting accross domains.
So we are stuck wonder - What is DNN doing during these high CPU utilization time periods?
Until we can find an answer - have have modifed the app pools of each of our DNN sites to Kill the W3WP process if the site is consuming 23% or more of CPU for more than 5 minutes. That is our only recourse.
We would greatly appreciate it If someone on the DNN team could offer some insight, and also tell us if our strategy of configuring the app pool to kill itself is going to cause problems (besides an error response to a user who might be accessing the site).
Of course we would like to address this in a different way than we have - but we need information from the DNN team as to what is going on and how we can control it / and or deal with it effectively.