I've reported a similiar problem in the following thread (sorry about the cross post):

http://www.dotnetnuke.com/Community/Forums/tabid/795/forumid/108/threadid/145050/threadpage/1/scope/posts/Default.aspx

My problem also seems to be occuring during the retrieval of the navigation nodes but is unpredictable. I have not been able to reproduce this problem in my dev environment which suggests the problem is load based. I will post in my thread if a solution emerges.

We discovered that our problem was related to a custom module implementation of ISearchable. Since this code was only firing when the search scheduler was triggered it was difficult to identify.

Some points that we discovered in the process of debugging:

• Ensure your worker processes (configured under IIS) has sufficient memory allocated. Under normal functioning DNN will consume upto 300MB of memory (this is based on our DNN implementation comprising 3 portals and about 500 pages). If the worker process hits its memory limit you are likely to see the error message you provided.
• Do NOT configure your IIS processes to run in web gardens. DNN does not support web gardens and you will experience eratic caching behaviour if you do.
• Do NOT disable cahcing. DNNs caching model significantly reduces calls to the database. If caching is disabled more resources will be consumed in the short term. However, in our experience the garbage collector catches up quickly.
• If you are utilising a webfarm do NOT use file based provider (use the the db provider). In our experience the file based provider causes contention problems and is slower.

Hope this helps