Just to be clear:  
"Disallowing" search via robots.txt does not protect data, it just "asks" search engines not to index the data.  By adding a reference in robots.txt you are actually providing a direct, publicly-accessible link to anyone who may be inclined to seek out pages that webmasters wish to keep out of search results.
 
In DNN you can truly prevent data from being indexed / accessed by using user accounts and setting page permissions.
 
  
Regarding the built-in DNN search engine:  
DNN search returns data from modules that implement iSearchable.  That means that you don't set search permissions by page but rather by module (if at all).  Some modules may not implement iSearchable at all.  Many modules implement iSearchable but do not allow you to turn on/off the feature (i.e., they're always indexed but only with the info the module creator decided to expose via iSearchable).

 
Assuming you protect sensitive pages by requiring a login for those pages, your choices are:

1. Do nothing!  Your DNN search results should only include pages/modules that the current user has permissions to see.  Of course, search results for all other modules will still be limited to whatever each module developer decided to expose via iSearchable.
    
2. Pay close attention and use only modules that allow you to "turn off" search indexing.  (This is probably not practical.)
    
3. Drop use of the internal search engine and switch to an external engine that cannot index protected pages.  One example of swapping to an external search provider in DNN is here: http://googleAJAXsearch.com  
    
4. Drop DNN Search and replace with an internal, third-party search provider that allows you to specify what does and does not get indexed.  A few such providers do exist but I've not used them and cannot say how easy/effective they are.

I most often use option #3 when I need a good DNN search for public-facing web pages (pages that do not require a login).  Note also that you can use option #3 for all public-facing pages and still provide, only on the "private pages", the default DNN search so your "private" users have a means of searching your protected, sensitive info.
  
-mamlin