I was surprised to see the Register page for one of my sites rank fairly highly for "outfits for weddings":

Outfits for weddings, parties, occasions or the office; trousers ...

I don't want Register (or Login) pages to get indexed by search engines - is there a way to instruct them to skip over them?

I wrote a big article about this problem a short time ago - Removing duplicated content in DotNetNuke ..

The solution is to manually add 4 lines to your robots.txt file for every single page in your website (because robots.txt doesnt support wildcards, you have to list them all explictly).

Since this is a very time consuming process for any site with more than a few pages, I have also created and released the SEO Robots.txt generator module for DotNetNuke (4.x.x and above) - this module automatically generates the contents of your robots.txt file, it's a great time and effort saver.

JK.

Jeff - everything I have read about robots.txt says that wildcards are not recognised. How do you know that "Disallow: /*/ctl/" is working?

JK - I am interested in your module to stop the register page getting indexed, but I am confused about the other benefit you say it offers - stopping getting penalised by Google for duplicate content. In the copy of your article at http://www.bestwebsites.co.nz/webcontent/bws/remove-duplicated-dotnetnuke-content.aspx you say duplicate content within a site "... probably wont get penalized". However in the copy of the article at http://www.codeproject.com/useritems/DotNetNuke_SEO_pt2.asp you leave that out implying that maybe it is penalized. Which is it?