Hey all,

Just starting to mess with a Robots.txt file.  I am using Human friendly urls, but still want to block the indexing of the terms and log in pages.  How does this look to the more experienced eyes??

 User-agent: *

Disallow: /admin/

Disallow: /App_Browsers/

Disallow: /App_Code/

Disallow: /App_Data/

Disallow: /App_GlobalResources/

Disallow: /aspnet_client/

Disallow: /bin/

Disallow: /Components/

Disallow: /controls/

Disallow: /DesktopModules/

Disallow: /Documentation/

Disallow: /house/

Disallow: /Install/

Disallow: /js/

Disallow: /Providers/

Disallow: /Resources/

Disallow: /login.aspx*/*

Disallow: /Register.aspx*/*

Disallow: /Terms.aspx*/*

Disallow: /Privacy.aspx*/*

Disallow: /SendPassword/*

Crawl-delay: 5

Very nice post... i do have a question though... about your GoogleSiteMap.axd file.

How did you generate your GoogleSiteMap.axd?  I'm using SiteMap.aspx on a friend's website. Is that basically the same with just the extension different? Just curious and sorry if that question seemed juvinile... just trying to get a better understanding of linking to sitemap via Robots.txt

Rodney,

Thanks for the help.  Nice catch :-)

Doug