Robots txt - General Discussion

New Community Website

Ordinarily, you'd be at the right spot, but we've recently launched a brand new community website... For the community, by the community.

Yay... Take Me to the Community!

Welcome to the DNN Community Forums, your preferred source of online community support for all things related to DNN.
In order to participate you must be a registered DNNizen

Home All Forums

1
2
3
4
5

7/9/2008 6:32 AM

Albertramsbottom

www.classcampers.com
Joined: 12/8/2006

Posts: 506

Robots txt

Can anyone point me to a robots text file that has all or most of the necessary DNN exclusions.

And also, hows does one exclude different diectories from different portals. My current robots.txt file looks like this:

User-agent: *
Disallow: /SearchResults/tabid/37/Default.aspx
Disallow: /SearchResults/tabid/69/Default.aspx
Disallow: /XmodTest/tabid/312/Default.aspx
Disallow: /xmodtest2/tabid/313/Default.aspx
Disallow: /SearchResults/tabid/37/Default.aspx
Disallow: /SearchResults/tabid/37/Default.aspx
Disallow: /SearchResults/tabid/37/Default.aspx

Disallow: /Admin/
Disallow: /logfiles/

Sitemap: http://www.MyMainDomain.com/MyMainDomainSiteMap.xml
Sitemap: http://www.MyMainDomain.com/MyOtherParentPortal1Sitemap.xml
Sitemap: http://www.MyMainDomain.com/MyOtherParentPortal2SiteMap.xml
Sitemap: http://www.MyMainDomain.com/MyOtherParentPortal3SiteMap.xml
Sitemap: http://www.MyMainDomain.com/MyOtherParentPortal4SiteMap.xml

I have 5 parent portals running

Thanks for your help

Class Campers -The Home of the Volkswagen camper & Bus Enthusiast
Worlds Cheapest Homes - Need a Cheap Property, cant afford a home
Beeb iPlayer - Need help with the BBC's new iPlayer software
Internet Insurance Quotes - For the the information you will ever require about Insurance

7/10/2008 4:29 AM

Albertramsbottom

www.classcampers.com
Joined: 12/8/2006

Posts: 506

Re: Robots txt

Well I have just found out that various controls can create duplicate content as far as Google is concerned.

http://www.bestwebsites.co.nz/dotnetnuke/modules/robots.txt-generator-module

Now has nayone else had any of these problems - Any comments

Cheers

Class Campers -The Home of the Volkswagen camper & Bus Enthusiast
Worlds Cheapest Homes - Need a Cheap Property, cant afford a home
Beeb iPlayer - Need help with the BBC's new iPlayer software
Internet Insurance Quotes - For the the information you will ever require about Insurance

Page 1 of 1

These Forums are dedicated to discussion of DNN Platform and Evoq Solutions.

For the benefit of the community and to protect the integrity of the ecosystem, please observe the following posting guidelines:

No Advertising. This includes promotion of commercial and non-commercial products or services which are not directly related to DNN.
No vendor trolling / poaching. If someone posts about a vendor issue, allow the vendor or other customers to respond. Any post that looks like trolling / poaching will be removed.
Discussion or promotion of DNN Platform product releases under a different brand name are strictly prohibited.
No Flaming or Trolling.
No Profanity, Racism, or Prejudice.
Site Moderators have the final word on approving / removing a thread or post or comment.
English language posting only, please.

Products

Solutions

Resources

Partners

Community

Blog

About

QA

Ideas Test

New Community Website

DNN Corp. (DotNetNuke)