Products

Solutions

Resources

Partners

Community

Blog

About

QA

Ideas Test

New Community Website

Ordinarily, you'd be at the right spot, but we've recently launched a brand new community website... For the community, by the community.

Yay... Take Me to the Community!

Welcome to the DNN Community Forums, your preferred source of online community support for all things related to DNN.
In order to participate you must be a registered DNNizen

HomeHomeUsing DNN Platf...Using DNN Platf...Administration ...Administration ...Googlebot cannot access CSS and JS filesGooglebot cannot access CSS and JS files
Previous
 
Next
New Post
8/3/2015 11:00 AM
 

Hello DNN Community!

We develop and host quite a number of websites for our clients, and part of our services include SEO. We recently got a big blast for the following message on almost all of our DNN sites: 

Googlebot cannot access CSS and JS files on www.domain.com

Has anyone else seen this? Any ideas on how to resolve this issue within the DNN environment? 

 
New Post
8/12/2015 3:31 AM
 
Same problem! All of your sites using this Robots.txt setting are showing this problem.
I think the problem occurs because we excluded Disallow: /Components
Strange enough if we use the "Fetch as Google" option in google webmaster tools. The only thing he will say is this:

http://XXXXX/resources/shared/scripts/dnn.jquery.js?cdv=243 temporary not available

If i use the robots.txt-tester. It show no warnings and no Errors.
A week a go I did remove the /resources from your Robots.txt to check if that was a solution but still it will show this error.

----------------Robots.txt------------------------------------

User-Agent: *
Disallow: /admin
Disallow: /App_Code
Disallow: /App_Data
Disallow: /App_GlobalResources
Disallow: /bin
Disallow: /Components
Disallow: /Config
Disallow: /controls
Disallow: /Install
Disallow: /obj
Disallow: /Properties
Disallow: /Providers
Disallow: /upload
 
New Post
8/12/2015 1:09 PM
 
You need to modify your robots.txt file:


User-Agent: Googlebot
Allow: .js
Allow: .css

http://www.thesempost.com/how-to-unbl...
 
New Post
8/12/2015 1:52 PM
 

Hi Everyone- thanks for your replies

After several different tries, I was able to figure out a solution. Basically, between the modules, and the portals, I had to allow the following in order to resolve all of the errors coming up after running a google fetch and render report- Here is the robots.txt I ended up with. Note that not only could it not access the JS of the skins themselves, but also for the modules. 

User-agent: Googlebot

Disallow: /*/ctl/ # Googlebot permits *

Disallow: /admin/

Disallow: /App_Browsers/

Disallow: /App_Code/

Disallow: /App_Data/

Disallow: /App_GlobalResources/

Disallow: /bin/

Disallow: /Components/

Disallow: /Config/

Disallow: /contest/

Disallow: /controls/

Allow: /DesktopModules/

Disallow: /Documentation/

Disallow: /HttpModules/

Disallow: /images/

Disallow: /Install/

Allow: /js/

Allow: /portals/

Disallow: /Providers/

Disallow: /Resources/ContentRotator/

Disallow: /Resources/ControlPanel/

Disallow: /Resources/Dashboard/

Disallow: /Resources/FeedBrowser/

Disallow: /Resources/OpenForceAd/

Disallow: /Resources/Search/

Allow: /Resources/Shared/

Disallow: /Resources/SkinWidgets/

Disallow: /Resources/TabStrip/

Disallow: /Resources/Widgets/

Disallow: /Activity-Feed/userId/ # Do not index user profiles

 
New Post
8/12/2015 5:43 PM
 
I had this problem, but now is ok for me!
Anyone else?
 
Previous
 
Next
HomeHomeUsing DNN Platf...Using DNN Platf...Administration ...Administration ...Googlebot cannot access CSS and JS filesGooglebot cannot access CSS and JS files


These Forums are dedicated to discussion of DNN Platform and Evoq Solutions.

For the benefit of the community and to protect the integrity of the ecosystem, please observe the following posting guidelines:

  1. No Advertising. This includes promotion of commercial and non-commercial products or services which are not directly related to DNN.
  2. No vendor trolling / poaching. If someone posts about a vendor issue, allow the vendor or other customers to respond. Any post that looks like trolling / poaching will be removed.
  3. Discussion or promotion of DNN Platform product releases under a different brand name are strictly prohibited.
  4. No Flaming or Trolling.
  5. No Profanity, Racism, or Prejudice.
  6. Site Moderators have the final word on approving / removing a thread or post or comment.
  7. English language posting only, please.
What is Liquid Content?
Find Out
What is Liquid Content?
Find Out
What is Liquid Content?
Find Out