Products

Solutions

Resources

Partners

Community

Blog

About

QA

Ideas Test

New Community Website

Ordinarily, you'd be at the right spot, but we've recently launched a brand new community website... For the community, by the community.

Yay... Take Me to the Community!

Welcome to the DNN Community Forums, your preferred source of online community support for all things related to DNN.
In order to participate you must be a registered DNNizen

HomeHomeUsing DNN Platf...Using DNN Platf...Administration ...Administration ..."Crawling" DNN Websites"Crawling" DNN Websites
Previous
 
Next
New Post
5/26/2016 6:18 PM
 

So, my "simple" question is ... is there anything that actually indexes a DNN site by actually CRAWLING the site?




Joe Craig
Patapsco Research Group, Ellicott City, MD
DotNetNuke Development and Services (http://patapscorg.com)
 
New Post
5/26/2016 7:04 PM
 
yes Craig,
it is called "Google" ;)

Cheers from Germany,
Sebastian Leupold

dnnWerk - The DotNetNuke Experts   German Spoken DotNetNuke User Group

Speed up your DNN Websites with TurboDNN
 
New Post
5/27/2016 3:45 AM
 
Sebastian Leupold wrote:
yes Craig,
it is called "Google" ;)

LOL!

I guess this would not really help Joe, but maybe he could tell us the reason why he asked this question - so we can answer in a less "stupid" way.

Happy DNNing!
Michael


Michael Tobisch
DNN★MVP

dnn-Connect.org - The most vibrant community around the DNN-platform
 
New Post
5/31/2016 1:42 PM
 
Thanks, both of you.

Yes, my question begged for that answer, but there was a more serious question behind it. I will explain.

I'm working with what I think is a typical situation, and one that doesn't, to my mind, have a good solution. The client has two modules that are used together to display a list of employees, each has a link to a detail page where details about a specific employee are shown. So, the first page is something like mysite.com/our-team, and that page has links to the detail page that look like mysite.com/our-team/detail/name/BobSmith. In the latter, this link is a friendly url that really is ?name=BobSmith, and the detail module uses the querystring variable to determine what is shown.

Both pages are dynamic. The our-team page is constructed from users in a specific security role. The detail page is built from the specified user's profile. So, everything is based on user profiles.

Now, the big issue is that neither the our-team nor the detail module implement Searchable (or iSearchable). {The fact that a reputable module developer built this and didn't include search is another to-be-written rant! They should be ashamed.]

But my problem is "search isn't working."

This is when it hit home that the DNN "Site Crawler" really isn't "crawling" the site and indexing the contents directly. Both Searchable and the older iSearchable provide the content to the search indexer indirectly, so this depends on two things: search must be supported at the module level, and also it must be done in a useful way. It also seems to me that there isn't a good way to handle the querystring. Further, even though the content of one module includes links that have the querystrings in them, these do not seem to be "followed" in the crawler sense.

So my options are:
Have the client go back to the original site developer and demand that the modules get fixed: not going to happen.
Fortunately I do have the source code, could dig through how it works, and add search. I think that this still fails to solve my problem, due to the querystring issue mentioned.
Find a replacement search indexer that can handle this situation - I haven't.
So I'm in search of a solution. Hence my badly formed question. But, you've got to admit it was easier to read that this version!

Oh, and with respect to #2 above, I think that the only real feasible solution is that the detail module's Searchable implementation should return "search documents" for all possible team members? That probably should work.

But, the larger question now is search. Search really don't index the contents of a site. This was somewhat of an eye opener, as I'd never really dug in the intricacies of search, and some of the design issues.

(Some years ago I used the old Xepeient search on a DNN site. I solved problems like this one because it really crawled the site. There were other issues, but I think I solved most of those!

Would it be possible (or advisable) to have a module that self-identified as "I don't support searchable, so index me directly"?

So, help and advice for you folks will be appreciated.



Joe Craig
Patapsco Research Group, Ellicott City, MD
DotNetNuke Development and Services (http://patapscorg.com)
 
New Post
5/31/2016 2:48 PM
 
Joe,
there are different approaches with search: DNN site search requires modules to implement an interface, but is is able to take permission into account, while an external search crawler would (usually) index public content only. If you want content to show up in site search, you need to implement the search interface, otherwise you may just submit the search to Google and display the results of this search on a local page.

Cheers from Germany,
Sebastian Leupold

dnnWerk - The DotNetNuke Experts   German Spoken DotNetNuke User Group

Speed up your DNN Websites with TurboDNN
 
Previous
 
Next
HomeHomeUsing DNN Platf...Using DNN Platf...Administration ...Administration ..."Crawling" DNN Websites"Crawling" DNN Websites


These Forums are dedicated to discussion of DNN Platform and Evoq Solutions.

For the benefit of the community and to protect the integrity of the ecosystem, please observe the following posting guidelines:

  1. No Advertising. This includes promotion of commercial and non-commercial products or services which are not directly related to DNN.
  2. No vendor trolling / poaching. If someone posts about a vendor issue, allow the vendor or other customers to respond. Any post that looks like trolling / poaching will be removed.
  3. Discussion or promotion of DNN Platform product releases under a different brand name are strictly prohibited.
  4. No Flaming or Trolling.
  5. No Profanity, Racism, or Prejudice.
  6. Site Moderators have the final word on approving / removing a thread or post or comment.
  7. English language posting only, please.
What is Liquid Content?
Find Out
What is Liquid Content?
Find Out
What is Liquid Content?
Find Out