Hexagora Forum
Hexagora Forum
Home | Profile | Active Topics | Members | Search | FAQ
Username:
 Password:
Save Password
Forgot your Password?

 All Forums
 Dynamic HTML Editor
 Dynamic HTML Editor
 Robots		  New Topic  Reply to Topic
 Printer Friendly
Author Previous Topic Topic Next Topic  

T00N
Super User

573 Posts
Posted - 30 May 2006 :  15:33:43  Show Profile  Visit T00N's Homepage  Reply with Quote
In the metatags there used to be the following line for search engines:

<META NAME="robots" CONTENT="INDEX,FOLLOW">

DHE doesn't produce this?
Isn't this line necesasary nowadays?

s.dav
Site Admin

Italy
3364 Posts
Posted - 30 May 2006 :  16:37:20  Show Profile  Visit s.dav's Homepage  Reply with Quote
I've inserted this line using the "Header Code" property of the page. D.H.E. cannot produce this code for now, maybe in future.
It is not necessary, this should be the default.
Regards, Davide
Go to Top of Page

Martin1
Super User

Netherlands
626 Posts
Posted - 30 May 2006 :  17:04:31  Show Profile  Reply with Quote
As Davide says it's not necessary and the example you gave is the default setting. You are better of creating a seperate robots.txt file that contains the settings you want.

For example
User-agent: *
Disallow: /mytestfiles/
Disallow: /images/
Disallow: /cms/

This example shows you that all robots may crawl everything except for the folders named mytestfiles, images and cms.

For more information on robots.txt you can look here:
http://www.robotstxt.org/

For help with creating a robots.txt file you can go here:
http://www.webtoolcentral.com/webmaster/tools/robots_txt_file_generator/

Martin
Edited by - Martin1 on 30 May 2006 17:08:25
Go to Top of Page

T00N
Super User

573 Posts
Posted - 30 May 2006 :  19:30:16  Show Profile  Visit T00N's Homepage  Reply with Quote
Ok thanks, I was just wondering.
Go to Top of Page
  Previous Topic Topic Next Topic  
 New Topic  Reply to Topic
 Printer Friendly
Jump To:
Hexagora Forum © s.dav Go To Top Of Page
Snitz Forums 2000