|
|
|
Forum Member
Group: Forum Members
Last Login: 12/28/2008 6:14:43 PM
Posts: 13,
Visits: 33
|
|
| After my recent tests of TopStyle 3.5 I found that I had inadvertently triggered some side effects when TopStyle created a site. In other words TopStyle fired a page invisible to browsers and Search Engines that I considered "safe". Fortunately I have a "checking robot" that detected the problem and sent me an email alert. (There may be a feature that already does what follows. I didn't find it. If it's there my apologies in advance.) It seems to me that TopStyle acts like a robot, but with privileged access: it can see the filing system. The web site in question has a robots.txt which prohibits access to the files I don't want touched. To guard against such problems l suggest: - TopStyle reads the robots.txt file and, by default, adhere to it instructions. (When using server mappings to access pages.)
- TopStyle should be individually addressable as a user agent in robots.txt.
(There are possible applications of the robots meta tag, for a similar purpose. My current concern cannot be addressed that way. Sitemaps are not a mechanism for doing this as they only list pages you can access.) I'm usually careful about this sort of thing but let my guard down when testing!!
|
|
|
|
|
NewsGator
Group: Administrators
Last Login: 12/18/2008 8:43:14 AM
Posts: 2,264,
Visits: 54,129
|
|
Thanks for the feedback - I've passed it through to the developer to take a look at it. (113735)
Jonathon McDougall
NewsGator Support
|
|
|
|
|
NewsGator
Group: Administrators
Last Login: 12/18/2008 8:43:14 AM
Posts: 2,264,
Visits: 54,129
|
|
At this point we are not likely to implement this feature.
Jonathon McDougall
NewsGator Support
|
|
|
|
|
Forum Member
Group: Forum Members
Last Login: 12/28/2008 6:14:43 PM
Posts: 13,
Visits: 33
|
|
| Thanks Jonathon. I'll keep Topstyle away from web sites that have this issue until it's sorted.
|
|
|
|