Posted by Candego You are right, Medusa does not currently look at the robots.txt or robots meta-tags.
on 11/26/2005, 6:18 pm, in reply to "robots behaviour and net etiquette"
Board Administrator
Hi AJ!
We understand your situation, but presently the only way to stop Medusa is to use the user-agent field to identify it. This field sent in the http headers by Medusa 1.2 includes the text "Medusa" so you can prevent access by using this as filter.
Message Thread:
« Back to thread