|
Click a screen shot below to see a full size image and a brief description.
Step 1 - Spiders
Step 2 - Disallow
Step 3 - Robots.txt
|
 |
Web Spider Software
The premier web spider software that helps you streamline and enhance your search engine optimization efforts.
- Easily generate industry standard robots.txt files
- Identify malicious and unwanted spiders and exclude them from your site
- Direct search engine spiders to correct content for multi lingual sites
- Keep spider our of sensitive and private areas of your web site
- Uploads the correctly formatted robots.txt file to your web site
Robot-Manager Standard is a specialized web spider software you can use to precisely direct search engine robots and spiders to the pages which enhance your overall theme, thus eliminating pages which might dilute your search term relevancy.
Key Features of the Standard Edition
- Automatically creates correctly formatted, industry standard Robot Exclusion Files in real-time
- Uploads the robots.txt file to your web server
- Allows you to manually edit the robots.txt file and export those changes
- Allows grouping of spiders with similar attributes for smaller robots.txt files
- Reads your web site structure either locally or from the web server
- Database of most major US and international spider user-agent identifiers
- Allows you to Add, Edit, and Delete your own spiders
- Can be used in some situations to direct search engine spiders to optimized pages
- Easy-to-learn-and-use step-by-step user interface with Quick Help
Download Trial Edition • Download User's Guide
|
