Robots.txt Generator

Default - All Robots are:	Allowed Refused
Crawl-Delay:	Default - No Delay 5 Seconds 10 Seconds 20 Seconds 60 seconds 120 Seconds
Sitemap: (leave blank if you don't have)
Search Robots:	Google	Same as Default Allowed Refused
	Google Image	Same as Default Allowed Refused
	Google Mobile	Same as Default Allowed Refused
	MSN Search	Same as Default Allowed Refused
	Yahoo	Same as Default Allowed Refused
	Yahoo MM	Same as Default Allowed Refused
	Yahoo Blogs	Same as Default Allowed Refused
	Ask/Teoma	Same as Default Allowed Refused
	GigaBlast	Same as Default Allowed Refused
	DMOZ Checker	Same as Default Allowed Refused
	Nutch	Same as Default Allowed Refused
	Alexa/Wayback	Same as Default Allowed Refused
	Baidu	Same as Default Allowed Refused
	Naver	Same as Default Allowed Refused
	MSN PicSearch	Same as Default Allowed Refused
Restricted Directories:	The path is relative to root and must contain a trailing slash "/"

Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.

About Robots.txt Generator

How to Create a Robots.txt File Using a Robots-txt-Generator

Introduction:
A robots.txt file is an important part of website optimization as it informs web crawlers which pages they should and should not index. A robots-txt-generator is a tool that allows users to easily create robots.txt files, which can help improve the SEO of their website. This article will explain how to use a robots-txt-generator to create a robots.txt file.

Table of Contents:
I. What is a Robots.txt File? 
II. What is a Robots-txt-Generator? 
III. Benefits of Using a Robots-txt-Generator 
IV. How to Create a Robots.txt File Using a Robots-txt-Generator 
V. Conclusion 

I. What is a Robots.txt File? 
A robots.txt file is a text file that informs web crawlers which pages they should and should not index. It is used to optimize a website and can help improve its ranking in search engine results. It is important to note that robots.txt files are not always required, and that they should only be used when necessary.

II. What is a Robots-txt-Generator? 
A robots-txt-generator is a tool that allows users to easily create robots.txt files. It eliminates the need for manual coding and makes it easy for users to create a robots.txt file in just a few simple steps.

III. Benefits of Using a Robots-txt-Generator 
There are many benefits to using a robots-txt-generator. It is a fast and easy way to create a robots.txt file, as it eliminates the need for manual coding. Additionally, it is also more reliable than manual coding and can help ensure that the robots.txt file is created correctly. Additionally, it can help improve a website's SEO by allowing users to exclude certain pages from being indexed by search engine crawlers.

IV. How to Create a Robots.txt File Using a Robots-txt-Generator 
Creating a robots.txt file using a robots-txt-generator is a simple process. First, the user needs to select the file type they want to generate. Most robots-txt-generators will offer two options: a standard robots.txt file or a custom robots.txt file. The standard file type is recommended for most users, as it contains the most commonly used directives for web crawlers. After selecting the file type, the user will be prompted to enter the directives they want to include in the robots.txt file. The directives are usually written in a simple, easy-to-understand syntax and can be edited as necessary. Once the user has entered the directives, they can then generate the robots.txt file.

V. Conclusion 
A robots-txt-generator is a useful tool for creating robots.txt files. It eliminates the need for manual coding and makes it easy for users to create a robots.txt file in just a few simple steps. Additionally, it can help improve a website's SEO by allowing users to exclude certain pages from being indexed by search engine crawlers.