Disallow Command
Posted: Sat Feb 22, 2025 10:34 am
The Disallow command is responsible for describing which directory pages or websites should not be included in search results.
Just like the User-agent command, simply enter the page address after the command.
To direct robots not to access the "beta.php" page on your site, the command would be this:
Disallow: /beta.php
You can still prevent access to specific folders.
If you needed to block access to the "files" folder, philippines mobile database the command would be this:
Disallow: /files/
You can also block access to content that begins with a specific letter.
To block access to all folders and files beginning with the letter "a", this would be the command:
Disallow: / a
Allow command
The Allow command allows you to determine for search robots which pages or directories on your site you want to index .
By default, all pages on your site will be indexed, except when you use the Allow command.
Therefore, it is recommended to use the Allow command only when you need to block a folder or directory via the Allow command, but you would like to have indexed a specific file or folder that is inside the blocked directory.
Just like the User-agent command, simply enter the page address after the command.
To direct robots not to access the "beta.php" page on your site, the command would be this:
Disallow: /beta.php
You can still prevent access to specific folders.
If you needed to block access to the "files" folder, philippines mobile database the command would be this:
Disallow: /files/
You can also block access to content that begins with a specific letter.
To block access to all folders and files beginning with the letter "a", this would be the command:
Disallow: / a
Allow command
The Allow command allows you to determine for search robots which pages or directories on your site you want to index .
By default, all pages on your site will be indexed, except when you use the Allow command.
Therefore, it is recommended to use the Allow command only when you need to block a folder or directory via the Allow command, but you would like to have indexed a specific file or folder that is inside the blocked directory.