Remove disallowed in robots.txt urls from XML site map
Remove disallowed in robots.txt urls from XML site map
This script will clean site map from links disalowed in robots.txt (and some tags)
Works both in browser and in command line
Usage in windos .bat file:
@echo OFF
@chcp 65001
"C:\OpenServer\modules\php\PHP-5.6\php.exe" smap_clean.php %*
pause
Usage in Linux crone:
php sm_clean.php
more info about php cli usage: http://php.net/manual/ru/features.commandline.usage.php