Name: | perl-WWW-RobotRules |
---|---|
Version: | 6.02 |
Release: | 18.module+el8.6.0+20613+17ccb46e |
Architecture: | noarch |
Module: | perl-libwww-perl:6.34:8030020200428102957:b947e2fe perl-libwww-perl:6.34:8060020210901111951:b947e2fe |
Group: | Development/Libraries |
Size: | 25394 |
License: | GPL+ or Artistic |
RPM: | perl-WWW-RobotRules-6.02-18.module+el8.6.0+20613+17ccb46e.noarch.rpm |
Source RPM: | perl-WWW-RobotRules-6.02-18.module+el8.6.0+20613+17ccb46e.src.rpm |
Build Date: | Fri Apr 08 2022 |
Build Host: | build-ol8-x86_64.oracle.com |
Vendor: | Oracle America |
URL: | http://search.cpan.org/dist/WWW-RobotRules/ |
Summary: | Database of robots.txt-derived permissions |
Description: | This module parses /robots.txt files as specified in "A Standard for Robot Exclusion", at <http://www.robotstxt.org/wc/norobots.html>. Webmasters can use the /robots.txt file to forbid conforming robots from accessing parts of their web site. |