A fast and very simple to use PHP library to parse «robots.txt» files and check paths against the rules they define, optionally using “Google interpretation” of the Robots Exclusion Protocol (REP) as defined by RFC 9309.
https://git.lattuga.net/Jones/robtxt.php
Marcie
npub1mv…s49kt
2026-08-29 18:37:14 UTC
Author Public Key
npub1mvs355mr47375e8p6cfnm9fxcyp8w0w0wypkyzhhj76e7q9ljglqws49ktSeen on
wss://relay.ditto.pubPublished at
2026-08-29 18:37:14 UTCEvent JSON
{
"id": "0916639380682d042ab14f7d064b0932608588e5e93cf51d40fbd3a1cd3c4aed",
"pubkey": "db211a5363afa3ea64e1d6133d9526c102773dcf7103620af797b59f00bf923e",
"created_at": 1788028634,
"kind": 1,
"tags": [
[
"proxy",
"https://todon.nl/ap/users/116504000946206292/statuses/117180244591242850",
"activitypub"
],
[
"client",
"Mostr",
"31990:6be38f8c63df7dbf84db7ec4a6e6fbbd8d19dca3b980efad18585c46f04b26f9:mostr",
"wss://relay.ditto.pub"
]
],
"content": "A fast and very simple to use PHP library to parse «robots.txt» files and check paths against the rules they define, optionally using “Google interpretation” of the Robots Exclusion Protocol (REP) as defined by RFC 9309.\n\nhttps://git.lattuga.net/Jones/robtxt.php",
"sig": "e8e2a34aea4d98ff6414d00092f847500a2306a73e4fb5358cfc6de62ebcbf1a6c5dcbb31c2edaa17fcd5969d4cd01ddf6d4f32499719327a553a772d58b2864"
}