The following is the closest thing I have to Jim Flanagan's zeitcode.pm module.
| file | purpose |
|---|---|
| Zeitgeist.pm | Jim's original module with newer search engine evaluation. |
| searchengines.txt | Source file of engines the module recognises. |
| searchengines.new.txt | Referrers the engine thinks might be searches. |
Jim's Original Instructions are still valid.
What I did was extend the module to take in a file searchengines.txt that contains a list of known search engines. In the XML or HTML output, it includes only results from the specified referers.
zeitcode.pm also increments a counter in searchengines.txt associated with each referer it found, as a way of distinguishing active referers from obsolete ones.
Finally, it writes a new file, searchengines.new.txt, with suggestions for new referers that might be from search engines. This file is meant to be reviewed by a human, and for certain referers to be manually added to searchengines.txt.