# robots.txt file # 31 Oct 2008 # 02 Dec 2008 # 09 Nov 2012 # Exclude Files From All Robots: User-agent: Twitterbot Disallow: /BBS docs/ Disallow: /catalog/ Disallow: /cgi/ Disallow: /contact/ Disallow: /data/ Disallow: /esp/ Disallow: /flab/ Disallow: /gif/ Disallow: /grief/ Disallow: /html/glyphcharts/ Disallow: /html/info/ Disallow: /index.html/ Disallow: /logs/ Disallow: /mom/ Disallow: /mom/gallery/ Disallow: /music/ Disallow: /newsite/ Disallow: /notfound.html/ Disallow: /papers/ Disallow: /pdf/ Disallow: /psych/ Disallow: /research/ Disallow: /styles/ Disallow: /thumbnails/ Disallow: /tropicals/ # End robots.txt file