KEYSTONESURVEYOR
The crawler in your logs
KeystoneSurveyor is TEKTON’s survey tool. It measures how a public website performs and how it is secured, the same checks a browser can make, so we can tell a business what its systems are costing it. Every request it makes carries KeystoneSurveyor/1.0 in the user agent, with a link to this page.
It makes one short visit per survey and never signs in, submits a form or tests your defences.
WHAT ONE SURVEY REQUESTS
- robots.txt
- Read first. If it disallows us, we stop there.
- 3 timed requests
- To your home page, 800ms apart, to measure the time to first byte.
- 1 plain-HTTP request
- To check that it redirects to HTTPS.
- 1 TLS handshake
- To read the certificate and the protocol version.
- 1 page load
- Your home page in a browser, with the scripts and images it loads itself.
TO BLOCK IT
Add this to your robots.txt. It is read before every survey.
User-agent: KeystoneSurveyor Disallow: /
To be taken off our list altogether, email samuel@houseofdangelo.co.uk with your domain. More on how we work is on the security page.