The Pew Pew Sports crawler

What it is

PewPewSportsAggregatoris the crawler for Pew Pew Sports’ aggregated listings. It indexes public firearms-for-sale listings from other Australian sites to bring them into one structured search. Every result links back to the original listing.

We may show a preview image from your site and a short, scrubbed extract of the listing — make, model, calibre, price, location. We don’t host the sale. Buyers click through to you. We don’t republish seller contact details.

How we crawl

Identifiable user-agent: PewPewSportsAggregator/1.0 (+https://pewpewsports.com/aggregator). We honour robots.txt, crawl slowly, and use conditional requests.

Opt out

Add this to your robots.txt and we stop on our next crawl:

User-agent: PewPewSportsAggregator
Disallow: /