Robots.txt file For domain: http://newcastlecarpetcleaner.co.uk # All robots will spider the domain User-agent: *