[ic] Spiders and affiliate tracking
Jamie Neil
jamie at versado.net
Wed Jan 7 16:11:31 EST 2004
Ed LaFrance wrote:
>> I can't stop the spiders following affiliate links, so I'm just trying
>> to stop them from indexing the page they land on without affecting
>> normal crawling behavoir.
>
>
> That will be tough. The spiders are probably following these links from
> the affiliate sites and indexing as they go. You might try a [bounce] to
> a page with no args when the landing page is called with an affiliate
> code, that might prevent the index of the mv_pc string - but I am no
> expert at these things, just a suggestions.
A "noindex" robots metatag on a page _should_ stop that page being
indexed by a search engine when it is crawled (in fact it should remove
the entry from it's index if already there). However I still don't like
this as a solution.
After thinking about it some more last night, I've actually ended up
using a bounce method like you mentioned. I've created a simple referral
page:
pages/referral.html
-------------------
[tmp page][cgi name=ref][/tmp]
[if scratch page]
[bounce page=|[scratch page]| status="301 Moved Permanently"]
[else]
[bounce page=|ref_error| status="301 Moved Permanently"]
[/else]
[/if]
and an error page:
pages/ref_error.html
--------------------
[tag op=header]
Status: 404 Not found
Content-type: text/html
[/tag]
[tmp page_title]Sorry, the page was not found[/tmp]
@_NOLEFT_TOP_@
<H1>[scratch page_title]</H1>
<p>The requested page was not found. You can [page index]return to
browsing our catalog</a>, if you wish.</p>
@_NOLEFT_BOTTOM_@
Instead of referrals using a URL like:
http://www.oursite.com/pagename.html?mv_pc=ref1
they will now use:
http://www.oursite.com/referral.html?mv_pc=ref1&ref=pagename
which sets the session source value for affiliate tracking and then 301s
the user to the correct page:
http://www.oursite.com/pagename.html
AFAIK, Google (and other SEs) will not index a page that gives a 301
error. Instead they will follow the redirect and index that page, thus
ignoring the original referral URL. This should also stop people
bookmarking affiliate links.
So far it seems to be working ok. Time will tell whether these new
affliate links appear in the SERPs!
All I need to work out now is how to get the links that have already
been indexed to get dropped.
--
Jamie Neil | <jamie at versado.net> | 0870 7777 454
Versado I.T. Services Ltd. | http://versado.net/ | 0845 450 1254
More information about the interchange-users
mailing list