Saturday, December 13, 2008

How to Deal With Stolen Content

If you ever wondered why your once popular post suddenly stops getting hits, first thing to check it Google Webmaster Tools. If you don’t use it, get it now. Here is how you install it:

Go to Google – My Account – Webmaster Tools

On the dashboard where you see “add a site” you will need to insert your blog URL and hit the button “Add Site” after which you will be asked to verify the ownership of your blog which you can easily do by adding a piece of code to your blog HTML. Just follow instructions.

Next, wait till Google bot crawls your blog and check for errors. If this is the first time you are submitting you will have to wait several hours. However, chances are your blog is already included in Google index and you can check your stats right away.

In this case what you are looking for is URL’s restricted by robots.txt or URL’s not found. If you see any URL’s restricted by robots.txt, click on the link that shows as restricted. Unless you set robots.txt yourself, there should not be any restricted URL’s. To find restricted URL’s type in your browser your blog URL/robots.txt and you will see everything that is disallowed/restricted. Here is what you will see:

User-agent: Mediapartners-Google
Disallow:

User-agent: *
Disallow: /search
Noindex: /feedReaderJson

Sitemap: http://YOURBLOGSITEMAP (whatever your sitemap URL is)

You may not see restricted URL’s here. (Note that in blogger you can’t change robots.txt manually).

To find the problem with a particular URL where you suspect stolen content, go to the url in your webmaster tools that shows as restricted. Next, copy the first sentence in that post and insert the whole sentence in Google search with quotations just like this “the first sentence goes here” and hit search button. If someone has stolen your content, you will see their exact URL.

Chances are, your own URL will not be shown in the search. Don’t panic. Here is how you fix it:

Republish your post by simply going to “edit post” and “publish post”. It can take anywhere from a few minutes to a few days for your URL to appear in the search depending on how often Google normally crawls your site which is determined by the frequency of your posts. Next, chances are the webmaster who stole your content is using it to redirect to some spam site. What you do is, go back to your webmaster tools and click on “report spam in our index”. Follow the instructions to report the site that stole your content. Check everything that applies (deceptive redirects, doorway pages, cloaked pages – those are pages that install cookies, etc.). In the box provided for additional info you may want to note that content was stolen and shows up in search results instead of the original URL.

Google staff will typically review your inquiry within a couple of days and remove the deceptive URL from the index. Don’t expect to hear from them, just re-check in a couple of days and it should be fixed.