IP and URL Canonicalization Checker Services and Tools: What is canonicalization and how to check it?
First, I’d like to explain the meaning of “IP and URL Canonicalization” terms.
Then, I’ll show you some mistakes to avoid when submitting your site to Google and Bing (Webmasters sections).
For the appropriate definition – as the most real authentic source of IP and URL Canonicalization -, I’ll quote Mr. Matt Cutts (Google): “Canonicalization is the process of picking the best url when there are several choices, and it usually refers to home pages. For example, most people would consider these the same urls:
But technically all of these urls are different. A web server could return completely different content for all the urls above. When Google “canonicalizes” a url, we try to pick the url that seems like the best representative from that set.” – taken from Matt Cutts: Gadgets, Google, and SEO; SEO advice: url canonicalization; Posted January 4, 2006 in Google/SEO, https://www.mattcutts.com/blog/seo-advice-url-canonicalization/.
Note: About http://example.com: “This domain is established to be used for illustrative examples in documents. You may use this domain in examples without prior coordination or asking for permission. IANA-managed Reserved Domains: Certain domains are set aside, and nominally registered to “IANA”, for specific policy or technical purposes…As described in RFC 2606, a number of domains such as example.com and example.org for documentation purposes. These domains may be used as illustrative examples in documents without prior coordination with us. They are not available for registration or transfer.” – taken from http://www.iana.org/domains/reserved.
This may seem a bit complicated at the first sight, and really not as easy as e.g. publishing a post. However, our task is for today is to find a Canonicalization Checker Tool, I’d like to explain it first from the eye point of an average Internet user for the better understanding.
When you search the Internet, you’ll find there various sites beginning with “www.”, without www, “http://www.”,”htp://”, “htps://”. As an average user, usually you even don’t bother with understanding the difference but the result only. If you are a webmaster, it is more than advisable to know the difference. The main goal of the IP canonicalization is to redirect the web pages which are labeled as the duplicate content.
Now you can see the importance of the IP and URL Canonicalization process: instead of relying on the “good decision” of the particular search engine – like Google, Bing and Yahoo -, you should define the most appropriate URL for your content. To cut the long story short, in the case your target pages have the same issues and if you had decided which URL was the first and most relevant one, you can give a definiton with embedding the
line into the section of all site pages.
Please take care, that the IP and URL Canonicalization process is not the same as a “301” – Permanent Redirect command. However, there can be similarities. Canonicalization will tell the search Engines that multiple pages should be considered to one like 301 Redirect command does, but it works only within the root directory of your domain and with the pages belong to this particular domain, based on your own intention and decision. On the other 301 has a so-called cross-domain functionality, that is you can permanently redirect one page – or the entire site – to domain1, domain2 names an so on.
On WordPress sites, the software itself will give you a solution for the Canonicalization of your homepage. But, it is another story, I’ll deal it with in a post soon. Now, I’d like to introduce some Services dealing with checking IP and URL Canonicalization:
Which SEO Tools can help us in Checking IP and URL Canonicalization?
For checking HTTP headers – and IP Canonicalization – I found a very useful site, titled as: Web Sniffer – View HTTP Request and Response Header – http://web-sniffer.net. This specialized site’s very sophisticated tool is able to check all parameters of your site regarding to HTTP Headers, Canonicalization (see the screenshot, please). Another great advantage of Web-Sniffer.net, that you can download a really FREE small standalone application either for your Windows or Mac computer (you can read more about this app in my post: “IP And URL Canonicalization Test And Check – Another Way” if interested).
However, you can find more sites with similar tools such as Statscrop.com, SEOSiteCheckUp.com, WOORank.com (you can follow a test on WooRank in my post “IP And URL Canonicalization Test And Check – Some Other Ways“) and more – see SEO Site Status Checker Tools page on this site -, this site itself can give to most appropriate and direct results in this specific question. This is why I use it myself, too.
Let’s see our work now!
Checking IP Canonicalization
As the definition states, your site’s IP, that is xxx.xxx.xxx.xxx number should redirect – the IP points – to your site’s domain name. If not, this could cause duplicate content problems if a search engine indexes your site under both, that is its IP and domain name.
Here come some difficulties what many people may misunderstand. I’d like to explain from my point of view, based on my previous experiences.
If your site is a “standalone” one (dedicated IP, assigned only to this domain name), that is its IP address IS the same as your domain name, this definition is right. If your site is hosted on a Shared IP address, this rule can be true only within the root directory of the domain, and not for the IP address. How to check and solve it? This is our job for today.
Checking IP and URL Canonicalization:
When you create a new account with a domain hosting provider for your domain, you’ll be given an IP address. In this case, we’ll meet two ways concerning your IP address. I’ll begin with the most common, the shared acoount:
1. Shared hosting account – IP and URL Canonicalization Check:
If your account is a shared account – most of them are shared -, your domain will become a “part” of the group of other domains which use the same IP address. You can check these domains out – with using the “Reverse IP Lookup” command, it depends on the offered service of several SEO Site Status Checker sites – within a minute.
In order to check IP and URL Canonicalization, we’d need a domain name. I’d suggest you trying it on your own domain (it doesn’t hurt al all). Please follow these simple steps below:
Step 1 – Checking you domain without “www”:
Be sure, that if you type in your browser the name of your domain, you’ll land on your own page. Now, open a new browser tab (or window), and go to http://web-sniffer.net – View HTTP Request and Response Header site. Type in your domain name, first WITHOUT www.
Fill the captcha correctly and click on SUBMIT button.
Now, you’ll get the results. If your site – domain name – is registered and used WITHOUT www, AND you check this situation with Web.Sniffer, you’ll get the following:
Status: HTTP/1.1 200 OK. What does it mean? Your basic setting is: using the domain WITHOUT www.
Step 2- Check the same domain with “www”:
Go back to the HTTP(S)-URL box, and type in your domain name WITH www. Solve the captcha and submit.
Now, supposing the correct setting of your domain, you’ll get:
Status: HTTP/1.1 301 Moved Permanently. What does it mean? Your domain containing www IS REDIRECTED correctly to the domain without www.
Now, finishing these tests, check out your IP address. When you are on Web Sniffer, you’ll see the your IP, just under the HTTP Request Header title. It will look like xxx.xxx.xxx.xxx (currently, in the case of IPv4). Let’s move to Step 3.
Step 3 – Checking IP Canonicalization
After picking up your IP address (xxx.xxx.xxx.xxx), type in your IP using the same box you did in Step 2 and Step 3. Solve the captcha, and see the results.
You’ll be surprised!
The result of checking (see the content box where the content of the examined site is displayed) shows, that you landed on a completely different site (except for the case, when your domain is the first on the hosting list, and your cPanel is connected to this IP directly)!
How can it happen? Because you are on a Shared Hosting Account, your IP is also shared (this explains its name). Together with this recognition, a simple question might come into your mind at once:
Can a Shared IP be harmful from SEO acpects? With other words: do you really need a dedicated IP address for SEO benefits?
Mainly, not your IP address is the most important from SEO aspects, but the relation of your site to other web sites. Put it the other way around, Google really does care in regards to the IP addresses of the web sites linking to you. This is a crucial distinction because your IP address DOES NOT matter. That is, basically, hosting on a shared IP has zero impact to your SEO results. However, the IP addresses of the parents linking to you DOES matter. Google doesn’t like the so-called link farms, that is link network. These link farms can be easily discovered by Google’s advanced, sophisticated and enhanced internet spiders, the so-called web crawlers.
What Google wants to see that you are simply popular around the world (a whole bunch of various web sites with completely different owners and unique IP addresses linking to you), not just amongst the 30 different web sites all hosted in the same place. Otherwise, you can quickly lose your entire site’s PageRank and your links can be removed even from their index as well.
According to my experience, on Shared Accounts your domain host itself offers solution for the IP canonicalization with redirection within the the main IP.
How can they do this? They use so-called AddOn Domains within a given Hosting Account, and they will take of the correct Domain Name Redirection. But, it is another long story and not our topic today (nonetheless, if you were still interested how it looks like from inside, simply visit Robtex.com (their slogan: “Robtex Swiss Army Knife Internet Tool”) or DNSstuff.com/tools (main site title: “DNS tools | Manage Monitor Analyze | DNSstuff”) and make some tests with their extremely sophisticated SEO tools).
Back to the tests: don’t worry on Shared accounts. This can be the main reason why the classic IP Canonicalization test will not give you authentic results, that is “Failed” test results. But, this kind of IP redirection (Canonicalization) will not substitute the Canonicalization within your root directory, so you can still have issues there. Consequently, as an advice and in order to avoid any issues, double check the Canonicalization concerning your domain in your root directory as well.
2. Dedicated IP – IP and URL Canonicalization Check:
If your domain is the only one on the hosting account – and the DNS setting are correct -, its IP number is surely the same as your domain name. In this case, when you carry out the test I wrote above (in the case of the Shared IP Hosting Account), you’ll get back your own site as the result. According to the definition of IP Canonicalization, if you enter your IP (xxx.xxx.xxx.xxx) on your browser, you should land on your own page. If it doesn’t happen, this event really can cause issues, so you would need to check it out more carefully and investigate the possible reasons.
Advice: What are the mistakes you should avoid during the first registration of your site to Google and Bing?
First of all, after you had purchased your domain, contracted with a domain host, built and published your site, you must visit Google and Bing Webmaster to “introduce” it. The verification process both on Google and Bing consist of the following steps and slightly similar to each other:
1. Adding your url
2. Adding your sitemap (generally sitemap.xml) address
3. Verifying your site with choosing the most appropriate way
(Notice: Yahoo mostly uses the data of Bing, so it is not necessary to submit on Yahoo, as it is known today (2017)).
The Mistakes you have to avoid:
Canonicalization rules on Google Webmasters site:
When you are on Google, you can add your site’s address either with www, or without it, it is completely up to you at the very first moment. But, if you have added it, the syntax of your submitted links MUST follow the same way (that is, if you began it without www, you must follow itin the future), including the sitemap address.
NEVER submit your site on Google using www AND without www – that is both at the same time -, because it will cause immediate duplicate content! You decide: either this or that way will be the most appropriate solution for you, but never forget your choice and insist on this decision consistently in the future!
Canonicalization rules on Bing Webmasters site:
On Bing – unlike on Google -, it is REQUIRED to submit your site as with www, and without www. It means, that you have to give 2 sitemap addresses as well:with and without www. During single link submissions,it is advisable to use the form you are using on Google.
How to check if your site content is not duplicated?
As I’ve mentioned it on my SEO Site Status Checker Tools page, you can use e.g. StatsCrop.com for this purpose the easiest: it will display the fact of a duplicate site content in its SEO section in a minute. You can see now why IP and URL Canonicalization are so important in our Internet life nowadays.
From the Search Engine Optimization perspective IP and URL Canonicalization are essential for a web site. When you find yourself having a web site which has some internal pages also (for example, it may happen promoting almost same product with the same description, content) and the content on the other internal web page is same or similar with another URL on your web site then it may be an issue to your web site, and you should find a solution to solve this problem.
Additionally, in order to avoid any issues and possibly penalties, please take care of the correct IP and URL Canonicalization on your web site.
I hope I could help you in your work with sharing my ideas and experience, together with presenting some Tools for Checking IP and URL Canonicalization.
Thank you for reading!
Gyula Olah electrical engineer, author, the owner of SEO Service Guide – Search Engine Optimization Basics site.
SEOprofiler is a cool web-based SEO software tool that helps you to get high rankings on Google, Yahoo and Bing. You can also check the backlinks, Google rankings and Google AdWords of any website for free. Just enter a domain name in the following search box:
Just another complex SEO Solution – SEO Rank Monitor (Try it for 14 Days FREE!):