Having duplicate content on the website is a common practice and it’s not considered spamming or black hat SEO techniques. But, if you are using the same keywords in your meta description tags for multiple pages, then that will be penalized by Google.
Google has recently announced that it will start to penalize websites with duplicate content. This means that if you have more than one page with the exact same content, such as an About Us page and Contact Us page, then Google may consider this as “spam” and lower your rankings.
Why does Duplicate content matter?
Duplicate content can present three major issues for search engines: They don't know which versions to index; they don't know whether to treat each version separately or combine them into one document; and they don't know how to assign trust, authority, anchor text, link equity, etc., to each version.
The good news is that there are ways to solve each issue individually. First, you can use canonical URLs to tell search engines which version of a page is most authoritative. Second, you can use 301 redirects to redirect traffic from old URLs to newer ones. Third, you can use rel-alternate tags to indicate where a URL points to within another site. Finally, you can use to index meta tags to prevent search engines from indexing certain versions of a page.
How To Fix Duplicate Content on Your Website?
Fixing duplicate content issues all comes down to the same central concept: specifying which of the copies is the "correct" version. If you don't specify what the correct copy is, search engines might think both versions are equally valid.
There are three primary methods for doing this:
- Using a 301 redirection to the correct URL,
- adding the rel"canonical" tag to the original page, or
- using the Meta Robots Noindex
301 Redirects:
The best way to combat duplicate pages is to set up a permanent redirect from the duplicate page to the original content. A 301 redirect tells Google that the old URL no longer points to the same resource it used to.
Rel="canonical"
The rel"canonical" tag helps combat duplicate content problems. A common problem occurs when there are multiple versions of a single webpage. For instance, let’s say you run a blog about dogs and another person runs a different blog about cats. Both blogs contain dog pictures and both contain cat pictures. If one of those bloggers decides to add a picture of a cat, he or she might end up creating a duplicate page. In addition, if one of those blogs gets linked to by a third party, that third party could mistakenly believe that the page containing the cat image belongs to the other blogger.
As a result, each page will receive credit for the same number of links and PageRank, and no one will confuse the two sites.
Meta Robots Noindex
The meta robots tag allows search engine crawlers to know what to do when crawling a particular webpage. In fact, it's one of the most powerful tools you can use to deal with duplicate content problems. You might think that having multiple copies of a single page on your site is no big deal. After all, you already have plenty of other ways to optimize your content to ensure people find it easily. But there are some cases where duplicate pages can actually hurt your rankings.
For example, imagine that you have several different versions of a product page, such as a mobile version, a tablet version, and a desktop version. If you don't tell search engines about these variations, they'll treat all three pages as duplicates of one another, and they won't rank them differently. This could mean that your mobile and tablet pages aren't getting much traffic.
So how exactly does the meta robots tag work?
Well, let's say that you want to exclude a certain category of pages from being indexed. Say you have a page named /products/iPhone-5c/. This tells Google to ignore this page while crawling your site but to include it in its index.
Now, if someone searches for the term "iPhone 5c," Google will return the URL for your products page along with a link to the iPhone 5C page, but it won't show up in the search results.
The meta robots tag allows search engines to crawl the links on a page but keeps them from including those links in their indices.
Each method has pros and cons, and there isn't necessarily a single best way to approach fixing duplicate content issues. However, each method offers different benefits depending on your situation.
If you want to avoid having duplicate content on your site, there are some things you can do:
1) Use different titles for each of your pages.
2) Make sure that all of your pages use unique meta descriptions.
3) Include a unique URL for each page.
4) Don’t repeat any text across multiple pages.
5) Create new content for each page.
6) Write unique content for each page.
If you want to avoid getting penalized by Google, then make sure that all of your pages have unique content. You can use tools like Copyscape (add nofollow tag) to check whether there are any duplicates.
Help Google choose the right canonical URL for your duplicate pages
Google makes sure that every page it crawls has a unique URL. This helps ensure that no one else is indexing the same information twice. However, sometimes you'll find yourself in a situation where there are several URLs pointing to the exact same page. For instance, maybe you've built a responsive site that includes a mobile and desktop version of each page. Or perhaps you're working with a CMS that allows you to build separate pages for each device type. In either case, it's possible that you could end up with duplicate pages indexed by Google.
If you have a single page accessible by multiple URLs, such as /home/about us/, Google will choose one URL as the "canonical" version, and crawl that, and any other URLs will be seen as duplicates. So what happens when Google finds a duplicate URL?
The answer depends on whether you've told Google which URL is canonical. If you haven't, Google will choose one URL itself. But if you do that, Google won't use the canonical URL as a reference point when crawling your site. Instead, it will treat them both as equally important.
In some cases, however, you want Google to pick one URL over another. Maybe you want to rank high for queries like "how to write a resume." Or maybe you just want to avoid duplicate content penalties. In those situations, you should tell Google which URL is canonical.
What is a canonical URL?
A canonical URL is the URL of the page that Google thinks is most representative of a set of duplicate pages on your site. For example, if you have URLs for the same page (example.com/?dress1234 andexample.com/dresses/?dress1234), Google chooses one as canonical. The pages don't need to be absolutely identical; minor changes in sorting or filtering of list pages don't make the page unique (for example, sorting by price or filtering by item colour). The canonical URL can be in a different domain than a duplicate URL.
How Google indexes and chooses the canonical URL?
Google determines what is the best version of a web page by looking at many different signals. These include things like whether the page is served via HTTP or HTTPS, the type of browser used, the language of the page, and the presence of the URL in the sitemap. There are also some technical aspects to consider, such as whether the page is served with 301 redirects, how much of the page is cached, and whether the page uses cache control headers.
If there are multiple pages on the same domain that seems to be the exact same thing, Google will pick one of those pages as the canonical version. This happens because Google wants to make sure that people don't end up seeing the same information twice.
Google uses several methods to decide what page is the best choice for being the canonical version of a given page. Most importantly, Google looks at the content of the page itself.
If Google determines that one page contains a lot more information than another page, it might choose that page as the canonical version. This happens especially when there’s a link from a high-traffic page to a lower-ranking page because Google assumes that people will want to read the more important page.
It also considers things like whether the URLs match up, how often the URLs show up in a sitemap (a list of pages on a site), and how many times the URLs appear in a robots.txt file (which tells Google what pages aren’t supposed to be indexed).
Reasons to choose a canonical URL
There are a number of reasons you might want to explicitly choose a canonical page in a set of duplicates. This ensures that visitors don't end up getting confused about what URL they're supposed to use.
By choosing a canonical URL, you make it easier for search engines to understand how to track your site's performance across multiple pages.
If you have a lot of similar pages - such as those for different colours of your product - you might find it helpful to choose one "master" version of each page. This makes it easier for search engines and webmasters to keep track of the links pointing to your pages.
You could also choose a canonical URL because you want to tell search engines which version of a page you'd like to show in search results. For example, if you've got lots of product listings for a particular product, you might choose a page that includes the most important information about the product.
- To specify which URL you want people to see in search results.
- To consolidate link signals for similar or duplicate pages.
- To simplify tracking metrics for a single product or topic.
- To manage syndicated content.
- To avoid spending crawling time on duplicate pages
Learn which page Google considers canonical
URLInspector helps you understand how Google views the canonicalization process. You can use it to see what page Google thinks is the most important one and why. This information is useful for deciding whether to make changes to your site’s structure or content.
How can Webplanners help?
If you are worried about duplicate content issues, contact us today! We'll work with you to create a strategy that works for your business. Please write to us at This email address is being protected from spambots. You need JavaScript enabled to view it. or call us on (03) 9510 0717. We'd love to hear any questions you may have about this article.
Suggested Read: A Guaranteed Effective Guide to Managing a Content Audit
