Google Webmaster Central Blog - Official news on crawling and indexing sites for the Google index
Showing newest posts with label beginner. Show older posts
Showing newest posts with label beginner. Show older posts

SEO Starter Guide updated

Tuesday, September 28, 2010 at 2:00 PM

Webmaster Level: Beginner

Update on October 3, 2010: We have fixed the issue causing the highlighted text to be obscured on Linux PDF readers.

About two years ago we published our first SEO Starter Guide, which we have since translated into 40 languages. Today, we’re very happy to share with you the new version of the guide with more content and examples.

Here’s what’s new:
  • Glossary to define terms throughout the guide
  • More example images to help you understand the content
  • Ways to optimize your site for mobile devices
  • Clearer wording for better readability
You may remember getting to see what Googlebot looks like in our “First date with Googlebot” post. In this version of the SEO Starter Guide, Googlebot is back to provide you with some more SEO tips.

You can download the new version here [PDF]. Entertain and impress your friends by leaving a printed copy on your coffee table.

Googlebot

Finding Places on the Web: Rich Snippets for Local Search

Wednesday, September 22, 2010 at 11:08 AM

Webmaster Level: All
Cross-posted from the Lat Long Blog.

We’re sharing some news today that we hope webmasters will find exciting. As you know, we’re constantly working to organize the world’s information - be it textual, visual, geographic or any other type of useful data. From a local search perspective, part of this effort means looking for all the great web pages that reference a particular place. The Internet is teeming with useful information about local places and points of interest, and we do our best to deliver relevant search results that help shed light on locations all across the globe.

Today, we’re announcing that your use of Rich Snippets can help people find the web pages you’ve created that may reference a specific place or location. By using structured HTML formats like hCard to markup the business or organization described on your page, you make it easier for search engines like Google to properly classify your site, recognize and understand that its content is about a particular place, and make it discoverable to users on Place pages.

You can get started by reviewing these tips for using Rich Snippets for Local Search. Whether you’re creating a website for your own business, an article on a newly opened restaurant, or a guide to the best places in town, your precise markup helps associate your site with the search results for that particular place. Though this markup does not guarantee that your site will be shown in search results, we’re excited to expand support for making the web better organized around real world places.

Video Sitemaps: Is your video part of a gallery?

Friday, September 17, 2010 at 3:45 PM

Webmaster Level: All

Often a website which hosts videos will have a common top-level page that groups conceptually related videos together. Such a page may be of interest to a user searching on that subject. Sites with many videos about a single subject can group these videos together on a top-level page, often known as a gallery. This can make it easier for users to find exactly what they're looking for. In this case, you can use a Sitemap to tell Google the URL of the gallery page on which each video appears.


You can specify the URL of the gallery level page using the optional tag <video:gallery_loc> on a per-video basis. Note that only one gallery_loc is allowed per video.

For more information on Google Videos, including Sitemap specifications, please visit our Help Center. To post questions and search for answers, check out our Help Forum.

Tips for getting help with your site

Wednesday, September 15, 2010 at 9:35 AM

Webmaster Level: All

As a search company, we at Google try to develop scalable solutions to problems. In fact, Webmaster Tools was born out of this instinct: rather than fighting the losing battle of trying to respond to questions via email (and in multiple languages!), we developed an automated, scalable product that gives webmasters like you information about your sites and lets you handle many requests yourself. Now you can streamline the crawling of your site, improve your sitelinks, or clean up after a malware attack all on your own.

Of course, our Help Forum still gets hundreds of questions from site owners every week — everything from "Why isn't my site in Google?" to very specific questions about a particular API call or a typo in our documentation. When we see patterns—such as a string of questions about one particular topic—we continue to use that information in scalable ways, such as to help us decide which parts of the product need work, or what new features we should develop. But we also still answer a lot of individual questions in our forum, on our blog, and at industry events. However, we can't answer them all.

So how do we decide which questions to tackle? We have a few guiding principles that help us make the most of the time we spend in places like our forum. We believe that there are many areas in which Google’s interests and site owners’ interests overlap, and we’re most motivated by questions that fall into these areas. We want to improve our search results, and improve the Internet; if we can help you make your site faster, safer, more compelling, or more accessible, that’s good for both of us, and for Internet users at large. We want to help as many people at a time as we can, so we like questions that are relevant to more than just one person, and we like to answer them publicly. We want to add value with the time we spend, so we prefer questions where we can provide more insight than the average person, rather than just regurgitating what’s already written in our Help Center.

The reason I tell you all this is because you can greatly increase your chances of getting an answer if you make it clear how your question helps us meet these goals. Here are some tips for increasing the likelihood that someone will answer your question:
  1. Ask in public.
    If you post your question in our forum, the whole world gets to see the answer. Then when Betty has the same question a week later, she benefits because she can find the answer instantly in our forum, and I benefit because it saves me from having to answer the same question twice (or ten times, or fifty times, or...). We have a very strong preference for answering questions publicly (in a forum, on a blog, at a conference, in a video...) so that many people can benefit from the answer.
  2. Do your homework.
    We put a lot of effort into writing articles, blog posts and FAQs to help people learn about search and site-building, and we strongly encourage you to search our Help Center, blog and/or forum for answers before asking a question. You may find an answer on the spot. If you don’t, when you post your question be sure to indicate what resources you’ve already read and why they didn’t meet your needs: for example, “I read the Help Center article on affiliate websites but I’m still not sure whether this particular affiliate page on my site has enough added value; can I get some feedback?” This shows that you’ve taken the time to try to help yourself, it saves everyone from reiterating the obvious solutions if you’ve already ruled those out, and it will help get you a more specific and relevant answer. It can also help us improve our documentation if something’s missing.
  3. Be specific.
    If you ask a vague question, you’re likely to get a vague answer. The more details and context you can give, the more able someone will be to give you a relevant, personalized answer. For example, “Why was my URL removal request denied?” is likely to get you a link to this article, as removals can be denied for a variety of reasons. However, if you say what type of removal you requested, what denial reason you got, and/or the URL in question, you’re more likely to get personalized advice on what went wrong in your case and what you can do differently.
  4. Make it relevant to others.
    As I said earlier, we like to help as many people at a time as we can. If you make it clear how your question is relevant to more people than just you, we’ll have more incentive to look into it. For example: “How can site owners get their videos into Google Video search? In particular, I’m asking about the videos on www.example.com.”
  5. Let us know if you’ve found a bug.
    As above, the more specific you can be, the better. What happened? What page or URL were you on? If it’s in Webmaster Tools, what site were you managing? Do you have a screenshot? All of these things help us track down the issue sooner. We appreciate your feedback, but if it’s too vague we won’t understand what you’re trying to tell us!
  6. Stay on-topic.
    Have a question about Google Analytics? iGoogle? Google Apps? That’s great; go ask it in the Analytics / iGoogle / Apps forum. Not every Googler is familiar with every product Google offers, so you probably won’t get an answer if you’re asking a Webmaster Central team member about something other than Web Search or Webmaster Tools.
  7. Stay calm.
    Trust me, we’ve heard it all. Making threats, being aggressive or accusatory, YELLING IN ALL CAPS, asking for “heeeeeeeeeeeeeeelp!!!!!1!!,” or claiming Google is involved in a mass conspiracy against you & your associates because your sites aren’t ranked on page one... Rather than making others want to help you, these things are likely to turn people off. The best way to get someone to help is by calmly explaining the situation, giving details, and being clear about what you’re asking for.
  8. Listen, even when it’s not what you wanted to hear.
    The answer to your question may not always be the one you wanted; but that doesn’t mean that answer isn’t correct. There are many areas of SEO and website design that are as much an art as a science, so a conclusive answer isn’t always possible. When in doubt, feel free to ask people to cite their sources, or to explain how/where they learned something. But keep an open mind and remember that most people are just trying to help, even if they don’t agree with you or tell you what you wanted to hear.
Bonus tip: Are you more comfortable communicating in a language other than English? We have Webmaster Help Forums available in 18 other languages; you can find the list here.

Google Instant: Impact on Search queries

Wednesday, September 08, 2010 at 10:15 AM

Webmaster Level: All

Webmasters, you may notice some changes in you Search queries data due to the launch of Google Instant. With Google Instant, the page updates dynamically to show results for the top completion of what the user has typed, so this means people could be seeing and visiting your website much faster than before, and often without clicking the search button or hitting “enter.”


While the presentation of the search results may change, our most important advice to webmasters remains the same: Users want to visit pages with compelling content and a great user experience.

With Google Instant, you may notice an increase in impressions because your site will appear in search results as users type.


Impressions are measured in three ways with Google Instant:
  1. Your site is displayed in search results as a response to a user’s completed query (e.g. by pressing “enter” or selecting a term from autocomplete). This is the traditional model.

    With Google Instant, we also measure impressions in these new cases:

  2. The user begins to type a term on Google and clicks on a link on the page, such as a search result, ad, or a related search.

  3. The user stops typing, and the results are displayed for a minimum of 3 seconds.
To give an example, let’s say your site has lots of impressions for [hotels] and [hotels in santa cruz]. Now, because Instant is quickly fetching results as the user types, the user could see your site in the search results for [hotels] after typing only the partial query [hote]. If a user types the partial query [hote] and then clicks on any result on the page for [hotels], that counts as an impression for your site. That impression will appear in Webmaster Tools for the query [hotels]. The term 'hotels' would also be included in the HTTP referrer when the user clicks through to visit your website.

It’s likely that your site will still see impressions for queries like [hotels in santa cruz], but because Instant is helping the user find results faster, your site may see an increase in impressions for shorter terms as well.

Let us know if we can help you better understand how these changes impact Webmaster Tools, measured Search queries and impressions, CTR, or anything else. We’d love to hear from you in our blog comments or Webmaster Help Forum.

New ways to view Webmaster Tools messages

Friday, September 03, 2010 at 10:15 AM

Webmaster Level: All

Now there’s a new way to see just the messages for a specific site. A new Messages feature will appear on all site pages. The feature is just like the Message Center on the home page, except it‘ll show only messages for the currently selected site. This gives you more freedom to choose how you want to view your messages: either for all your sites, or for just one site at a time.


Alerts (formally known as SiteNotice messages) will now be more prominent in the Message Center. These messages tell you about significant changes we’ve noticed related to your site which may indicate serious problems. For instance, alerts may warn you about an increase in crawl errors, an increase in 404 errors, or about possible outages. With their newfound prominence comes a new name: what used to be “SiteNotice messages” will now simply be known as “alerts.”

Messages containing alerts will be marked with an icon to make them quickly distinguishable from other messages. Each site’s Dashboard will display a notification whenever the site has unread alerts. The Dashboard notification will lead to the new site Message Center with a filter enabled to show only alerts for the current site.


You can also enable the alerts filter yourself. On the home page, enabling the alerts filter across all your sites is a great way to see alerts you may have missed and may help you find problems common across multiple sites. Even with these changes we recommend you use the email forwarding feature to receive these important alerts without having to visit Webmaster Tools.

We hope these new features make it easier to manage your messages. If you have any questions, please post them in our Webmaster Help Forum or leave your comments below.

Rich snippets: testing tool improvements, breadcrumbs, and events

Thursday, September 02, 2010 at 4:57 PM

Webmaster Level: All

Since the initial roll-out of rich snippets in 2009, webmasters have shown a great deal of interest in adding markup to their web pages to improve their listings in search results. When webmasters add markup using microdata, microformats, or RDFa, Google is able to understand the content on web pages and show search result snippets that better convey the information on the page. Thanks to steady adoption by webmasters, we now see more than twice as many searches with rich snippets in the results in the US, and a four-fold increase globally, compared to one year ago. Here are three recent product updates.

Testing tool improvements

Despite the healthy adoption rate by webmasters so far, implementing the rich snippets markup correctly can still be a major challenge. To help address this, we’ve added new error messages to the rich snippets testing tool to help you better identify and fix any problems with the markup.


If you’ve added markup in the past but haven’t seen rich snippets appear for your site, we encourage you to take a few minutes to try testing the markup again on the updated testing tool.

Rich snippets markup for breadcrumbs

Last year, Google announced a modification to search results to begin showing site hierarchies (typically referred to as "breadcrumbs") rather than standard URLs in cases where it helped users to better understand a website:


We are now adding support for a Breadcrumbs markup format that allows webmasters to explicitly identify the breadcrumb hierarchy on their pages.

If the breadcrumbs UI is already showing for your site, we'll continue to show it even if you don't do the markup, so don't worry about any existing UI disappearing. Note that this new format is experimental. Based on feedback and on other available standards, this format may be modified or replaced in the future. As with other rich snippet types, while markup helps us to better understand the content on your site, it does not guarantee that the breadcrumbs UI will be shown for your web pages in search results.

Events

In January, we added support for rich snippets for events. If a web page containing events listings showed up in search results, up to three links to specific events could be shown in the search result snippet.

This works well for general queries like [concerts in seattle], but we also wanted to improve the search experience when searching for a specific event. We will now show rich snippets when pages containing a single event show up in search results. Single event rich snippets now contain the date and location of the event:


For instructions on adding events markup, refer to the events page in the rich snippets documentation.

Google now indexes SVG

Tuesday, August 31, 2010 at 11:30 PM

Webmaster Level: All

You can now use Google search to find SVG documents. SVG is an open, XML-based format for vector graphics with support for interactive elements. We’re big fans of open standards, and our mission is to organize the world’s information, so indexing SVG is a natural step.

We index SVG content whether it is in a standalone file or embedded directly in HTML. The web is big, so it may take some time before we crawl and index most SVG files, but as of today you may start seeing them in your search results. If you want to see it yourself, try searching for [sitemap site:fastsvg.com] or [HideShow site:svg-whiz.com]

If you host SVG files and you wish to exclude them from Google’s search results, you can use the “X-Robots-Tag: noindex” directive in the HTTP header.

Check out Webmaster Central for a full list of file types we support.

Showing more results from a domain

Friday, August 20, 2010 at 9:40 AM

Webmaster Level: All

Today we’ve launched a change to our ranking algorithm that will make it much easier for users to find a large number of results from a single site. For queries that indicate a strong user interest in a particular domain, like [exhibitions at amnh], we’ll now show more results from the relevant site:



Prior to today’s change, only two results from www.amnh.org would have appeared for this query. Now, we determine that the user is likely interested in the Museum of Natural History’s website, so seven results from the amnh.org domain appear. Since the user is looking for exhibitions at the museum, it’s far more likely that they’ll find what they’re looking for, faster. The last few results for this query are from other sites, preserving some diversity in the results.

We’re always reassessing our ranking and user interface, making hundreds of changes each year. We expect today’s improvement will help users find deeper results from a single site, while still providing diversity on the results page.


Verification time savers —  Analytics included!

Wednesday, August 18, 2010 at 1:12 PM

Webmaster Level: All

Nobody likes to duplicate effort. Unfortunately, sometimes it's a fact of life. If you want to use Google Analytics, you need to add a JavaScript tracking code to your pages. When you're ready to verify ownership of your site in other Google products (such as Webmaster Tools), you have to add a meta tag, HTML file or DNS record to your site. They're very similar tasks, but also completely independent. Until today.

You can now use a Google Analytics JavaScript snippet to verify ownership of your website. If you already have Google Analytics set up, verifying ownership is as simple as clicking a button.


This only works with the newer asynchronous Analytics JavaScript, so if you haven't migrated yet, now is a great time. If you haven't set up Google Analytics or verified yet, go ahead and set up Google Analytics first, then come verify ownership of your site. It'll save you a little time — who doesn't like that? Just as with all of our other verification methods, the Google Analytics JavaScript needs to stay in place on your site, or your verification will expire. You also need to remain an administrator on the Google Analytics account associated with the JavaScript snippet.

Don't forget that once you've verified ownership, you can add other verified owners quickly and easily through the Verification Details page. There's no need for each owner to manually verify ownership. More effort and time saved!


We've also introduced an improved interface for verification. The new verification page gives you more information about each verification method. In some cases, we can now provide detailed instructions about how to complete verification with your specific domain registrar or provider. If your provider is included, there's no need to dig through their documentation to figure out how to add a verification DNS record — we'll walk you through it.


The time you save using these new verification features might not be enough to let you take up a new hobby, but we hope it makes the verification process a little bit more pleasant. As always, please visit the Webmaster Help Forum if you have any questions.

To err is human, Video Sitemap feedback is divine!

Monday, August 16, 2010 at 10:06 PM

Webmaster Level: All

You can now check your Video Sitemap for even more errors right in Webmaster Tools! It’s a new Labs feature to signal issues in your Video Sitemap such as:
  • URLs disallowed by robots.txt
  • Thumbnail size errors (160x120px is ideal. Anything smaller than 90x50 will be rejected.)



Video Sitemaps help us to better crawl and extract information about your videos, so we can appropriately feature them in search results.

Totally new to Video Sitemaps? Check out the Video Sitemaps center for more information. Otherwise, take a look at this new Labs feature in Webmaster Tools.

Video Sitemaps: Understanding location tags

Sunday, August 15, 2010 at 10:41 PM

Webmaster Level: All

If you want to add video information to a Sitemap or mRSS feed you must specify the location of the video. This means you must include one of two tags, either the video:player_loc or video:content_loc. In the case of an mRSS feed, these equivalent tags are media:player or media:content, respectively. We need this information to verify that there is actually a live video on your landing page and to extract metadata and signals from the video bytes for ranking. If one of these tags is not included we will not be able to verify the video and your Sitemap/mRSS feed will not be crawled. To reduce confusion, here is some more detail about these elements.

Video Locations Defined

Player Location/URL: the player (e.g., .swf) URL with corresponding arguments that load and play the actual video.

Content Location/URL: the actual raw video bytes (e.g., .flv, .avi) containing the video content.

The Requirements

One of either the player video:player_loc or content video:content_loc location is required. However, we strongly suggest you provide both, as they each serve distinct purposes: player location is primarily used to help verify that a video exists on the page, and content location helps us extract more signals and metadata to accurately rank your videos.

URL extensions at a glance:



















Sitemap:mRSS:Contents:
<loc><link>The playpage URL
<video:player_loc>

<media:player> (url attribute)The SWF URL
<video:content_loc><media:content> (url attribute)The FLV or other raw video URL

NOTE: All URLs should be unique (every URL in your entire Video Sitemap and mRSS feed should be unique)

If you would like to better ensure that only Googlebot accesses your content, you can perform a reverse DNS lookup.

For more information on Google Videos please visit our Help Center, and to post questions and search for answers check out our Help Forum.

New Message Center notifications for detecting an increase in Crawl Errors

Monday, July 26, 2010 at 2:50 PM

Webmaster Level: All

When Googlebot crawls your site, it’s expected that most URLs will return a 200 response code, some a 404 response, some will be disallowed by robots.txt, etc. Whenever we’re unable to reach your content, we show this information in the Crawl errors section of Webmaster Tools (even though it might be intentional and not actually an error). Continuing with our effort to provide useful and actionable information to webmasters, we're now sending SiteNotice messages when we detect a significant increase in the number of crawl errors impacting a specific site. These notifications are meant to alert you of potential crawl-related issues and provide a sample set of URLs for diagnosing and fixing them.

A SiteNotice for a spike in the number of unreachable URLs, for example, will look like this:


We hope you find SiteNotices helpful for discovering and dealing with issues that, if left unattended, could negatively affect your crawl coverage. You’ll only receive these notifications if you’ve verified your site in Webmaster Tools and we detect significant changes to the number of crawl errors we encounter on your site. And if you don't want to miss out on any these important messages, you can use the email forwarding feature to receive these alerts in your inbox.

If you have any questions, please post them in our Webmaster Help Forum or leave your comments below.

Video Sitemaps 101: Making your videos searchable

Friday, July 16, 2010 at 9:20 AM

Webmaster Level: All

We know that some of you, or your clients or colleagues, may be new to online video publishing. To make it easier for everyone to understand video indexing and Video Sitemaps, we’ve created a video -- narrated by Nelson Lee, Video Search Product Manager -- that explains everything in basic terms:



Also, last month we wrote about some best practices for getting video content indexed on Google. Today, to help beginners better understand the whys and hows of implementing a Video Sitemap, we added a starting page to the information on Video Sitemaps in the Webmaster Help Center. Please take a look and share your thoughts.

Sitemaps: One file, many content types

Tuesday, June 29, 2010 at 4:48 PM

Webmaster Level: All

Have you ever wanted to submit your various content types (video, images, etc.) in one Sitemap? Now you can! If your site contains videos, images, mobile URLs, code or geo information, you can now create—and submit—a Sitemap with all the information.

Site owners have been leveraging Sitemaps to let Google know about their sites’ content since Sitemaps were first introduced in 2005. Since that time additional specialized Sitemap formats have been introduced to better accommodate video, images, mobile, code or geographic content. With the increasing number of specialized formats, we’d like to make it easier for you by supporting Sitemaps that can include multiple content types in the same file.

The structure of a Sitemap with multiple content types is similar to a standard Sitemap, with the additional ability to contain URLs referencing different content types. Here's an example of a Sitemap that contains a reference to a standard web page for Web search, image content for Image search and a video reference to be included in Video search:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
xmlns:image="http://www.sitemaps.org/schemas/sitemap-image/1.1"
xmlns:video="http://www.sitemaps.org/schemas/sitemap-video/1.1">
<url>
<loc>http://www.example.com/foo.html</loc>
<image:image>
<image:loc>http://example.com/image.jpg</image:loc>
</image:image>
<video:video>
<video:content_loc>http://www.example.com/videoABC.flv</video:content_loc>
<video:title>Grilling tofu for summer</video:title>
</video:video>
</url>
</urlset>

Here's an example of what you'll see in Webmaster Tools when a Sitemap containing multiple content types is submitted:



We hope the capability to include multiple content types in one Sitemap simplifies your Sitemap submission. The rest of the Sitemap rules, like 50,000 max URLs in one file and the 10MB uncompressed file size limit, still apply. If you have questions or other feedback, please visit the Webmaster Help Forum.

Quality links to your site

Monday, June 21, 2010 at 10:00 AM

A popular question on our Webmaster Help Forum is in regard to best practices for organic link building. There seems to be some confusion, especially among less experienced webmasters, on how to approach the topic. Different perspectives have been shared, and we would also like to explain our viewpoint on earning quality links.

If your site is rather new and still unknown, a good way marketing technique is to get involved in the community around your topic. Interact and contribute on forums and blogs. Just keep in mind to contribute in a positive way, rather than spamming or soliciting for your site. Just building a reputation can drive people to your site. And they will keep on visiting it and linking to it. If you offer long-lasting, unique and compelling content -- something that lets your expertise shine -- people will want to recommend it to others. Great content can serve this purpose as much as providing useful tools.

A promising way to create value for your target group and earn great links is to think of issues or problems your users might encounter. Visitors are likely to appreciate your site and link to it if you publish a short tutorial or a video providing a solution, or a practical tool. Survey or original research results can serve the same purpose, if they turn out to be useful for the target audience. Both methods grow your credibility in the community and increase visibility. This can help you gain lasting, merit-based links and loyal followers who generate direct traffic and "spread the word." Offering a number of solutions for different problems could evolve into a blog which can continuously affect the site's reputation in a positive way.

Humor can be another way to gain both great links and get people to talk about your site. With Google Buzz and other social media services constantly growing, entertaining content is being shared now more than ever. We've seen all kinds of amusing content, from ASCII art embedded in a site's source code to funny downtime messages used as a viral marketing technique to increase the visibility of a site. However, we do not recommend counting only on short-lived link-bait tactics. Their appeal wears off quickly and as powerful as marketing stunts can be, you shouldn't rely on them as a long-term strategy or as your only marketing effort.

It's important to clarify that any legitimate link building strategy is a long-term effort. There are those who advocate for short-lived, often spammy methods, but these are not advisable if you care for your site's reputation. Buying PageRank-passing links or randomly exchanging links are the worst ways of attempting to gather links and they're likely to have no positive impact on your site's performance over time. If your site's visibility in the Google index is important to you it's best to avoid them.

Directory entries are often mentioned as another way to promote young sites in the Google index. There are great, topical directories that add value to the Internet. But there are not many of them in proportion to those of lower quality. If you decide to submit your site to a directory, make sure it's on topic, moderated, and well structured. Mass submissions, which are sometimes offered as a quick work-around SEO method, are mostly useless and not likely to serve your purposes.

It can be a good idea to take a look at similar sites in other markets and identify the elements of those sites that might work well for yours, too. However, it's important not to just copy success stories but to adapt them, so that they provide unique value for your visitors.


Social bookmarks on YouTube enable users to share content easily


Finally, consider making linking to your site easier for less tech savvy users. Similar to the way we do it on YouTube, offering bookmarking services for social sites like Twitter or Facebook can help spread the word about the great content on your site and draw users' attention.

As usual, we'd like to hear your opinion. You're welcome to comment here in the blog, or join our Webmaster Help Forum community.

Google Videos best practices

Friday, June 11, 2010 at 5:00 PM

Webmaster Level: All

We'd like to highlight three best practices that address some of the most common problems found when crawling and indexing video content. These best practices include ensuring your video URLs are crawlable, stating what countries your videos may be played in, and that if your videos are removed, you clearly indicate this state to search engines.

  • Best Practice 1: Verify your video URLs are crawlable: check your robots.txt
    • Sometimes publishers unknowingly include video URLs in their Sitemap that are robots.txt disallowed. Please make sure your robots.txt file isn't blocking any of the URLs specified in your Sitemap. This includes URLs for the:
      • Playpage
      • Content and player
      • Thumbnail
      More information about robots.txt.

  • Best Practice 2: Tell us what countries the video may be played in
    • Is your video only available in some locales? The optional attribute “restriction” has recently been added (documentation at http://www.google.com/support/webmasters/bin/answer.py?answer=80472), which you can use to tell us whether the video can only be played in certain territories. Using this tag, you have the option of either including a list of all countries where it can be played, or just telling us the countries where it can't be played. If your videos can be played everywhere, then you don't need to include this.

  • Best Practice 3: Indicate clearly when videos are removed -- protect the user experience
    • Sometimes publishers take videos down but don't signal to search engines that they've done so. This can result in the search engine's index not accurately reflecting content of the web. Then when users click on a search result, they're taken to a page either indicating that the video doesn't exist, or to a different video. Users find this experience dissatisfying. Although we have mechanisms to detect when search results are no longer available, we strongly encourage following community standards.

      To signal that a video has been removed,
      1. Return a 404 (Not found) HTTP response code, you can still return a helpful page to be displayed to your users. Check out these guidelines for creating useful 404 pages.
      2. Indicate expiration dates for each video listed in a Video Sitemap (use the <video:expiration_date> element) or mRSS feed (<dcterms:valid> tag) submitted to Google.
For more information on Google Videos please visit our Help Center, and to post questions and search answers check out our Help Forum.

Crawl Errors now reports soft 404s

Monday, June 07, 2010 at 10:17 AM

Webmaster Level: All

Today we’re releasing a feature to help you discover if your site serves undesirable "soft” or “crypto” 404s. A "soft 404" occurs when a webserver responds with a 200 OK HTTP response code for a page that doesn't exist rather than the appropriate 404 Not Found. Soft 404s can limit a site's crawl coverage by search engines because these duplicate URLs may be crawled instead of pages with unique content.

The web is infinite, but the time search engines spend crawling your site is limited. Properly reporting non-existent pages with a 404 or 410 response code can improve the crawl coverage of your site’s best content. Additionally, soft 404s can potentially be confusing for your site's visitors as described in our past blog post, Farewell to Soft 404s.    

You can find the new soft 404s reporting feature under the Crawl errors section in Webmaster Tools.



Here’s a list of steps to correct soft 404s to help both Google and your users:
  1. Check whether you have soft 404s listed in Webmaster Tools
  2. For the soft 404s, determine whether the URL:
    1. Contains the correct content and properly returns a 200 response (not actually a soft 404)
    2. Should 301 redirect to a more accurate URL
    3. Doesn’t exist and should return a 404 or 410 response
  3. Confirm that you’ve configured the proper HTTP Response by using Fetch as Googlebot in Webmaster Tools
  4. If you now return 404s, you may want to customize your 404 page to aid your users. Our custom 404 widget can help.

We hope that you’re now better enabled to find and correct soft 404s on your site. If you have feedback or questions about the new "soft 404s" reporting feature or any other Webmaster Tools feature, please share your thoughts with us in the Webmaster Help Forum.

Grab bag videos are back!

Tuesday, June 01, 2010 at 12:36 PM

We’re kicking off June with the start of a new round of webmaster Q&A on the Webmaster Central YouTube channel. You submitted and voted on questions for Matt Cutts to answer, and Matt sat in the studio for a full day sharing advice for webmasters.

For those of you who watch each video (and who doesn’t?), we’ve worked hard to keep things interesting. Not only did Matt wear different colored shirts, we changed the backgrounds as well! Just don’t submit any screen grabs to We Have Lasers, okay?

To get you started, here’s the first video, which addresses a question about geographic targeting in Webmaster Tools:

We’ll be posting links to new videos as they’re posted on our Twitter account, so follow us there or subscribe to our YouTube channel to be notified of new answers.

Chrome Extensions for web development

Thursday, May 27, 2010 at 10:00 AM

Webmaster Level: All

The Chrome Developer Tools are great for debugging HTML, JavaScript and CSS in Chrome. If you're writing a webpage or even a web app for the Chrome Web Store, you can inspect elements in the DOM, debug live JavaScript, and edit CSS styles directly in the current page. Extensions can make Google Chrome an even better web development environment by providing additional features that you can easily access in your browser. To help developers like you, we created a page that features extensions for web development. We hope you’ll find them useful in creating applications and sites for the web.


For example, Speed Tracer is an extension to help you identify and fix performance issues in your web applications. With Speed Tracer, you can get a better idea of where time is being spent in your application and troubleshoot problems in JavaScript parsing and execution, CSS style, and more.


Another useful extension is the Resolution Test that changes the size of the browser window, so web developers can preview websites in different screen resolutions. It also includes a list of commonly used resolutions, as well as a custom option to input your own resolution.


With the Web Developer extension, you can access additional developer tools such as validation options, page resizing and a CSS elements viewer; all from an additional button in the toolbar.


Another extension you should check out is the Chrome Editor that allows you to easily code within your browser, so you don’t have to flip between your browser and code editor. You can also save a code reference locally to your computer for later use.

These are just a few of the extensions you can find in our extensions for web development page. You can also look for more in the extensions gallery.