Pages in search results

Page information is presented as follows:

• Added and removed — The ratio of the pages included and excluded from the search.
• Excluded — The dynamics of pages excluded from the search.
• History — The dynamics of pages included in the search. Each segment of the graph corresponds to a site section.
• Dictribution — The number of pages included in the search. Each segment of the pie chart corresponds to a site section.

The total number of pages included in the search can exceed the total number of pages in the diagrams (only pages and sections listed on the Site structure page in Yandex.Webmaster are counted).

Yandex.Webmaster informs you of changes in the search results over the past three months:

• Date of the search results update.
• Whether a page was added to in the search results (the page URL is displayed in green) or removed from them (displayed in blue).
• The reason why the page was removed from the search.

A dramatic change in the number of pages added or participating in search might be related to changes on the site. For example, with changing the website structure or the robots.txt file.

In addition, the service displays general information about the page:

• The date of the last robot crawl.
• The page path from the root directory of the site.
• The page title (the title element).

To view the changes, set the option to Recent changes. Up to 50,000 changes can be displayed.

You can view the list of pages included in the search and the following information about them:

• The date of the last robot crawl.
• The page path from the root directory of the site.
• Page title (the HTML title element).
• Availability of Turbo pages for the URL.

To view the list of pages, set the option to All pages. The list can contain up to 50,000 pages.

If the list doesn't contain any pages that should be included in the search results, use the Reindex pages tool to inform Yandex about them.

If the list contains pages that shouldn't be included in the search results, see the Deleting a site section.

Site pages can disappear from the search results. For more information about why this may happen, see the Why are pages excluded from the search? section.

To view the list of excluded pages (up to 50,000), set the option to Excluded pages in Indexing → Pages in search. The following information about the pages is displayed:

• The date of the last robot crawl.
• The page path from the root directory of the site.
• Status (the reason why the page was excluded).

A page is no longer listed as excluded from the search some time after the following conditions are fulfilled:

• The page is unavailable for the indexing robot (HTTP server returns 404 Not Found), or there is a 301 redirect set on the page.
• Other resources don't refer to the excluded page.

A page that is removed from the site or doesn't exist can be found in the list of pages excluded from the search, probably because other resources refer to it. To remove the page from the list, disallow indexing in the robots.txt file.

You can quickly find information about a page using filters. To do this, click the  icon. You can filter data by all available parameters. For example, you can use URL filtering:

Enter part of a URL.

To make a list of pages with a certain text in the URL, choose URL contains from the list and enter the URL fragment.

Use special characters.

You can use special characters to match the beginning of the string or a substring, and set more complex conditions using regular expressions. To do it, choose URL matches from the list and enter the condition in the field. You can add multiple conditions by putting each of them on a new line.

For conditions, the following rules are available:

• Match any of the conditions (corresponds to the “OR” operator).
• Match all conditions (corresponds to the “AND” operator).

Characters used for filtering

Character	Description	Example
*	Matches any number of any characters	Display data for all pages that start with https://example.com/tariff/, including the specified page: / tariff / * Using the * character The * character can be useful when searching for URLs that contain two specific elements or more. For example, you can find news or announcements for a certain year: /news/*/2017/.
@	The filtered results contain the specified string (but don't necessarily strictly match it)	Display information for all pages with URLs containing the specified string: @tariff
~	Condition is a regular expression	Display data for pages with URLs that match a regular expression. For example, you can filter all pages with address containing the fragment ~table|sofa|bed repeated once or several times.
!	Negative condition	Exclude pages with URLs starting with https://example.com/tariff/: !/tariff/*

Characters used for filtering

Character	Description	Example
*	Matches any number of any characters	Display data for all pages that start with https://example.com/tariff/, including the specified page: / tariff / * Using the * character The * character can be useful when searching for URLs that contain two specific elements or more. For example, you can find news or announcements for a certain year: /news/*/2017/.
@	The filtered results contain the specified string (but don't necessarily strictly match it)	Display information for all pages with URLs containing the specified string: @tariff
~	Condition is a regular expression	Display data for pages with URLs that match a regular expression. For example, you can filter all pages with address containing the fragment ~table|sofa|bed repeated once or several times.
!	Negative condition	Exclude pages with URLs starting with https://example.com/tariff/: !/tariff/*

The use of characters isn't case sensitive.

The @,!, ~ characters can be used only at the beginning of the string. The following combinations are available:

Operator	Example
!@	Exclude pages with URLs containing tariff: !tariff
!~	Exclude pages with URLs that match the regular expression

Operator	Example
!@	Exclude pages with URLs containing tariff: !tariff
!~	Exclude pages with URLs that match the regular expression

You can download the pages information in the XLS or CSV format.