Alexa Web Information

Alexa is gathering Web information and learning from content and paths to create the Alexa Service.

Gathering

Alexa is continually crawling all publicly available web sites to create a series of snapshots of the Web. They use the data they collect to create features and services that are integrated parts of the Alexa service:

Site Data: Provides information, news, and statistics about web sites
Archive of the Web: Serves pages that are no longer available, virtually eliminating dead links
Related Links: Sites that are similar to the one you are currently viewing
They are currently gathering in excess of 250 gigabytes of information per day. Our crawling does not interfere with a server's normal activity. Alexa has been gathering this information since early 1996, and we have continually increased the amount of information that we gather, as you will see in the graph below.

Finding Patterns in Data

The Alexa service is derived from the data they collect. They use a number of techniques to find information about web sites to create our Site Data and Related Links features.They also use the collection to offer access to dead links that are no longer available on the Web.

Data Components of the Alexa Service

As an Alexa user navigates to a web site or web page, the Alexa service retrieves data about the page the user is requesting. They gather information from a number of sources, then organize it to be presented by the Alexa Service, helping the user to be better informed about the site he/she is viewing.

The service lists who registered the domain name and when they registered it, giving our users an idea of who is behind the site they are viewing. By analysing the data we receive from our updated snapshots of the Web, they can tell our users how large individual web sites are, how their traffic ranks in comparison to other web sites, how many other sites link to that site, and how fast the site is. They can also derive a popularity rating from our user base and share that rating with our community of users.

Our goal for the "Site Stats" feature of Alexa is to help our users navigate the Web more efficiently by giving them all the information they need to make informed decisions about the sites they visit.

Related Links

Whenever an Alexa user visits a web page, the Alexa service retrieves information from the Alexa servers to suggest other pages that might be of interest to the user. To find Related Links, they use several techniques, including:

The usage paths of the collective Alexa community- this is the most important source of their information, since these paths show them which web sites our users believe are important and interesting.
Clustering - the hundreds of millions of links on the Web can be used to find clusters of sites that are similar and relevant to one another. They mine this data by using custom databases to find and identify these clusters.
Text analysis - we compare the text on web pages to find related sites.
Users' suggestions - they consider their users' suggestions to augment our Related Links recommendations. Many of our users help point the service to similar pages that are relevant and interesting.

Every time a user visits a web page, Alexa checks to see if a copy of the page exists in the Archive. If the Alexa servers do have the page, the Archive link on the toolbar is blue and clickable.
If a user receives a "404 Not Found" message, or if the original page is not available, the user can click on this link and request an archived version of the page from the Alexa servers. To provide "out of print" web documents and graphics, they use the previous snapshots of the Web that they have collected and archived. When a document is no longer available, or a web server is down, they are often able to supply Alexa users with the information they are looking for.

The Alexa Toolbar

The Alexa toolbar is created by a program running on the user's personal computer called the "client program." The client can be a separate application that communicates with the browser to determine when a new URL is requested or an integrated component of the browser. Every time the user changes pages, the Alexa client communicates with Alexa's servers to retrieve the data it displays. Since the Alexa service works in parallel with the browser, it does not interfere with the browser's operation. Alexa starts up when the computer starts and exits when the machine is shut down.

Source (will open in a new window)

Back to Alexa ranking