Thread: Google Site Search

Hi,
Im implemening search for my site using google site search API. We use a search button and paging control for our search page, and on search button event handler i create the url, exceute the request and get xml response as follows,
private string urlFormat = "https://www.googleapis.com/customsearch/v1?key={0}&cx={1}&q={2}&alt=atom&start={3}";
int startIndex = 1;
try
{
string url = string.Format(this.urlFormat,
HttpUtility.UrlEncode("AIzaSyA4TJfARVSIT6R00f8Eavs 51BUcuv1K1Xc"),
HttpUtility.UrlEncode("009064404879215115542:vqh-zxduchq"),
HttpUtility.UrlEncode("privacy"),
HttpUtility.UrlEncode(startIndex.ToString())
);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Method = WebRequestMethods.Http.Get;
request.Accept = "application/json";
request.ContentType = "application/json; charset=utf-8";
// Execute the request and obtain the response stream
WebResponse response = request.GetResponse();
using (var sr = new StreamReader(response.GetResponseStream()))
{
this.Output.Text = sr.ReadToEnd();
}

When user clicks on the any of the pagenumber links, we execute the same above code, changing the startindex as
startIndex = (PgNo * PageCount) – 9
ex: if user clicks the pagenumber 4, i calculate the start index as (4 * 10) – 9 = 31

when i execute the results first time(with start vlaue as 1, it shows total results as 80 and when i click on 5th page(start index 41), it shows only 2 results, concluding the actual total results are only 42.

Please help with these wierd totalresults..im attaching my response here..

<?xml version="1.0" encoding="UTF-8"?>

<feed gd:kind="customsearch#search" xmlns="http://www.w3.org/2005/Atom" xmlns:cse="http://schemas.google.com/cseapi/2010" xmlns:gd="http://schemas.google.com/g/2005" xmlnspensearch="http://a9.com/-/spec/opensearch/1.1/">

<title>Google Custom Search - privacy</title>

<id>tag:www.googleapis.com,2010-09-29:/customsearch/v1?q=privacy&amp;cx=009064404879215115542:vqh-zxduchq&amp;num=2&amp;start=41&amp;safe=off</id>

<author>

<name>Dev test - Google Custom Search</name>

</author>

<updated>2011-06-15T21:13:22.903Z</updated>

<opensearch:Url type="application/atom+xml" template="https://www.googleapis.com/customsearch/v1?q={searchTerms}&amp;num={count?}&amp;start={sta rtIndex?}&amp;hr={language?}&amp;safe={cse:safe?}& amp;cx={cse:cx?}&amp;cref={cse:cref?}&amp;sort={cs e:sort?}&amp;alt=atom"/>

<opensearch:Query role="request" title="Google Custom Search - privacy" totalResults="42" searchTerms="privacy" count="2" startIndex="41" inputEncoding="utf8" outputEncoding="utf8" cse:safe="off" cse:cx="009064404879215115542:vqh-zxduchq"/>

<opensearch:Query role="csereviousPage" title="Google Custom Search - privacy" totalResults="42" searchTerms="privacy" count="10" startIndex="31" inputEncoding="utf8" outputEncoding="utf8" cse:safe="off" cse:cx="009064404879215115542:vqh-zxduchq"/>

We use the same kind of system as yourself on our searchadv page with Google.

We have ran into this problem a few times. The problem is with Google. We can get say 40 results returned yet there are only 24 results. I can carry on clicking pages all the way to page 5 where nothing will be returned. On the 3 pages page before the end they will all just show the same 4 results.

Sadly to make this method completely fool prove you need to not only look at what Google says it is returning but also what is being returned.

I highly suggest having a look at BFG's website 3M Select, they use Google for their search and it is extremely impressive, however a ton of work has gone into that.

Google search is not always for everyone, sometimes it is better to improve the ASPDNSF search, but that is going to be up to you.