Thread: Spider Simulators can't see site

We have tested more than a few spider simulator tools that can't view our new site. Testing a competitors site in the same tool seems fine. We are also having a problem with Google Adwords giving us a message for keywords "Rarely shown due to low quality score".

We have plenty of keywords and understand all about text vs images but somthing in our setup must be breaking the site from an SEO point of view. The Robot.txt is default and this is a V9013 store. Anyone recognize these telltale signs.

I am not sure what to do and without Adwords we are very limited.

Thanks...

MSD

Hi,

What's your site URL and I'll take a look?

Ok, we found the error to be in the /App_Code/SkinBase.cs file in the CheckUserAgentForMobile method: (see the Compunix header/fix below in bold) - basically the UserAgent object is null with a spider/crawler


public bool CheckUserAgentForMobile
{
get
{
using (SqlConnection conn = new SqlConnection(DB.GetDBConn()))
{
conn.Open();

//COMPUNIX String usrAgent = HttpContext.Current.Request.UserAgent.ToLowerInvar iant();
String usrAgent = "";
try
{
usrAgent = HttpContext.Current.Request.UserAgent.ToLowerInvar iant();
}
catch { }

using (IDataReader rs = DB.GetRS("select UserAgent from dbo.MobileDevice with(NOLOCK)", conn))
{
while (rs.Read())
{
String mDevice = DB.RSField(rs, "UserAgent").ToLowerInvariant();

if (usrAgent.Contains(mDevice))
{
rs.Close();
rs.Dispose();

return true;
}
}
}
conn.Close();
conn.Dispose();
}

return false;
}
}

Nice catch Jan!