Thread: Weird links in google crawl errors

We're experiencing these weird urls in our google crawl errors. They look like this:

Code:

http://www.bikersden.com/(f(deqysrspjtyc31mwliaf7z2mc1_l5on58lkqgjx_kbhzicau3y_iaa2fohwswr06iudlvjyqptqeoacrik7oibeofslzpxj5fpmbqylmfozvwnjbqwz9gext2qxgp3rhejw1rx_bgfpidxom-1qg7lalgljsixld8psvzuxdlo4snrmyahqousydybqksbgluotunflb_5oc2aslvlouu_6mohmbfxdxuiw681btue7y7bes0))/t-security.aspx

the end aspx page is different and are pages on our site. category pages, product pages, topic pages. Needless to say if you click on this url it goes nowhere of course.

We have experienced seeing these urls on rare occasions while browsing the site, as well, in the breadcrumbs displayed on the top of the site ...

where are these coming from? I can't even begin to figure out how to debug this because I have no idea how to reproduce it on demand.

I was coming to the forums to post another question, and I noticed your post. We are experiencing the same issue with the weird crawl errors. Below is an example of one from our site.

Code:

http://www.intechequipment.com/store/(f(y3q6w_c60rmyk3ojel49r9qbayz5g0wdqbe-z03lfo7qulmooo7mundgifq7ln5q4nztcr0c59s9iwvr3bryr5rowilp6zgtvnvnltf7pxt2v_xrb8rmrl6gluue02g0azoysmk_3jbwsdcnobkx6g3zrfs7tprmhhx-gszhmev-kh5ez3b9alchwhalvrc1y0ju7dyz1ycstelbrylzbw3yqqayvsc-3n2n9icovtpwzqks0))/c-5-roofing-equipment-parts-accessories.aspx

Everything with the site url is correct except the directory after /store/.

Let me know if you find a fix to this, and thanks in advance if you do.

Looks like its a session variable being stored within the URL when the site is being crawled.

One would think that, however I have first hand reports of it showing up during normal web browsing as well.

That looks like what the .NET platform will do to URLs if you have cookieless sessions enabled.

I've thought that before, and I checked and we definitely have it set to use cookies.

Same issue here. V9200

Do you have .browser files on your site that modify behavior based on the browser or user agent? The only times we've ever seen this happen have been when cookieless sessions are on, which can be done in the web.config, in the 'higher' .NET config files, in IIS, or 'forced' on particular sessions by .browser files.

Why in the world would it show up in google webmaster tools under crawl errors if it were a browser issue?

The reports we saw of this in real browsers, btw, were in breadcrumb links.

Mentioning a .browser files doesn't mean it's 'a browser issue' - those files can be modified to handle different user agents (such as the Googlebot agent) differently, which is what we've seen before. Once the customers removed the 'extra' .browser files that had been added to their App_Browsers folder (we only ship with one) or corrected the handling of different user agents, the issue went away.

Okay. I looked in the .browser files for the classic .net that this store is running on and am wondering what .browser file googlebot triggers. the Default.browser file has enabled cookies as a capability. I am assuming that is the issue with the weird session ID urls, the cookie capability?

Originally Posted by AspDotNetStorefront Staff - Scott

Mentioning a .browser files doesn't mean it's 'a browser issue' - those files can be modified to handle different user agents (such as the Googlebot agent) differently, which is what we've seen before. Once the customers removed the 'extra' .browser files that had been added to their App_Browsers folder (we only ship with one) or corrected the handling of different user agents, the issue went away.

So I checked the webconfig, and cookieless is set to "false", I also checked in the App_browser folder, and the only thing I see is "safari.browser" which I am 99.9% sure is the one that came with the original store. So if you do not mind Scott, what would be the next thing I need to check or what do I need to verify with our Hoster to get this resolved?

The issue is in the application.dll and they way sessions are crated; this is the file that CONTROLS THE ENTIRE session system in aspdotnetstorefront. Sadly the source is not available. We have the same issue on about 10-15 other sites. The support staff kept trying to say its a hosting environment issue or some code mods, even though BONE STOCK sites are having the same problem.

Google isn't any help either they keep saying its a problem with the site.

I'm currently hosted on a public cloud at SI and I'm being told that it may be a result of how the URL rewriting rules work due to the nature of the public infrastructure. It's been suggested that I move to a private cloud solution but that will be 3-4 times current cost. I have the APSC (auto parts shopping cart) extension on my site. I'd sure like to find a solution that allows me to stay on the public cloud.

Is there a solution for this issue. We are facing this issue for some of our clients.
I have checked and in web.config cookielss is set to false.

We fixed this issue by adding cookieless="UseCookies" to the web.config authentication mode

We are having the same problem.
We've seen this type of error in the Bing Webmaster Tools, Reports & Data, Crawl Information - some are 400 some are 414 some are 301.

Similar errors in our server event logs as well...

Exception type: PathTooLongException
Exception message: The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.

Request URL: http://www.portlandcompressor.com/st...-electric.aspx
Request path: /store/(f(ign5nzkcgcfcuctw2u-aqr_ey2bk4xsx2_nm9eh_sdsnb5akowswo8eqkxh-qfoswse47eho9onso6-davwl0iinxrs5mo6yqgpvtcqdbyz0bsgvamviob3fyxf5pf53f _-wvxiidixxsmdlffygrkd14iv2v1hrcr2squ4jrg6jeh8xqdky8 nekuexomod7y-8fj-6bj7qmjynnimo5acwlnbo1))/c-8-electric.aspx

As seen in this thread
http://forums.aspdotnetstorefront.co...and-over-again

It looks like the answer by baba should do the trick. After researching this it looks like cookieless="false" is .NET 1.1 where as cookieless="UseCookies" is .NET 2.0. I just updated our web.config - we'll see what happens.