Avoiding Google Captcha and Geolocation Issues


Avoiding Google Captcha and Geolocation Issues

Google search is the world's most popular search engine with billions of web searches performed every day. Google also provides a suite of productivity apps, G Suite, along with many other consumer tools, such as Maps and Gmail. By default, the Google search home page and other Google Enterprise services are accessed over an SSL connection.

Sporadically, Zscaler customers have reported the following issues while accessing Google Search services through the Zscaler platform:

  • Captcha on google.com:  When a user types a search string in google.com, instead of displaying search results, users are subject to a Captcha screen that they must resolve before the search results are displayed.
  • Geolocation error: The google.com home page and search results displayed do not match with user's location. For example, a user in Miami might be shown search results in Spanish and the Google homepage is https://google.com.mx

Zscaler has taken multiple steps to mitigate these issues:

  • Google and Zscaler are in constant communication regarding Zscaler's multi-tenant data center footprint and IP address ranges.  
  • The Zscaler Operations team performs automated monitoring for Google Captcha and geolocation issues.
  • Zscaler has an agreement with Google to use XFF headers to isolate offending Zscaler clients.

Resolving Google Captcha Issues

Google displays a Captcha screen when it needs to verify that a search query is being performed by an actual user and not by an automated bot or program with malicious intent. Google has built a proprietary algorithm to detect such behavior. While Zscaler forbids the use of its proxy services for automated testing as part of its contract, some customers doing automated queries on Google trigger the search engine to respond with a Captcha. This can affect other users sending traffic through the same Zscaler IP address.

Zscaler recommends that customers enable SSL inspection, so Zscaler can insert an XFF header in each request. Inserting an XFF header allows Google to isolate the offending client, without affecting the traffic of other clients.

In addition to working with Google, Zscaler has also added monitoring capabilities to dynamically detect Google Captcha on our Zscaler Enforcement Nodes (ZENs). To do this, Zscaler added a signature pattern to detect Google Captcha incidents and the Zscaler Operations team is alerted in real-time when a Captcha incident happens. When a Captcha is detected, Zscaler might add IP addresses to the affected ZEN or rotate IP addresses to reduce the likelihood of this issue occurring again.

Resolving Geolocation Issues

Geolocation errors might occur when the Zscaler data center IP address ranges are incorrectly interpreted by Google's Geo-IP mapping system. This might lead to an IP address being incorrectly mapped to a different region. For example, an IP address in Miami might map to an IP address in Mexico City.

To resolve this issue, Zscaler has shared its IP ranges with Google and barring a rare corruption of the Geo-IP system, this issue is fixed.

Also, in certain regions, geolocation errors might be attributed to the fact that Zscaler customers are sending data to Zscaler from countries where Zscaler doesn't have a data center. In this case, Google returns content based on the IP address of the Zscaler data center.

To resolve this issue Zscaler recommends that customers turn on SSL inspection. This allows Google to read a customer's actual IP address along with the Zscaler data center IP address and return geolocalized content. Customers who cannot enable SSL might run into this issue.  Additionally, to avoid as geolocation errors, you might ask your users to use Google's NCR plugin for the Chrome browser to bookmark google.com/ncr to redirect your traffic to google.com instead of a local, geolocalized Google page.

You can go to https://support.google.com/websearch/contact/ip to log any inconsistencies with Geo-IP lookups directly with Google.

The suggestions provided in this article reduces the likelihood that these issues will occur, but they might not completely resolve them. Zscaler will continue to work with Google to resolve these issues permanently.