当前位置: 代码迷 >> Java Web开发 >> 运用HttpURLConnection抓取google搜索结果,总是报错:java.net.SocketException: Connection reset
  详细解决方案

运用HttpURLConnection抓取google搜索结果,总是报错:java.net.SocketException: Connection reset

热度:10581   发布时间:2013-02-25 21:14:34.0
使用HttpURLConnection抓取google搜索结果,总是报错:java.net.SocketException: Connection reset
抓取百度或者其他网站数据没问题,可是谷歌的总是报错:java.net.SocketException: Connection reset


抓取代码如下:
  HttpURLConnection connection = null;
BufferedInputStream in = null;
BufferedReader read = null;

try {
URL url = new URL(strURL); 

String cookie = "";
int iii=0;
do {
iii++;
connection = (HttpURLConnection)url.openConnection();
System.out.println("11111111:"+strURL);
Thread.sleep(5000);
if(cookie.length() != 0) {
connection.setRequestProperty("Cookie", cookie);
}

connection.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 8.0)");
connection.setInstanceFollowRedirects(false);
System.out.println("9999999999"+connection.getHeaderField("Set-Cookie"));
int code = connection.getResponseCode();

if(code == HttpURLConnection.HTTP_MOVED_TEMP) {
cookie += connection.getHeaderField("Set-Cookie") + ";";
}

if((connection.getResponseCode() == HttpURLConnection.HTTP_OK) || iii==4)
break;

}
while(true);

in = new BufferedInputStream(connection.getInputStream());
read = new BufferedReader(new InputStreamReader(in,"GB2312"));

------解决方案--------------------------------------------------------
连接重置,意思是说你的链接设置非法。可能谷歌有特别的设置吧。
------解决方案--------------------------------------------------------
貌似要申请一个谷歌的一个账号吧 不然就会有这个问题
------解决方案--------------------------------------------------------
google是做Web服务起家的,你想用程序获取人家服务的结果,google能让你这么轻易拿劳动成果吗?

google map,google translate,google doc...几乎所有的google产品你都可以集成到你自己的web应用中,但是:要给钱!!!!

记得以前集成google map的地图搜索的时候,社区版还是免费的,但是一天内只能搜索20次....
------解决方案--------------------------------------------------------
不是吧。是不是你有代理没配什么的。我自己写的抓取的小例子,怎么抓都行啊。。。
  相关解决方案