写了一个抓取程序,抓一个网站的。跑了快半年了,一直正常运行,最近老是爆出这个问题,实在整不明白。请CSDN的各位大神解答一下:
24 17:18:52 - I/O exception (org.apache.http.NoHttpResponseException) caught when processing request: The target server failed to respond
24 17:18:52 - Retrying request
24 17:18:56 - I/O exception (org.apache.http.NoHttpResponseException) caught when processing request: The target server failed to respond
24 17:18:56 - Retrying request
24 17:18:56 - ===============
org.apache.http.NoHttpResponseException: The target server failed to respond
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:95) ~[tu2.jar:na]
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:61) ~[tu2.jar:na]
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254) ~[tu2.jar:na]
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289) ~[tu2.jar:na]
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252) ~[tu2.jar:na]
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191) ~[tu2.jar:na]
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300) ~[tu2.jar:na]
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127) ~[tu2.jar:na]
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:715) ~[tu2.jar:na]
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:520) ~[tu2.jar:na]
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) ~[tu2.jar:na]
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:827) ~[tu2.jar:na]
程序运行环境:
java version "1.7.0_45"
Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)
使用httpClient:
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.2.6</version>
</dependency>
访问方式:
约800个代理服务器
有一定可能性是代理服务器性能下降了,或者是对方主机访问量提升了。
但是无法达到突变效果:无法访问是突然情况。
这种情况出现的几率高达98%
------解决思路----------------------
一般情况下好像只有服务突然down了,才会出现这种异常,但是这么高的几率确实有点奇怪,不知道对方是采用什么手段来达到这情况,楼主有新的发现的话,可以分享一下。