- 论坛徽章:
- 0
|
在用golang做模拟登录某个网站的功能时发现了一个问题:如何获取该网站带有重定向信息的Response?或者说我只需要
这个Response中的location信息即可,但是发现对于go的标准库net/http来说如果不hack这个库是没办法直接获取带有重
定向信息的Response,而且目前大多第三方的http库也是基于标准库net/http的,貌似就要改net/http库或库本身就提供了
对于重定向的设置?
那么我们就得先查看一下net/http库的源码了,通常我们都会用http.Client的Do方法发起一个请求,首先看一下它的源码:- // Do sends an HTTP request and returns an HTTP response, following
- // policy (e.g. redirects, cookies, auth) as configured on the client.
- //
- // An error is returned if caused by client policy (such as
- // CheckRedirect), or if there was an HTTP protocol error.
- // A non-2xx response doesn't cause an error.
- //
- // When err is nil, resp always contains a non-nil resp.Body.
- //
- // Callers should close resp.Body when done reading from it. If
- // resp.Body is not closed, the Client's underlying RoundTripper
- // (typically Transport) may not be able to re-use a persistent TCP
- // connection to the server for a subsequent "keep-alive" request.
- //
- // The request Body, if non-nil, will be closed by the underlying
- // Transport, even on errors.
- //
- // Generally Get, Post, or PostForm will be used instead of Do.
- func (c *Client) Do(req *Request) (resp *Response, err error) {
- if req.Method == "GET" || req.Method == "HEAD" {
- return c.doFollowingRedirects(req, shouldRedirectGet)
- }
- if req.Method == "POST" || req.Method == "PUT" {
- return c.doFollowingRedirects(req, shouldRedirectPost)
- }
- return c.send(req)
- }
复制代码 由源码的上方的注释可发现Do方法是不会返回有重定向信息的Response的,只会返回已经重定向跳转成功后的Response,而且会根据是否CheckRedirect返回error云云
那么假设我们要发起的是一个Get请求,所以进入c.doFollowingRedirects()方法中去看看- func (c *Client) doFollowingRedirects(ireq *Request, shouldRedirect func(int) bool) (resp *Response, err error) {
- var base *url.URL
- redirectChecker := c.CheckRedirect
- if redirectChecker == nil {
- redirectChecker = defaultCheckRedirect
- }
- var via []*Request
-
- if ireq.URL == nil {
- ireq.closeBody()
- return nil, errors.New("http: nil Request.URL")
- }
-
- var reqmu sync.Mutex // guards req
- req := ireq
-
- var timer *time.Timer
- if c.Timeout > 0 {
- type canceler interface {
- CancelRequest(*Request)
- }
- tr, ok := c.transport().(canceler)
- if !ok {
- return nil, fmt.Errorf("net/http: Client Transport of type %T doesn't support CancelRequest; Timeout not supported", c.transport())
- }
- timer = time.AfterFunc(c.Timeout, func() {
- reqmu.Lock()
- defer reqmu.Unlock()
- tr.CancelRequest(req)
- })
- }
-
- urlStr := "" // next relative or absolute URL to fetch (after first request)
- redirectFailed := false
- for redirect := 0; ; redirect++ {
- if redirect != 0 {
- nreq := new(Request)
- nreq.Method = ireq.Method
- if ireq.Method == "POST" || ireq.Method == "PUT" {
- nreq.Method = "GET"
- }
- nreq.Header = make(Header)
- nreq.URL, err = base.Parse(urlStr)
- if err != nil {
- break
- }
- if len(via) > 0 {
- // Add the Referer header.
- lastReq := via[len(via)-1]
- if ref := refererForURL(lastReq.URL, nreq.URL); ref != "" {
- nreq.Header.Set("Referer", ref)
- }
-
- err = redirectChecker(nreq, via)
- if err != nil {
- redirectFailed = true
- break
- }
- }
- reqmu.Lock()
- req = nreq
- reqmu.Unlock()
- }
-
- urlStr = req.URL.String()
- if resp, err = c.send(req); err != nil {
- break
- }
-
-
- if shouldRedirect(resp.StatusCode) {
-
- // Read the body if small so underlying TCP connection will be re-used.
- // No need to check for errors: if it fails, Transport won't reuse it anyway.
- const maxBodySlurpSize = 2 << 10
- if resp.ContentLength == -1 || resp.ContentLength <= maxBodySlurpSize {
- io.CopyN(ioutil.Discard, resp.Body, maxBodySlurpSize)
- }
- resp.Body.Close()
- if urlStr = resp.Header.Get("Location"); urlStr == "" {
- err = errors.New(fmt.Sprintf("%d response missing Location header", resp.StatusCode))
- break
- }
-
- base = req.URL
- via = append(via, req)
- continue
- }
- if timer != nil {
- resp.Body = &cancelTimerBody{timer, resp.Body}
- }
- return resp, nil
- }
-
- method := ireq.Method
- urlErr := &url .Error{
- Op: method[0:1] + strings.ToLower(method[1:]),
- URL: urlStr,
- Err: err,
- }
-
- if redirectFailed {
- // Special case for Go 1 compatibility: return both the response
- // and an error if the CheckRedirect function failed.
- // See http://golang.org/issue/3795
- return resp, urlErr
- }
-
- if resp != nil {
- resp.Body.Close()
- }
- return nil, urlErr
- }
复制代码 在上述代码中发现了有检查是否进行重定向的代码:- redirectChecker := c.CheckRedirect
- if redirectChecker == nil {
- redirectChecker = defaultCheckRedirect
- }
复制代码 redirectChecker是一个这样的函数:func(req Request, via []Request) error
发现了可在client中设置自己的redirectChecker,就是只要实现了func(req Request, via []Request) error即可,其功能是由源码的
defaultCheckRedirect可知是控制重定向跳转的次数而已。- func defaultCheckRedirect(req *Request, via []*Request) error {
- if len(via) >= 10 {
- return errors.New("stopped after 10 redirects")
- }
- return nil
- }
复制代码 defaultCheckRedirect中规定重定向跳转不能超过10次,否则返回error,那么如果我们要禁止跳转重定向的话自己实现一个
CheckRedirect,把其中的10改成0即可。但是即使这样设置后也是不能返回有重定向信息的Response,而是返回一个跳转停
止的error,不过如果你需要的只是带有重定向信息Response中的location信息的话只需要从返回的error中提取即可。http库
client中的doFollowingRedirects中传入了一个shouldRedirect函数,这个函数正是根据各种http协议的代码返回是否进行跳
转的信息,所以如果不改动这部分代码没法直接获取有重定向信息的Response,这样看来http库的设置不够灵活,得自己改或
另外实现了。 |
|