When I make a request with Go’s net/http/client
package, and the server responds with 301 or 302 status code, but doesn’t provide a Location
header, we get an error object, and no response object. Example ReplIt session: https://replit.com/@sharat87/RedirectWithoutLocationGo#main.go. Including source here in case that session goes away in the future:
package main
import (
"log"
"net/http"
)
func main() {
client := &http.Client{}
resp, err := client.Get("https://httpbun.com/status/301")
if err != nil {
log.Println("Error getting", err)
} else {
log.Println(resp.Status)
}
}
This prints the error message: Error getting Get "https://httpbun.com/status/301": 301 response missing Location header
.
However, the Location header in 301 or 302 status responses is optional. Servers are not required to provide it, and clients are not obliged to redirect to it. In fact, in my case, I need to look at the other headers as well as the response body for what I need. But when the Location
header is missing, Go just returns an error and discards the whole response. For example, curl
doesn’t throw an error in this case, and actually responds with the other headers:
$ curl -v https://httpbun.com/status/301
[... TLS logs redacted]
> GET /status/301 HTTP/1.1
> Host: httpbun.com
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Server: nginx/1.18.0 (Ubuntu)
< Date: Sun, 25 Jul 2021 02:18:53 GMT
< Content-Type: text/plain; charset=utf-8
< Content-Length: 21
< Connection: keep-alive
< X-Powered-By: httpbun
<
I dug into the source for the net/http/client
module to try and see what might be happening. The error message comes up from this line in the source. Here, for the redirected request, we first check if a Location
header is available, and if not, we are returning an error. This is very limiting especially because the spec doesn’t describe this to be an error condition and in fact, lists a legitimate reason for doing this (i.e., the server doesn’t know where to redirect to) (reference, links to sources).
So, my question is, is this intended? Am I missing something here? Would a PR to change this be invited? (Asking here because looks like Golang repo issues on GitHub are reserved for proposals?).
I’d love any advise here since I’m essentially blocked here.
Edit: Fixed in https://github.com/golang/go/issues/49281.
2
Answers
The default client follows redirects. You can bypass this by using
RoundTrip directly:
Create a transport wrapper that fixes up the response for the client. The following example sets the location header when the header is missing. Another option is to change the status code.
Use a http.Client with the wrapped transport. Here’s how to create a wrapper for the net/http default client: