End-to-end testing
In the last chapter we talked through the general pattern for how to unit test your HTTP handlers in isolation.
But — most of the time — your HTTP handlers aren’t actually used in isolation. So in this chapter we’re going to explain how to run end-to-end tests on your web application that encompass your routing, middleware and handlers. In most cases, end-to-end testing should give you more confidence that your application is working correctly than unit testing in isolation.
To illustrate this, we’ll adapt our TestPing function so that it runs an end-to-end test on our code. Specifically, we want the test to ensure that a GET /ping request to our application calls the ping handler function and results in a 200 OK status code and "OK" response body.
Essentially, we want to test that our application has a route like this:
| Route pattern | Handler | Action |
|---|---|---|
| … | … | … |
| GET /ping | ping | Return a 200 OK response |
Using httptest.Server
The key to end-to-end testing our application is the httptest.NewTLSServer() function, which spins up a httptest.Server instance that we can make HTTPS requests to.
The whole pattern is a bit too complicated to explain upfront, so it’s probably best to demonstrate by first writing the code and then we’ll talk through the details afterwards.
With that in mind, head back to your handlers_test.go file and update the TestPing test so that it looks like this:
package main import ( "bytes" "io" "log/slog" // New import "net/http" "net/http/httptest" "testing" "snippetbox.alexedwards.net/internal/assert" ) func TestPing(t *testing.T) { // Create a new instance of our application struct. For now, this just // contains a structured logger (which uses the slog.DiscardHandler handler // and will discard anything written to it with no action). app := &application{ logger: slog.New(slog.DiscardHandler), } // We then use the httptest.NewTLSServer() function to create a new test // server, passing in the value returned by our app.routes() method as the // handler for the server. This starts up an HTTPS server which listens on a // randomly-chosen port of your local machine for the duration of the test. // Notice that we defer a call to ts.Close() so that the server is shut down // when the test finishes. ts := httptest.NewTLSServer(app.routes()) defer ts.Close() // The network address that the test server is listening on is contained in // the ts.URL field. We can use this to construct a new HTTP request for the // GET /ping route. req, err := http.NewRequest(http.MethodGet, ts.URL+"/ping", nil) if err != nil { t.Fatal(err) } // Use the ts.Client().Do() method to execute the request against the test // server. This returns an http.Response struct containing the response. res, err := ts.Client().Do(req) if err != nil { t.Fatal(err) } defer res.Body.Close() // We can then check the value of the response status code and body using // the same pattern as before. assert.Equal(t, res.StatusCode, http.StatusOK) body, err := io.ReadAll(res.Body) if err != nil { t.Fatal(err) } body = bytes.TrimSpace(body) assert.Equal(t, string(body), "OK") }
There are a few things about this code to point out and discuss.
When we call
httptest.NewTLSServer()to initialize the test server we need to pass in anhttp.Handleras the parameter — and this handler is called each time the test server receives an HTTPS request. In our case, we’ve passed in the return value from ourapp.routes()method, meaning that a request to the test server will use all our real application routes, middleware and handlers.This is a big upside of the work that we did earlier in the book to isolate all our application routing in the
app.routes()method.Please note that if you’re testing an HTTP (not HTTPS) server you should use the
httptest.NewServer()function to create the test server instead.The
ts.Client()method returns the test server client — which has the typehttp.Client— and then we use thets.Client().Do()method to actually send the request to the test server. It’s possible to configure the client to tweak its behavior, and we’ll explain how to do that at the end of this chapter.You might be wondering why we have set the
loggerfield of ourapplicationstruct, but none of the other fields. The reason for this is that the logger is needed by thelogRequestandrecoverPanicmiddlewares, which are used by our application on every route. Trying to run this test without setting these two dependencies will result in a panic.
Anyway, let’s try out the new test:
$ go test ./cmd/web/
--- FAIL: TestPing (0.00s)
handlers_test.go:41: got 404; want 200
handlers_test.go:51: got: Not Found; want: OK
FAIL
FAIL snippetbox.alexedwards.net/cmd/web 0.007s
FAIL
If you’re following along, you should get a failure at this point.
We can see from the test output that the response from our GET /ping request has a 404 status code, rather than the 200 we expected. And that’s because we haven’t actually registered a GET /ping route with our router yet.
Let’s fix that now:
package main ... func (app *application) routes() http.Handler { mux := http.NewServeMux() mux.Handle("GET /static/", http.FileServerFS(ui.Files)) // Add a new GET /ping route. mux.HandleFunc("GET /ping", ping) dynamic := alice.New(app.sessionManager.LoadAndSave, preventCSRF, app.authenticate) mux.Handle("GET /{$}", dynamic.ThenFunc(app.home)) mux.Handle("GET /snippet/view/{id}", dynamic.ThenFunc(app.snippetView)) mux.Handle("GET /user/signup", dynamic.ThenFunc(app.userSignup)) mux.Handle("POST /user/signup", dynamic.ThenFunc(app.userSignupPost)) mux.Handle("GET /user/login", dynamic.ThenFunc(app.userLogin)) mux.Handle("POST /user/login", dynamic.ThenFunc(app.userLoginPost)) protected := dynamic.Append(app.requireAuthentication) mux.Handle("GET /snippet/create", protected.ThenFunc(app.snippetCreate)) mux.Handle("POST /snippet/create", protected.ThenFunc(app.snippetCreatePost)) mux.Handle("POST /user/logout", protected.ThenFunc(app.userLogoutPost)) standard := alice.New(app.recoverPanic, app.logRequest, commonHeaders) return standard.Then(mux) }
And if you run the tests again everything should now pass.
$ go test ./cmd/web/ ok snippetbox.alexedwards.net/cmd/web 0.008s
Using test helpers
Our TestPing test is now working nicely. But there’s a good opportunity to break out some of this code into helper functions, which we can reuse as we add more end-to-end tests to our project.
There’s no hard-and-fast rules about where to put helper methods for tests. If a helper is only used in a specific *_test.go file, then it probably makes sense to include it inline in that file alongside your tests. At the other end of the spectrum, if you plan to use a helper in tests across multiple packages, then you might want to put it in a reusable package called internal/testutils (or similar) which can be imported by your test files.
In our case, the helpers will be used for testing code throughout our cmd/web package but nowhere else, so it seems reasonable to put them in a new cmd/web/testutils_test.go file.
If you’re following along, please go ahead and create this now…
$ touch cmd/web/testutils_test.go
And then add the following code:
package main import ( "bytes" "io" "log/slog" "net/http" "net/http/httptest" "testing" ) // Create a newTestApplication helper which returns an instance of our // application struct containing mocked dependencies. func newTestApplication(t *testing.T) *application { return &application{ logger: slog.New(slog.DiscardHandler), } } // Define a custom testServer type which embeds an httptest.Server instance. type testServer struct { *httptest.Server } // Create a newTestServer helper which initializes and returns a new instance // of our custom testServer type. func newTestServer(t *testing.T, h http.Handler) *testServer { ts := httptest.NewTLSServer(h) return &testServer{ts} } // Define a testResponse struct to hold data about responses from the test // server. Note that this struct includes fields for the HTTP response headers // and cookies, as well as the status code and body. type testResponse struct { status int headers http.Header cookies []*http.Cookie body string } // Implement a get() method on our custom testServer type. This makes a GET // request to a given url path using the test server client and it returns a // testResponse struct containing the response data. func (ts *testServer) get(t *testing.T, urlPath string) testResponse { req, err := http.NewRequest(http.MethodGet, ts.URL+urlPath, nil) if err != nil { t.Fatal(err) } res, err := ts.Client().Do(req) if err != nil { t.Fatal(err) } defer res.Body.Close() body, err := io.ReadAll(res.Body) if err != nil { t.Fatal(err) } return testResponse{ status: res.StatusCode, headers: res.Header, cookies: res.Cookies(), body: string(bytes.TrimSpace(body)), } }
By and large, these helpers are just a generalization of the code we’ve already written in this chapter for spinning up a test server and making a GET request against it. The main difference is the addition of the testResponse struct, which acts as a convenient single container for the response details we’re likely to want to check in our tests.
Let’s head back to our TestPing handler and put these new helpers to work:
package main import ( "net/http" "testing" "snippetbox.alexedwards.net/internal/assert" ) func TestPing(t *testing.T) { app := newTestApplication(t) ts := newTestServer(t, app.routes()) defer ts.Close() res := ts.get(t, "/ping") assert.Equal(t, res.status, http.StatusOK) assert.Equal(t, res.body, "OK") }
And, again, if you run the tests again everything should still pass.
$ go test ./cmd/web/ ok snippetbox.alexedwards.net/cmd/web 0.013s
This is shaping up nicely now. We have a neat pattern in place for spinning up a test server and making requests to it, encompassing our routing, middleware and handlers in an end-to-end test. We’ve also broken apart some of the code into helpers, which will make writing future tests quicker and easier.
Cookies and redirects
So far in this chapter we’ve been using the default test server client settings. But to make it better suited for testing our web application, there are a couple of changes we need to make.
By default, the test server client does not persist cookies across multiple requests. This will become a problem later in the book when we need to test functionality — like our anti-CSRF measures — that rely on cookie persistence. To change this, we can customize the client to use a non-nil
cookiejar.Jar, which will then allow cookies to be stored and reused across requests. While we’re at it, we’ll also add a helper method that we can use in our tests to reset the cookie jar if we need to.By default, the test server client automatically follows redirects. But in our tests, we normally want to check the initial response sent by our server — the one the server sends before any redirect takes place. To do that, we can change the default behavior by customizing the client’s
CheckRedirectfield.
Let’s go back to the testutils_test.go file we just created and update the code like so:
package main import ( "bytes" "io" "log/slog" "net/http" "net/http/cookiejar" // New import "net/http/httptest" "testing" ) ... func newTestServer(t *testing.T, h http.Handler) *testServer { // Initialize the test server as normal. ts := httptest.NewTLSServer(h) // Initialize a new cookie jar. jar, err := cookiejar.New(nil) if err != nil { t.Fatal(err) } // Add the cookie jar to the test server client. Any response cookies will // now be stored in the jar and sent with subsequent requests when using // this client. ts.Client().Jar = jar // Prevent the test server client from following redirects by setting a // custom CheckRedirect function. This function runs whenever a 3xx // response is received. By returning http.ErrUseLastResponse, it tells // the client to stop and immediately return the received response. ts.Client().CheckRedirect = func(req *http.Request, via []*http.Request) error { return http.ErrUseLastResponse } return &testServer{ts} } // And we also add a helper function to reset the test server client to use a // new and empty cookie jar. func (ts *testServer) resetClientCookieJar(t *testing.T) { jar, err := cookiejar.New(nil) if err != nil { t.Fatal(err) } ts.Client().Jar = jar } ...