Let's Go Testing › End-to-end testing
Previous · Contents · Next
Chapter 13.3.

End-to-end testing

In the last chapter we talked through the general pattern for how to unit test your HTTP handlers in isolation.

But — most of the time — your HTTP handlers aren’t actually used in isolation. So in this chapter we’re going to explain how to run end-to-end tests on your web application that encompass your routing, middleware and handlers. In most cases, end-to-end testing should give you more confidence that your application is working correctly than unit testing in isolation.

To illustrate this, we’ll adapt our TestPing function so that it runs an end-to-end test on our code. Specifically, we want the test to ensure that a GET /ping request to our application calls the ping handler function and results in a 200 OK status code and "OK" response body.

Essentially, we want to test that our application has a route like this:

Route pattern Handler Action
GET /ping ping Return a 200 OK response

Using httptest.Server

The key to end-to-end testing our application is the httptest.NewTLSServer() function, which spins up a httptest.Server instance that we can make HTTPS requests to.

The whole pattern is a bit too complicated to explain upfront, so it’s probably best to demonstrate by first writing the code and then we’ll talk through the details afterwards.

With that in mind, head back to your handlers_test.go file and update the TestPing test so that it looks like this:

File: cmd/web/handlers_test.go
package main

import (
    "bytes"
    "io"
    "log/slog" // New import
    "net/http"
    "net/http/httptest"
    "testing"

    "snippetbox.alexedwards.net/internal/assert"
)

func TestPing(t *testing.T) {
    // Create a new instance of our application struct. For now, this just
    // contains a structured logger (which uses the slog.DiscardHandler handler
    // and will discard anything written to it with no action).
    app := &application{
        logger: slog.New(slog.DiscardHandler),
    }

    // We then use the httptest.NewTLSServer() function to create a new test
    // server, passing in the value returned by our app.routes() method as the
    // handler for the server. This starts up an HTTPS server which listens on a
    // randomly-chosen port of your local machine for the duration of the test.
    // Notice that we defer a call to ts.Close() so that the server is shut down
    // when the test finishes.
    ts := httptest.NewTLSServer(app.routes())
    defer ts.Close()

    // The network address that the test server is listening on is contained in
    // the ts.URL field. We can use this to construct a new HTTP request for the 
    // GET /ping route.
    req, err := http.NewRequest(http.MethodGet, ts.URL+"/ping", nil)
    if err != nil {
        t.Fatal(err)
    }

    // Use the ts.Client().Do() method to execute the request against the test 
    // server. This returns an http.Response struct containing the response.
    res, err := ts.Client().Do(req)
    if err != nil {
        t.Fatal(err)
    }
    defer res.Body.Close()

    // We can then check the value of the response status code and body using
    // the same pattern as before.
    assert.Equal(t, res.StatusCode, http.StatusOK)

    body, err := io.ReadAll(res.Body)
    if err != nil {
        t.Fatal(err)
    }
    body = bytes.TrimSpace(body)

    assert.Equal(t, string(body), "OK")
}

There are a few things about this code to point out and discuss.

Anyway, let’s try out the new test:

$ go test ./cmd/web/
--- FAIL: TestPing (0.00s)
    handlers_test.go:41: got 404; want 200
    handlers_test.go:51: got: Not Found; want: OK
FAIL
FAIL    snippetbox.alexedwards.net/cmd/web      0.007s
FAIL

If you’re following along, you should get a failure at this point.

We can see from the test output that the response from our GET /ping request has a 404 status code, rather than the 200 we expected. And that’s because we haven’t actually registered a GET /ping route with our router yet.

Let’s fix that now:

File: cmd/web/routes.go
package main

...

func (app *application) routes() http.Handler {
    mux := http.NewServeMux()

    mux.Handle("GET /static/", http.FileServerFS(ui.Files))

    // Add a new GET /ping route.
    mux.HandleFunc("GET /ping", ping)

    dynamic := alice.New(app.sessionManager.LoadAndSave, preventCSRF, app.authenticate)

    mux.Handle("GET /{$}", dynamic.ThenFunc(app.home))
    mux.Handle("GET /snippet/view/{id}", dynamic.ThenFunc(app.snippetView))
    mux.Handle("GET /user/signup", dynamic.ThenFunc(app.userSignup))
    mux.Handle("POST /user/signup", dynamic.ThenFunc(app.userSignupPost))
    mux.Handle("GET /user/login", dynamic.ThenFunc(app.userLogin))
    mux.Handle("POST /user/login", dynamic.ThenFunc(app.userLoginPost))

    protected := dynamic.Append(app.requireAuthentication)

    mux.Handle("GET /snippet/create", protected.ThenFunc(app.snippetCreate))
    mux.Handle("POST /snippet/create", protected.ThenFunc(app.snippetCreatePost))
    mux.Handle("POST /user/logout", protected.ThenFunc(app.userLogoutPost))

    standard := alice.New(app.recoverPanic, app.logRequest, commonHeaders)
    return standard.Then(mux)
}

And if you run the tests again everything should now pass.

$ go test ./cmd/web/
ok      snippetbox.alexedwards.net/cmd/web    0.008s

Using test helpers

Our TestPing test is now working nicely. But there’s a good opportunity to break out some of this code into helper functions, which we can reuse as we add more end-to-end tests to our project.

There’s no hard-and-fast rules about where to put helper methods for tests. If a helper is only used in a specific *_test.go file, then it probably makes sense to include it inline in that file alongside your tests. At the other end of the spectrum, if you plan to use a helper in tests across multiple packages, then you might want to put it in a reusable package called internal/testutils (or similar) which can be imported by your test files.

In our case, the helpers will be used for testing code throughout our cmd/web package but nowhere else, so it seems reasonable to put them in a new cmd/web/testutils_test.go file.

If you’re following along, please go ahead and create this now…

$ touch cmd/web/testutils_test.go

And then add the following code:

File: cmd/web/testutils_test.go
package main

import (
    "bytes"
    "io"
    "log/slog"
    "net/http"
    "net/http/httptest"
    "testing"
)

// Create a newTestApplication helper which returns an instance of our
// application struct containing mocked dependencies.
func newTestApplication(t *testing.T) *application {
    return &application{
        logger: slog.New(slog.DiscardHandler),
    }
}

// Define a custom testServer type which embeds an httptest.Server instance.
type testServer struct {
    *httptest.Server
}

// Create a newTestServer helper which initializes and returns a new instance
// of our custom testServer type.
func newTestServer(t *testing.T, h http.Handler) *testServer {
    ts := httptest.NewTLSServer(h)
    return &testServer{ts}
}

// Define a testResponse struct to hold data about responses from the test 
// server. Note that this struct includes fields for the HTTP response headers 
// and cookies, as well as the status code and body.
type testResponse struct {
    status  int
    headers http.Header
    cookies []*http.Cookie
    body    string
}

// Implement a get() method on our custom testServer type. This makes a GET
// request to a given url path using the test server client and it returns a 
// testResponse struct containing the response data.
func (ts *testServer) get(t *testing.T, urlPath string) testResponse {
    req, err := http.NewRequest(http.MethodGet, ts.URL+urlPath, nil)
    if err != nil {
        t.Fatal(err)
    }

    res, err := ts.Client().Do(req)
    if err != nil {
        t.Fatal(err)
    }
    defer res.Body.Close()

    body, err := io.ReadAll(res.Body)
    if err != nil {
        t.Fatal(err)
    }

    return testResponse{
        status:  res.StatusCode,
        headers: res.Header,
        cookies: res.Cookies(),
        body:    string(bytes.TrimSpace(body)),
    }
}

By and large, these helpers are just a generalization of the code we’ve already written in this chapter for spinning up a test server and making a GET request against it. The main difference is the addition of the testResponse struct, which acts as a convenient single container for the response details we’re likely to want to check in our tests.

Let’s head back to our TestPing handler and put these new helpers to work:

File: cmd/web/handlers_test.go
package main

import (
    "net/http"
    "testing"

    "snippetbox.alexedwards.net/internal/assert"
)

func TestPing(t *testing.T) {
    app := newTestApplication(t)

    ts := newTestServer(t, app.routes())
    defer ts.Close()

    res := ts.get(t, "/ping")
    assert.Equal(t, res.status, http.StatusOK)
    assert.Equal(t, res.body, "OK")
}

And, again, if you run the tests again everything should still pass.

$ go test ./cmd/web/
ok      snippetbox.alexedwards.net/cmd/web    0.013s

This is shaping up nicely now. We have a neat pattern in place for spinning up a test server and making requests to it, encompassing our routing, middleware and handlers in an end-to-end test. We’ve also broken apart some of the code into helpers, which will make writing future tests quicker and easier.

Cookies and redirects

So far in this chapter we’ve been using the default test server client settings. But to make it better suited for testing our web application, there are a couple of changes we need to make.

Let’s go back to the testutils_test.go file we just created and update the code like so:

File: cmd/web/testutils_test.go
package main

import (
    "bytes"
    "io"
    "log/slog"
    "net/http"
    "net/http/cookiejar" // New import
    "net/http/httptest"
    "testing"
)

...

func newTestServer(t *testing.T, h http.Handler) *testServer {
    // Initialize the test server as normal.
    ts := httptest.NewTLSServer(h)

    // Initialize a new cookie jar.
    jar, err := cookiejar.New(nil)
    if err != nil {
        t.Fatal(err)
    }

    // Add the cookie jar to the test server client. Any response cookies will
    // now be stored in the jar and sent with subsequent requests when using 
    // this client.
    ts.Client().Jar = jar

    // Prevent the test server client from following redirects by setting a
    // custom CheckRedirect function. This function runs whenever a 3xx
    // response is received. By returning http.ErrUseLastResponse, it tells
    // the client to stop and immediately return the received response.
    ts.Client().CheckRedirect = func(req *http.Request, via []*http.Request) error {
        return http.ErrUseLastResponse
    }

    return &testServer{ts}
}

// And we also add a helper function to reset the test server client to use a 
// new and empty cookie jar.
func (ts *testServer) resetClientCookieJar(t *testing.T) {
	jar, err := cookiejar.New(nil)
	if err != nil {
		t.Fatal(err)
	}

	ts.Client().Jar = jar
}
...