🚧 Testing GraphQL applications with minimal mock data 🚧

There are three criteria for a great test:

  1. It's realistic, meaning it will fail in valuable ways
  2. It's easy to maintain (easy to read, runs fast)
  3. It's easy to write

Codegen tools & mocking tools in the GraphQL ecosystem have made it possible to write tests that are unbelievably good at #1 and #2. But #3 needs some love. In this blog post I'm gonna show how typechecking can make #3 easy peasy.

An example test

Say you've got a function that takes some data fetched from GraphQL as an argument:

export const GetCardsDocument = graphql("
    query GetCards {
        cards {
            id
            abilities {
                effect
                name
            }
        }
    }
")

export function app(data: ResultOf<typeof GetCardsDocument>) {
    return data.cards?.[0]?.abilities?.[0]?.effect
}

and you want to test this function, with something like:

import { expect, test } from 'vitest'
import { app, GetCardsDocument } from './app'
import { createMockData } from 'src/api/mock-data-utils/create-mock-data'

test('app', () => {
    const data = magicalFunctionThatCreatesExactlyTheMockDataIWant(GetCardsDocument, "paralyzed")
    const result = app(data)
    expect(result).toBe("paralyzed")
})

This is a short test! Sure, the example is simplistic, but to recap, this test:

  1. creates realistic mock data with a specific value that I'm going to test for
  2. calls a function with that data
  3. asserts that the function did the right thing with the value I specified (paralyzed)

That's a lot to do in only 3 lines of code--in fact, it's hard to imagine a shorter test; we've got one each for arrange / act / assert). For the three criteria I laid out earlier, it's a 10/10 for each one.

Welp, the whole point of this post is to explain how magicalFunctionThatCreatesExactlyTheMockDataIWant doesn't have to be magical, but it does take some effort to set up.

Implementing magicalFunctionThatCreatesExactlyTheMockDataIWant

We can use the @graphql-tools/mock package to write a function that, given a schema and a document node, outputs mock data:

import { buildASTSchema  } from "graphql"
import { addMocksToSchema } from "@graphql-tools/mock"
import SDL from "src/api/graphql/schema.graphql"

const schema = buildASTSchema(SDL)

export function magicalFunctionThatCreatesExactlyTheMockDataIWant(documentNode) {
  const executableSchema = addMocksToSchema({ schema })
  const result = executeSync({ schema: executableSchema, document: documentNode })
  return result.data
}

Alright, let's break down what this function is doing.

  1. We're importing a schema definition language document (as SDL) that describes our schema. In this case it's a schema file I generated by introspecting a public api for pokémon cards.
  2. Next, we're using buildASTSchema to parse the SDL into a GraphQLSchema which is the format that @graphql-tools/mock wants.
  3. Now we're ready to implement our function: we're creating an executable schema by feeding our schema to addMocksToSchema. Now we can run queries against this mocked schema and get realistic responses: any field we request in a query will receive a mock value of the type described by the schema. ie if we request a field called name, and it's typed as String in the GraphQL API that we used to create our schema, we'll get a response like { data: { myQuery: { name: "Hello, World" } } }
  4. Calling executeSync with our executable schema and the document node that describes our operation gives us a realistic response!

Straightforward enough, right?

Well, we haven't fulfilled our original promise--there's no way to add specific values to the mock response. Let's add that real quick:

import { buildASTSchema  } from "graphql"
import { addMocksToSchema } from "@graphql-tools/mock"
import SDL from "src/api/graphql/schema.graphql"

const schema = buildASTSchema(SDL)

export function magicalFunctionThatCreatesExactlyTheMockDataIWant(documentNode, mocks) {
  const executableSchema = addMocksToSchema({ schema, mocks })
  const result = executeSync({ schema: executableSchema, document: documentNode })
  return result.data
}

Is it really that easy? Kind of. The hard part about this function is typing it! addMocksToSchema's mocks argument is supposed to be a Resolvers object, which would look something like this:

const resolvers = {
 Query: () => ({ someQuery: () => ({ someField: () => generateRandomValue() })}),
 Mutation: () => ({ someMutation: () => ({ someField: () => generateRandomValue() })})
 Entity: () => ({ someField: () => generateRandomValue() })
}

NOTE: anything you don't define in the mock resolvers object is created automatically for you by addMocksToSchema. You only need to provide overrides for specific fields in your tests, e.g.

const resolvers = { Entity: () => ({ someField: "paralyzed" }) }

and addMocksToSchema takes care of the rest. Problem is, we're working on a client. We don't have resolvers on the client, so how are we supposed to write a type for magicalFunctionThatCreatesExactlyTheMockDataIWant's mocks argument?

Luckily, we can use code generation to create a resolvers type, even if no resolvers exist.

import { CodegenConfig } from "@graphql-codegen/cli"

export default {
    overwrite: true,
    schema: ["src/api/graphql/schema.graphql"],
    documents: ["src/**/*.graphql", "!src/api/*", "src/**/*.tsx", "src/**/*.ts"],
    generates: {
        "src/api/graphql/resolvers.ts": {
            plugins: ["typescript", "typescript-resolvers"],
        },
    },
} satisfies CodegenConfig

This codegen configuration will generate a resolvers type for us! Let's talk about what it's doing:

  1. schema: this should point to your SDL document (same one we used for created the executable schema), but it could also point to an API introspection endpoint.
  2. generates: each key in this object should be a path to a file you want to generate. In this case, we've got one key (realistic projects will have several) that leads to resolvers.ts
  3. plugins: for the resolvers.ts file we want to generate, we're using the typescript plugin first, which translates the SDL types to TypeScript types. Second, we're using the "typescript-resolvers" plugin to generate resolver types (it references the types from the "typescript" plugin to define its own types)

Alright! How do we use the Resolvers type generated in resolvers.ts to typecheck our magical function?

We could try naively applying it to the addMocksToSchema mocks parameter, but if we look carefully at addMocksToSchema's signature, we'll notice something:

export declare function addMocksToSchema<TResolvers = IResolvers>({
  schema, 
  store: maybeStore, 
  mocks, 
  typePolicies, 
  resolvers: resolversOrFnResolvers, 
  preserveResolvers
}: IMockOptions<TResolvers>): GraphQLSchema;

Ok, this is some fancy footwork--we've got to look at IMockOptions to understand what mocks is supposed to be.

type IMockOptions<TResolvers = IResolvers> = {
    schema: GraphQLSchema;
    store?: IMockStore;
    mocks?: IMocks<TResolvers>;
    typePolicies?: {
        [typeName: string]: TypePolicy;
    };
    resolvers?: Partial<TResolvers> | ((store: IMockStore) => Partial<TResolvers>);
    preserveResolvers?: boolean;
};

Ok, mocks is typed by two types:

  1. IMocks, which @graphql-tools/mock defines, and
  2. TResolvers, which is a generic type argument, and in this case that means we're supposed to provide it.

That's how our generated Resolvers type fits in:

import { buildASTSchema  } from "graphql"
import { addMocksToSchema, IMocks } from "@graphql-tools/mock"
import { Resolvers } from "src/api/graphql/resolvers"
import SDL from "src/api/graphql/schema.graphql"

const schema = buildASTSchema(SDL)

export function magicalFunctionThatCreatesExactlyTheMockDataIWant(documentNode, mocks: IMocks<Resolvers>) {
  const executableSchema = addMocksToSchema({ schema, mocks })
  const result = executeSync({ schema: executableSchema, document })
  return result.data
}

Cool! Now we have a type that tells us how to define the mocks object. Let's try applying that to our test:

import { expect, test } from 'vitest'
import { app, GetCardsDocument } from './app'
import { createMockData } from 'src/api/mock-data-utils/create-mock-data'

test('app', () => {
    const data = magicalFunctionThatCreatesExactlyTheMockDataIWant(GetCardsDocument, { AbilitiesListItem: { effect: () => "paralyzed" } })
    const result = app(data)
    expect(result).toBe("paralyzed")
})

Holy shit! This actually works, and it's still only three lines (if slightly more complex). Let's break down what's happening:

  1. We're providing our mock data function with a DocumentNode just as before
  2. In our mocks object, we're defining an entity resolver for AbilitiesListItem, to ensure its effect property has the specific value we're looking for in our test.
  3. When we call that function, it produces an object that looks like this:
{
  "cards": [
    {
      "id": "Hello World",
      "abilities": [
        {
          "effect": "paralyzed",
          "name": "Hello World",
          "__typename": "AbilitiesListItem"
        },
        {
          "effect": "paralyzed",
          "name": "Hello World",
          "__typename": "AbilitiesListItem"
        }
      ],
      "__typename": "Card"
    },
    {
      "id": "Hello World",
      "abilities": [
        {
          "effect": "paralyzed",
          "name": "Hello World",
          "__typename": "AbilitiesListItem"
        },
        {
          "effect": "paralyzed",
          "name": "Hello World",
          "__typename": "AbilitiesListItem"
        }
      ],
      "__typename": "Card"
    }
  ]
}

You're probably thinking "whoa, that's a lot of stuff!". It is! Let's also break down why there's so much data here!

  1. First we see the cards array property, which corresponds to the cards query in the GraphQL API.
  2. The cards array has 2 entries in it. Why? This is the default behavior for @graphql-tools/mock. When it encounters an array field, it produces an array of length 2 by default. You can change this.
  3. We see "Hello World" being used a lot--this is the default value that @graphql-tools/mock uses for any String field it encounters.
  4. effect has our "paralyzed" value just as we wanted!

Hooray! That was a lot work, but we wrote a dream interface for writing tests. Are we done? Kind of.

Let's try writing another test. Remember how I said you can change how @graphql-tools/mock resolves arrays? Let's do that:

test('app', () => {
    const data = createMockData(GetCardsDocument, { Query: { cards: () => [{ abilities: [{ effect: "paralyzed" }] }] }})
    const result = app(data)
    expect(result).toBe("paralyzed")
})

Here, we're using the Query property in the mocks object to tell @graphql-tools/mock exactly how many Cards we want returned in the cards array. We're also defining a specific value for our card's AbilityListItem's name, same as before but through the Query resolver instead of the AbilityListItem resolver.

Oh no! We have a TypeScript error on cards:

Type '() => { abilities: { effect: string; }[]; }[]' is not assignable to type 'Resolver<Maybe<Maybe<ResolverTypeWrapper<Card>>[]>, {}, any, Partial<QueryCardsArgs>> | undefined'.
  Type '() => { abilities: { effect: string; }[]; }[]' is not assignable to type 'ResolverFn<Maybe<Maybe<ResolverTypeWrapper<Card>>[]>, {}, any, Partial<QueryCardsArgs>>'.
    Type '{ abilities: { effect: string; }[]; }[]' is not assignable to type 'Maybe<Maybe<ResolverTypeWrapper<Card>>[]> | Promise<Maybe<Maybe<ResolverTypeWrapper<Card>>[]>>'.
      Type '{ abilities: { effect: string; }[]; }[]' is not assignable to type 'Maybe<ResolverTypeWrapper<Card>>[]'.
        Type '{ abilities: { effect: string; }[]; }' is not assignable to type 'Maybe<ResolverTypeWrapper<Card>>'.
          Type '{ abilities: { effect: string; }[]; }' is missing the following properties from type 'Card': category, id, legal, localId, and 3 more.ts(2322)

Bleh, that's a lot of text. Always remember: TypeScript errors should be read from the bottom up:

Type '{ abilities: { effect: string; }[]; }' is missing the following properties from type 'Card': category, id, legal, localId, and 3 more.ts(2322)

Ok, so the object we're providing in the cards array is missing some properties. To fix this TypeScript error, we could add all those properties, but that would have a disastrous effect on test criteria #2 and #3. Besides, if we needed these properties for our query, we know @graphql-tools/mock would provide them automatically for us. So the type must be wrong! Let's fix it:

import { CodegenConfig } from "@graphql-codegen/cli"

export default {
    overwrite: true,
    schema: ["src/api/graphql/schema.graphql"],
    documents: ["src/**/*.graphql", "!src/api/*", "src/**/*.tsx", "src/**/*.ts"],
    generates: {
        "src/api/graphql/resolvers.ts": {
          config: {
            resolverTypeWrapperSignature: "RecursivePartial<T>",
          },
          plugins: [
            "typescript", 
            "typescript-resolvers",
            { add: { content: "type RecursivePartial<T> = T extends object ? { [K in keyof T]?: RecursivePartial<T[K]> } : T" } },
          ],
        },
    },
} satisfies CodegenConfig

Ok this is starting to look a little complicated, but bear with me! Let's break it down:

  1. We added a config property to our src/api/graphql/resolvers.ts object. This lets us define custom behavior.
  2. In that config property, we've defined resolverTypeWrapperSignature, which is supposed to be a TypeScript type that wraps the resolver's return type. In this case we're saying we want the return type to be a recursive partial type (so we won't have to define all the properties; @graphql-tools/mock does that for us at runtime)
  3. In plugins we've added an object that lets us simply add a line to the generated resolvers.ts file, and it's a definition for RecursivePartial. That way when the resolvers' return types are wrapped with our resolverTypeSignature, the definition of RecursivePartial is available in the file

Now if we run codegen to re-generate resolvers.ts we find that our mocks object no longer has a type error! Hooray!

Ok, that was even more work. NOW, are we done? Depending on your perspective, I have either bad news or good news for you. TypeScript types, like great works of art, are never finished, only abandoned.

You could leave well enough alone and use this function in your codebase as it is! However, here are some missing pieces of functionality to consider before you abandon this blog post:

@graphql-tools/mock's IMock type is too permissive

A test like this will fail because the mocks object is defined wrong, but you won't see a TypeScript error:

test('app', () => {
    const data = createMockData(GetCardsDocument, { Qeury: { cards: () => [{ abilities: [{ effect: "paralyzed" }] }] }})
    const result = app(data)
    expect(result).toBe("paralyzed")
})

This is something we can fix! What's going on here is that IMocks type is an intersection between two types:

export type IMocks<TResolvers = IResolvers> = {
    [TTypeName in keyof TResolvers]?: {
        [TFieldName in keyof TResolvers[TTypeName]]: TResolvers[TTypeName][TFieldName] extends (args: any) => any ? () => ReturnType<TResolvers[TTypeName][TFieldName]> | ReturnType<TResolvers[TTypeName][TFieldName]> : TResolvers[TTypeName][TFieldName];
    };
} & {
    [typeOrScalarName: string]: IScalarMock | ITypeMock;
};

The first type is a mapped object type. It's taking the keys of TResolvers and defining types for each resolver that make sense for the interface addMocksToSchema needs. This one isn't the problem.

The second type is an object type, and its English equivalent says something like "you can also define any property you want inside an object typed by IMocks, as long as its key is a string". This is flexible to the point of being unhelpful! If I mistype a top-level resolver property name, I want TypeScript to tell me about it!

You can ensure this happens by copying this code in your codebase and omitting the second type in the intersection:

export type IMocks<TResolvers> = {
    [TTypeName in keyof TResolvers]?: {
        [TFieldName in keyof TResolvers[TTypeName]]: TResolvers[TTypeName][TFieldName] extends (args: any) => any ? () => ReturnType<TResolvers[TTypeName][TFieldName]> | ReturnType<TResolvers[TTypeName][TFieldName]> : TResolvers[TTypeName][TFieldName];
    };
}

I need to mock custom scalar types

Let's say your API has fields that are ISO8601 date strings (accept no substitutes!). Well, if they're typed as String in the GraphQL API, our createMockData function is going to provide values that make sense for a String field: "Hello, World!". This is really irritating because your API should be providing you type information to trust; you shouldn't have to validate date fields in your API's responses.

Here's what you can do: add a custom scalar type to your API like DateTime, which returns an ISO8601 format string, and ensure your API follows through with its promise, of course.

Next, we need to provide a scalar mock to our createMockData function, which ensures that every field typed with DateTime gets a ISO8601 format string by default:

import { Resolvers } from "src/api/graphql/resolvers"
const globalMocks: MockResolvers = {
    DateTime: () => new Date().toISOString()
}

export function magicalFunctionThatCreatesExactlyTheMockDataIWant(documentNode, mocks: IMocks<Resolvers>) {
  const executableSchema = addMocksToSchema({ schema, mocks: {...globalMocks, ...mocks} })
  const result = executeSync({ schema: executableSchema, document })
  return result.data
}

Yay! Our test will pass, but uh-oh! The MockResolvers type does not like the implementation we provided to our DateTime resolver. That's just another case where the Resolvers type, intended for server-side use, isn't quite a good match for what addMocksToSchema really wants. We can use codegen tools to fix this!

First, add a scalar type mapping to your codegen config, which will generate a Scalars type we'll use in the next step:

import { CodegenConfig } from "@graphql-codegen/cli"

export default {
    // ...
    generates: {
        "src/api/graphql/resolvers.ts": {
          config: {
            // ...
            scalars: {
              DateTime: "string"
            },
          },  
        },
    },
} satisfies CodegenConfig

The generated Scalars type looks something like this:

export type Scalars = {
  ID: { input: string; output: string; }
  String: { input: string; output: string; }
  Boolean: { input: boolean; output: boolean; }
  Int: { input: number; output: number; }
  Float: { input: number; output: number; }
  DateTime: { input: string; output: string; } // this is the only custom one; defining the scalars map just makes predefined scalar types explicit
};

Now we can modify our custom IMocks type to ask for the type of Scalar mock that addMocksToSchema really wants:

import { Resolvers, Scalars } from "src/api/graphql/resolvers"

type IMocks<TResolvers> = {
  [TTypeName in keyof TResolvers]?: TTypeName extends keyof Scalars
  ? () => Scalars[TTypeName]["output"]
  : {
    [TFieldName in keyof TResolvers[TTypeName]]: TResolvers[TTypeName][TFieldName] extends (...args: any) => any
    ? (() => ReturnType<TResolvers[TTypeName][TFieldName]>) | ReturnType<TResolvers[TTypeName][TFieldName]>
    : TResolvers[TTypeName][TFieldName];
  }
}

In plain English, this type is saying: for each top-level key of the Resolvers type, let's define a new value for the property. If the top-level key also exists in the generated Scalars type, then the value ought to be a function that returns the "output" type for that key's entry in the Scalars type. If top-level key doesn't exist in the Scalars map, carry on as we did before.

Now our globalMocks object is correctly typed!

Ugh, the IMocks type (as well as our custom version of it) permits a mock type that addMocksToSchema can't actually use!

This part is particularly annoying, I'm not even gonna front. But basically this test fails because addMocksToSchema doesn't actually use mock values if at least one of the top 2 nested properties isn't a function:

test('app', () => {
    const data = createMockData(GetCardsDocument, { AbilitiesListItem: { effect: "paralyzed" } })
    const result = app(data)
    expect(result).toBe("paralyzed")
})

My reaction after putting several days' worth of effort into creating a nicely typed interface for tests, only to learn that there's a bizarre edge case: Gif of Tim Robinson from I Think You Should Leave yelling "NOT EVERYONE KNOWS HOW TO DO EVERYTHING" from his car window

Luckily, dear reader, I have endured this pain, and can share with you the not-obvious-at-all solution. We can naively start in our custom IMocks type--just make the object mapping portion of the type be a function that returns an object instead of an object:

import type { Scalars } from "src/api/graphql/resolvers"

type IMocks<TResolvers> = {
  [TTypeName in keyof TResolvers]?: TTypeName extends keyof Scalars
  ? () => Scalars[TTypeName]["output"]
  : () => {
    [TFieldName in keyof TResolvers[TTypeName]]: TResolvers[TTypeName][TFieldName] extends (...args: any) => any
    ? (() => ReturnType<TResolvers[TTypeName][TFieldName]>) | ReturnType<TResolvers[TTypeName][TFieldName]>
    : TResolvers[TTypeName][TFieldName];
  }
}

This will lead you to some weird TypeScript errors. Be not afraid. I'll save you the tedium of explaining why (I am getting ready to abandon this blog post), but the way you fix it is, once again, in codegen:

import type { CodegenConfig } from "@graphql-codegen/cli"

export default {
    // ...
    generates: {
        "src/api/graphql/resolvers.ts": {
          config: {
            // ...
            customResolverFn: "TResult | (() => TResult)",
          },  
        },
    },
} satisfies CodegenConfig

This has the side effect of disallowing one flavor of mock object type that addMocksToSchema actually accepts, but, disallowing things that should be disallowed is of a higher value in my book.

Is this starting to get complicated? A bit! Does it work well? You bet your ass it does. I wrote this interface (and some more complicated ones for creating Apollo MockedResponses about 2 years ago, and now, when I search for usages of these functions in our codebase, I get 375 results in 58 files.

Check out a repo with fully-working examples (and tests you can run!) here