Manage Contentful Models with Migration Script

Use transformEntriesToType and transformEntries to apply automatic and predictable content migration

Photo by Jessy Paston on Unsplash

Not long ago, I was tasked with an project to do a Spring Cleaning to our Contentful usage, with the aim to tidy up some old models and replace them with the more organised one (does it sound familiar in all restructure project?). Well, designing and optimising the model proved to be fun and intellectual enjoyable, what was difficult was the migration stage, as we obviously didn’t want to migrate 1000+ entries one by one.

Luckily, Contentful came up with a migration toolkit that is easy to understand and simple to use.

There are essentially three steps to run a migration (or two, depends on if you have the set-up already).

• Install and configure the Contentful CLI and the environment
• Write migration script
• Run the script using the CLI

Install and configure the Contentful CLI and the environment

First, install the CLI globally:

npm install -g contentful-cli

You also need a Contentful space and a few API keys. You can find it under your settings dashboard, or in your .contentful.json file. For your convenience, you could export the space id, so you won’t need to add the --space-id argument to every command.

export SPACE_ID=my-space-id contentful space use --space-id $SPACE_ID

Since we are running migrations, it is good to test it and make sure you are happy with the result before you apply the changes in the production environment.

To do that, create a sandbox environment — a full copy of your model and all the content in your space — ready for safe manipulation.

contentful space environment create --environment-id 'dev' --name 'Development'

You can refer to the following workflow for a proper set up. This is a bit like Git.

How should I merge changes from sandbox environments into master?

1. Test your changes in a target environment (Environment B)
2. Create a migration script to apply the changes to the master content model
3. Clone the current master environment (Environment A)
4. Run the migration script against the freshly cloned environment (Environment C)
5. Run your tests
6. If the tests pass, update the master alias to target the new environment (Environment C)

https://www.contentful.com/faq/environments/

Write migration script

Next step if writing the migration script. In my case, I need to

• For the given source content type B, transforms all its entries into new entries of a specific different target content type A. Namely, I will transform entryTitle”, “name”, “age”, “title” on type B to “entryTitle”, “name”, “age”, “title” on type A. But I will leave “status” on type B untouched.
• Since B originally reference to C as an array of Bb field, namely, Bb is a field on type C, and Bb can reference multiple B, I need to convert A to reference to C as well through Aa field.
• After the transition, I would like to delete B and Bb entirely (both its entries and its content type).

Note that in above scenario, A already exists. But it’s highly likely that you need to create A and make it attachable to C before all the other work.

Here is the script which I will go through it soon:

const uuid62 = require("uuid62");
//Contentful uses uuid behind the scene to create the contentful_id //field which we would need for the identity key below
module.exports = (migration) => {
  // a customised function to create new EntryTitle 
function getNewEntryTitle(oldEntryTitle) {
    let list = oldEntryTitle.split("/");
    list.splice(0, 1, "typeA");
    list.push("typeB-typeA");
    return list.join("/");
  }

  migration.transformEntriesToType({
    sourceContentType: "B",
    targetContentType: "A",
    from: ["entryTitle", "name", "age", "title"],
    shouldPublish: "preserve",
    updateReferences: true,
    removeOldEntries: true,
    identityKey: function(fromFields) {
      if (!fromFields) {
        return;
      }
      const id = uuid62.v4();
      return id;
    },
    transformEntryForLocale: function(fromFields, currentLocale) {
      if (!fromFields || currentLocale !== "en-US") {
        return;
      }

      return {
        entryTitle: !fromFields.entryTitle
          ? undefined
          : getNewEntryTitle(fromFields.entryTitle[currentLocale]),
        name: !fromFields.name ? undefined : fromFields.name[currentLocale],
        age: !fromFields.age ? undefined : fromFields.age[currentLocale],
        title: !fromFields.title ? undefined : fromFields.title[currentLocale],
        status: undefined
      };
    },
  });

  migration.transformEntries({
    contentType: "C",
    from: ["Bb"],
    to: ["Aa"],
    shouldPublish: "preserve",
    transformEntryForLocale: function(fromFields, currentLocale) {
      if (!fromFields || currentLocale !== "en-US") {
        return;
      }

      return {
        Aa: !fromFields.Bb
          ? undefined
          : fromFields.Bb[currentLocale],
      };
    },
  });

  const cc = migration.editContentType("C");
  cc.deleteField("Bb");

  migration.deleteContentType("B");
};

Firstly, transformEntriesToType . What it does is to transforms all entries of a source content type according to the user-provided transformEntryForLocale function into a new entry of a specific different (target) content type.

For each entry, the CLI will call the function transformEntryForLocale once per locale (in our case, it’s “en-US”) in the space, passing in the from fields and the locale as arguments. The transform function is expected to return an object with the desired target fields. If it returns undefined, this entry locale will be left untouched.

sourceContentType – Content type ID of source entries: B

targetContentType – Targeted Content type ID: A

from – Array of the source field IDs, returns complete list of fields if not configured: [“entryTitle”, “name”, “age”, “title”]

identityKey - Function to create a new entry ID for the target entry. Think of it as unique identifier for each entry, just like primary key for each data entry in the database. We return the uuid in this case.

shouldPublish – Flag that specifies publishing of target entries, preserve will keep current states of the source entries. We 'preserve' in our case.

updateReferences– Flag that specifies if linking entries should be updated with target entries (default false)

removeOldEntries – Flag that specifies if source entries should be deleted (default false). But in our case, we want it to be removed, so true .

transformEntryForLocale – Transformation function to be applied.

• fields is an object containing each of the from fields. Each field will contain their current localized values (i.e. fields == {myField: {'en-US': 'my field value'}})
• locale one of the locales in the space being transformed
• The return value must be an object with the same keys as specified in the targetContentType. Their values will be written to the respective entry fields for the current locale (i.e. {nameField: 'myNewValue'}). If it returns undefined, this the values for this locale on the entry will be left untouched. So you notice that the status field has undefined as we don’t have a matching field on type B but we have to explicitly mention it.

Once you understand above, the next function is almost self-explanatory.

transformEntries: For the given content type, transforms all its entries according to the user-provided transformEntryForLocale function. For each entry, the CLI will call this function once per locale in the space, passing in the from fields and the locale as arguments. The transform function is expected to return an object with the desired target fields. If it returns undefined, this entry locale will be left untouched. So in our case, we transform the Bb field with a list of B to Aa field with a list of A .

Run the script using the CLI

The last step is to run the script. Name the file and run the command:

contentful space migration — environment-id ‘dev’ migrations/script_1.js

You will see the console:

The following migration has been planned

Environment: dev
Migrate entries from B
  - from: B
  - to: A
? Do you want to apply the migration Yes
  ✔ Migrate entries from B
Transform entries for C
 — from: Bb
 — to: Aa
? Do you want to apply the migration Yes
 ❯ Transform entries for C
 ✔ Transform entries for C
Update Content Type C
Delete field Bb
Publish Content Type C
? Do you want to apply the migration Yes
  ✔ Update Content Type C
🎉 Migration successful

If there’s any error happens in the interim, don’t worry, Contentful has some sanity check along the way that will help you debug. If not, the community is very vivid and friendly.

That’s so much of it today!

Happy Reading!