Dependency Hell with Yarn & Jest

21 Jun 2020 - JavaScript, TypeScript, Yarn, Jest

Let’s discuss dependency hell when mocking Node module with Jest while using Yarn as the package manager.

Context

To illustrate the issue let’s say we have a monorepo (handled using Yarn Workspaces) with three packages:

• A: provides a Test class containing an UUID created using the uuid package in the constructor;
• B: consumes that class. Currently, it only contains a test and, for testing purpose, we mock the uuid package so Test always contains the same UUID;
• C: independent of the above projects, it has some external dependencies.

Here are the associated files:

1
2
3
4
5
6

{
  "name": "my-project-top",
  "version": "1.0.0",
  "private": true,
  "workspaces": ["packages/*"]
}

1
2
3
4
5
6
7
8
9

const v4 = require('uuid').v4;

class Test {
  constructor() {
    this.id = v4();
  }
}

module.exports = Test;

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

{
  "name": "my-project-a",
  "version": "1.0.0",
  "main": "index.js",
  "private": true,
  "dependencies": {
    "jest": "^26.0.1",
    "uuid": "8.1.0"
  }
}

1
2
3
4
5
6
7
8
9

{
  "name": "my-project-b",
  "version": "1.0.0",
  "main": "index.js",
  "private": true,
  "scripts": {
    "start": "node index.js"
  }
}

1
2
3
4
5
6
7
8
9

const v4 = require('uuid').v4;
const Test = require('my-project-a');

jest.mock("uuid", () => ({ v4: () => "00000000-0000-0000-0000-000000000000" }));

test('Test UUID mock', () => {
    expect(v4()).toBe("00000000-0000-0000-0000-000000000000");
    expect(new Test().id).toBe("00000000-0000-0000-0000-000000000000");
});

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

{
  "name": "my-project-c",
  "version": "1.0.0",
  "private": true,
  "dependencies": {
    "aws-sdk": "^2.701.0",
    "request": "^2.88.2",
    "sockjs": "^0.3.20"
  }
}

I used the following versions:

• Node: 12.16.3;
• Yarn: 1.22.4.

Unfortunately if we run the test from project B then the mock does not seem to work very well:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

> yarn workspace my-project-b jest

[...]
FAIL  test/index.test.ts
 ✕ Test UUID mock (7 ms)

 ● Test UUID mock

   expect(received).toBe(expected) // Object.is equality

   Expected: "00000000-0000-0000-0000-000000000000"
   Received: "dc615a07-7132-4e17-8dde-bf299aa542de"

      6 | test('Test UUID mock', () => {
      7 |     expect(v4()).toBe("00000000-0000-0000-0000-000000000000");
   >  8 |     expect(new Test().id).toBe("00000000-0000-0000-0000-000000000000");
        |                           ^
      9 | });
     10 |

     at Object.<anonymous> (test/index.test.ts:8:27)

[...]

It seems that locally (line 7) we used the mocked version while Test (line 8) relied on the actual uuid package. Having done that a couple of times in the past, it was an unexpected result 😔.

Dependency Hell

After investigating for few hours, I found the following for the uuid package:

1
2
3
4
5
6
7
8

> find . -type d -name uuid -exec bash -c '
  echo {}, version $(jq -r ".version" {}/package.json)
' \;

./node_modules/node-notifier/node_modules/uuid, version 7.0.3
./node_modules/aws-sdk/node_modules/uuid, version 3.3.2
./node_modules/uuid, version 3.4.0
./packages/my-project-a/node_modules/uuid, version 8.1.0

The package uuid seems to be installed several times, with different versions. This is due to Yarn:

• our projects depend on version 8.1.0;
• aws-sdk on version 3.3.2;
• node-notifier on version 7.0.3;
• other dependencies on version 3.4.0.

One of the version (here 3.4.0) has been hoisted (from Yarn maintainers, the algorithm choosing the hoisted version should be considered as a blackbox, and might change in the future).

From my understanding, when the test from project B mocks uuid it currently mocks the version 3.4.0, due to Node module resolution strategy, while Test from project A will use version 8.1.0 (due to the resolution strategy, again).

Workaround

I am a bit annoyed as there is no official way to ensure my version of uuid is hoisted. So far, I was able to “force” Yarn by using this crappy hack 💩:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

diff --git a/package.json b/package.json
index 1fdf11a..dade45e 100644
--- a/package.json
+++ b/package.json
@@ -2,5 +2,8 @@
   "name": "my-project-top",
   "version": "1.0.0",
   "private": true,
+  "dependencies": {
+    "uuid": "8.1.0"
+  },
   "workspaces": ["packages/*"]
 }

as I know how the algorithm works from a Yarn contributor:

Normally yarn will take whichever version has the most references to it and make it the hoisted top-level version, because that would result in the least duplication (least amount of wasted drive space)

After running yarn, I now have:

1
2
3
4
5
6
7
8
9

> find . -type d -name uuid -exec bash -c '
  echo {}, version $(jq -r ".version" {}/package.json)
' \;

./node_modules/node-notifier/node_modules/uuid, version 7.0.3
./node_modules/aws-sdk/node_modules/uuid, version 3.3.2
./node_modules/sockjs/node_modules/uuid, version 3.4.0
./node_modules/request/node_modules/uuid, version 3.4.0
./node_modules/uuid, version 8.1.0

Fortunately I have a test to validate this behavior. I am more worried that the workaround will not be needed anymore at some point, which would result in dead code (and complicated code for other engineers as I can’t easily comment it).

Note: nohoist is not a solution. On the contrary, it would ensure that a mocked version of the uuid package will never be used in a different package.