Catch more SSR issues and improve compat tests (#353)

This will cause the build to fail, as this will catch an SSR issue, which will be fixed after merging #369

.github/workflows/compatibility-tests.yml

@@ -42,4 +42,9 @@ jobs:
             ${{ runner.os }}-${{ matrix.node-version }}-yarn-
 
       - run: yarn install
-      - run: yarn test:compatibility:ci
+      # This will test whether `npm install`/`yarn install` can actually run on each compatibility test fixture. See `compatibility-tests/README.md` for more info.
+      - run:
+          yarn workspace @theatre/compatibility-tests run install-fixtures
+          --verbose
+      # after that, we run the jest tests for each fixture
+      - run: yarn test:compat

compatibility-tests/.gitignore

@@ -1,5 +1,5 @@
 # these lock files include checksums of the @theatre/*@compat packages, which
 # would change after every edit to their source, so we better not keep them
 # in the repo
-/*/package-lock.json
+/fixtures/*/package/package-lock.json
-/*/yarn.lock
+/fixtures/*/package/yarn.lock

compatibility-tests/README.md

@@ -1,34 +1,21 @@
 # Compatibility tests
 
-This setup helps us test whether Theatre.js is compatible with popular tools in the JS ecosystem, such as Vite, webpack, react, vue, etc.
+This setup helps us test whether Theatre.js is compatible with popular tools in the JS ecosystem, such as Vite, Next.js, webpack, react, vue, etc.
 
-> This setup is 3/10 complete.
+## The directory structure
 
-The currentr workflow is:
+- `./fixtures` (contains the fixtures - read on for more details)
+  - `parcel2-react18/`: The name of the fixture. This name means we're testing a minimal setup of Theatre.js alongside `parcel2` and `react18`. 
+    - `package/`: This is the npm package that contains a minimal setup of `theatre+parcel2+react18`.
+    - `production.compat-test.ts`: This is a jest test for creating a production build of this setup.
+    - `*.compat-test.ts`: Any `.compat-test.ts` file will be picked up by jest, so you can use more files to test different aspects of the fixture.
 
-1. Run a local npm registry, and build and publish the @theatre/* packages to it. All packages will have the 0.0.1-COMPAT.1 version
+## How to run the tests
-   
-  ```sh
-  yarn workspace @theatre/compatibility-tests run registry:start
-  ```
 
-  This script will keep running until it is terminated. While running, npm will be configured to point to the local registry that contains the `@theatre/*@0.0.1-COMPAT.1` packages.
+1. First, we run `yarn run install-fixtures`, which tries to install Theatre.js on a fixture as if `@theatre/core|studio|r3f` were installed through npm. This script runs a [local npm registry](https://github.com/verdaccio/verdaccio) and publishes a production build of all the Theatre.js packages to it. Then, it iterates through `./fixtures/*/package` and runs `$ npm install` on them, using that local npm registry. 
-
+  **If this step fails**, that usually means one of `@theatre/*` packages has a `dependency/peerDependency` that cannot be satisfied by `npm/yarn`. So this is always the first thing to fix.
-2. `cd` into any test project (either use those in `./test-*` or make a new npm package elsewhere via `npm init`.)
+1. Then, we run `$ yarn test:compat`, which will run jest on all of `*.compat-test.ts` files, each of which tests an aspect of a test setup.
-3. Install `@theatre/*` from the local registry:
+2. Most of our fixtures don't actually have `.compat-test.ts` files, so we'll have to run them manually and see if Theatre still works in them, jut like a manual QA pass.
-
-  ```sh
-  npm install @theatre/core@0.0.1-COMPAT.1 @theatre/studio@0.0.1-COMPAT.1 @theatre/r3f@0.0.1-COMPAT.1
-  ```
-
-4. Test your local build of Thatre.js against your setup by manually building, running a dev server, and using Theatre.js on a browser pointing to the dev server.
-
-
-Inspired by the
-[#help channel on our Discord](https://discord.com/channels/870988717190426644/870988717190426647)
-we collect examples for including `Theatre.js` in project that use different
-tools (`parcel`, `Next.js`, `vanilla Rollup`, etc...) to build them in the CI
-(these are the _compatibility tests_).
 
 > **Gotchas**
 > Some bundlers like webpack are not configured to work well with yarn workspaces by default. For example, the webpack config of create-react-app, tries to look up the node_modules chain to find missing dependencies, which is not a behavior that we want in build-tests setups. So if a setup doesn't work, try running it outside the monorepo to see if being in the monorepo is what's causing it to fail.

compatibility-tests/fixtures/cra-react18/package/package.json

@@ -13,25 +13,18 @@
     "@testing-library/jest-dom": "^5.11.4",
     "@testing-library/react": "^11.1.0",
     "@testing-library/user-event": "^12.1.10",
-    "@theatre/core": "^0.0.1-COMPAT.1",
+    "@theatre/core": "0.0.1-COMPAT.1",
-    "@theatre/r3f": "^0.0.1-COMPAT.1",
+    "@theatre/r3f": "0.0.1-COMPAT.1",
-    "@theatre/studio": "^0.0.1-COMPAT.1",
+    "@theatre/studio": "0.0.1-COMPAT.1",
     "react-scripts": "^5.0.1",
     "three": ">0.132.0",
     "web-vitals": "^1.0.1"
   },
   "eslintConfig": {
-    "extends": [
+    "extends": ["react-app", "react-app/jest"]
-      "react-app",
-      "react-app/jest"
-    ]
   },
   "browserslist": {
-    "production": [
+    "production": [">0.2%", "not dead", "not op_mini all"],
-      ">0.2%",
-      "not dead",
-      "not op_mini all"
-    ],
     "development": [
       "last 1 chrome version",
       "last 1 firefox version",

compatibility-tests/fixtures/next/__snapshots__/production.compat-test.ts.snap

@@ -0,0 +1,17 @@
+// Jest Snapshot v1, https://goo.gl/fbAQLP
+
+exports[`next / production \`$ next build\` should succeed and have a predictable output 1`] = `
+"
+> build
+> next build
+
+info  - Linting and checking validity of types...
+info  - Creating an optimized production build...
+info  - Compiled successfully
+info  - Collecting page data...
+info  - Generating static pages (0/3)
+info  - Generating static pages (3/3)
+info  - Finalizing page optimization...
+
+"
+`;

compatibility-tests/fixtures/next/package/package.json

@@ -6,9 +6,9 @@
     "start": "next start"
   },
   "dependencies": {
-    "@theatre/core": "^0.0.1-COMPAT.1",
+    "@theatre/core": "0.0.1-COMPAT.1",
-    "@theatre/r3f": "^0.0.1-COMPAT.1",
+    "@theatre/r3f": "0.0.1-COMPAT.1",
-    "@theatre/studio": "^0.0.1-COMPAT.1",
+    "@theatre/studio": "0.0.1-COMPAT.1",
     "next": "latest",
     "react": "^18.2.0",
     "react-dom": "^18.2.0"

compatibility-tests/fixtures/next/package/pages/index.js

@@ -4,12 +4,17 @@ import {Canvas} from '@react-three/fiber'
 import studio from '@theatre/studio'
 import {editable as e, SheetProvider} from '@theatre/r3f'
 import extension from '@theatre/r3f/dist/extension'
+import playgroundState from './playgroundState.json'
 
 if (process.env.NODE_ENV === 'development' && typeof window !== 'undefined') {
   studio.extend(extension)
   studio.initialize({usePersistentStorage: false})
 }
 
+const sheet = getProject('Playground - R3F', {state: playgroundState}).sheet(
+  'R3F-Canvas',
+)
+
 // credit: https://codesandbox.io/s/camera-pan-nsb7f
 
 function Plane({color, theatreKey, ...props}) {
@@ -30,7 +35,7 @@ function App() {
       dpr={[1.5, 2]}
       style={{position: 'absolute', top: 0, left: 0}}
     >
-      <SheetProvider sheet={getProject('Playground - R3F').sheet('R3F-Canvas')}>
+      <SheetProvider sheet={sheet}>
         {/* @ts-ignore */}
         <e.orthographicCamera makeDefault theatreKey="Camera" />
         <ambientLight intensity={0.4} />
@@ -82,10 +87,6 @@ function App() {
   )
 }
 
-const project = getProject('Project')
-const sheet = project.sheet('Sheet')
-const obj = sheet.object('Obj', {str: 'some string', num: 0})
-
 export default function Home() {
-  return <App obj={obj}>hi</App>
+  return <App></App>
 }

compatibility-tests/fixtures/next/package/pages/playgroundState.json

@@ -0,0 +1,24 @@
+{
+  "sheetsById": {
+    "R3F-Canvas": {
+      "staticOverrides": {
+        "byObject": {
+          "plane1": {
+            "position": {
+              "x": -0.06000000000000002
+            }
+          },
+          "plane2": {
+            "position": {
+              "x": 0,
+              "y": -1.1043953439330743,
+              "z": 6.322692591942688
+            }
+          }
+        }
+      }
+    }
+  },
+  "definitionVersion": "0.4.0",
+  "revisionHistory": ["lSnZ_QVusR3qNnVN"]
+}

compatibility-tests/fixtures/next/production.compat-test.ts

@@ -0,0 +1,67 @@
+// @cspotcode/zx is zx in CommonJS
+import {$, cd, path} from '@cspotcode/zx'
+import {chromium, devices} from 'playwright'
+
+$.verbose = true
+
+const PATH_TO_PACKAGE = path.join(__dirname, `./package`)
+
+describe(`next / production`, () => {
+  test(`\`$ next build\` should succeed and have a predictable output`, async () => {
+    cd(PATH_TO_PACKAGE)
+    const {exitCode, stdout} = await $`npm run build`
+    // at this point, the build should have succeeded
+    expect(exitCode).toEqual(0)
+    // now let's check the output to make sure it's what we expect
+
+    // all of stdout until the line that contains "Route (pages)". That's because what comes after that
+    // line is a list of all the pages that were built, and we don't want to snapshot that because it changes every time.
+    const stdoutUntilRoutePages = stdout.split(`Route (pages)`)[0]
+
+    // This test will fail if `next build` outputs anything unexpected.
+    expect(stdoutUntilRoutePages).toMatchSnapshot()
+  })
+
+  // this test is not ready yet, so we'll skip it
+  describe.skip(`$ next start`, () => {
+    let browser, page
+    beforeAll(async () => {
+      browser = await chromium.launch()
+    })
+    afterAll(async () => {
+      await browser.close()
+    })
+    beforeEach(async () => {
+      page = await browser.newPage()
+    })
+    afterEach(async () => {
+      await page.close()
+    })
+
+    // just a random port I'm hoping is free everywhere.
+    const port = 30978
+
+    test('`$ next start` serves the app, and the app works', async () => {
+      // run the production server but don't wait for it to finish
+      cd(PATH_TO_PACKAGE)
+      const p = $`npm run start -- --port ${port}`
+      // await p
+
+      try {
+        page.on('console', (msg) => console.log('PAGE LOG:', msg.text()))
+        await page.goto(`http://localhost:${port}`)
+        // wait three seconds
+        await page.waitForTimeout(3000)
+      } finally {
+        p.kill()
+      }
+      try {
+        await p
+      } catch (e) {
+        if (e.signal !== 'SIGKILL' && e.signal !== 'SIGTERM') {
+          throw e
+        }
+      }
+    })
+  })
+})

compatibility-tests/fixtures/parcel1-react17/package/package.json

@@ -1,14 +1,14 @@
 {
-  "name": "@compat/parcel1-react18",
+  "name": "@compat/parcel1-react17",
   "scripts": {
     "dev": "parcel serve ./index.html"
   },
   "dependencies": {
     "@react-three/drei": "^7.3.1",
     "@react-three/fiber": "^7.0.6",
-    "@theatre/core": "^0.0.1-COMPAT.1",
+    "@theatre/core": "0.0.1-COMPAT.1",
-    "@theatre/r3f": "^0.0.1-COMPAT.1",
+    "@theatre/r3f": "0.0.1-COMPAT.1",
-    "@theatre/studio": "^0.0.1-COMPAT.1",
+    "@theatre/studio": "0.0.1-COMPAT.1",
     "parcel-bundler": "^1.12.5",
     "react": "^17.0.2",
     "react-dom": "^17.0.2",

compatibility-tests/fixtures/parcel1-react18/package/package.json

@@ -4,9 +4,9 @@
     "dev": "parcel serve ./index.html"
   },
   "dependencies": {
-    "@theatre/core": "^0.0.1-COMPAT.1",
+    "@theatre/core": "0.0.1-COMPAT.1",
-    "@theatre/r3f": "^0.0.1-COMPAT.1",
+    "@theatre/r3f": "0.0.1-COMPAT.1",
-    "@theatre/studio": "^0.0.1-COMPAT.1",
+    "@theatre/studio": "0.0.1-COMPAT.1",
     "parcel-bundler": "^1.12.5",
     "react": "^18.1.0",
     "react-dom": "^18.1.0"

compatibility-tests/fixtures/parcel2-react18/package/package.json

@@ -4,9 +4,9 @@
     "dev": "parcel serve ./index.html"
   },
   "dependencies": {
-    "@theatre/core": "^0.0.1-COMPAT.1",
+    "@theatre/core": "0.0.1-COMPAT.1",
-    "@theatre/r3f": "^0.0.1-COMPAT.1",
+    "@theatre/r3f": "0.0.1-COMPAT.1",
-    "@theatre/studio": "^0.0.1-COMPAT.1",
+    "@theatre/studio": "0.0.1-COMPAT.1",
     "parcel": "^2.5.0",
     "react": "^18.1.0",
     "react-dom": "^18.1.0"

compatibility-tests/fixtures/vite-react18/package/package.json

@@ -10,9 +10,9 @@
   "dependencies": {
     "@react-three/drei": "^9.11.3",
     "@react-three/fiber": "^8.0.19",
-    "@theatre/core": "^0.0.1-COMPAT.1",
+    "@theatre/core": "0.0.1-COMPAT.1",
-    "@theatre/r3f": "^0.0.1-COMPAT.1",
+    "@theatre/r3f": "0.0.1-COMPAT.1",
-    "@theatre/studio": "^0.0.1-COMPAT.1",
+    "@theatre/studio": "0.0.1-COMPAT.1",
     "react": "^18.0.0",
     "react-dom": "^18.0.0",
     "three": "^0.141.0"

compatibility-tests/package.json

@@ -1,18 +1,20 @@
 {
   "name": "@theatre/compatibility-tests",
   "private": true,
+  "type": "module",
   "scripts": {
-    "registry:start": "zx ./scripts/start-registry.mjs",
+    "install-fixtures": "zx ./scripts/install-fixtures.mjs",
-    "test:ci": "zx ./scripts/ci.mjs",
     "clean": "zx ./scripts/clean.mjs"
   },
   "dependencies": {
+    "@cspotcode/zx": "^6.1.2",
     "node-cleanup": "^2.1.2",
+    "playwright": "^1.28.1",
     "prettier": "^2.6.2",
     "verdaccio": "^5.10.2",
     "verdaccio-auth-memory": "^10.2.0",
     "verdaccio-memory": "^10.2.0",
-    "zx": "^6.1.0"
+    "zx": "^7.1.1"
   },
   "version": "0.0.1-COMPAT.1"
 }

compatibility-tests/scripts/build-setups.mjs

@@ -3,10 +3,10 @@
  */
 
 import path from 'path'
-import {colorize, getCompatibilityTestSetups} from './utils.mjs'
+import {cd, fs, $} from 'zx'
+import {getCompatibilityTestSetups} from './utils.mjs'
 
-const root = path.resolve(__dirname, '../..')
+const absPathOfCompatibilityTestSetups = getCompatibilityTestSetups()
-const absPathOfCompatibilityTestSetups = getCompatibilityTestSetups(root)
 
 const setupsWithErros = []
 
@@ -15,7 +15,7 @@ const setupsWithErros = []
   for (const setupDir of absPathOfCompatibilityTestSetups) {
     try {
       cd(setupDir)
-      const pathToSetup = path.join(absPathOfCompatibilityTestSetups, setupDir)
+      const pathToSetup = path.join(setupDir, setupDir)
       fs.removeSync(path.join(pathToSetup, 'node_modules'))
       fs.removeSync(path.join(pathToSetup, 'package-lock.json'))
       fs.removeSync(path.join(pathToSetup, 'yarn.lock'))
@@ -31,7 +31,7 @@ const setupsWithErros = []
   // and print all of them to the console.
   if (setupsWithErros.length !== 0) {
     throw new Error(
-      `The following setups had problems when their dependencies were being installed:\n${colorize.red(
+      `The following setups had problems when their dependencies were being installed:\n${(
         setupsWithErros.join('\n'),
       )}`,
     )

compatibility-tests/scripts/ci.mjs

@@ -1,5 +1,5 @@
-import {startRegistry} from './utils.mjs'
+import {installTests} from './scripts.mjs'
 ;(async function runCI() {
-  await startRegistry()
+  await installTests()
   process.exit(0)
 })()

compatibility-tests/scripts/clean.mjs

@@ -1,3 +1,3 @@
-import {clean} from './utils.mjs'
+import {clean} from './scripts.mjs'
 
 await clean()

compatibility-tests/scripts/install-fixtures.mjs

@@ -0,0 +1,3 @@
+import {installFixtures} from './scripts.mjs'
+
+installFixtures()

compatibility-tests/scripts/scripts.mjs

@@ -0,0 +1,351 @@
+/**
+ * Utility functions for the compatibility tests
+ */
+
+import prettier from 'prettier'
+import path from 'path'
+import {globby, argv, YAML, $, fs, cd, os, within} from 'zx'
+import onCleanup from 'node-cleanup'
+import * as verdaccioPackage from 'verdaccio'
+import {chromium, devices} from 'playwright'
+
+if (!argv['verbose']) {
+  $.verbose = false
+  console.log(
+    'Running in quiet mode. Add --verbose to see the output of all commands.',
+  )
+}
+
+// 'verdaccio' is not an es module so we have to do this:
+// @ts-ignore
+const startVerdaccioServer = verdaccioPackage.default.default
+
+const config = {
+  VERDACCIO_PORT: 4823,
+  VERDACCIO_HOST: `localhost`,
+  get VERDACCIO_URL() {
+    return `http://${config.VERDACCIO_HOST}:${config.VERDACCIO_PORT}/`
+  },
+  PATH_TO_COMPAT_TESTS_ROOT: path.join(__dirname, '..'),
+  MONOREPO_ROOT: path.join(__dirname, '../..'),
+}
+
+/**
+ * Set environment variables so that yarn and npm use verdaccio as the registry.
+ * These are only set for the current process.
+ */
+process.env.YARN_NPM_PUBLISH_REGISTRY = config.VERDACCIO_URL
+process.env.YARN_UNSAFE_HTTP_WHITELIST = config.VERDACCIO_HOST
+process.env.YARN_NPM_AUTH_IDENT = 'test:test'
+process.env.NPM_CONFIG_REGISTRY = config.VERDACCIO_URL
+
+const tempVersion =
+  '0.0.1-COMPAT.' +
+  // a random integer between 1 and 50000
+  (Math.floor(Math.random() * 50000) + 1).toString()
+
+/**
+ * This script starts verdaccio and publishes all the packages in the monorepo to it, then
+ * it runs `npm install` on all the test packages, and finally it closes verdaccio.
+ */
+export async function installFixtures() {
+  onCleanup((exitCode, signal) => {
+    onCleanup.uninstall()
+    restoreTestPackageJsons()
+    process.kill(process.pid, signal)
+    return false
+  })
+
+  console.log('Using temporary version: ' + tempVersion)
+  console.log('Patching package.json files in ./test-*')
+  const restoreTestPackageJsons = await patchTestPackageJsons()
+
+  console.log('Starting verdaccio')
+  const verdaccioServer = await startVerdaccio(config.VERDACCIO_PORT)
+  console.log(`Verdaccio is running on ${config.VERDACCIO_URL}`)
+
+  console.log('Releasing @theatre/* packages to verdaccio')
+  await releaseToVerdaccio()
+
+  console.log('Running `$ npm install` on test packages')
+  await runNpmInstallOnTestPackages()
+  console.log('All tests installed successfully')
+  await verdaccioServer.close()
+  restoreTestPackageJsons()
+  console.log('Done')
+}
+
+async function runNpmInstallOnTestPackages() {
+  const packagePaths = await getCompatibilityTestSetups()
+
+  for (const pathToPackageDir of packagePaths) {
+    cd(pathToPackageDir)
+    try {
+      console.log('Running npm install on ' + pathToPackageDir + '...')
+      await $`npm install --registry ${config.VERDACCIO_URL} --loglevel error --fund false`
+    } catch (error) {
+      console.error(`Failed to install dependencies for ${pathToPackageDir}
+Try running \`npm install\` in that directory manually via:
+cd ${pathToPackageDir}
+npm install --registry ${config.VERDACCIO_URL}
+Original error: ${error}`)
+    }
+  }
+}
+
+/**
+ * Takes an absolute path to a package.json file and replaces all of its
+ * dependencies on `@theatre/*` packatges to `version`.
+ *
+ * @param {string} pathToPackageJson absolute path to the package.json file
+ * @param {string} version The version to set all `@theatre/*` dependencies to
+ */
+async function patchTheatreDependencies(pathToPackageJson, version) {
+  const originalFileContent = fs.readFileSync(pathToPackageJson, {
+    encoding: 'utf-8',
+  })
+  // get the package.json file's content
+  const packageJson = JSON.parse(originalFileContent)
+
+  // find all dependencies on '@theatre/*' packages and replace them with the local version
+  for (const dependencyType of [
+    'dependencies',
+    'devDependencies',
+    'peerDependencies',
+  ]) {
+    const dependencies = packageJson[dependencyType]
+    if (dependencies) {
+      for (const dependencyName of Object.keys(dependencies)) {
+        if (dependencyName.startsWith('@theatre/')) {
+          dependencies[dependencyName] = version
+        }
+      }
+    }
+  }
+  // run the json through prettier
+  const jsonStringPrettified = prettier.format(
+    JSON.stringify(packageJson, null, 2),
+    {
+      parser: 'json',
+      filepath: pathToPackageJson,
+    },
+  )
+
+  // write the modified package.json file
+  fs.writeFileSync(pathToPackageJson, jsonStringPrettified, {encoding: 'utf-8'})
+}
+
+async function patchTestPackageJsons() {
+  const packagePaths = (await getCompatibilityTestSetups()).map(
+    (pathToPackageDir) => path.join(pathToPackageDir, 'package.json'),
+  )
+
+  // replace all dependencies on @theatre/* packages with the local version
+  for (const pathToPackageJson of packagePaths) {
+    patchTheatreDependencies(pathToPackageJson, tempVersion)
+  }
+
+  return () => {
+    // replace all dependencies on @theatre/* packages with the 0.0.1-COMPAT.1
+    for (const pathToPackageJson of packagePaths) {
+      patchTheatreDependencies(pathToPackageJson, '0.0.1-COMPAT.1')
+    }
+  }
+}
+
+/**
+ * Starts the verdaccio server and returns a promise that resolves when the serve is up and ready
+ *
+ * Credit: https://github.com/storybookjs/storybook/blob/92b23c080d03433765cbc7a60553d036a612a501/scripts/run-registry.ts
+ */
+const startVerdaccio = (port) => {
+  let resolved = false
+  return Promise.race([
+    new Promise((resolve) => {
+      const config = {
+        ...YAML.parse(
+          fs.readFileSync(path.join(__dirname, '../verdaccio.yml'), 'utf8'),
+        ),
+      }
+
+      const onReady = (webServer) => {
+        webServer.listen(port, () => {
+          resolved = true
+          resolve(webServer)
+        })
+      }
+
+      startVerdaccioServer(
+        config,
+        6000,
+        undefined,
+        '1.0.0',
+        'verdaccio',
+        onReady,
+      )
+    }),
+    new Promise((_, rej) => {
+      setTimeout(() => {
+        if (!resolved) {
+          resolved = true
+          rej(new Error(`TIMEOUT - verdaccio didn't start within 10s`))
+        }
+      }, 10000)
+    }),
+  ])
+}
+
+const packagesToPublish = [
+  '@theatre/core',
+  '@theatre/studio',
+  '@theatre/dataverse',
+  '@theatre/react',
+  '@theatre/browser-bundles',
+  '@theatre/r3f',
+]
+
+/**
+ * Assigns a new version to each of @theatre/* packages. If there a package depends on another package in this monorepo,
+ * this function makes sure the dependency version is fixed at "version"
+ *
+ * @param {{name: string, location: string}[]} workspacesListObjects - An Array of objects containing information about the workspaces
+ * @param {string} version - Version of the latest commit (or any other string)
+ * @returns {Promise<() => void>} - An async function that restores the package.json files to their original version
+ */
+async function writeVersionsToPackageJSONs(workspacesListObjects, version) {
+  /**
+   * An array of functions each of which restores a certain package.json to its original state
+   * @type {Array<() => void>}
+   */
+  const restores = []
+  for (const workspaceData of workspacesListObjects) {
+    const pathToPackage = path.resolve(
+      config.MONOREPO_ROOT,
+      workspaceData.location,
+      './package.json',
+    )
+
+    const originalFileContent = fs.readFileSync(pathToPackage, {
+      encoding: 'utf-8',
+    })
+    const originalJson = JSON.parse(originalFileContent)
+
+    restores.push(() => {
+      fs.writeFileSync(pathToPackage, originalFileContent, {encoding: 'utf-8'})
+    })
+
+    let {dependencies, peerDependencies, devDependencies} = originalJson
+
+    // Normally we don't have to override the package versions in dependencies because yarn would already convert
+    // all the "workspace:*" versions to a fixed version before publishing. However, packages like @theatre/studio
+    // have a peerDependency on @theatre/core set to "*" (meaning they would work with any version of @theatre/core).
+    // This is not the desired behavior in pre-release versions, so here, we'll fix those "*" versions to the set version.
+    for (const deps of [dependencies, peerDependencies, devDependencies]) {
+      if (!deps) continue
+      for (const wpObject of workspacesListObjects) {
+        if (deps[wpObject.name]) {
+          deps[wpObject.name] = version
+        }
+      }
+    }
+    const newJson = {
+      ...originalJson,
+      version,
+      dependencies,
+      peerDependencies,
+      devDependencies,
+    }
+    fs.writeFileSync(pathToPackage, JSON.stringify(newJson, undefined, 2), {
+      encoding: 'utf-8',
+    })
+  }
+  return () =>
+    restores.forEach((fn) => {
+      fn()
+    })
+}
+
+/**
+ * Builds all the @theatre/* packages with version number 0.0.1-COMPAT.1 and publishes
+ * them all to the verdaccio registry
+ */
+async function releaseToVerdaccio() {
+  cd(config.MONOREPO_ROOT)
+
+  // @ts-ignore ignore
+  process.env.THEATRE_IS_PUBLISHING = true
+
+  const workspacesListString = await $`yarn workspaces list --json`
+  const workspacesListObjects = workspacesListString.stdout
+    .split(os.EOL)
+    // strip out empty lines
+    .filter(Boolean)
+    .map((x) => JSON.parse(x))
+
+  const restorePackages = await writeVersionsToPackageJSONs(
+    workspacesListObjects,
+    tempVersion,
+  )
+
+  // Restore the package.json files to their original state when the process is killed
+  process.on('SIGINT', async function cleanup(a) {
+    restorePackages()
+  })
+
+  try {
+    await $`yarn clean`
+    await $`yarn build`
+
+    await Promise.all(
+      packagesToPublish.map(async (workspaceName) => {
+        const npmTag = 'compat'
+        await $`yarn workspace ${workspaceName} npm publish --access public --tag ${npmTag}`
+      }),
+    )
+  } finally {
+    restorePackages()
+  }
+}
+
+/**
+ * Get all the setups from `./compatibility-tests/`
+ *
+ * @returns {Promise<Array<string>>} An array containing the absolute paths to the compatibility test setups
+ */
+export async function getCompatibilityTestSetups() {
+  const fixturePackageJsonFiles = await globby(
+    './fixtures/*/package/package.json',
+    {
+      cwd: config.PATH_TO_COMPAT_TESTS_ROOT,
+      gitignore: false,
+      onlyFiles: true,
+    },
+  )
+
+  return fixturePackageJsonFiles.map((entry) => {
+    return path.join(config.PATH_TO_COMPAT_TESTS_ROOT, entry, '../')
+  })
+}
+
+/**
+ * Deletes ../test-*\/(node_modules|package-lock.json|yarn.lock)
+ */
+export async function clean() {
+  const toDelete = await globby(
+    './fixtures/*/package/(node_modules|yarn.lock|package-lock.json)',
+    {
+      cwd: config.PATH_TO_COMPAT_TESTS_ROOT,
+      // node_modules et al are gitignored, but we still want to clean them
+      gitignore: false,
+      // include directories too
+      onlyFiles: false,
+    },
+  )
+
+  return await Promise.all(
+    toDelete.map((fileOrDir) => {
+      console.log('deleting', fileOrDir)
+      return fs.remove(path.join(config.PATH_TO_COMPAT_TESTS_ROOT, fileOrDir))
+    }),
+  )
+}

compatibility-tests/scripts/start-registry.mjs

@@ -1,3 +0,0 @@
-import {startRegistry} from './utils.mjs'
-
-startRegistry()

compatibility-tests/scripts/utils.mjs

@@ -1,284 +0,0 @@
-/**
- * Utility functions for the compatibility tests
- */
-
-import path from 'path'
-import {globby, YAML} from 'zx'
-import onCleanup from 'node-cleanup'
-import * as verdaccioPackage from 'verdaccio'
-
-// 'verdaccio' is not an es module so we have to do this:
-const startVerdaccioServer = verdaccioPackage.default.default
-
-export const VERDACCIO_PORT = 4823
-export const VERDACCIO_HOST = `localhost`
-export const VERDACCIO_URL = `http://${VERDACCIO_HOST}:${VERDACCIO_PORT}/`
-export const PATH_TO_COMPAT_TESTS_ROOT = path.join(__dirname, '..')
-export const MONOREPO_ROOT = path.join(__dirname, '../..')
-export const PATH_TO_YARNRC = path.join(MONOREPO_ROOT, '.yarnrc.yml')
-
-/**
- * This script will:
- *  1. Start verdaccio (a local npm registry),
- *  2. Configure npm (and _not_ yarn) to use verdaccio as its registry
- *  3. It will _not_ affect the global yarn installation yet. That is a TODO
- *  4. It does _not_ affect the monorepo yarnrc file.
- *
- * If the script is interrupted, it'll attempt to restore the npm/yarn
- * registry config to its original state, but that's not guaranteed.
- */
-export async function startRegistry() {
-  const npmOriginalRegistry = (
-    await $`npm config get registry --location=global`
-  ).stdout.trim()
-  onCleanup((exitCode, signal) => {
-    onCleanup.uninstall()
-    $`npm config set registry ${npmOriginalRegistry} --location=global`.then(
-      () => {
-        process.kill(process.pid, signal)
-      },
-    )
-    return false
-  })
-
-  await $`echo "Setting npm registry url to verdaccio's"`
-  await $`npm config set registry ${VERDACCIO_URL} --location=global`
-
-  await $`echo Running verdaccio on ${VERDACCIO_URL}`
-  const verdaccioServer = await startVerdaccio(VERDACCIO_PORT)
-
-  await releaseToVerdaccio()
-}
-
-/**
- * Starts the verdaccio server and returns a promise that resolves when the serve is up and ready
- *
- * Credid: https://github.com/storybookjs/storybook/blob/92b23c080d03433765cbc7a60553d036a612a501/scripts/run-registry.ts
- */
-const startVerdaccio = (port) => {
-  let resolved = false
-  return Promise.race([
-    new Promise((resolve) => {
-      const config = {
-        ...YAML.parse(
-          fs.readFileSync(path.join(__dirname, '../verdaccio.yml'), 'utf8'),
-        ),
-      }
-
-      const onReady = (webServer) => {
-        webServer.listen(port, () => {
-          resolved = true
-          resolve(webServer)
-        })
-      }
-
-      startVerdaccioServer(
-        config,
-        6000,
-        undefined,
-        '1.0.0',
-        'verdaccio',
-        onReady,
-      )
-    }),
-    new Promise((_, rej) => {
-      setTimeout(() => {
-        if (!resolved) {
-          resolved = true
-          rej(new Error(`TIMEOUT - verdaccio didn't start within 10s`))
-        }
-      }, 10000)
-    }),
-  ])
-}
-
-const packagesToPublish = [
-  '@theatre/core',
-  '@theatre/studio',
-  '@theatre/dataverse',
-  '@theatre/react',
-  '@theatre/browser-bundles',
-  '@theatre/r3f',
-]
-
-/**
- * Assigns a new version to each of @theatre/* packages. If there a package depends on another package in this monorepo,
- * this function makes sure the dependency version is fixed at "hash"
- *
- * @param {{name: string, location: string}[]} workspacesListObjects - An Array of objects containing information about the workspaces
- * @param {string} hash - Hash of the latest commit (or any other string)
- * @returns {Promise<() => void>} - An async function that restores the package.json files to their original version
- */
-async function writeVersionsToPackageJSONs(workspacesListObjects, hash) {
-  /**
-   * An array of functions each of which restores a certain package.json to its original state
-   * @type {Array<() => void>}
-   */
-  const restores = []
-  for (const workspaceData of workspacesListObjects) {
-    const pathToPackage = path.resolve(
-      MONOREPO_ROOT,
-      workspaceData.location,
-      './package.json',
-    )
-
-    const originalFileContent = fs.readFileSync(pathToPackage, {
-      encoding: 'utf-8',
-    })
-    const originalJson = JSON.parse(originalFileContent)
-
-    restores.push(() => {
-      fs.writeFileSync(pathToPackage, originalFileContent, {encoding: 'utf-8'})
-    })
-
-    let {dependencies, peerDependencies, devDependencies} = originalJson
-    const version = hash
-
-    // Normally we don't have to override the package versions in dependencies because yarn would already convert
-    // all the "workspace:*" versions to a fixed version before publishing. However, packages like @theatre/studio
-    // have a peerDependency on @theatre/core set to "*" (meaning they would work with any version of @theatre/core).
-    // This is not the desired behavior in pre-release versions, so here, we'll fix those "*" versions to the set version.
-    for (const deps of [dependencies, peerDependencies, devDependencies]) {
-      if (!deps) continue
-      for (const wpObject of workspacesListObjects) {
-        if (deps[wpObject.name]) {
-          deps[wpObject.name] = hash
-        }
-      }
-    }
-    const newJson = {
-      ...originalJson,
-      version,
-      dependencies,
-      peerDependencies,
-      devDependencies,
-    }
-    fs.writeFileSync(pathToPackage, JSON.stringify(newJson, undefined, 2), {
-      encoding: 'utf-8',
-    })
-  }
-  return () =>
-    restores.forEach((fn) => {
-      fn()
-    })
-}
-
-/**
- * Builds all the @theatre/* packages with version number 0.0.1-COMPAT.1 and publishes
- * them all to the verdaccio registry
- */
-async function releaseToVerdaccio() {
-  const version = '0.0.1-COMPAT.1'
-  cd(MONOREPO_ROOT)
-
-  // @ts-ignore ignore
-  process.env.THEATRE_IS_PUBLISHING = true
-
-  const workspacesListString = await $`yarn workspaces list --json`
-  const workspacesListObjects = workspacesListString.stdout
-    .split(os.EOL)
-    // strip out empty lines
-    .filter(Boolean)
-    .map((x) => JSON.parse(x))
-
-  const restorePackages = await writeVersionsToPackageJSONs(
-    workspacesListObjects,
-    version,
-  )
-
-  process.on('SIGINT', async function cleanup(a) {
-    restorePackages()
-    process.exit(0)
-  })
-
-  // set verdaccio as the publish registry, and add it to the whitelist
-  const restoreYarnRc = patchYarnRcToUseVerdaccio()
-
-  await $`yarn clean`
-  await $`yarn build`
-
-  await Promise.all(
-    packagesToPublish.map(async (workspaceName) => {
-      const npmTag = 'compat'
-      await $`yarn workspace ${workspaceName} npm publish --access public --tag ${npmTag}`
-    }),
-  )
-
-  restorePackages()
-  restoreYarnRc()
-}
-
-/**
- * Temporarily patches the yarnrc file to sue verdaccio as its publish registry.
- *
- * Restores yarnrc to the old version when restoreYarnRc() is called.
- */
-function patchYarnRcToUseVerdaccio() {
-  const originalYarnrcContent = fs.readFileSync(PATH_TO_YARNRC, {
-    encoding: 'utf-8',
-  })
-
-  const newYarnRcContent = YAML.stringify({
-    ...YAML.parse(originalYarnrcContent),
-    unsafeHttpWhitelist: [VERDACCIO_HOST],
-    npmPublishRegistry: VERDACCIO_URL,
-    npmAuthIdent: 'test:test',
-  })
-
-  fs.writeFileSync(PATH_TO_YARNRC, newYarnRcContent, {encoding: 'utf-8'})
-
-  return function restoreYarnRc() {
-    fs.writeFileSync(PATH_TO_YARNRC, originalYarnrcContent, {encoding: 'utf-8'})
-  }
-}
-
-/**
- * Get all the setups from `./compatibility-tests/`
- *
- * @returns {Array<string>} An array containing the absolute paths to the compatibility test setups
- */
-export function getCompatibilityTestSetups() {
-  const buildTestsDir = path.join(MONOREPO_ROOT, 'compatibility-tests')
-  let buildTestsDirEntries
-
-  try {
-    buildTestsDirEntries = fs.readdirSync(buildTestsDir)
-  } catch {
-    throw new Error(
-      `Could not list directory: "${buildTestsDir}" Is it an existing directory?`,
-    )
-  }
-  const setupsAbsPaths = []
-
-  // NOTE: We assume that every directory matching `compatibility-tests/test-*` is
-  // a test package
-  for (const entry of buildTestsDirEntries) {
-    if (!entry.startsWith('test-')) continue
-    const entryAbsPath = path.join(buildTestsDir, entry)
-    if (fs.lstatSync(entryAbsPath).isDirectory()) {
-      setupsAbsPaths.push(entryAbsPath)
-    }
-  }
-
-  return setupsAbsPaths
-}
-
-/**
- * Deletes ../test-*\/(node_modules|package-lock.json|yarn.lock)
- */
-export async function clean() {
-  const toDelete = await globby(
-    './test-*/(node_modules|yarn.lock|package-lock.json)',
-    {
-      cwd: PATH_TO_COMPAT_TESTS_ROOT,
-      // node_modules et al are gitignored, but we still want to clean them
-      gitignore: false,
-      // include directories too
-      onlyFiles: false,
-    },
-  )
-  toDelete.forEach((fileOrDir) => {
-    console.log('deleting', fileOrDir)
-    fs.removeSync(path.join(PATH_TO_COMPAT_TESTS_ROOT, fileOrDir))
-  })
-}

compatibility-tests/verdaccio.yml

@@ -21,4 +21,7 @@ packages:
   '**':
     access: $all
     proxy: npmjs
-log: {type: stdout, format: pretty, level: http}
+logs:
+  type: stdout
+  format: pretty
+  level: error

jest.compat-tests.config.js

@@ -0,0 +1,23 @@
+/** @type {import('jest').Config} */
+module.exports = {
+  testMatch: ['<rootDir>/compatibility-tests/fixtures/*/*.compat-test.ts'],
+  moduleNameMapper: {},
+  // setupFiles: ['./theatre/shared/src/setupTestEnv.ts'],
+  automock: false,
+  // transform: {
+  //   '^.+\\.tsx?$': [
+  //     'esbuild-jest',
+  //     {
+  //       sourcemap: true,
+  //     },
+  //   ],
+  //   '^.+\\.js$': [
+  //     'esbuild-jest',
+  //     {
+  //       sourcemap: true,
+  //     },
+  //   ],
+  // },
+  moduleFileExtensions: ['ts', 'tsx', 'js', 'jsx', 'json', 'node'],
+  testTimeout: 1000 * 60,
+}

jest.config.js

@@ -10,7 +10,10 @@ module.exports = {
     'lodash-es/(.*)': 'lodash/$1',
     'react-use/esm/(.*)': 'react-use/lib/$1',
     'lodash-es': 'lodash',
-    // An ES module that jest can't handle at the moment.
+    // ES modules that jest can't handle at the moment.
+    uuid: '<rootDir>/node_modules/uuid/dist/index.js',
+    nanoid: '<rootDir>/node_modules/nanoid/index.cjs',
+    'nanoid/non-secure': '<rootDir>/node_modules/nanoid/non-secure/index.cjs',
     'react-icons/(.*)': 'identity-obj-proxy',
   },
   setupFiles: ['./theatre/shared/src/setupTestEnv.ts'],

package.json

@@ -12,12 +12,12 @@
     "playground": "yarn workspace playground run serve",
     "test:e2e": "yarn workspace playground run test",
     "test:e2e:ci": "yarn workspace playground run test:ci",
-    "test:compatibility:ci": "yarn workspace @theatre/compatibility-tests run test:ci",
     "typecheck": "yarn run build:ts",
     "build": "zx scripts/build.mjs",
     "clean": "zx scripts/clean.mjs",
     "build:ts": "tsc --build ./devEnv/typecheck-all-projects/tsconfig.all.json",
     "test": "jest",
+    "test:compat": "jest --config jest.compat-tests.config.js",
     "postinstall": "husky install",
     "release": "zx scripts/release.mjs",
     "lint:all": "eslint . --ext ts,tsx --ignore-path=.gitignore --rulesdir ./devEnv/eslint/rules"
@@ -42,7 +42,7 @@
     "@microsoft/api-extractor": "^7.28.6",
     "@typescript-eslint/eslint-plugin": "^5.30.7",
     "@typescript-eslint/parser": "^5.30.7",
-    "esbuild": "^0.14.49",
+    "esbuild": "^0.16.7",
     "esbuild-jest": "^0.5.0",
     "eslint": "^8.20.0",
     "eslint-plugin-jsx-a11y": "^6.6.1",
@@ -50,7 +50,8 @@
     "eslint-plugin-tsdoc": "^0.2.16",
     "eslint-plugin-unused-imports": "^2.0.0",
     "husky": "^6.0.0",
-    "jest": "^27.1.0",
+    "jest": "^29.3.1",
+    "jest-environment-jsdom": "^29.3.1",
     "jsonc-parser": "^3.1.0",
     "lint-staged": "^13.0.3",
     "node-gyp": "^9.1.0",

packages/playground/package.json

@@ -14,12 +14,12 @@
     "build:static": "echo 'building for vercel' && yarn run build",
     "typecheck": "tsc --noEmit",
     "test": "playwright test --config=devEnv/playwright.config.ts",
-    "test:ci": "percy exec -- playwright test --reporter=dot --config=devEnv/playwright.config.ts --project=chromium"
+    "test:ci": "playwright test --reporter=dot --config=devEnv/playwright.config.ts --project=chromium"
   },
   "devDependencies": {
-    "@percy/cli": "^1.6.0",
+    "@percy/cli": "^1.16.0",
     "@percy/playwright": "^1.0.4",
-    "@playwright/test": "^1.23.1",
+    "@playwright/test": "^1.29.1",
     "@react-three/drei": "^7.2.2",
     "@react-three/fiber": "^7.0.6",
     "@theatre/core": "workspace:*",

packages/playground/src/tests/setting-static-props/test.e2e.ts

@@ -61,6 +61,7 @@ test.describe('setting-static-props', () => {
     await expect(secondInput).toHaveAttribute('value', '2')
 
     // Our first visual regression test
+    // @ts-ignore - probably percy uses a different version of playwright
     await percySnapshot(page, test.info().titlePath.join('/') + '/After redo')
   })
 })

theatre/studio/src/utils/tightJsonStringify.test.ts

@@ -3,25 +3,25 @@ describe('tightJsonStringify', () => {
   it('matches a series of expectations', () => {
     expect(tightJsonStringify({a: 1, b: 2, c: {y: 4, z: 745}}))
       .toMatchInlineSnapshot(`
-      "{ \\"a\\": 1,
+      "{ "a": 1,
-        \\"b\\": 2,
+        "b": 2,
-        \\"c\\": {
+        "c": {
-          \\"y\\": 4,
+          "y": 4,
-          \\"z\\": 745 } }"
+          "z": 745 } }"
     `)
     expect(tightJsonStringify(true)).toMatchInlineSnapshot(`"true"`)
     expect(tightJsonStringify('Already a string')).toMatchInlineSnapshot(
-      `"\\"Already a string\\""`,
+      `""Already a string""`,
     )
     expect(tightJsonStringify({a: 1, b: {c: [1, 2, {d: 4}], e: 8}}))
       .toMatchInlineSnapshot(`
-      "{ \\"a\\": 1,
+      "{ "a": 1,
-        \\"b\\": {
+        "b": {
-          \\"c\\": [
+          "c": [
             1,
             2,
-            { \\"d\\": 4 } ],
+            { "d": 4 } ],
-          \\"e\\": 8 } }"
+          "e": 8 } }"
     `)
   })
 })