PKG-JS-TOOLS(7)

📦
version 0.17.8 · 4 versions
, in debian trixie, ubuntu questing, ubuntu resolute, ubuntu stonking

PKG-JS-TOOLS(7) — the bestie edition 💖

okay so welcome to the Miscellaneous Information Manual, section 7, and it's literally all about pkg-js-tools — a whole lil collection of tools that helps you package Node modules in Debian. we love to see it 🎀

✨ NAME ✨

pkg-js-tools — collection of tools to aid packaging Node modules in Debian.

no cap, that's the whole vibe. moving on 💅

🌸 WORKING WITH SALSA.DEBIAN.ORG REPOSITORY 🌸

so salsa(1) can use the pkg-javascript config, and to make that happen you just add something like this to your .bashrc:

alias js-salsa='salsa --conf-file +/usr/share/pkg-js-tools/pkg-js-salsa.conf'

and then bestie, you can just use salsa like it's nothing. some examples:

  • if you made a local repo and you wanna create it and push it on salsa, you literally just launch:
js-salsa push_repo .
  • to configure a repo that's already been pushed:
js-salsa update_safe node-foobar
  • to clone a js-team package locally:
js-salsa co node-foobar

peep salsa(1) for more. iykyk 🦋

💜 DEBHELPER ADDON 💜

the pkg-js-tools debhelper addon loads itself automatically if a package build-depends on dh-sequence-nodejs — or, the old fashion way, if dh is called with --with nodejs in debian/rules.

🌈 Quick use

examples of basic migration to pkg-js-tools:

Switch test and install to pkg-js-tools

⭐ How it works

pkg-js-tools provides hooks for these steps:

  • configure — populate node_modules/
  • build — build components and main module
  • test — test components and main module
  • install — install components and main module
  • installdocs — can auto-generate docs for each component
  • clean — clean pkg-js-tools stuff

technically (and this is the part that's giving "how does she know that"), it adds --buildsystem=nodejs to the corresponding dh_auto_<step> command.

Important note, bestie. Here's the vocab you gotta lock in: - component is the directory name of a submodule (uscan(1) component or additional components listed in debian/nodejs/additional_components). Example: types-glob - module is the npmjs name. Example: @types/glob

See Group Sources Tutorial

for more about embedding components. 👑

details, and yes i'm listing all of them:

  • dh_auto_clean, cleans files automatically installed by pkg-js-tools itself
  • dh_auto_configure, automatically populates node_modules directory:
    • links embedded components
    • links global modules listed in debian/nodejs/extlinks
    • copies global modules listed in debian/nodejs/extcopies
  • dh_auto_build, Remember to add a dh_auto_build --buildsystem=nodejs in override_dh_auto_build section if your debian/rules file has such section, else this step will be ignored. (not me warning you fr fr 😭). Builds:
    • components by launching sh -ex debian/nodejs/<component-name>/build in this file exists
    • main module by launching sh -ex debian/nodejs/build if exists
  • dh_auto_test, tests:
    • components by launching sh -ex debian/nodejs/<component-name>/test if this file exists
    • main module by launching sh -ex debian/tests/pkg-js/test if this file exists. This test is also used by pkg-js-autopkgtest(7) during autopkgtest. If you want to have a different test during build, set this test in debian/nodejs/test.
  • dh_auto_install: installs modules in the good directories and provides some debhelper variables to be used in debian/control. Note that if your package provides more that one binary package, you have to use some debian/<package-name>.install files to distribute the files. Steps:
    • components: determine files to install using the same algorithm than main module and install them:
      • nowhere if component if debian/nodejs/submodules exists and component isn't listed in it (an empty debian/nodejs/submodules drops all components)
      • nowhere if component is listed in debian/nodejs/additional_components with a "!" prefix
      • in main nodejs directories if component is listed in debian/nodejs/root_modules
      • else: in a node_modules subdirectory of main module
    • main module, determine files to install (see below) and install them in the "good" directory:
      • if "architecture" is "all": /usr/share/nodejs
      • else: /usr/lib/${DEB_HOST_MULTIARCH}/nodejs
    • links: builds symlinks listed in debian/nodejs/links. Same usage as debian/links except that source or destination not starting with / are related to arch path (/usr/share/nodejs or /usr/lib/<gnu-arch>/nodejs)
    • Build pkgjs-lock.json files: if package "maybe a bundle" (built with webpack, browserify,...), pkg-js-tools builds a pkgjs-lock.json for each module. This files may help in Debian transitions
    • Variables usable in debian/control:
      • ${nodejs:Version}: the version of nodejs used during build
      • ${nodejs:Provides}: virtual names to be added into "Provides:" field. This lists all modules installed in nodejs root directories
      • ${nodeFoo:Provides}: for a source package that provides several binary packages, dh-sequence-nodejs filters ${nodejs:Provides} for each binary package. The package name is converted into its camelcase name: node-jest-worker becomes nodeJestWorker
      • ${nodejs:BuiltUsing}: when package "maybe a bundle", lists packages and versions used to build package. Use it in XB-Javascript-Built-Using field
  • dh_installdocs: dh-sequence-nodejs provides a tool named dh_nodejs_autodocs which can be used in a override_dh_installdocs to automatically generate documentation for each component. See related manpage
  • dh_install: dh-sequence-nodejs provides a tool named dh_nodejs_build_debug_package which can be used to build a separate debug package with sourcemap files when package size is too big. See related manpage

🦋 Automatically detect some additional components 🦋

starting from 0.12.0, dh-sequence-nodejs automatically reads lerna.conf and reads "packages" field to find additional components.

starting from 0.12.7, it does the same when package.json has a "workspaces" field.

this auto-detection automatically drops "test" and "tests" directories. You can override this behavior using debian/additional_components.

if a component should not be considered, insert its name preceded by a "!" in debian/nodejs/additional_components.

to disable this feature, use dh-sequence-nodejs-no-lerna. periodt. 💅

🎀 Algorithm to determine files to install 🎀

pkg-js-tools tries to reproduce npm(1) behavior: it reads package.json and/or .npmignore files to determine files to install except that it drops licenses, *.md, doc*, example*, test*, makefiles,....

this behavior is overridden if:

  • debian/nodejs/install (or debian/nodejs/<component-name>/install) exists. This file uses the same format than debian/install.
  • debian/nodejs/files (or debian/nodejs/<component-name>/files) exists. the content of this file replaces "files" entry of package.json

💖 pkg-js-tools files 💖

  • all steps:
    • debian/nodejs/additional_components is used to set some subdirectories that should be considered as components even if they are not listed in debian/watch. Content example: packages/*. Important note: in this example, component name is packages/foo in every other files, including paths
    • debian/nodejs/main is used to indicates where is the main module. In a package containing only components (bundle package), you should choose one of them as main component
  • configure step:
    • debian/build_modules additional modules needed to build, will be linked in node_modules directory
    • debian/nodejs/component_links lists needed links between components: links ../../component-src in component-dst/node_modules/component-src-name
    • debian/nodejs/<component-name>/nolink avoids node_modules links creation for this component (configuration step)
    • debian/nodejs/extlinks lists installed node modules that should be linked into node_modules directory (modules are searched using nodejs algorithm). You can mark them with "test" to avoid errors when build profile contains nocheck
    • debian/nodejs/extcopies lists installed node modules that should be copied into node_modules directory. You can also mark them with "test"
    • debian/nodejs/<component>/extlinks lists installed node modules that should be linked in <component>/node_modules directory (test flag available)
    • debian/nodejs/<component>/extcopies lists installed node modules that should be copied in <component>/node_modules directory (test flag available)
  • build step:
    • debian/nodejs/build custom build. An empty file stops auto build
    • debian/nodejs/<component>/build: same for components

🌸 more parts incoming, don't go anywhere bestie 🌸

  • debian/nodejs/build_order — this is the ✨build order✨ list, one component per line. If you don't make one, components get built in alphabetic order… except the ones declared in debian/nodejs/component_links, because a component that depends on another one has to be built after it. Dependencies first, bestie, that's just how it is 💅
  • 🧪 test step:
    • debian/tests/test_modules/ — extra modules you need for running tests go in here as subdirectories, and they get linked into the node_modules directory during the test step only. So they're not hanging around in the final package, ngl very clean of us 🧼
    • debian/tests/pkg-js/test — the script that gets launched during test (and yes, it's launched with set -e, so the second something fails, everything stops. no covering things up 🙈)
    • debian/tests/pkg-js/files — lists the other files, i.e. anything besides debian/tests/test_modules/* and the installed files, that you need for autopkgtest. Default is test* in case you were wondering 🎀
    • debian/nodejs/test — overwrites debian/tests/pkg-js/test during build, for when the test is different in build vs autopkgtest
    • debian/nodejs/<component-name>/test — same idea but for components (and this one is launched during build only)
    • autopkgtest files:
      • debian/tests/autopkgtest-pkg-nodejs.conf — an autodep8 configuration file, which lets you add packages or restrictions during autopkgtest only
        • extra_depends=p1, p2, p3 — permits one to add p1, p2 and p3 packages 🫶
        • extra-restrictions=needs-internet — permits one to add additional restrictions during autopkgtest
      • debian/tests/pkg-js/require-name — contains the name to use in the autopkgtest require test, instead of whatever the package.json value says 👑
  • 📦 install step:
    • debian/nodejs/submodules — lists the components to install (all of them if the file is missing)
    • debian/nodejs/root_modules — lists components to install in the nodejs root directory, instead of the node_modules subdirectory. And if this file contains *, literally all components get installed in the root directory. asterisk ate and left no crumbs ⭐
    • debian/nodejs/files — overwrites the package.json#files field
    • debian/nodejs/<component-name>/files — overwrites the package.json#files field, per component. An empty file means avoid any install at all (it's giving "we're not doing that today" 💀)
    • debian/nodejs/name — overwrites the package.json#name field
    • debian/nodejs/<component-name>/name — overwrites package.json#name for that component
    • debian/nodejs/install — overwrites debian/nodejs/files: same usage as debian/install, except that destinations not starting with / are related to the arch path (/usr/share/nodejs or /usr/lib/<gnu-arch>/nodejs)
    • debian/nodejs/<component-name>/install — same as debian/nodejs/install but for components
  • 🔗 link step:
    • debian/nodejs/links — same usage as debian/links, except that a source or destination not starting with / is related to the arch path (/usr/share/nodejs or /usr/lib/<gnu-arch>/nodejs)
NB: To install a component in another directory, set its files in debian/install. ✍️

Example:

  • debian/control
...
Testsuite: autopkgtest-pkg-nodejs
Build-Depends: dh-sequence-nodejs
...
  • debian/tests/pkg-js/test
mocha -R spec

See also pkg-js-autopkgtest README. 📖

🌈 Multiple binary packages 🌈

When debian/control provides more than one binary package, dh_auto_install populates a debian/tmp, and then dh_install installs files into each package. In that case you must write a debian/<package>.install for every binary package, bestie, no skipping 😤 And each line with only one argument is related to debian/tmp.

Examples:

  • debian/node-arch-indep.install — pick files from debian/tmp:
usr/share/nodejs/foo/
  • debian/node-arch-dep.install — pick files from debian/tmp:
usr/lib/*/nodejs/foo/
  • debian/libjs.install — pick files from sources:
dist/* usr/share/javascript/foo/

Since the path is not fixed for an arch-dependent package, you gotta write debian/nodejs/links:

# debian/nodejs/links
foo/bin/cli.js  /usr/bin/foo

With an arch independent package, pkg-js-tools transforms this into:

/usr/share/nodejs/foo/bin/cli.js    /usr/bin/foo

and for an arch dependent package, it uses DEB_GNU_ARCH. Example with amd64:

/usr/lib/x86_64-linux-gnu/foo/bin/cli.js  /usr/bin/foo

All fields that do not start with / are prefixed with the good nodejs path. automagic, literally ✨

📚 Component docs 📚

Starting from version 0.13.0, pkg-js-tools provides dh_nodejs_autodocs. This tool automatically installs README.md, CONTRIBUTING.md,… for each root component into its /usr/share/doc/node-<name> directory. And if no debian/*docs is found, it does the same for the main component too. To use it:

override_dh_installdocs:
 dh_installdocs
 dh_nodejs_autodocs

🚫 .eslint* files 🚫

The pkg-js-tools auto installer always removes .eslint* files — unless it's explicitly specified in debian/nodejs/**/files or debian/nodejs/**/install. Not up for debate, sorry 💅

😵‍💫 Having different test between build and autopkgtest 😵‍💫

When debian/nodejs/test exists, that file is used during the build test instead of debian/tests/pkg-js/test. This permits one to have a different test. You can also just overwrite the dh_auto_test step in debian/rules:

override_dh_auto_test:
      # No test during build (test needs Internet)

🌐 Autopkgtest additional test packages or test restrictions 🌐

autodep8 allows one to add additional packages during autopkgtest (and/or additional restrictions) by using a debian/tests/autopkgtest-pkg-nodejs.conf file:

extra_depends=mocha, npm
extra-restrictions=needs-internet

🔍 LINTIAN PROFILES 🔍

pkg-js-tools provides a lintian profile:

  • pkg-js-extra — launches additional checks (repo consistency, see debcheck-node-repo below) 👀

To use them:

lintian --profile pkg-js-extra ../node-foo_1.2.3-1.changes

🛠️ OTHER TOOLS 🛠️

See related manpages. 💖

  • add-node-component — automatically modifies gbp.conf and debian/watch to add a node component. See JS Group Sources Tutorial. It can also list components or modules (real names)
  • getFromNpmCache — export npm cache content to standard output
  • github-debian-upstream — launches it in the source repo to create a debian/upstream/metadata (works only if the upstream repo is on GitHub)
  • nodepath — shows the path of a node module (npm name). You can use -p to show also the Debian package. Option -o shows only the Debian package name
  • debcheck-node-repo — checks repo consistency: compares the vcs repo registered in the npm registry with the source repo declared in debian/watch"
  • dh_nodejs_autodocs — automatically select and install documentation files to install for each component
  • dh_nodejs_build_debug_package — move sourcemap files from binary packages to a separated debug package
  • mjs2cjs — build commonjs file using rollup
  • pkgjs-audit — creates a temporary package-lock.json file using the Debian package values used by the module to analyze, and launches a npm audit. If the module is a bundle (and then has a pkgjs-lock.json), pkgjs-audit uses pkgjs-lock.json, else it generates its package-lock.json using available values 🔎
  • pkgjs-depends — search recursively the dependencies of the given module name (if not given, uses the current package.json) and displays related Debian packages and missing dependencies
  • pkgjs-install — same as npm install but uses Debian JS modules when available
  • pkgjs-install-minimal — same as pkgjs-install but uses only available Debian modules. It is included in dh-nodejs so it can be used during build
  • pkgjs-ls — same as npm ls but it searches also in global nodejs paths
  • pkgjs-run — same as npm run
  • pkgjs-lerna run — same as lerna run (only the run command is implemented)
  • pkgjs-utils, pkgjs-ln, pkgjs-main, pkgjs-pjson — various utilities. See pkgjs-utils(1) manpage
  • pkgjs — tries to reproduce yarnpkg(1) commands with pkgjs-* scripts 🧚

👀 SEE ALSO 👀

debhelper(7), pkg-js-autopkgtest(7), uscan(1), add-node-component(1), github-debian-upstream(1), nodepath(1), mjs2cjs(1), pkgjs-ls(1), pkgjs-depends(1), pkgjs-audit(1), pkgjs-utils(1), pkgjs-install(1)

✨ FEATURES HISTORY ✨

Okay bestie, this is the receipts section 💅 — literally a whole timeline of when each tool showed up and when each feature dropped. Think of it as the "since when does this exist" cheat sheet, iykyk. Every version number here is the minimal version that has the thing, so if your package is older than that, it does not have it, periodt.

🛠️ TOOL ✨

Here's the glow-up timeline for the actual commands. If you're running something older than the number next to it, that tool (or that flag!) isn't in your life yet 😭

  • add-node-component — 0.8.14
  • add-node-component --cmp-tree — 0.9.22
  • debcheck-node-repo — 0.8.14
  • dh_nodejs_autodocs — 0.13.0
  • dh_nodejs_autodocs auto_dispatch — 0.14.5
  • dh_nodejs_build_debug_package — 0.15.5
  • dh_nodejs_substvars — 0.14.5
  • dh-make-node — 0.9.18
  • getFromNpmCache — 0.14.32
  • mjs2cjs — 0.12.3
  • mjs2cjs -a — 0.14.14
  • pkgjs — 0.15.21
  • pkgjs-audit — 0.11.2
  • pkgjs-depends — 0.9.54
  • pkgjs-depends --graph — 0.14.34
  • pkgjs-install — 0.14.20
  • pkgjs-install-minimal — 0.14.27
  • pkgjs-ln — 0.9.76
  • pkgjs-lerna — 0.15.13
  • pkgjs-ls — 0.9.30
  • pkgjs-main — 0.9.76
  • pkgjs-pjson — 0.9.76
  • pkgjs-run — 0.14.22
  • pkgjs-utils — 0.9.75

🌈 FEATURE ✨

And now the vibes timeline 🦋 — the behaviours and conventions, with the oldest version that supports each one. Lowkey the most useful part if you're tryna figure out why your build is doing the thing.

  • additional_components — 0.9.11
  • auto build (grunt) — 0.9.3
  • autopkgtest skip — 0.9.30
  • auto-install (arch-dep) — 0.9.27
  • build order — 0.9.10
  • dh-sequence-nodejs — 0.9.41
  • follow lerna.json#useWorkspaces — 0.14.8
  • .npmignore support — 0.9.53
  • support lerna.conf — 0.12.0
  • support workspaces — 0.12.7
  • debian/nodejs/main — 0.9.11
  • debian/tests/test_modules — 0.9.33
  • debian/build_modules — 0.9.33
  • ${nodejs:BuiltUsing} — 0.11.8
  • ${nodejs:Provides} — 0.9.10
       │ ${nodejs:Version}                           0.9.38          │
       ├─────────────────────────────────────────────────────────────┤
       │ ${nodeFoo:Provides}                         0.14.5          │
       ├─────────────────────────────────────────────────────────────┤
       │ ordered_components_list                     0.15.13         │
       ├─────────────────────────────────────────────────────────────┤
       │ packages_list                               0.15.13         │
       └─────────────────────────────────────────────────────────────┘

okay bestie, legal o'clock, but it's actually a slay 💅

Copyright Yadd <yadd@debian.org> 🫶

This library is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2, or (at your option) any later version. ✨

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. 😭 See the GNU General Public License for more details. (yeah, the all-caps is literally screaming at you, that's how licenses do)

On Debian systems, the complete text of version 2 of the GNU General Public License can be found in /usr/share/common-licenses/GPL-2. If not, see GNU licenses. 📄

ngl if you're shipping this in your own package, read the actual license text fr fr — don't just vibe off this page 💀

and that's the whole page, bestie!! 🎀 you made it all the way through — iconic behavior 👑

this is the tail end of pkg-js-tools(7), dated June 2024, and it's giving main character energy from the very last line to the very first. go forth and package some JS 💖✨🦋🌸🍓🧚💜

📦 where this page comes from