npm test (a sample case study of selected libraries)

What are people using as 'Test a package' script for their node packages ? I have taken several node modules from my test project and searched for scripts.test in their package.json.

Here is complete listing of script.test values from package.json files:

assert-plus
aws-sign2
bindings
byline
cli
combined-stream
core-util-is
cssom
ctype
delayed-stream
difflib
domelementtype
forever-agent
heap
inherits
jshint
json-diff.bin
mime
nan
node-uuid
nwmatcher
tunnel-agent
xml2js
xmlhttprequest
xpath
dreamopt             ./node_modules/.bin/mocha
json-diff            ./node_modules/mocha/bin/mocha
json-diff-patch      echo Error: no test specified && exit 1
wordwrap             expresso
when                 jshint . && buster-test -e node && promises-aplus-tests test/promises-aplus-adapter.js
json                 make test
printf               make test
qs                   make test
boom                 make test-cov
cryptiles            make test-cov
hawk                 make test-cov
hoek                 make test-cov
sntp                 make test-cov
entities             mocha && npm run lint
htmlparser2          mocha && npm run lint
domhandler           mocha -R list && jshint index.js test/
domutils             mocha test/tests/**.js && jshint index.js test/**/*.js lib/*.js
cli-color            node ./node_modules/tad/bin/tad lib
es5-ext              node ./node_modules/tad/bin/tad lib
jsdom                node ./test/runner
inherits             node test
marked               node test
json-stringify-safe  node test.js
oauth-sign           node test.js
sax                  node test/index.js
combined-stream      node test/run.js
form-data            node test/run.js
request              node tests/run.js
punycode             node tests/tests.js
open                 node_modules/mocha/bin/mocha
contextify           nodeunit test/
async                nodeunit test/test-async.js
cssstyle             nodeunit tests
xmldom               proof platform win32 && proof test */*/*.t.js || t/test
JSONStream           set -e; for t in test/*.js; do echo '***'  '***'; node ; done
through              set -e; for t in test/*.js; do node ; done
asn1                 tap ./tst
lru-cache            tap test
minimatch            tap test
glob                 tap test/*.js
graceful-fs          tap test/*.js
isarray              tap test/*.js
jsonparse            tap test/*.js
sigmund              tap test/*.js
readable-stream      tap test/simple/*.js
string_decoder       tap test/simple/*.js
http-signature       tap tst/*.js
tough-cookie         vows test.js

No test specified

25 modules have no test specified. It means they have no test or if they have it, you must read the manual how to run it.

tap (Test Anything Protocol tools for node)

11 modules uses tap.

mocha

7 modules uses mocha as their test runner. Also here are differencies, some people rely on global mocha installed, some use local node module.

dreamopt             ./node_modules/.bin/mocha
json-diff            ./node_modules/mocha/bin/mocha
entities             mocha && npm run lint
htmlparser2          mocha && npm run lint
domhandler           mocha -R list && jshint index.js test/
domutils             mocha test/tests/**.js && jshint index.js test/**/*.js lib/*.js
open                 node_modules/mocha/bin/mocha

make

8 uses make

vows

1 uses vows - Asynchronous BDD & continuous integration for node.js

Linting as part of test process

5 uses linting as part of their test process, some more can be hidden in make files (TODO:)

when                 jshint . && buster-test -e node && promises-aplus-tests test/promises-aplus-adapter.js
entities             mocha && npm run lint
htmlparser2          mocha && npm run lint
domhandler           mocha -R list && jshint index.js test/
domutils             mocha test/tests/**.js && jshint index.js test/**/*.js lib/*.js

npm run lint and scripts.lint

Some projects also use scrpts.lint property in their package.json file.

This is arbitrary script and can be run with npm run lint

scripts:{
    ....
    lint: jshint index.js lib/*.js test/*.js,
    ....
    lint: jshint lib/*.js test/*.js test/*/*.js
    ....
}

Naming your test folder

test vs. tests vs. tst or even using directories: in package.json

test modules and devDependencies

Please do not put test harnesses or transpilers in your dependencies object. See devDependencies, below.

Most of used modules use this correctly anyway, for mocha and tap based project.

Other patterns seen

TODO: finish the texts

directories: {
  test: tests
}

// in domutils/package.json

Lessons learned

How npm handles the scripts field

Must read: https://docs.npmjs.com/misc/scripts

Flexible definition of test frameworks

{
  devDependencies: {
    tap: ,
    mocha:
  },
  scripts: {
    test-tap: tap tst/*.js,
    test-mocha:mocha -R list .,

    test:npm run test-mocha
  }
}

Combining test from multile steps

{
  "devDependencies": {
    "tap": "",
    "mocha": "",
    "nsp": "",
    "jshint": ""
  },
  "scripts": {
    "test-tap": "tap tst/*.js",
    "test-mocha": "mocha -R list .",
    "lint": "jshint test/test.js",
    "test-security": "npm shrinkwrap && nsp audit-shrinkwrap",


    "test": "npm run test-mocha && npm run lint && npm run test-security"
  }
}   

This features also nsp, nodesecurity project, to check your dependencies for known vulnerabilities as part of test process.

tap vs. mocha vs. vows vs. ...

Currently I use mocha on my projects. Based on my dependency modules analyzed the tap seems more popular however.

In global mocha seems to be most popular an raising. see www.npm-stats.com for charts: vows, tap and mocha