Testing Piscosour

There are to ways to test piscosour recipes: unit testing and functional testing (integration). Both could be functional but the main difference is that unit testing runs as a node library inside the test and functional testing (integration) runs the recipe as a system executable using require('child_process').exec.

Unit Testing
Functional Testing (integration)

Needed to test steps, plugins (coming soon) and contexts (coming soon)

These tests reside on the test folder of a recipe module.
It just needs the recipe to be tested and its dependencies.
You could generate coverage with this kind of tests

Testing steps

Steps only need dependency to piscosour on devDependencies. Add dependencies to mocha and chai too.

NOTE: If you need a context definition for the step execution, then the dependency to the context has to be on devDependencies too. for example pisco-contexts

package.json:

{
  "...":"...",
  "scripts": {
    "deps": "npm install",
    "test": "node_modules/.bin/mocha -u tdd --recursive"
  },
  "devDependencies": {
    "chai": "*",
    "mocha": "*",
    "piscosour": "^1.0.0"
  },
  "...":"..."
}

Writing a test for piscosour. First require stepTester

 const tester = require('piscosour/lib/tests/stepTester');

command object
setLoggerLevel(level) method
loadStep(command) method
runStep(command) method

command object.

The command object is the configuration parameter that piscosour uses to run the test.

Param	Type	Optional	Description
name	String	No	The name of the step
context	Array	Yes	Array of the contexts runned
baseDir	String	Yes	Folder where the step is going to be runned. Default is the root of the recipe.
params	Object	Yes	All params passed for the execution

setLoggerLevel(level) method

logger.setLoggerLevel(level) set logger level for piscosour. See logger for more information

Param	Type	Optional	Description
level	Number	No	0...5 log levels

loadStep(command) method

logger.loadStep(command) Load local step with all plugins and all piscosour configuration. Returns step object with all piscosour features loaded.

Param	Type	Optional	Description
command	Object	No	See command object

runStep(command) method

logger.runStep(command) Run one local step from the recipe. Returns a promise with the execution of the step.

Param	Type	Optional	Description
command	Object	No	See command object

This is a example of an entire test file.

'use strict';

const path = require('path');
const tester = require('piscosour/lib/tests/stepTester');
const expect = require('chai').expect;
const assert = require('assert');

/* global define, it, describe, before, beforeEach, afterEach, after */

// configure

tester.setLoggerLevel(0);

// constants

const stepName = 'askHello';
const contexts = ['world'];
const message = 'Unit Framework hola!';

describe('Unit testing framework for askHello step', () => {
  it('Should return the step to test', (done) => {
    const step = tester.loadStep({
      name: stepName,
      context: contexts
    });
    expect(step).not.equal(null);
    expect(step.name).to.equal(stepName);
    done();
  });
  it('Should run the step to test', (done) => {
    tester.setLoggerLevel(0);
    tester.runStep({
        name: stepName,
        context: contexts,
        baseDir: path.join(__dirname, 'world'),
        params: {
          paramInquire: message
        }
      })
      .then(() => {
        done();
      })
      .catch((err) => {
        done(err);
      })
  });
  it('Should run the step with plugins to test', (done) => {
    tester.setLoggerLevel(0);
    tester.runStep({
        name: 'emittingHello',
        context: contexts,
        baseDir: path.join(__dirname, 'world'),
        params: {
          paramInquire: message
        }
      })
      .then(() => {
        done();
      })
      .catch((err) => {
        done(err);
      })
  });
});

Testing plugins

(comming soon)

Testing contexts

(comming soon)

Functional testing (integration)

Configure the recipe
Writing tests
Running tests

The functional testing features are:

It is used to test only recipes by executing externally all its commands.
It is possible to test flows and steps.
Executes the recipe command so a configured environment is needed.
Tests are located on an external module, outside of the recipe. Test modules have to be dependencies of the recipe.

Configuring the recipe for testing

Add "test" : "bin/pisco.js -ft" to the scripts object in package.json.
Add dependencies to one or more functional-testing modules.

Fragment of package.json of a recipe.

{
  "scripts": {
    "deps": "npm install",
    "test": "bin/pisco.js -ft"
  },
  "keywords": [
    "piscosour-recipe"
  ],
  "bin": {
    "cells": "bin/pisco.js"
  },
  "dependencies": {
    "piscosour": "^1.1.0"
  },
  "devDependencies": {
    "pisco-functional-tests": "^1.0.1-beta"
  },
  "...":"..."
}

Writing the tests (the functional testing module)

This module must have functional-tests keyword
Dependencies to testing frameworks should be on dependencies not on devDependencies
Dependency to piscosour is not necessary.

Fragment of package.json of the functional testing module:

{
  "...":"...",
  "keywords": [
    "functional-tests"
  ],
  "dependencies": {
    "chai": "*",
    "mocha": "*"
  },
  "...":"..."
}

Now tests must execute the command piscoExec that is injected as an environment variable.

This is an example of tests that we can be written. This test proves the creation of an app using cells-cli recipe.

'use strict';

const pctp = require('pisco-callback-to-promise');
const u = require('../utils');

const fs = require('fs-extra');
const path = require('path');
const exec = require('child_process').exec;

/* global define, it, describe, before, beforeEach, afterEach, after */

// constants

const baseApp = `${__dirname}/../tmp`;

// ---- Tests --------

describe('Run cells app:create', function() {

  const allOk = '[ create ] finished';
  this.timeout(6000);

  it(`Should create a list of apps and say "${allOk}" on all apps`, function(done) {
    pctp.c2p(fs.ensureDir, baseApp)
      .then(() => pctp.c2p(exec, `${process.env.piscoExec} app:create --scaffoldDir scaffold --appName test-app`, {cwd: baseApp}))
      .catch((err) => pctp.logError(err, done));
  });
  afterEach('Should delete the tmp directory', (done) => {
    u.remove([ baseApp ])
      .then(() => done());
  });
});

Let's see this example:

Note that executable is a env variable: process.env.piscoExec
We are using pctp module that converts callbacks to promises.

Running functional tests

From the recipe just with npm test or bin/pisco.js -ft
In the functional testing module just executing mocha. First is needed to export the piscoExec parameter to environment:

export piscoExec="node /Users/sbonacho/projects/cells-cli/bin/pisco.js"
mocha -u tdd --recursive test --timeout 5000 --grep "Unit testing framework for askHello step"

NOTE: Is possible to test docker by changing piscoExec value:

export piscoExec="docker run -ti --rm -p 8000-8100:8000-8100 -p 3000-3100:3000-3100 -v ~/.gradle:/home/pisco/.gradle -v ~/.bowerrc:/home/pisco/.bowerrc -v ~/.npmrc:/home/pisco/.npmrc -v ~/.netrc:/home/pisco/.netrc -v ~/.ssh:/home/pisco/.ssh -v `pwd`:/home/pisco/workspace piscosour/cells-bundle"

The execution of pisco -ft or pisco --functionalTests starts with these messages on stdout. Showing the number of functional testing modules in the recipe, theirs names and versions.


[14:32:49] Number of functional testing modules detected:  1
[14:32:49] Executing piscosour functional tests from pisco-functional-tests ( 1.0.15 )


  Pisco context world validation
    ✓ Should return ....