Synchronizing Scenario Outlines

Scenario Outlines can be used to specify business rules that can be represented with a table of input-output pairs. The following example shows a scenario outline for specifying the addition function of a calculator.

Scenario Outline: Add two numbers
    Given I have entered <a> into the calculator
    And I have entered <b> into the calculator
    When I press add
    Then the result should be <result> on the screen

Examples: 
    | case          | a  | b  | result |
    | classic       | 50 | 70 | 120    |
    | commutativity | 70 | 50 | 120    |
    | zero          | 0  | 42 | 42     |

This scenario outline represents three executable tests, one for each row of the Examples table.

In Team Foundation Server, parametrized test cases can be created for the similar problems. A test case automatically becomes parametrized, once you use a parameter (e.g. @myparam) in the test case steps. The parameter values can be specified in a separate table, similarly to scenario outlines: one row in the parameter values table represents one test, called iteration.

SpecSync syncronizes scenario outlines automatically to parametrized test cases. Once the scenario outline above is synchronized, it produces a test case like this.

Associating automation for a scenario outline test case

Note: Associating automation for scenario outline test cases is only required if you use SpecSync with SpecFlow and would like to use automated test cases. For synchronizing automated and non-automated test cases, check the related documentation page.

Each normal scenario represents a single test and SpecFlow generates a single test method to be able to execute them locally. This test method is associated to the synchronized test case to be able to execute the test case in TFS too.

For scenario outlines, SpecFlow generates multiple test methods, one for each row in the Examples table. These methods cannot be used for associating automation to the synchronized parametrized test cases.

Note: some unit test runners, eg. NUnit SpecFlow generates a single, data-driven test method for scenario outlines. But MsTest does not support in-line test data specifications through attributes, this is not possible for MsTest. TFS requires the associated test methods to use MsTest.

In order to be able to associate automation for these test cases, SpecSync also contains a SpecFlow plugin. The plugin extends the test method generation process and generates an additional method for the scenario outlines, which is configured in a way that it takes the test data from the synchronized test case.

This additional method is sufficient for associating automation for the test case and executing it from TFS, but unfortunately it cannot be executed locally. Therefore this test method has to be filtered out for local executions. To make this easier, the test method is associated with a special test category, called MtmWrapper, so it can be easily filtered out with a category filter.

Filter out "MtmWrapper" test methods

In the Visual Studio Test Explorer window, you can filter out these tests by specifying -Trait:MtmWrapper in the "Search" box.

In TFS build, the TestCategory!=MtmWrapper expression has to be provided in the "Test Filter criteria" option.