Synchronizing automated test cases

The TFS Test Cases can have associated automation, so that they can be executed as a part of the TFS build, release or by executing a test plan.

Please note that tests can be executed from build and release even without associating automation to test cases, e.g. by running all tests from an assembly. The difference is that if the tests are not executed through the test cases, these executions will not be trackable from the test cases.

TFS Test Case automation currently supports MsTest-bases automation only. This means that with SpecSync, this feature can only be used with SpecFlow projects using MsTest unit test runner. (There is a workaround for projects using SpecFlow+ Runner, see here.)

For all other cases (e.g. for Cucumber projects or xUnit-based SpecFlow projects), SpecSync can synchronize the scenarios to non-automated test cases. The synchronized non-automated test cases can be managed, linked and structured in TFS. You can run them manually as well.

For synchronizing automated test cases, SpecSync uses a SpecFlow plugin to generate a compatible automation method for Scenario Outlines (see Synchronizing Scenario Outlines). For non-automated test cases, this plugin is not necessary.

The following sections describe the most common usage patterns.

Using SpecSync with SpecFlow for automated test cases

In order to synchronize automated test cases with SpecSync, you need to have a SpecFlow project that matches the prerequisites.

Installing the SpecSync NuGet package downloads the syncronization tool and configures the SpecSync SpecFlow plugin, so there is no additional setup required.

The SpecSync plugin generates additional test methods (for Scenario Outlines) that are only used to associate the automation for the test cases and cannot be executed locally. For local execution, these have to be filtered out (see Synchronizing Scenario Outlines for details.)

Using SpecSync with SpecFlow for non-automated test cases

When automated test cases are not required, SpecSync can be setup to synchronize the scenarios to non-automated test cases.

To achieve this, one option is to download the synchronization tool as an executable file and invoke it from the project folder.

Another option is to use the NuGet package, but disable the unnecessary SpecFlow plugin. This can be done by simply removing (or commenting out) the following line from the app.config file of the SpecFlow project:

<!-- remove this line if automated test cases are not required -->
<add name="SpecSync.MTM" type="Generator" />

For SpecFlow projects, SpecSync will try to associate automation for the test cases by default. This has to be switched off as well, by specifying the --skipAutomation ALL option.

Using SpecSync for non-SpecFlow projects (e.g. Cucumber)

Scenarios in non-SpecFlow projects are synchronized as non-automated test cases by default, there is no additional configuration steps needed.

For using SpecSync with non-SpecFlow projects, like Cucumber, please check page Using SpecSync with Cucumber.