Running test plans
Last updated
Last updated
Plans which run on one runner generally should run be usable on all other runners as well. The following table describes the features of different runners.
It is common practice when developing a test plan to use a local runner (local:exec or local:docker) in order to iterate quickly and then move to the Kubernetes cluster:k8s runner when you want to run your test plan with many more test instances.
local:exec
✅
❌
❌
❌
✅
local:docker
✅
❌
✅
✅
✅
cluster:k8s
❌
✅
✅
✅
❌
exec:go
✅
❌
❌
✅
❌
docker:go
❌
✅
✅
✅
❌
docker:generic
❌
✅
✅
❌
✅
A single run, is one in which a single version of the code is being tested. This is the simplest way to run a test plan. If you have run through the tutorial, you have already encountered this:
A single test run can be used to benchmark, validate, or observe the behaviour of a peer-to-peer system.
A composition run, is one in which multiple versions of the same software can be tested simultaneously. Where single runs allow you the performance of a single version of code, compositions allow you to test the way new developments interact with the existing network. This feature has been used heavily during libp2p DHT development.
To define a composition run, a manifest for the __single plan must already exist. Indeed, a composition run is just multiple single runs being executed simultaneously with different versions of code being imported.
composition.toml
Code is divided into two groups. These groups can have any name. In this example, the groups are called group1 and group2.
Both groups will execute the same plan, which is defined in the global section of this file.
Dependencies can be explicitly versioned. To force group1 to use a specific software version, override the dependencies.
Before we can execute this example, make sure we have a test plan we want to run. Create a new plan, or import an existing plan.
Copy the file above and into a file called composition.toml. Because of the way this file is written, plans will use the local:docker runner.
?> Be aware that the composition file will be overwritten to include the names of built artifacts!
Here is how we go about creating a composition test run. The following is a simple example. For a more complete example used by the libp2p project, I recommend you have a look at .