Table of contents
The Caliper benchmark generator is a Yeoman generator for generating the configuration and callback files used to perform benchmarks on deployed smart contracts. This page will take you through installing and using the generator.
You must first have Yeoman installed to be able to install and use the generator. You can do this using the following command:
npm install -g yo
Once Yeoman is installed, use the following command to install the generator:
npm install -g @hyperledger/generator-caliper
Using the Generator
To use the generator, run the following command
If successful, you should get the following output where you will be prompted to choose a generator - choose Benchmark to run the Caliper benchmark generator:
Welcome to the Hyperledger Caliper generator! ? Which generator would you like to run? (Use arrow keys) ❯ Benchmark
Alternatively, you can run the benchmark generator using:
You will then get the following output where you will be prompted to name your workspace:
Welcome to the Hyperledger Caliper benchmark generator! Let's start off by creating a workspace folder! ? What would you like to call your workspace? myWorkspace
The benchmark generator will inititally take you through generating the callback file and you will be prompted for:
- the name of your smart contract,
- the version of your smart contract,
- a smart contract function
- the argument variables of your smart contract function, which must be entered in array format
By the end, you should have something similar to the following:
Now for the callback file... ? What is the name of your smart contract? fabcar ? What is the version of your smart contract? 0.0.1 ? Which smart contract function would you like to perform the benchmark on? changeCarOwner ? What are the arguments of your smart contract function? (e.g. ["arg1", "arg2"]) ["CAR001", "Tom"]
Next, you will be taken through generating the configuration file and you will be prompted for:
- the name of the benchmark
- a description of the benchmark
- the number of workers
Note: On an invalid input value for workers, a default value will be used.
- a label for differentiating between multiple rounds
- the rate controller you would like to use. The generator currently provides the rate controllers displayed below as options. The generated configuration file will use default
optsfor whichever rate controller is chosen.
? Which rate controller would you like to use? (Use arrow keys) ❯ Fixed Rate Fixed Backlog Linear Rate Fixed Feedback Rate
- the method of which you should like to measure the length of the round. The round may be measured using either transaction duration, which defines the length of the round in seconds, or transaction number, which defines the length of the round using the number of transactions to be generated in the round.
? How would you like to measure the length of the round? (Use arrow keys) ❯ Transaction Duration Transaction Number
- a value for either
txDurationdepending on the answer to previous prompt.
Note: On an invalid input value for either txDuration or txNumber, a default value will be used.
By the end, you should have something similar to the following:
Now for the benchmark configuration file... ? What would you like to name your benchmark? Fabcar benchmark ? What description would you like to provide for your benchamrk? Benchmark for performance testing fabcar contract modules ? How many workers would you like to have? 5 ? What label (hint for test) would you like to provide for your benchmark? Round for changing car owner ? Which rate controller would you like to use? Fixed Rate ? How would you like to measure the length of the round? Transaction Number ? How many transactions would you like to have in this round? 60
On successful generation, you should see the following:
Generating benchmarking files... create myBenchmark/benchmarks/callbacks/changeCarOwner.js create myBenchmark/benchmarks/config.yaml Finished generating benchmarking files
The generator can also be run non-interactively from the command line using the following command line options:
||A workspace to put all the generate benchmark files.|
||The name of your smart contract.|
||The version of your smart contract.|
||Your smart contract function.|
||||The arguments of your smart contract function. These must be in an array format.|
||A name for your benchmark.|
||A description for your benchmark.|
||5||A value for the number of workers.|
||A label for the round.|
||The rate controller.|
||The way you would like to measure the length of the round - either “txDuration” or “txNumber”.|
||50||The value for transaction duration if “txDuration” was entered for txType.|
||50||The value for transaction number if “txNumber” was entered for txType.|
Below is an example of the generator being run non-interactively from the command line using the options above:
yo caliper:benchmark -- --workspace 'myWorkspace' --contractId 'fabcar' --version '0.0.1' --contractFunction 'changeCarOwner' --contractArguments '["CAR001", "Tom"]' --benchmarkName 'Fabcar benchmark' --benchmarkDescription 'Benchmark for performance testing fabcar contract modules' --workers 5 --label 'Round for changing car owner' --rateController 'fixed-rate' --txType 'txDuration' --txDuration 50
Note: All the options above are required when using the generator non-interactively.
The generated files will be placed within the workspace directory you named at the beginning of the generator, and you should have a directory structure similar to the one shown below:
.myWorkspace └── benchmarks │ callbacks │ └── changeCarOwner.js └─ config.yaml
Currently, the generator does not provide
contractArguments as inputs to your callback file. Should these be required, you will need to provide these in the
run function of your callback file.
The generator only generates a single callback file for a single smart contract function. If you would like to test other smart contract functions, you may create more callback files under the callbacks directory. You will also need to update your benchmnark configuration file to take into account the extra callbacks.
Note: The benchmark generator will only create the benchmark configuration file and the callback file. You will still need to provide a network configuration file to be able to perform the benchmark.
The Caliper codebase is released under the Apache 2.0 license. Any documentation developed by the Caliper Project is licensed under the Creative Commons Attribution 4.0 International License. You may obtain a copy of the license, titled CC-BY-4.0, at http://creativecommons.org/licenses/by/4.0/.