v1.1.0 Migration Guide¶

Overview¶

Hyperledger FireFly v1.1.0 is a feature release that includes significant new functionality around namespaces and plugins, as detailed in FIR-12. As a result, upgrading an existing FireFly environment from any prior release may require special steps (depending on the functionality used).

If seamless data preservation is not required, you can simply create a new network from scratch using FireFly v1.1.0.

If you want to preserve data from an existing 1.0.x network, significant care has been taken to ensure that it is possible. Most existing environments can be upgraded with minimal extra steps. This document attempts to call out all potentially breaking changes (both common and uncommon), so that you can easily assess the impact of the upgrade and any needed preparation before proceeding.

Before Upgrading¶

These steps are all safe to do while running FireFly v1.0.x. While they do not have to be done prior to upgrading, performing them ahead of time may allow you to preemptively fix some problems and ease the migration to v1.1.0.

Common Steps¶

Upgrade to latest v1.0.x patch release

Before upgrading to v1.1.0, it is strongly recommended to upgrade to the latest v1.0.x patch release (v1.0.4 as of the writing this document). Do not proceed any further in this guide until all nodes are successfully running the latest patch release version.

Fix any deprecated config usage

All items in FireFly's YAML config that were deprecated at any time in the v1.0.x line will be unsupported in v1.1.0. After upgrading to the latest v1.0.x patch release, you should therefore look for any deprecation warnings when starting FireFly, and ensure they are fixed before upgrading to v1.1.0. Failure to do so will cause your config file to be rejected in v1.1.0, and FireFly will fail to start.

You can utilize the ffconfig tool to automatically check and fix deprecated config with a command such as:

ffconfig migrate -f <input-file> -o <output-file> --to 1.0.4

This should ensure your config file is acceptable to 1.0.x or 1.1.x.

Note that if you are attempting to migrate a Dockerized development environment (such as one stood up by the firefly-cli), you may need to edit the config file inside the Docker. Environments created by a v1.0.x CLI do not expose the config file outside the Docker container.

Less Common Situations¶

Record all broadcast namespaces in the config file

Expand for migration details only if your application uses non-default namespaces.

FireFly v1.0 allowed for the dynamic creation of new namespaces by broadcasting a namespace definition to all nodes. This functionality is _removed_ in v1.1.0. If your network relies on any namespaces that were created via a broadcast, you must add those namespaces to the `namespaces.predefined` list in your YAML config prior to upgrade. If you do not, they will cease to function after upgrading to v1.1.0 (all events on those namespaces will be ignored by your node).

Identify queries for organization/node identities

Expand for migration details only if your application queries /network/organizations or /network/nodes.

Applications that query `/network/organizations` or `/network/nodes` will temporarily receive _empty result lists_ after upgrading to v1.1.0, just until all identities have been re-registered (see steps in "After Upgrading"). This is because organization and node identities were broadcast on a global "ff_system" namespace in v1.0, but are no longer global in v1.1.0. The simplest solution is to shut down applications until the FireFly upgrade is complete on all nodes and all identities have been re-broadcast. If this poses a problem and you require zero downtime from these APIs, you can proactively mitigate with the following steps in your application code: - Applications that query the `/network/organizations` may be altered to _also_ query `/namespaces/ff_system/network/organizations` and combine the results (but should disregard the second query if it fails). - Applications that query the `/network/nodes` may be altered to _also_ query `/namespaces/ff_system/network/nodes` and combine the results (but should disregard the second query if it fails). Further details on the changes to `/network` APIs are provided in the next section.

Identify usage of changed APIs

Expand for migration details on all changes to /namespaces, /status, and /network APIs.

The primary API change in this version is that the "global" paths beginning with `/network` and `/status` have been relocated under the `/namespaces/{ns}` prefix, as this data is now specific to a namespace instead of being global. At the same time, the API server has been enhanced so that omitting a namespace from an API path will _query the default namespace_ instead. That is, querying `/messages` is now the same as querying `/namespaces/default/messages` (assuming your default namespace is named "default"). This has the effect that most of the moved APIs will continue to function without requiring changes. See below for details on the affected paths. These global routes have been moved under `/namespaces/{ns}`. Continuing to use them without the namespace prefix **will still work**, and will simply query the default namespace.

/network/diddocs/{did}
/network/nodes
/network/nodes/{nameOrId}
/network/nodes/self
/network/organizations
/network/organizations/{nameOrId}
/network/organizations/self
/status
/status/batchmanager

These global routes have been moved under `/namespaces/{ns}` and have also been deprecated in favor of a new route name. Continuing to use them without the namespace prefix **will still work**, and will simply query the default namespace. However, it is recommended to switch to the new API spelling when possible.

/network/identities - replaced by existing /namespaces/{ns}/identities
/network/identities/{did} - replaced by new /namespaces/{ns}/identities/{did}

These global routes have been have been permanently renamed. They are deemed less likely to be used by client applications, but any usage **will be broken** by this release and must be changed after upgrading.

/status/pins - moved to /namespaces/{ns}/pins (or /pins to query the default namespace)
/status/websockets - moved to /websockets

The response bodies of the following APIs have also had fields removed. Any usage of the removed fields **will be broken** by this release and must be changed after upgrading.

/namespaces - removed all fields except "name", "description", "created"
/namespaces/{ns} - same as above
/namespaces/{ns}/status - removed "defaults"

Adjust or remove usage of admin APIs

Expand for migration details on all changes to /admin and /spi.

FireFly provides an administrative API in addition to the normal API. In v1.1.0, this has been renamed to SPI (Service Provider Interface). Consequently, all of the routes have moved from `/admin` to `/spi`, and the config section has been renamed from `admin` to `spi`. There is no automatic migration provided, so any usage of the old routes will need to be changed, and your config file will need to be adjusted if you wish to keep the SPI enabled (although it is perfectly fine to have both `admin` and `spi` sections if needed for migration). The ability to set FireFly config via these routes has also been removed. Any usage of the `/admin/config` routes must be discontinued, and config should be set exclusively by editing the FireFly config file. The only route retained from this functionality was `/admin/config/reset`, which has been renamed to `/spi/reset` - this will continue to be available for performing a soft reset that reloads FireFly's config.

Performing the Upgrade¶

Backup current data

Before beginning the upgrade, it is recommended to take a full backup of your FireFly database(s). If you encounter any serious issues after the upgrade, you should revert to the old binary and restore your database snapshot. While down-migrations are provided to revert a database in place, they are not guaranteed to work in all scenarios.

Upgrade FireFly and all dependencies

Bring FireFly down and replace it with the new v1.1.0 binary. You should also replace other runtimes (such as blockchain, data exchange, and token connectors) with the supported versions noted in the v1.1.0 release. Once all binaries have been replaced, start them up again.

After Upgrading¶

Ensure nodes start without errors

Ensure that FireFly starts without errors. There will likely be new deprecation warnings for config that was deprecated in v1.1.0, but these are safe to ignore for the moment. If you face any errors or crashes, please report the logs to the FireFly channel on Discord, and return your nodes to running the previous version of FireFly if necessary.

Re-broadcast organization and node identities

Once all nodes in the multiparty network have been upgraded and are running without errors, each node should re-broadcast its org and node identity by invoking /network/organizations/self and /network/nodes/self (or, if your application uses a non-default namespace, by invoking the /namespace/{ns}-prefixed versions of these APIs).

This will ensure that queries to /network/organizations and /network/nodes return the expected results, and will register the identities in a way that can be supported by both V1 and V2 multiparty contracts (see "Upgrading the Multi-Party Contract").

Update config file to latest format

Once the network is stable, you should update your config file(s) again to remove deprecated configuration and set yourself up to take advantage of all the new configuration options available in v1.1.0.

You can utilize the ffconfig tool to automatically check and fix deprecated config with a command such as:

ffconfig migrate -f <input-file> -o <output-file>

Upgrading the Multi-Party Contract¶

FireFly v1.1.0 includes a new recommended version of the contract used for multi-party systems (for both Ethereum and Fabric). It also introduces a versioning method for this contract, and a path for migrating networks from one contract address to a new one.

After upgrading FireFly itself, it is recommended to upgrade your multi-party system to the latest contract version by following these steps.

Compile and deploy an instance of the new FireFly contract (linked above) to your blockchain, using ff deploy or a similar method.
Edit the config file on each node in your network, to add the new contract to the multi-party contract list like so:

namespaces:
  predefined:
  - name: default
    multiparty:
      enabled: true
      contract:
      - location:
          address: 0x09f107d670b2e69a700a4d9ef1687490ae1568db
      - location:
          address: 0x1bee32b37dc48e99c6b6bf037982eb3bee0e816b

This example assumes 0x09f1... represents the address of the original contract, and 0x1bee... represents the new one. Note that if you have multiple namespaces, you must repeat this step for each namespace in the config - and you must deploy a unique contract instance per namespace (in the new network rules, multiple namespaces cannot share a single contract).

After updating each node's configuration, restart the node and ensure it starts without issues.
Have any member of the multi-party network invoke the /namespaces/{ns}/network/action FireFly API with a body of {"type": "terminate"}. This will terminate the old contract and instruct all members to move simultaneously to the newly configured one.
Verify success by querying /namespaces/{ns}/status on each node and checking that the active multi-party contract matches the new address.