Data

Data is a uniquely identified piece of data available for retrieval or transfer.

Multiple data items can be attached to a message when sending data off-chain to another party in a multi-party system. Note that if you pass data in-line when sending a message, those data elements will be stored separately to the message and available to retrieve separately later.

An UUID is allocated to each data resource.

A hash is also calculated as follows:

If there is only data, the hash is of the value serialized as JSON with no additional whitespace (order of the keys is retained from the original upload order).
If there is only a blob attachment, the hash is of the blob data.
There is is both a blob and a value, then the hash is a hash of the concatenation of a hash of the value and a hash of the blob.

Value - JSON data stored in the core database¶

Each data resource can contain a value, which is any JSON type. String, number, boolean, array or object. This value is stored directly in the FireFly database.

If the value you are storing is not JSON data, but is small enough you want it to be stored in the core database, then use a JSON string to store an encoded form of your data (such as XML, CSV etc.).

Datatype - validation of agreed data types¶

A datatype can be associated with your data, causing FireFly to verify the value against a schema before accepting it (on upload, or receipt from another party in the network).

These datatypes are pre-established via broadcast messages, and support versioning. Use this system to enforce a set of common data types for exchange of data across your business network, and reduce the overhead of data verification\ required in the application/integration tier.

More information in the Datatype section

Blob - binary data stored via the Data Exchange¶

Data resources can also contain a blob attachment, which is stored via the Data Exchange plugin outside of the FireFly core database. This is intended for large data payloads, which might be structured or unstructured. PDF documents, multi-MB XML payloads, CSV data exports, JPEG images video files etc.

A Data resource can contain both a value JSON payload, and a blob attachment, meaning that you bind a set of metadata to a binary payload. For example a set of extracted metadata from OCR processing of a PDF document.

One special case is a filename for a document. This pattern is so common for file/document management scenarios, that special handling is provided for it. If a JSON object is stored in value, and it has a property called name, then this value forms part of the data hash (as does every field in the value) and is stored in a separately indexed blob.name field.

The upload REST API provides an autometa form field, which can be set to ask FireFly core to automatically set the value to contain the filename, size, and MIME type from the file upload.

Example¶

{
    "id": "4f11e022-01f4-4c3f-909f-5226947d9ef0",
    "validator": "json",
    "namespace": "ns1",
    "hash": "5e2758423c99b799f53d3f04f587f5716c1ff19f1d1a050f40e02ea66860b491",
    "created": "2022-05-16T01:23:15Z",
    "datatype": {
        "name": "widget",
        "version": "v1.2.3"
    },
    "value": {
        "name": "filename.pdf",
        "a": "example",
        "b": {
            "c": 12345
        }
    },
    "blob": {
        "hash": "cef238f7b02803a799f040cdabe285ad5cd6db4a15cb9e2a1000f2860884c7ad",
        "size": 12345,
        "name": "filename.pdf"
    }
}

Field Descriptions¶

Field Name	Description	Type
`id`	The UUID of the data resource	`UUID`
`validator`	The data validator type	`FFEnum`:
`namespace`	The namespace of the data resource	`string`
`hash`	The hash of the data resource. Derived from the value and the hash of any binary blob attachment	`Bytes32`
`created`	The creation time of the data resource	`FFTime`
`datatype`	The optional datatype to use of validation of this data	`DatatypeRef`
`value`	The value for the data, stored in the FireFly core database. Can be any JSON type - object, array, string, number or boolean. Can be combined with a binary blob attachment	`JSONAny`
`public`	If the JSON value has been published to shared storage, this field is the id of the data in the shared storage plugin (IPFS hash etc.)	`string`
`blob`	An optional hash reference to a binary blob attachment	`BlobRef`

DatatypeRef¶

Field Name	Description	Type
`name`	The name of the datatype	`string`
`version`	The version of the datatype. Semantic versioning is encouraged, such as v1.0.1	`string`

BlobRef¶

Field Name	Description	Type
`hash`	The hash of the binary blob data	`Bytes32`
`size`	The size of the binary data	`int64`
`name`	The name field from the metadata attached to the blob, commonly used as a path/filename, and indexed for search	`string`
`path`	If a name is specified, this field stores the '/' prefixed and separated path extracted from the full name	`string`
`public`	If the blob data has been published to shared storage, this field is the id of the data in the shared storage plugin (IPFS hash etc.)	`string`