Aries RFC 0011: Decorators

• Authors: Daniel Hardman
• Status: ADOPTED
• Since: 2019-01-31
• Status Note: broadly used in Indy, but not yet harmonized with DIF work on hubs.
• Start Date: 2018-12-14
• Tags: concept, decorator

Summary

Explain how decorators work in DID communication.

Motivation

Certain semantic patterns manifest over and over again in communication. For example, all communication needs the pattern of testing the type of message received. The pattern of identifying a message and referencing it later is likely to be useful in a high percentage of all protocols that are ever written. A pattern that associates messages with debugging/tracing/timing metadata is equally relevant. And so forth.

We need a way to convey metadata that embodies these patterns, without complicating schemas, bloating core definitions, managing complicated inheritance hierarchies, or confusing one another. It needs to be elegant, powerful, and adaptable.

Tutorial

A decorator is an optional chunk of JSON that conveys metadata. Decorators are not declared in a core schema but rather supplementary to it. Decorators add semantic content broadly relevant to messaging in general, and not so much tied to the problem domain of a specific type of interaction.

You can think of decorators as a sort of mixin for agent-to-agent messaging. This is not a perfect analogy, but it is a good one. Decorators in DIDComm also have some overlap (but not a direct congruence) with annotations in Java, attributes in C#, and both decorators and annotations in python.

Simple Example

Imagine we are designing a protocol and associated messages to arrange meetings between two people. We might come up with a meeting_proposal message that looks like this:

{
  "@id": "e2987006-a18a-4544-9596-5ad0d9390c8b",
  "@type": "did:sov:8700e296a1458aad0d93;spec/meetings/1.0/proposal",
  "proposed_time": "2019-12-23 17:00",
  "proposed_place": "at the cathedral, Barfüsserplatz, Basel",
  "comment": "Let's walk through the Christmas market."
}

Now we tackle the meeting_proposal_response messages. Maybe we start with something exceedingly simple, like:

{
  "@id": "d9390ce2-8ba1-4544-9596-9870065ad08a",
  "@type": "did:sov:8700e296a1458aad0d93;spec/meetings/1.0/response",
  "agree": true,
  "comment": "See you there!"
}

But we quickly realize that the asynchronous nature of messaging will expose a gap in our message design: if Alice receives two meeting proposals from Bob at the same time, there is nothing to bind a response back to the specific proposal it addresses.

We could extend the schema of our response so it contains an thread that references the @id of the original proposal. This would work. However, it does not satsify the DRY principle of software design, because when we tackle the protocol for negotiating a purchase between buyer and seller next week, we will need the same solution all over again. The result would be a proliferation of schemas that all address the same basic need for associating request and response. Worse, they might do it in different ways, cluttering the mental model for everyone and making the underlying patterns less obvious.

What we want instead is a way to inject into any message the idea of a thread, such that we can easily associate responses with requests, errors with the messages that triggered them, and child interactions that branch off of the main one. This is the subject of the message threading RFC, and the solution is the ~thread decorator, which can be added to any response:

{
  "@id": "d9390ce2-8ba1-4544-9596-9870065ad08a",
  "@type": "did:sov:8700e296a1458aad0d93;spec/meetings/1.0/response",
  "~thread": {"thid": "e2987006-a18a-4544-9596-5ad0d9390c8b"},
  "agree": true,
  "comment": "See you there!"
}

This chunk of JSON is defined independent of any particular message schema, but is understood to be available in all DIDComm schemas.

Basic Conventions

Decorators are defined in RFCs that document a general pattern such as message threading RFC or message localization. The documentation for a decorator explains its semantics and offers examples.

Decorators are recognized by name. The name must begin with the ~ character (which is reserved in DIDComm messages for decorator use), and be a short, single-line string suitable for use as a JSON attribute name.

Decorators may be simple key:value pairs "~foo": "bar". Or they may associate a key with a more complex structure:

"~thread": {
  "thid": "e2987006-a18a-4544-9596-5ad0d9390c8b",
  "pthid": "0c8be298-45a1-48a4-5996-d0d95a397006",
  "sender_order": 0
}

Decorators should be thought of as supplementary to the problem-domain-specific fields of a message, in that they describe general communication issues relevant to a broad array of message types. Entities that handle messages should treat all unrecognized fields as valid but meaningless, and decorators are no exception. Thus, software that doesn't recognize a decorator should ignore it.

However, this does not mean that decorators are necessarily optional. Some messages may intend something tied so tightly to a decorator's semantics that the decorator effectively becomes required. An example of this is the relationship between a general error reporting mechanism and the ~thread decorator: it's not very helpful to report errors without the context that a thread provides.

Because decorators are general by design and intent, we don't expect namespacing to be a major concern. The community agrees on decorators that everybody will recognize, and they acquire global scope upon acceptance. Their globalness is part of their utility. Effectively, decorator names are like reserved words in a shared public language of messages.

Namespacing is also supported, as we may discover legitimate uses. When namespaces are desired, dotted name notation is used, as in ~mynamespace.mydecoratorname. We may elaborate this topic more in the future.

Decorators are orthogonal to JSON-LD constructs in DIDComm messages.

Versioning

We hope that community-defined decorators are very stable. However, new fields (a non-breaking change) might need to be added to complex decorators; occasionally, more significant changes might be necessary as well. Therefore, decorators do support semver-style versioning, but in a form that allows details to be ignored unless or until they become important. The rules are:

1. As with all other aspects of DIDComm messages, unrecognized fields in decorators must be ignored.
2. Version information can be appended to the name of a decorator, as in ~mydecorator/1. Only a major version (never minor or patch) is used, since:
   • Minor version variations should not break decorator handling code.
   • The dot character . is reserved for namespacing within field names.
   • The extra complexity is not worth the small amount of value it might add.
3. A decorator without a version is considered to be synonymous with version 1.0, and the version-less form is preferred. This allows version numbers to be added only in the uncommon cases where they are necessary.

Decorator Scope

A decorator may be understood to decorate (add semantics) at several different scopes. The discussion thus far has focused on message decorators, and this is by far the most important scope to understand. But there are more possibilities.

Suppose we wanted to decorate an individual field. This can be done with a field decorator, which is a sibling field to the field it decorates. The name of decorated field is combined with a decorator suffix, as follows:

{
  "note": "Let's have a picnic.",
  "note~l10n": { ... }
}

In this example, taken from the localization pattern, note~l10n decorates note.

Besides a single message or a single field, consider the following scopes as decorator targets:

• Message Family: An decorator can be declared in documentation for the message family, explaining semantics that apply to all message instances in the family unless overridden at a narrower scope. This is not the same as declaring the decorator's structure to be part of the schema of a message family; this is saying that the field values in a concrete instance of a decorator applies to all instances of a message family. The localization pattern contemplates this usage as a way to declare default localization semantics.
• Message Type: A decorator can be declared in the documentation on a specific message type. If this happens, semantics of the decorator on the message type should be understood as overriding or enriching the semantics of that same decorator on the parent message family, since the scope is becoming more specific.
• Message Thread or thread tree: A participant in a thread could send a message containing nothing but the ~thread with one or more additional decorators inside it. This should be viewed as a statement of semantics that apply to subsequent communications from the sender, on that thread, until further notice or until overridden at the scope of an individual message instance. (While powerful, applying a decorator to this scope may be a burden on receivers because it requires them to be stateful. Therefore this usage, though theoretically possible, may be undesirable. If community members want to explore this, they are encouraged to discuss broadly, first.)
• Connection. It is conceivable that Alice could bundle up a set of decorators and send them to Bob as a way of saying "Hey, Bob: unless/until subsequent interactions give you more specific semantics, here's a set of semantics (settings for our connection) that you should assume for all interactions." (This may or may not be useful. It introduces ambiguity--should settings declared at this scope override those at the scope of a message family or message type? It also adds a burden of remembering to another party; some of what informs the interaction isn't present in the bits that flow later on. Thus, this style of decorator is also left as a possibility for future exploration and is not actively supported at present.)

Reference

This section of this RFC will be kept up-to-date with a list of globally accepted decorators, and links to the RFCs that define them.

• ~thread: provide request/reply and threading semantics
• ~timing: timestamps, expiration, elapsed time
• ~trace: collaborative debugging and monitoring
• ~l10n: localization support

Drawbacks

By having fields that are meaningful yet not declared in core schemas, we run the risk that parsing and validation routines will fail to enforce details that are significant but invisible. We also accept the possibility that interop may look good on paper, but fail due to different understandings of important metadata.

We believe this risk will take care of itself, for the most part, as real-life usage accumulates and decorators become a familiar and central part of the thinking for developers who work with agent-to-agent communication.

Rationale and alternatives

There is ongoing work in the #indy-semantics channel on Rocket.Chat to explore the concept of overlays. These are layers of additional meaning that accumulate above a schema base. Decorators as described here are quite similar in intent. There are some subtle differences, though. The most interesting is that decorators as described here may be applied to things that are not schema-like (e.g., to a message family as a whole, or to a connection, not just to an individual message).

We may be able to resolve these two worldviews, such that decorators are viewed as overlays and inherit some overlay goodness as a result. However, it is unlikely that decorators will change significantly in form or substance as a result. We thus believe the current mental model is already RFC-worthy, and represents a reasonable foundation for immediate use.

Prior art

See references to similar ../../features in programming languages like Java, C#, and Python, mentiond above.

See also this series of blog posts about semantic gaps and the need to manage intent in a declarative style: [ Lacunas Everywhere, Bridging the Lacuna Humana, Introducing Marks, Mountains, Molehills, and Markedness ]

Unresolved questions

• Are we doing enough about namespacing and collision avoidance with JSON-LD?
• What should we do, if anything, about versioning decorators?
• What should we do, if anything, about applying decorators to stateful scopes like a connection or a thread?

Implementations

The following lists the implementations (if any) of this RFC. Please do a pull request to add your implementation. If the implementation is open source, include a link to the repo or to the implementation within the repo. Please be consistent in the "Name" field so that a mechanical processing of the RFCs can generate a list of all RFCs supported by an Aries implementation.

Name / Link	Implementation Notes
Aries RFCs: RFC 0008, RFC 0017, RFC 0015, RFC 0023, RFC 0043, RFC 0056, RFC 0075	many implemented RFCs depend on decorators...
Indy Cloud Agent - Python	message threading
Aries Framework - .NET	message threading
Streetcred.id	message threading
Aries Cloud Agent - Python	message threading, attachments
Aries Static Agent - Python	message threading
Aries Framework - Go	message threading
Connect.Me	message threading
Verity	message threading
Aries Protocol Test Suite	message threading