Message Structure

Overview

The Message Structure defines the JSON format used by the Open Payments system to handle incoming messages. This structure is designed to be generic and adaptable, capable of supporting various message standards across different domains.

Key characteristics:

• Generic structure for multiple use cases
• Domain-specific content encapsulation
• Clear separation of concerns
• Support for multi-tenant deployments

Architecture

The structure is organized into distinct sections:

Root-Level Fields

Essential fields that the message processor uses for core operations:

• id: Unique message identifier
• payload: Original message content
• tenant: Multi-tenant identifier
• origin: Source information
• processing: Processing state
• auditlog: Processing history

Domain-Specific Sections

• data: Contains the standardized message content (e.g., ISO20022 for payments)
• metadata: Additional context and processing instructions

Standards Support

Payments Domain

When used for payment processing:

• data section contains ISO20022-compliant message structures
• Standard formats like pacs.008 for credit transfers are supported
• Maintains compliance with payment scheme requirements

Other Domains

The structure can accommodate various standards:

• Different message formats can be incorporated
• Domain-specific validations can be implemented
• Custom workflows can be defined

Schema Definition

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "Open Payments Message Schema",
  "description": "Generic message structure for processing financial and non-financial messages in the Open Payments system",
  "type": "object",
  "required": ["id", "payload", "tenant", "origin", "data", "processing"],
  "properties": {
    "id": {
      "type": "string",
      "description": "Unique identifier for the message, automatically generated by the system. Used for message tracking and correlation",
      "pattern": "^[a-zA-Z0-9-_]+$"
    },
    "payload": {
      "type": "object",
      "description": "Contains the original message content, either directly or as a reference to external storage",
      "required": ["type"],
      "properties": {
        "type": {
          "type": "string",
          "enum": ["inline", "file"],
          "description": "Specifies how the payload is stored: 'inline' for direct content storage, 'file' for external file reference"
        },
        "content": {
          "type": ["string", "object"],
          "description": "The actual message content when stored inline. Can be a string or structured object depending on the format"
        },
        "url": {
          "type": "string",
          "description": "URL pointing to the message content when stored externally. Used for large messages or when content needs to be fetched on demand"
        },
        "format": {
          "type": "string",
          "description": "Format of the message content (e.g., 'XML', 'JSON', 'ISO20022'). Helps in proper parsing and processing"
        },
        "encoding": {
          "type": "string",
          "description": "Character encoding of the content (e.g., 'UTF-8', 'ASCII'). Essential for proper text handling"
        },
        "size": {
          "type": "integer",
          "description": "Size of the payload in bytes. Useful for resource management and monitoring"
        }
      },
      "oneOf": [
        {
          "properties": {
            "type": { "const": "inline" },
            "content": { "type": ["string", "object"] }
          },
          "required": ["type", "content"],
          "description": "Configuration for inline payload storage"
        },
        {
          "properties": {
            "type": { "const": "file" },
            "url": { "type": "string" }
          },
          "required": ["type", "url"],
          "description": "Configuration for file-based payload storage"
        }
      ]
    },
    "tenant": {
      "type": "string",
      "description": "Identifier for the organization or business unit owning this message. Used in multi-tenant deployments for data isolation"
    },
    "origin": {
      "type": "object",
      "description": "Information about where and when the message originated",
      "required": ["system", "channel", "timestamp"],
      "properties": {
        "system": {
          "type": "string",
          "description": "Identifier of the source system that generated the message (e.g., 'CORE_BANKING', 'MOBILE_APP')"
        },
        "channel": {
          "type": "string",
          "description": "Specific channel within the source system (e.g., 'API', 'WEB', 'BATCH')"
        },
        "timestamp": {
          "type": "string",
          "format": "date-time",
          "description": "ISO 8601 timestamp when the message was created in the source system"
        }
      }
    },
    "processing": {
      "type": "object",
      "description": "Current processing state and progress information used by the workflow engine to manage message flow",
      "required": ["status", "workflow_id"],
      "properties": {
        "status": {
          "type": "string",
          "description": "Domain-specific status indicating the current state of message processing. For payments, typical flow might be: received -> processing -> completed -> acknowledged. Status values are defined by the specific domain implementation",
          "examples": ["received", "processing", "completed", "acknowledged", "failed", "rejected"]
        },
        "workflow_id": {
          "type": "string",
          "description": "Identifier of the workflow definition being used to process this message"
        },
        "progress": {
          "type": "object",
          "description": "Information about the last completed processing task",
          "properties": {
            "prev_task": {
              "type": "string",
              "description": "Identifier of the last successfully completed task in the workflow"
            },
            "prev_status_code": {
              "type": "string",
              "description": "Status code returned by the last completed task (e.g., 'SUCCESS', 'VALIDATION_ERROR')"
            },
            "timestamp": {
              "type": "string",
              "format": "date-time",
              "description": "ISO 8601 timestamp when the last task was completed"
            }
          }
        },
        "retries": {
          "type": "object",
          "description": "Information about retry attempts for failed processing",
          "properties": {
            "count": {
              "type": "integer",
              "minimum": 0,
              "description": "Number of times processing has been retried"
            },
            "last_retry": {
              "type": "string",
              "format": "date-time",
              "description": "ISO 8601 timestamp of the last retry attempt"
            }
          }
        }
      }
    },
    "data": {
      "type": "object",
      "description": "Domain-specific standardized content. For payments, contains ISO20022-compliant message data. Structure varies based on message type",
      "additionalProperties": true
    },
    "metadata": {
      "type": "object",
      "description": "Additional information about the message that aids in processing",
      "properties": {
        "format_version": {
          "type": "string",
          "description": "Version of the data format used in the data section (e.g., 'ISO20022:2019')"
        },
        "type": {
          "type": "string",
          "description": "Specific type of message within the chosen format (e.g., 'pacs.008.001.09')"
        },
        "priority": {
          "type": "string",
          "enum": ["real-time", "queued", "scheduled"],
          "description": "Processing priority level determining how the message should be handled: 'real-time' for immediate processing, 'queued' for batch processing, 'scheduled' for future processing"
        },
        "tags": {
          "type": "array",
          "items": {
            "type": "string"
          },
          "description": "Array of labels for message categorization and filtering"
        },
        "flags": {
          "type": "object",
          "additionalProperties": true,
          "description": "Dynamic flags indicating special processing requirements or states"
        }
      }
    },
    "auditlog": {
      "type": "array",
      "description": "Chronological record of all changes and processing tasks",
      "items": {
        "type": "object",
        "required": ["timestamp", "hash", "workflow", "task", "executor", "changes"],
        "properties": {
          "timestamp": {
            "type": "string",
            "format": "date-time",
            "description": "ISO 8601 timestamp when the audit entry was created"
          },
          "hash": {
            "type": "string",
            "description": "Cryptographic hash of the message state after changes, ensuring data integrity"
          },
          "workflow": {
            "type": "string",
            "description": "Identifier of the workflow being executed"
          },
          "task": {
            "type": "string",
            "description": "Identifier of the specific task within the workflow. Tasks are atomic units of work in the workflow"
          },
          "description": {
            "type": "string",
            "description": "Human-readable description of the processing task or change"
          },
          "executor": {
            "type": "object",
            "required": ["service", "instance", "version"],
            "description": "Information about the service that performed the task",
            "properties": {
              "service": {
                "type": "string",
                "description": "Name of the service that performed the task (e.g., 'payment-enricher', 'validator')"
              },
              "instance": {
                "type": "string",
                "description": "Unique identifier of the service instance (e.g., container ID, pod name)"
              },
              "version": {
                "type": "string",
                "description": "Version of the service that performed the task"
              },
              "host": {
                "type": "string",
                "description": "Hostname or IP address where the service was running"
              },
              "region": {
                "type": "string",
                "description": "Geographic region or data center where the service was deployed"
              }
            }
          },
          "changes": {
            "type": "array",
            "description": "List of specific changes made during this processing task",
            "items": {
              "type": "object",
              "required": ["field", "type", "timestamp"],
              "properties": {
                "field": {
                  "type": "string",
                  "description": "JSON path to the changed field (e.g., 'data.creditTransfer.amount')"
                },
                "type": {
                  "type": "string",
                  "enum": ["create", "update", "delete"],
                  "description": "Type of change: 'create' for new fields, 'update' for modifications, 'delete' for removals"
                },
                "timestamp": {
                  "type": "string",
                  "format": "date-time",
                  "description": "ISO 8601 timestamp when this specific change was made"
                },
                "old_value": {
                  "type": ["string", "number", "boolean", "object", "array", "null"],
                  "description": "Previous value of the field before the change (null for create operations)"
                },
                "new_value": {
                  "type": ["string", "number", "boolean", "object", "array", "null"],
                  "description": "New value of the field after the change (null for delete operations)"
                },
                "reason": {
                  "type": "string",
                  "description": "Explanation of why the change was made"
                },
                "rule_id": {
                  "type": "string",
                  "description": "Identifier of the business rule that triggered this change"
                }
              }
            }
          }
        }
      }
    }
  }
}

Example: FedNow Credit Transfer

This example demonstrates how the structure is used for a FedNow credit transfer:

{
  "id": "MSG-20241029-FEDNOW-123456",
  "payload": {
    "type": "inline",
    "content": {
      "creditTransferMessage": {
        "messageId": "BFFF/120928-FEDNOW/123456",
        "creditorAccount": "9876543210",
        "amount": "1000.00",
        "debtorAccount": "1234567890"
      }
    },
    "format": "JSON",
    "encoding": "UTF-8",
    "size": 428
  },
  "tenant": "BANK_USA_001",
  "origin": {
    "system": "CORE_BANKING",
    "channel": "API",
    "timestamp": "2024-10-29T14:30:00.000Z"
  },
  "processing": {
    "status": "processing",
    "workflow_id": "FEDNOW_CREDIT_TRANSFER_V1",
    "progress": {
      "prev_task": "validation",
      "prev_status_code": "SUCCESS",
      "timestamp": "2024-10-29T14:30:01.523Z"
    },
    "retries": {
      "count": 0,
      "last_retry": null
    }
  },
  "data": {
    "messageType": "pacs.008.001.09",
    "businessMessageIdentifier": "BFFF/120928-FEDNOW/123456",
    "creditTransferTransaction": {
      "paymentIdentification": {
        "instructionIdentification": "INSTR-ID-123456",
        "endToEndIdentification": "E2E-REF-123456",
        "transactionIdentification": "TXID-123456"
      },
      "paymentTypeInformation": {
        "localInstrument": "STANDARD",
        "categoryPurpose": {
          "code": "CASH"
        }
      },
      "interbankSettlementAmount": {
        "amount": "1000.00",
        "currency": "USD"
      },
      "chargeBearer": "SLEV",
      "debtor": {
        "name": "John Smith",
        "identification": {
          "privateIdentification": {
            "other": {
              "identification": "DEB-123456"
            }
          }
        },
        "contactDetails": {
          "emailAddress": "john.smith@email.com"
        }
      },
      "debtorAccount": {
        "identification": {
          "accountNumber": "1234567890"
        },
        "type": "CACC"
      },
      "debtorAgent": {
        "financialInstitutionIdentification": {
          "clearingSystemIdentification": {
            "code": "USABA"
          },
          "clearingSystemMemberIdentification": "021000021"
        }
      },
      "creditor": {
        "name": "Jane Doe",
        "identification": {
          "privateIdentification": {
            "other": {
              "identification": "CRED-789012"
            }
          }
        }
      },
      "creditorAccount": {
        "identification": {
          "accountNumber": "9876543210"
        },
        "type": "CACC"
      },
      "creditorAgent": {
        "financialInstitutionIdentification": {
          "clearingSystemIdentification": {
            "code": "USABA"
          },
          "clearingSystemMemberIdentification": "021000022"
        }
      },
      "purpose": {
        "code": "CASH"
      },
      "remittanceInformation": {
        "unstructured": "Invoice payment #INV-2024-001"
      },
      "settlementInformation": {
        "settlementMethod": "CLRG",
        "clearingSystem": "FEDNOW"
      }
    }
  },
  "metadata": {
    "format_version": "ISO20022:2019",
    "type": "credit_transfer",
    "priority": "real-time",
    "tags": ["fednow", "domestic", "credit_transfer"],
    "flags": {
      "requires_compliance_check": true,
      "is_cross_border": false,
      "requires_funds_check": true
    }
  },
  "auditlog": [
    {
      "timestamp": "2024-10-29T14:30:00.100Z",
      "hash": "abc123def456",
      "workflow": "FEDNOW_CREDIT_TRANSFER_V1",
      "task": "reception",
      "description": "Message received and initial parsing completed",
      "executor": {
        "service": "message-receiver",
        "instance": "receiver-pod-123",
        "version": "1.0.0",
        "host": "receiver-node-east-1",
        "region": "us-east-1"
      },
      "changes": [
        {
          "field": "processing.status",
          "type": "create",
          "timestamp": "2024-10-29T14:30:00.100Z",
          "old_value": null,
          "new_value": "received",
          "reason": "Initial message reception",
          "rule_id": "INIT_001"
        }
      ]
    },
    {
      "timestamp": "2024-10-29T14:30:00.500Z",
      "hash": "def456ghi789",
      "workflow": "FEDNOW_CREDIT_TRANSFER_V1",
      "task": "enrichment",
      "description": "Enriched payment with customer and compliance information",
      "executor": {
        "service": "payment-enricher",
        "instance": "enricher-pod-456",
        "version": "1.0.0",
        "host": "enricher-node-east-2",
        "region": "us-east-1"
      },
      "changes": [
        {
          "field": "data.creditTransferTransaction.debtor.contactDetails",
          "type": "create",
          "timestamp": "2024-10-29T14:30:00.500Z",
          "old_value": null,
          "new_value": {
            "emailAddress": "john.smith@email.com"
          },
          "reason": "Added customer contact details",
          "rule_id": "ENRICH_001"
        }
      ]
    },
    {
      "timestamp": "2024-10-29T14:30:01.523Z",
      "hash": "ghi789jkl012",
      "workflow": "FEDNOW_CREDIT_TRANSFER_V1",
      "task": "validation",
      "description": "Validated message against FedNow requirements",
      "executor": {
        "service": "payment-validator",
        "instance": "validator-pod-789",
        "version": "1.0.0",
        "host": "validator-node-east-1",
        "region": "us-east-1"
      },
      "changes": [
        {
          "field": "processing.status",
          "type": "update",
          "timestamp": "2024-10-29T14:30:01.523Z",
          "old_value": "received",
          "new_value": "processing",
          "reason": "Validation successful, proceeding with processing",
          "rule_id": "VAL_001"
        }
      ]
    }
  ]
}

Key Features

1. Flexibility

   • Adaptable to different message standards
   • Supports multiple use cases
   • Extensible structure

2. Traceability

   • Complete audit trail
   • Change tracking
   • Processing history

3. Processing Control

   • Workflow management
   • Status tracking
   • Error handling

4. Multi-tenant Support

   • Tenant isolation
   • Domain separation
   • Configuration flexibility

Implementation Considerations

1. Message Processing

   • Messages are processed according to their workflow definitions
   • Status updates reflect processing progress
   • Audit logs maintain processing history

2. Data Handling

   • Original payload is preserved
   • Transformations are tracked
   • Changes are audited

3. Security

   • Multi-tenant isolation
   • Audit trail integrity
   • Access control support

The Message Structure provides a robust foundation for processing various types of messages while maintaining the flexibility to adapt to different standards and domains. Its clear separation between core processing fields and domain-specific content enables effective multi-tenant, multi-domain operations within the Open Payments platform.