11.03 Payload, Raw Data And Retention Requirements

Статус: черновик для обсуждения

1. Назначение документа

Этот документ описывает product-level requirements для raw payloads and message data.

Это не database schema and not storage design.

Development team должна определить:

• exact storage;
• format;
• encryption;
• access control;
• indexing;
• compression;
• retention implementation;
• security controls.

Product requirement: raw payloads must be available for investigation, but must not leak secrets or hidden provider/internal data.

2. Основной принцип

Для расследования transaction система должна сохранять payloads, которые реально участвовали в processing.

Raw data нужна, чтобы platform/support мог понять:

• что мы отправили provider-у;
• что provider ответил;
• какой callback provider прислал;
• что мы отправили merchant-у;
• что merchant endpoint вернул на webhook.

Raw payloads не должны попадать в обычный transaction timeline as inline data.

Timeline should contain summary + payload reference/link.

3. Provider callback payload

Raw provider callback payload нужно сохранять полностью, если это технически возможно.

Это включает:

• request body;
• relevant headers;
• callback route/context;
• received timestamp;
• source metadata, если доступно;
• validation result;
• processing result.

Если по техническим причинам невозможно сохранить полный payload, development team должна явно описать limitation.

Product expectation: preserve as much as possible for investigation.

4. Provider request payload

Raw provider request payload, который мы отправили provider-у, нужно сохранять.

Это включает:

• request body;
• request URL/path, если visibility/security позволяет;
• method;
• relevant headers, excluding/masking secrets;
• timestamp;
• selected SUB MID / SUB MID AGGREGATOR reference;
• provider method reference.

Secret values must be removed/masked before storing or before showing, depending on technical approach.

5. Provider response payload

Raw provider response payload нужно сохранять.

Это включает:

• response body;
• HTTP status/code;
• relevant response headers;
• timestamp;
• parsed result, если applicable;
• mapped internal result/error.

Provider response может содержать provider-specific error, redirect URL, provider transaction ID or other references.

Visibility rules apply.

6. Merchant webhook payload

Merchant webhook payload, который мы отправили merchant-у, нужно сохранять.

Это включает:

• webhook request body;
• target webhook URL;
• relevant request headers;
• signature headers;
• timestamp;
• delivery attempt number;
• transaction status at the moment of sending.

Merchant user с permission может видеть merchant webhook payload, потому что это payload, который был отправлен merchant-у.

Platform user с permission также может видеть merchant webhook payload.

7. Merchant webhook response

Merchant webhook response нужно сохранять полностью.

Это включает:

• HTTP status/code;
• response body;
• relevant response headers;
• error/timeout details;
• response time/latency;
• delivery result;
• retry scheduling result, if failed.

Merchant user с permission может видеть:

• webhook payload sent to merchant;
• webhook response status/code;
• webhook response body;
• delivery attempts;
• retry attempts.

8. Raw provider data visibility

Raw provider callback/request/response могут видеть только platform users with permission.

Merchant users не должны видеть raw provider data by default.

Merchant user может видеть provider-related data only if:

• product explicitly allows it;
• user has permission;
• merchant has provider access;
• data does not reveal hidden platform-created provider/config internals.

Default rule:

Provider raw data = platform-only with permission.

9. Merchant webhook data visibility

Merchant user с permission может видеть merchant webhook data.

Это включает:

• payload that was sent to merchant;
• target webhook URL;
• response status/code;
• response body;
• delivery attempt history;
• retry status.

Reason:

• это их integration endpoint;
• это помогает merchant debug their webhook receiver;
• это не раскрывает hidden provider internals by itself.

If webhook payload contains fields hidden by permissions, UI must still respect visibility rules.

10. Secrets in raw payloads

Secrets must be hidden/masked/removed in raw payload storage and display.

Examples:

• API keys;
• provider credentials;
• provider secret keys;
• Authorization headers;
• signatures if considered secret;
• webhook signing secrets;
• session tokens;
• private keys;
• passwords;
• 2FA secrets.

Product requirement:

• secrets must never be exposed in UI;
• secrets must not appear in exports;
• secrets must not appear in logs;
• secrets must not appear in timeline metadata;
• secrets must not be visible even to platform user.

Development team/security team must define exact masking/removal strategy.

11. Customer sensitive data in raw payloads

Raw payloads may contain customer sensitive data.

Platform user with permission can see customer sensitive data in raw payloads if it is required for investigation.

Visibility must be permission-based.

Merchant user visibility should be limited to merchant-safe data and merchant-owned webhook data.

If future compliance requirements require masking customer sensitive data even for platform users, this must be described separately.

12. Retention

Product requirement for MVP:

Raw payloads should be retained indefinitely / always.

Reason:

• payment investigation can happen much later;
• historical transaction details should remain useful;
• provider disputes/support cases may require old payloads.

No 30/90/180/365 day retention limit is defined by product for MVP.

Technical/security/legal follow-up:

• if storage cost, compliance, GDPR, PCI or security constraints require retention policy, development/security/legal team must propose it.

13. CSV export

Raw payloads must not be included in CSV export.

CSV export should include structured transaction fields available to the user.

Raw payload export would be risky because:

• payloads can be large;
• payloads can contain sensitive data;
• payloads can contain provider/internal details;
• payloads are not useful as table columns.

If future support export for raw payload is needed, it must be separate permissioned action, not normal transaction CSV export.

14. Timeline payload references

Raw payloads should not be stored inline in transaction timeline events.

Timeline should store:

• event type;
• summary;
• status/result;
• short reason;
• payload reference/link, if applicable.

Payload reference should open payload details only if user has permission.

This keeps timeline:

• readable;
• fast;
• permission-safe;
• not overloaded by huge JSON/XML payloads.

15. Checksum/hash

Checksum/hash for raw payloads is not required in MVP.

Product does not require cryptographic proof that payload was not changed.

Development/security team can add checksum/hash if they consider it useful, but it is not product requirement.

16. Data ownership

Raw payload ownership depends on payload type.

Product-level expectation:

• provider callback raw payloads are owned by provider callback processing domain;
• provider request/response raw payloads are owned by provider integration domain;
• merchant webhook payload/response is owned by merchant webhook delivery domain;
• transaction timeline stores references, not full raw payloads.

Orchestrator Service should be able to link all payloads to transaction investigation.

17. Acceptance Criteria

Payload / Raw Data / Retention Requirements считаются согласованными для MVP, если:

• Raw provider callback payload is saved fully where technically possible.
• Raw provider request payload is saved.
• Raw provider response payload is saved.
• Merchant webhook request payload is saved.
• Merchant webhook response status/body/headers/error/timeout details are saved.
• Raw provider data is visible only to platform users with permission.
• Merchant user with permission can see merchant webhook payload and response.
• Secrets are hidden/masked/removed from raw payloads.
• Customer sensitive data in raw payloads is visible to platform users with permission.
• Raw payloads are retained indefinitely by product requirement.
• Raw payloads are not included in normal CSV export.
• Timeline stores payload reference/link, not inline raw payload.
• Payload reference opens only with permission.
• Checksum/hash is not required in MVP.

18. Open Questions

Product open questions отсутствуют.

Technical/security/legal follow-up:

1. Development/security team должна определить exact secret masking/removal strategy.
2. Development team должна определить raw payload storage design.
3. Development team должна определить payload reference model.
4. Security/legal team должна проверить indefinite retention against compliance requirements.
5. Development team должна определить access control for raw payload details.