Portable Storage Format

Production codec: rust/shekyl-portable-storage (LV-2a). Typed Levin command maps are LV-2b in shekyl-levin. Decision pin: docs/design/LV2_PORTABLE_STORAGE.md.

Shekyl integers on this format are little-endian. Encode order is lexicographic (std::map / BTreeMap). Known-answer tests for the codec live in rust/shekyl-portable-storage/tests/oracle_kats.rs. Handshake / timed-sync / ping / support-flags / network_address round-trips live in rust/shekyl-levin/tests/payload_kats.rs. Notify maps (2001–2004 / 2006–2010) live in rust/shekyl-levin/tests/notify_kats.rs (LV-2b).

Background

Shekyl makes use of a set of helper classes from a small library named epee (inherited from Monero). Part of this library implements a networking protocol called Levin, which internally uses a storage format called Portable Storage. This format (amongst the rest of the epee library), is undocumented - or rather relies on the code itself to serve as the documentation. Unfortunately, whilst the rest of the library is fairly straightforward to decipher, the Portable Storage is less-so. Hence this document.

String and Integer Encoding

Integers

Integers in Shekyl portable_storage are little-endian. The inherited sketch's "implementations may choose big-endian" hedge is false for this chain and is not a decoder option.

Varints

Varints are used to pack integers in an portable and space optimized way. Varints are stored as little-endian integers, with the lowest 2 bits storing the amount of bytes required, which means the largest value integer that can be packed into 1 byte is 63 (6 bits).

Byte Sizes

Lowest 2 bits	Size value	Value range
b00	1 byte	0 to 63
b01	2 bytes	64 to 16383
b10	4 bytes	16384 to 1073741823
b11	8 bytes	1073741824 to 4611686018427387903

Represenations of Example Values

Value	Byte Representation (hex)
0	00
7	1c
101	95 01
17,000	A2 09 01 00
7,942,319,744	03 BA 98 65 07 00 00 00

Strings

These are simply length (varint) prefixed char strings without a null terminator (though one can always add one if desired). There is no specific encoding enforced, and in fact, many times binary blobs are stored as these strings. This type should not be confused with the keys in sections, as those are restricted to a maximum length of 255 and do not use varints to encode the length.

"Howdy" => 14 48 6F 77 64 79

Section Keys

These are similar to strings except that they are length limited to 255 bytes, and use a single byte at the front of the string to describe the length (as opposed to a varint).

"Howdy" => 05 48 6F 77 64 79

Binary Format Specification

Header

The format must always start with the following header:

Field	Type	Value
Signature Part A	UInt32	0x01011101
Signature Part B	UInt32	0x01020101
Version	UInt8	0x01

In total, the 9 byte header will look like this (in hex): 01 11 01 01 01 01 02 01 01

Section

Next we have a root object (or section as the library calls it). This is a map of name-value pairs called entries. It starts with a count:

Section	Type
Entry count	varint

Which is followed by the section's name-value entries sequentially:

Entry

Entry	Type
Name	section key
Type	byte
Count<sup>1</sup>	varint
Value(s)	(type dependant data)

¹ Note, this is only present if the entry type has the array flag (see below).

Entry types

The types defined are:

#define SERIALIZE_TYPE_INT64                1
#define SERIALIZE_TYPE_INT32                2
#define SERIALIZE_TYPE_INT16                3
#define SERIALIZE_TYPE_INT8                 4
#define SERIALIZE_TYPE_UINT64               5
#define SERIALIZE_TYPE_UINT32               6
#define SERIALIZE_TYPE_UINT16               7
#define SERIALIZE_TYPE_UINT8                8
#define SERIALIZE_TYPE_DOUBLE               9
#define SERIALIZE_TYPE_STRING               10
#define SERIALIZE_TYPE_BOOL                 11
#define SERIALIZE_TYPE_OBJECT               12
#define SERIALIZE_TYPE_ARRAY                13

The entry type can be bitwise OR'ed with a flag:

#define SERIALIZE_FLAG_ARRAY              0x80

This signals there are multiple values for the entry. Since only one bit is reserved for specifying an array, we can not directly represent nested arrays. However, you can place each of the inner arrays inside of a section, and make the outer array type SERIALIZE_TYPE_OBJECT | SERIALIZE_FLAG_ARRAY. Immediately following the type code byte is a varint specifying the length of the array. Finally, the all the elements are serialized in sequence with no padding and without any type information. For example:

<p style="padding-left:1em; font:italic larger serif">type, count, value₁, value₂,..., value_n</p>

Entry values

POD integers and doubles are little-endian, with no padding. Bool is a single byte that must be 0 or 1. Strings are a varint length then that many bytes; there is no UTF-8 requirement on values (hashes, tx blobs, and other PODs travel as SERIALIZE_TYPE_STRING).

Entry values which are objects (SERIALIZE_TYPE_OBJECT) are stored as sections.

SERIALIZE_TYPE_ARRAY (tag 13, untyped) and array-of-array are a hard decode error until a captured C++ body emits one. Production arrays use (inner_type | SERIALIZE_FLAG_ARRAY).

Decode vs encode details

• Trailing bytes after the root section are ignored (load_from_binary does not require consuming the whole buffer).
• Duplicate keys in one section are rejected.
• Encode walks keys in lexicographic order. Decode accepts any order.
• Encode rejects empty keys and keys longer than 254 bytes. Decode accepts key lengths 1..=255 (C++ asymmetry).
• Section keys in the Rust decoder must be UTF-8. Production maps use ASCII identifiers.
• A header-only blob is truncated: C++ rejects a root with sz == 0 after a successful header read.

Overall example

Let's put it all together and see what an entire object would look like serialized. To represent our data, let's create a JSON object (since it's a format that most will be familiar with):

{
  "short_quote": "Give me liberty or give me death!",
  "long_quote": "Shekyl builds on proven CryptoNote lineage for privacy and resilience.",
  "signed_32bit_int": 20140418,
  "array_of_bools": [true, false, true, true],
  "nested_section": {
    "double": -6.9,
    "unsigned_64bit_int": 11111111111111111111
  }
}

This would translate to:

Limits

portable_storage::limits_t is a decode parameter, not a format constant. Encode does not consult limits. The Rust crate takes a Limits { objects, fields, strings }:

Caller	objects	fields	strings	C++ source
Levin invoke/notify	8192	16384	16384	default_levin_limits (levin_abstract_invoke2.h)
HTTP .bin RPC	196608	196608	196608	Rust Limits::HTTP_BIN (shekyl-portable-storage/src/limits.rs) — the C++ default_http_bin_limits (65536×3) left with http_abstract_invoke.h in the epee HTTP client deletion (2026-08-21)
Null limits pointer	usize::MAX	usize::MAX	usize::MAX	load_from_binary unrestricted

• objects — nested sections. The root section is not counted.
• fields — sum of per-section field counts.
• strings — scalar strings plus reserved string-array slots (a string array of n elements bumps the string budget by n once).
• Recursion: C++ EPEE_PORTABLE_STORAGE_RECURSION_LIMIT is 100, counted on almost every read (including raw memcpy). The Rust decoder counts nested section/entry/array structural depth against the same 100. Deep primitive-only blobs that would trip C++ but not structural depth are not a production shape.
• String length must be < MAX_STRING_LEN_POSSIBLE (2_000_000_000).

Schema layer (not the codec)

The binary codec has optional keys and SERIALIZE_TYPE_STRING blobs. The C++ KV_SERIALIZE* macros are a typed overlay. LV-2b owns that overlay for Levin command maps. Notes so the codec is not asked to invent them:

• KV_SERIALIZE_OPT. Store omits the field when the value equals the default; load uses the default when the field is absent. Missing this is how handshake peerlists and dandelionpp_fluff diverge. The codec just encodes whatever keys are present.
• KV_SERIALIZE_VAL_POD_AS_BLOB. A POD is one SERIALIZE_TYPE_STRING of sizeof bytes, not a section. The codec sees a blob.
• KV_SERIALIZE_CONTAINER_POD_AS_BLOB. A vector/list of PODs is one string of concatenated elements, not a typed array. Used by hash lists (commands 2003/2006/2007/2009/2010).
• network_address union, including ipv4's store-time SWAP32LE, plus block_complete_entry pruned vs unpruned txs and attestation_witness OPT, are LV-2b. See LV2_PORTABLE_STORAGE.md §6.

Shekyl / CryptoNote specifics

Entry values

Hashes, Keys, Blobs

These are stored as strings, SERIALIZE_TYPE_STRING.

STL containers (vector, list)

These can be arrays of standard integer types, strings or SERIALIZE_TYPE_OBJECT's for structs. When the C++ map uses CONTAINER_POD_AS_BLOB, the wire is one concatenated STRING instead (see Schema layer above).

Links to struct definitions in this repository

• Core RPC definitions: src/rpc/core_rpc_server_commands_defs.h (~343 KV maps; not LV-2b — HTTP JSON/binary RPC stays C++).
• CryptoNote protocol definitions: src/cryptonote_protocol/cryptonote_protocol_defs.h and src/p2p/net_node_common.h (Levin-wire subset is LV-2b).

Known-answer tests

Codec KATs (empty section, C++ two_keys / duplicate_key from tests/unit_tests/epee_serialization.cpp, nested object, uint64 array, tag-13 hard error, HTTP .bin request shape) live in rust/shekyl-portable-storage/tests/oracle_kats.rs. Captured handshake and NOTIFY_NEW_TRANSACTIONS bodies are LV-2b.