When deciding on data structures to use for the data of the application, it is important to minimize the amount of data read and written to storage but also the amount of data serialized and deserialized to minimize the cost of transactions. It is important to understand the tradeoffs of data structures in your smart contract because it can become a bottleneck as the application scales and migrating the state to the new data structures will come at a cost.
The collections within
near-sdk are designed to split the data into chunks and defer reading and writing to the store until needed. These data structures will handle the low-level storage interactions and aim to have a similar API to the
near_sdk::collectionswill be moving to
near_sdk::storeand have updated APIs. If you would like to access these updated structures as they are being implemented, enable the
It is important to keep in mind that when using
std::collections, that each time state is loaded, all entries in the data structure will be read eagerly from storage and deserialized. This will come at a large cost for any non-trivial amount of data, so to minimize the amount of gas used the SDK collections should be used in most cases.
The most up to date collections and their documentation can be found in the rust docs.
The following data structures that exist in the SDK are as follows:
|Optional value in storage. This value will only be read from storage when interacted with. This value will be |
|A growable array type. The values are sharded in memory and can be used for iterable and indexable values that are dynamically sized.|
|This structure behaves as a thin wrapper around the key-value storage available to contracts. This structure does not contain any metadata about the elements in the map, so it is not iterable.|
|Similar to |
|An ordered equivalent of |
|A set, which is similar to |
|An iterable equivalent of |
HashMap vs persistent
HashMapkeeps all data in memory. To access it, the contract needs to deserialize the whole map.
UnorderedMapkeeps data in persistent storage. To access an element, you only need to deserialize this element.
HashMap in case:
- Need to iterate over all elements in the collection in one function call.
- The number of elements is small or fixed, e.g. less than 10.
UnorderedMap in case:
- Need to access a limited subset of the collection, e.g. one or two elements per call.
- Can't fit the collection into memory.
The reason is
HashMap deserializes (and serializes) the entire collection in one storage operation.
Accessing the entire collection is cheaper in gas than accessing all elements through
N storage operations.
Persistent collections such as
contain more elements than the amount of gas available to read them all.
In order to expose them all through view calls, we can implement pagination.
Vector returns elements by index natively using
To access elements by index in
UnorderedSet we can use
.as_vector() that will return a
Vector of elements.
UnorderedMap we need to get keys and values as
Vector collections, using
Example of pagination for
UnorderedMapsupports iteration over keys and values, and also supports pagination. Internally, it has the following structures:
- a map from a key to an index
- a vector of keys
- a vector of values
LookupMaponly has a map from a key to a value. Without a vector of keys, it doesn't have the ability to iterate over keys.
LookupMap has a better performance and stores less data compared to
2storage reads to get the value and
3storage writes to insert a new entry.
LookupMaprequires only one storage read to get the value and only one storage write to store it.
UnorderedMap requires more storage for an entry compared to a
UnorderedMapstores the key twice (once in the first map and once in the vector of keys) and value once. It also has a higher constant for storing the length of vectors and prefixes.
LookupMapstores key and value once.
It's a type of persistent collection that only stores a single value. The goal is to prevent a contract from deserializing the given value until it's needed. An example can be a large blob of metadata that is only needed when it's requested in a view call, but not needed for the majority of contract operations.
It acts like an
Option that can either hold a value or not and also requires a unique prefix (a key in this case)
like other persistent collections.
Compared to other collections,
LazyOption only allows you to initialize the value during initialization.