Skip to main content

Trie

Functional key-value hash maps.

Functional maps (and sets) whose representation is "canonical", and independent of operation history (unlike other popular search trees).

The representation we use here comes from Section 6 of ["Incremental computation via function caching", Pugh & Teitelbaum](https://dl.acm.org/citation.cfm?id=75305).

User’s overview​

This module provides an applicative (functional) hash map. Notably, each put produces a new trie and value being replaced, if any.

Those looking for a more familiar (imperative, object-oriented) hash map should consider TrieMap or HashMap instead.

The basic Trie operations consist of: - put - put a key-value into the trie, producing a new version. - get - get a key’s value from the trie, or null if none. - iter - visit every key-value in the trie.

The put and get operations work over Key records, which group the hash of the key with its non-hash key value.

import Trie "mo:base/Trie";
import Text "mo:base/Text";

type Trie<K, V> = Trie.Trie<K, V>;
type Key<K> = Trie.Key<K>;

func key(t: Text) : Key<Text> { { key = t; hash = Text.hash t } };

let t0 : Trie<Text, Nat> = Trie.empty();
let t1 : Trie<Text, Nat> = Trie.put(t0, key "hello", Text.equal, 42).0;
let t2 : Trie<Text, Nat> = Trie.put(t1, key "world", Text.equal, 24).0;
let n : ?Nat = Trie.put(t1, key "hello", Text.equal, 0).1;
assert (n == ?42);

Trie​

type Trie<K, V> = {#empty; #leaf : Leaf<K, V>; #branch : Branch<K, V>}

Binary hash tries: either empty, a leaf node, or a branch node

Leaf​

type Leaf<K, V> = { size : Nat; keyvals : AssocList<Key<K>, V> }

Leaf nodes of trie consist of key-value pairs as a list.

Branch​

type Branch<K, V> = { size : Nat; left : Trie<K, V>; right : Trie<K, V> }

Branch nodes of the trie discriminate on a bit position of the keys' hashes. we never store this bitpos; rather, we enforce a style where this position is always known from context.

AssocList​

type AssocList<K, V> = AssocList.AssocList<K, V>

Key​

type Key<K> = { hash : Hash.Hash; key : K }

equalKey​

func equalKey<K>(keq : (K, K) -> Bool) : ((Key<K>, Key<K>) -> Bool)

Equality function for two `Key\<K>`s, in terms of equality of `K’s.

isValid​

func isValid<K, V>(t : Trie<K, V>, enforceNormal : Bool) : Bool

Checks the invariants of the trie structure, including the placement of keys at trie paths

Trie2D​

type Trie2D<K1, K2, V> = Trie<K1, Trie<K2, V>>

A 2D trie maps dimension-1 keys to another layer of tries, each keyed on the dimension-2 keys.

Trie3D​

type Trie3D<K1, K2, K3, V> = Trie<K1, Trie2D<K2, K3, V>>

A 3D trie maps dimension-1 keys to another layer of 2D tries, each keyed on the dimension-2 and dimension-3 keys.

empty​

func empty<K, V>() : Trie<K, V>

An empty trie.

size​

func size<K, V>(t : Trie<K, V>) : Nat
 Get the number of key-value pairs in the trie, in constant time.
Get size in O(1) time.

branch​

func branch<K, V>(l : Trie<K, V>, r : Trie<K, V>) : Trie<K, V>

Construct a branch node, computing the size stored there.

leaf​

func leaf<K, V>(kvs : AssocList<Key<K>, V>, bitpos : Nat) : Trie<K, V>

Construct a leaf node, computing the size stored there.

This helper function automatically enforces the MAX_LEAF_SIZE by constructing branches as necessary; to do so, it also needs the bitpos of the leaf.

fromList​

func fromList<K, V>(kvc : ?Nat, kvs : AssocList<Key<K>, V>, bitpos : Nat) : Trie<K, V>

Transform a list into a trie, splitting input list into small (leaf) lists, if necessary.

clone​

func clone<K, V>(t : Trie<K, V>) : Trie<K, V>

Clone the trie efficiently, via sharing.

Purely-functional representation permits O(1) copy, via persistent sharing.

replace​

func replace<K, V>(t : Trie<K, V>, k : Key<K>, k_eq : (K, K) -> Bool, v : ?V) : (Trie<K, V>, ?V)

Replace the given key’s value option with the given one, returning the previous one

put​

func put<K, V>(t : Trie<K, V>, k : Key<K>, k_eq : (K, K) -> Bool, v : V) : (Trie<K, V>, ?V)

Put the given key’s value in the trie; return the new trie, and the previous value associated with the key, if any

get​

func get<K, V>(t : Trie<K, V>, k : Key<K>, k_eq : (K, K) -> Bool) : ?V

Get the value of the given key in the trie, or return null if nonexistent

find​

func find<K, V>(t : Trie<K, V>, k : Key<K>, k_eq : (K, K) -> Bool) : ?V

Find the given key’s value in the trie, or return null if nonexistent

merge​

func merge<K, V>(tl : Trie<K, V>, tr : Trie<K, V>, k_eq : (K, K) -> Bool) : Trie<K, V>

Merge tries, preferring the right trie where there are collisions in common keys.

note: the disj operation generalizes this merge operation in various ways, and does not (in general) lose information; this operation is a simpler, special case.

mergeDisjoint​

func mergeDisjoint<K, V>(tl : Trie<K, V>, tr : Trie<K, V>, k_eq : (K, K) -> Bool) : Trie<K, V>

Merge tries like merge, except signals a dynamic error if there are collisions in common keys between the left and right inputs.

diff​

func diff<K, V, W>(tl : Trie<K, V>, tr : Trie<K, W>, k_eq : (K, K) -> Bool) : Trie<K, V>

Difference of tries. The output consists are pairs of the left trie whose keys are not present in the right trie; the values of the right trie are irrelevant.

disj​

func disj<K, V, W, X>(tl : Trie<K, V>, tr : Trie<K, W>, k_eq : (K, K) -> Bool, vbin : (?V, ?W) -> X) : Trie<K, X>

Map disjunction.

This operation generalizes the notion of "set union" to finite maps.

Produces a "disjunctive image" of the two tries, where the values of matching keys are combined with the given binary operator.

For unmatched key-value pairs, the operator is still applied to create the value in the image. To accomodate these various situations, the operator accepts optional values, but is never applied to (null, null).

Implements the database idea of an ["outer join"](https://stackoverflow.com/questions/38549/what-is-the-difference-between-inner-join-and-outer-join).

join​

func join<K, V, W, X>(tl : Trie<K, V>, tr : Trie<K, W>, k_eq : (K, K) -> Bool, vbin : (V, W) -> X) : Trie<K, X>

Map join.

Implements the database idea of an ["inner join"](https://stackoverflow.com/questions/38549/what-is-the-difference-between-inner-join-and-outer-join).

This operation generalizes the notion of "set intersection" to finite maps. The values of matching keys are combined with the given binary operator, and unmatched key-value pairs are not present in the output.

foldUp​

func foldUp<K, V, X>(t : Trie<K, V>, bin : (X, X) -> X, leaf : (K, V) -> X, empty : X) : X

This operation gives a recursor for the internal structure of tries. Many common operations are instantiations of this function, either as clients, or as hand-specialized versions (e.g., see , map, mapFilter, some and all below).

prod​

func prod<K1, V1, K2, V2, K3, V3>(tl : Trie<K1, V1>, tr : Trie<K2, V2>, op : (K1, V1, K2, V2) -> ?(Key<K3>, V3), k3_eq : (K3, K3) -> Bool) : Trie<K3, V3>

Map product.

Conditional catesian product, where the given operation op conditionally creates output elements in the resulting trie.

The keyed structure of the input tries are not relevant for this operation: all pairs are considered, regardless of keys matching or not. Moreover, the resulting trie may use keys that are unrelated to these input keys.

iter​

func iter<K, V>(t : Trie<K, V>) : I.Iter<(K, V)>

Returns an Iter over the key-value entries of the trie.

Each iterator gets a persistent view of the mapping, independent of concurrent updates to the iterated map.

Build​

let Build

Represent the construction of tries as data.

This module provides optimized variants of normal tries, for more efficient join queries.

The central insight is that for (unmaterialized) join query results, we do not need to actually build any resulting trie of the resulting data, but rather, just need a collection of what would be in that trie. Since query results can be large (quadratic in the DB size), avoiding the construction of this trie provides a considerable savings.

To get this savings, we use an ADT for the operations that would build this trie, if evaluated. This structure specializes a rope: a balanced tree representing a sequence. It is only as balanced as the tries from which we generate these build ASTs. They have no intrinsic balance properties of their own.

fold​

func fold<K, V, X>(t : Trie<K, V>, f : (K, V, X) -> X, x : X) : X

Fold over the key-value pairs of the trie, using an accumulator. The key-value pairs have no reliable or meaningful ordering.

some​

func some<K, V>(t : Trie<K, V>, f : (K, V) -> Bool) : Bool

Test whether a given key-value pair is present, or not.

all​

func all<K, V>(t : Trie<K, V>, f : (K, V) -> Bool) : Bool

Test whether all key-value pairs have a given property.

nth​

func nth<K, V>(t : Trie<K, V>, i : Nat) : ?(Key<K>, V)

Project the nth key-value pair from the trie.

Note: This position is not meaningful; it’s only here so that we can inject tries into arrays using functions like Array.tabulate.

toArray​

func toArray<K, V, W>(t : Trie<K, V>, f : (K, V) -> W) : [W]

Gather the collection of key-value pairs into an array of a (possibly-distinct) type.

isEmpty​

func isEmpty<K, V>(t : Trie<K, V>) : Bool

Test for "deep emptiness": subtrees that have branching structure, but no leaves. These can result from naive filtering operations; filter uses this function to avoid creating such subtrees.

filter​

func filter<K, V>(t : Trie<K, V>, f : (K, V) -> Bool) : Trie<K, V>

Filter the key-value pairs by a given predicate.

mapFilter​

func mapFilter<K, V, W>(t : Trie<K, V>, f : (K, V) -> ?W) : Trie<K, W>

Map and filter the key-value pairs by a given predicate.

equalStructure​

func equalStructure<K, V>(tl : Trie<K, V>, tr : Trie<K, V>, keq : (K, K) -> Bool, veq : (V, V) -> Bool) : Bool

Test for equality, but naively, based on structure. Does not attempt to remove "junk" in the tree; For instance, a "smarter" approach would equate #bin {left = #empty; right = #empty} with #empty. We do not observe that equality here.

replaceThen​

func replaceThen<K, V, X>(t : Trie<K, V>, k : Key<K>, k_eq : (K, K) -> Bool, v2 : V, success : (Trie<K, V>, V) -> X, fail : () -> X) : X

Replace the given key’s value in the trie, and only if successful, do the success continuation, otherwise, return the failure value

putFresh​

func putFresh<K, V>(t : Trie<K, V>, k : Key<K>, k_eq : (K, K) -> Bool, v : V) : Trie<K, V>

Put the given key’s value in the trie; return the new trie; assert that no prior value is associated with the key

put2D​

func put2D<K1, K2, V>(t : Trie2D<K1, K2, V>, k1 : Key<K1>, k1_eq : (K1, K1) -> Bool, k2 : Key<K2>, k2_eq : (K2, K2) -> Bool, v : V) : Trie2D<K1, K2, V>

Put the given key’s value in the 2D trie; return the new 2D trie.

put3D​

func put3D<K1, K2, K3, V>(t : Trie3D<K1, K2, K3, V>, k1 : Key<K1>, k1_eq : (K1, K1) -> Bool, k2 : Key<K2>, k2_eq : (K2, K2) -> Bool, k3 : Key<K3>, k3_eq : (K3, K3) -> Bool, v : V) : Trie3D<K1, K2, K3, V>

Put the given key’s value in the trie; return the new trie;

remove​

func remove<K, V>(t : Trie<K, V>, k : Key<K>, k_eq : (K, K) -> Bool) : (Trie<K, V>, ?V)

Remove the given key’s value in the trie; return the new trie

removeThen​

func removeThen<K, V, X>(t : Trie<K, V>, k : Key<K>, k_eq : (K, K) -> Bool, success : (Trie<K, V>, V) -> X, fail : () -> X) : X

Remove the given key’s value in the trie, and only if successful, do the success continuation, otherwise, return the failure value

remove2D​

func remove2D<K1, K2, V>(t : Trie2D<K1, K2, V>, k1 : Key<K1>, k1_eq : (K1, K1) -> Bool, k2 : Key<K2>, k2_eq : (K2, K2) -> Bool) : (Trie2D<K1, K2, V>, ?V)

remove the given key-key pair’s value in the 2D trie; return the new trie, and the prior value, if any.

remove3D​

func remove3D<K1, K2, K3, V>(t : Trie3D<K1, K2, K3, V>, k1 : Key<K1>, k1_eq : (K1, K1) -> Bool, k2 : Key<K2>, k2_eq : (K2, K2) -> Bool, k3 : Key<K3>, k3_eq : (K3, K3) -> Bool) : (Trie3D<K1, K2, K3, V>, ?V)

Remove the given key-key pair’s value in the 3D trie; return the new trie, and the prior value, if any.

mergeDisjoint2D​

func mergeDisjoint2D<K1, K2, V>(t : Trie2D<K1, K2, V>, k1_eq : (K1, K1) -> Bool, k2_eq : (K2, K2) -> Bool) : Trie<K2, V>

Like [mergeDisjoint](#mergedisjoint), except instead of merging a pair, it merges the collection of dimension-2 sub-trees of a 2D trie.