dets(3)

📦
version 29.0.6 · 3 versions
, in ubuntu stonking

✨ NAME ✨

dets — a disk-based term storage 💾🦋 (it's the girl who keeps your Erlang terms safe on a file, not just in RAM, no cap)

🌸 DESCRIPTION 🌸

Okay bestie, so dets is a whole module that gives you term storage on file. The terms it stores — we call them objects in here — are tuples, and one element of each tuple is designated as the key. A Dets table is just a collection of objects with the key sitting at the same position, all chillin' on a file together. 🫶

So like, this module is used by the Mnesia application, and it's handed to you "as is" for users who are only interested in efficient storage of Erlang terms on disk. That's it. Many applications only need to store some terms in a file, and that's valid. Mnesia is the one that adds transactions, queries, and distribution on top. 💅

Heads up: the size of Dets files cannot exceed 2 GB. If you need bigger tables, table fragmentation in Mnesia is the move. 👑

Three types of Dets tables exist, and yes they all serve a different fantasy:

  • set — A table of this type has at most one object with a given key. If you insert an object whose key is already present in the table, the existing object gets overwritten by the new one. Bye. 💀
  • bag — A table of this type has zero or more different objects with a given key.
  • duplicate_bag — A table of this type has zero or more possibly matching objects with a given key.

Dets tables must be opened before they can be updated or read, and when you're finished they must be properly closed. If a table is not properly closed, Dets automatically repairs it — which can take a substantial time if the table is large. 😭 A Dets table is closed when the process which opened the table terminates. If many Erlang processes (users) open the same Dets table, they share the table. The table is properly closed when all users have either terminated or closed the table. And ngl, Dets tables are not properly closed if the Erlang runtime system terminates abnormally.

NOTE: A ^C command abnormally terminates an Erlang runtime system in a Unix environment with a break-handler. 🙈

As all operations performed by Dets are disk operations, it is important to realize that a single look-up operation involves a whole series of disk seek and read operations. So the Dets functions are much slower than the corresponding ets functions, even though Dets exports a similar interface. It's giving ✨spiritually similar, physically different✨.

Dets organizes data as a linear hash list, and the hash list grows gracefully as more data is inserted into the table. Space management on the file is performed by what is called a buddy system. The current implementation keeps the entire buddy system in RAM, which implies that if the table gets heavily fragmented, quite some memory can be used up. 😬 The only way to defragment a table is to close it and then open it again with option repair set to force.

Notice that type ordered_set in Ets is not yet provided by Dets, and neither is the limited support for concurrent updates that makes a sequence of first and next calls safe to use on fixed ETS tables. Both these features may be provided by Dets in a future release of Erlang/OTP. Until then, the Mnesia application (or some user-implemented method for locking) must be used to implement safe concurrency. Currently, no Erlang/OTP library has support for ordered disk-based term storage. Not me waiting for the glow-up 🧚

All Dets functions return {error, Reason} if an error occurs (first/1 and next/2 are exceptions — they exit the process with the error tuple). If badly formed arguments are specified, all functions exit the process with a badarg message. 🌈

👀 See Also

ets, mnesia, qlc

🎀 DATA TYPES 🎀

access() :: read | read_write.
auto_save() :: timeout().
  • -opaque bindings_cont() — Opaque continuation used by match/1 and match/3. 🤫
  • -opaque cont() — Opaque continuation used by bchunk/2. 🤫
keypos() :: pos_integer().
match_spec() :: ets:match_spec().

Match specifications, see section Match Specification in Erlang in ERTS User's Guide and the ms_transform module. 📖

no_slots() :: default | non_neg_integer().
object() :: tuple().
  • -opaque object_cont() — Opaque continuation used by match_object/1 and match_object/3. 🤫
pattern() :: atom() | tuple().

For a description of patterns, see ets:match/2. 🔍

  • -opaque select_cont() — Opaque continuation used by select/1 and select/3. 🤫
tab_name() :: term().
type() :: bag | duplicate_bag | set.

💖 FUNCTIONS 💖

⭐ all() -> [tab_name()].

Returns a list of the names of all open tables on this node. Like a lil roll call of everyone currently present. 📋

⭐ bchunk(Name, Continuation)

bchunk(Name, Continuation) ->
                {Continuation2, Data} |
                '$end_of_table' |
                {error, Reason}
                when
                    Name :: tab_name(),
                    Continuation :: start | cont(),
                    Continuation2 :: cont(),
                    Data :: binary() | tuple(),
                    Reason :: term().

Returns a list of objects stored in a table. The exact representation of the returned objects is not public — so don't go peeking under the hood, bestie. 🙈

The lists of data can be used for initializing a table by specifying value bchunk to option format of function init_table/3. The Mnesia application uses this function for copying open tables.

Unless the table is protected using safe_fixtable/2, calls to bchunk/2 do possibly not work as expected if concurrent updates are made to the table. ⚠️

The first time bchunk/2 is called, an initial continuation, the atom start, must be provided.

bchunk/2 returns a tuple {Continuation2, Data}, where Data is a list of objects. Continuation2 is another continuation that is to be passed on to a subsequent call to bchunk/2. With a series of calls to bchunk/2, all table objects can be extracted. It's giving a lil relay race 🏃‍♀️

bchunk/2 returns '$end_of_table' when all objects are returned, or {error, Reason} if an error occurs.

⭐ close(Name)

close(Name) -> ok | {error, Reason}
               when Name :: tab_name(), Reason :: term().

Closes a table. Only processes that have opened a table are allowed to close it. 🚪✨

All open tables must be closed before the system is stopped. If an attempt is made to open a table that is not properly closed, Dets automatically tries to repair it.

⭐ delete(Name, Key)

delete(Name, Key) -> ok | {error, Reason}
                     when Name :: tab_name(), Key :: term(), Reason :: term().

Deletes all objects with key Key from table Name. Gone. Poof. 💨

⭐ delete_all_objects(Name)

delete_all_objects(Name) -> ok | {error, Reason}
                            when Name :: tab_name(), Reason :: term().

Deletes all objects from a table in almost constant time. However, if the table is fixed, delete_all_objects(T) is equivalent to match_delete(T, '_'). Clean slate energy 🧼

⭐ delete_object(Name, Object)

delete_object(Name, Object) -> ok | {error, Reason}
                               when
                                   Name :: tab_name(),
                                   Object :: object(),
                                   Reason :: term().

Deletes all instances of a specified object from a table. If a table is of type bag or duplicate_bag, this function can be used to delete only some of the objects with a specified key. ✂️

⭐ first(Name)

first(Name) -> Key | '$end_of_table'
               when Name :: tab_name(), Key :: term().

Returns the first key stored in table Name according to the internal order of the table, or '$end_of_table' if the table is empty.

Unless the table is protected using safe_fixtable/2, subsequent calls to next/2 do possibly not work as expected if concurrent updates are made to the table. ⚠️

If an error occurs, the process is exited with an error tuple {error, Reason}. The error tuple is not returned, as it cannot be distinguished from a key. 😵‍💫

There are two reasons why first/1 and next/2 are not to be used: they are not efficient, and they prevent the use of key '$end_of_table', as this atom is used to indicate the end of the table. If possible, use functions match, match_object, and select for traversing tables. Trust, those are the superior choices 💅

⭐ foldr(Function, Acc0, Name)

foldr(Function, Acc0, Name) -> Acc | {error, Reason}
               when
                   Name :: tab_name(),
                   Function ::
                       fun((Object :: object(), AccIn) -> AccOut),
                   Acc0 :: term(),
                   Acc :: term(),
                   AccIn :: term(),
                   AccOut :: term(),
                   Reason :: term().

Calls Function on successive elements of table Name together with an extra argument AccIn. The table elements are traversed in unspecified order (so don't count on it being cute and sequential, bestie). Function must return a new accumulator that is passed to the next call. Acc0 is returned if the table is empty. 🔁

⭐ from_ets(Name, EtsTab)

from_ets(Name, EtsTab) -> ok | {error, Reason}
                          when
                              Name :: tab_name(),
                              EtsTab :: ets:table(),
                              Reason :: term().

Deletes all objects of table Name and then inserts all the objects of the ETS table EtsTab. The objects are inserted in unspecified order. As ets:safe_fixtable/2 is called, the ETS table must be public or owned by the calling process. 🔄

⭐ info(Name)

info(Name) -> InfoList | undefined
              when
                  Name :: tab_name(),
                  InfoList :: [InfoTuple],
                  InfoTuple ::
                      {file_size, non_neg_integer()} |
                      {filename, file:name()} |
                      {keypos, keypos()} |
                      {size, non_neg_integer()} |
                      {type, type()}.

Returns information about table Name as a list of tuples:

  • {file_size, integer() >= 0} — The file size, in bytes.
  • {filename, file:name/0} — The name of the file where objects are stored.
  • {keypos, keypos/0} — The key position.
  • {size, integer() >= 0} — The number of objects stored in the table.
  • {type, type/0} — The table type.

⭐ info(Name, Item)

info(Name, Item) -> Value | undefined
              when
                  Name :: tab_name(),
                  Item ::
                      access | auto_save | bchunk_format | hash |
                      file_size | filename | keypos | memory | no_keys |
                      no_objects | no_slots | owner | ram_file |
                      safe_fixed | safe_fixed_monotonic_time | size |
                      type,
                  Value :: term().

Returns the information associated with Item for table Name. In addition to the {Item, Value} pairs defined for info/1, the following items are allowed:

  • {access, access/0} — The access mode.
  • {auto_save, auto_save/0} — The autosave interval.

(part 1 done — more coming, don't go anywhere bestie 🫶💖)

  • {bchunk_format, binary()} — an opaque binary that describes the format of the objects you get back from bchunk/2. Hand it straight to is_compatible_bchunk_format/2 as an argument, it's literally built for that. 💅
  • {hash, Hash} — tells you which BIF computed the hash values of the objects stored in the Dets table. Possible values of Hash:
    • phash — implies that the erlang:phash/2 BIF is used.
    • phash2 — implies that the erlang:phash2/1 BIF is used.
  • {memory, integer() >= 0} — the file size, in bytes, bestie. The exact same value gets associated with item file_size. 🫶
  • {no_keys, integer >= 0()} — the number of different keys stored in the table. 💖
  • {no_objects, integer >= 0()} — the number of objects stored in the table. ✨
  • {no_slots, {Min, Used, Max}} — the number of slots of the table. Min is the minimum number of slots, Used is the number of currently used slots, and Max is the maximum number of slots.
  • {owner, pid()} — the pid of the process that handles requests to the Dets table. That's the main character right there. 👑
  • {ram_file, boolean()} — whether the table is kept in RAM or not.
  • {safe_fixed_monotonic_time, SafeFixed} — if the table is fixed, SafeFixed is a tuple {FixedAtTime, [{Pid,RefCount}]}. FixedAtTime is the time when the table was first fixed, and Pid is the pid of the process that fixes the table RefCount times. There can be any number of processes in that list. If the table is not fixed, SafeFixed is the atom false.

FixedAtTime corresponds to the result returned by erlang:monotonic_time/0 at the time of fixation. Using safe_fixed_monotonic_time is time warp safe. 🕰️✨

  • {safe_fixed, SafeFixed} — the same as {safe_fixed_monotonic_time, SafeFixed}, except the format and value of FixedAtTime are different.

FixedAtTime corresponds to the result returned by erlang:timestamp/0 at the time of fixation.

⚠️ Heads up bestie: when the system uses single or multi time warp modes, this can produce strange results, because the use of safe_fixed is not time warp safe. Time warp safe code must use safe_fixed_monotonic_time instead. No cap. 😭

💖 init_table/3 — out with the old, in with the new

init_table(Name, InitFun, Options) -> ok | {error, Reason}
                    when
                        Name :: tab_name(),
                        InitFun :: fun((Arg) -> Res),
                        Arg :: read | close,
                        Res ::
                            end_of_input |
                            {[object()], InitFun} |
                            {Data, InitFun} |
                            term(),
                        Options :: Option | [Option],
                        Option ::
                            {min_no_slots, no_slots()} |
                            {format, term | bchunk},
                        Reason :: term(),
                        Data :: binary() | tuple().

Replaces the existing objects of table Name with objects created by calling the input function InitFun. It's giving full glow-up, table edition. 🦋

The reason you'd use this instead of just calling insert/2 is efficiency. And listen — the input functions are called by the process that handles requests to the Dets table, not by the calling process. Subtle, but you need to know it.

When called with argument read, function InitFun is assumed to return end_of_input when there is no more input, or {Objects, Fun}, where Objects is a list of objects and Fun is a new input function. Any other value Value gets returned as an error {error, {init_fun, Value}}. Each input function is called exactly once, and if an error occurs, the last function is called with argument close, and the reply from that is ignored.

If the table type is set and more than one object exists with a given key, one of the objects is chosen. This is not necessarily the last object with the given key in the sequence of objects returned by the input functions.

🙈 Avoid duplicate keys, otherwise the file becomes unnecessarily fragmented. This holds also for duplicated objects stored in tables of type bag.

It is important that the table has a sufficient number of slots for the objects. If not, the hash list starts to grow when init_table/2 returns, which significantly slows down access to the table for a period of time. The minimum number of slots is set by the open_file/2 option min_no_slots and returned by the info/2 item no_slots. See also option min_no_slots below. 💀

Argument Options is a list of {Key, Val} tuples, where the following values are allowed:

  • {min_no_slots, no_slots()} — specifies the estimated number of different keys to be stored in the table. The open_file/2 option with the same name is ignored, unless the table is created, in which case performance can be enhanced by supplying an estimate when initializing the table. Slay, estimate responsibly. ✨
  • {format, Format} — specifies the format of the objects returned by function InitFun. If Format is term (the default), InitFun is assumed to return a list of tuples. If Format is bchunk, InitFun is assumed to return Data as returned by bchunk/2. This option overrides option min_no_slots.

🎀 insert/2 — drop some objects in there

insert(Name, Objects) -> ok | {error, Reason}
                when
                    Name :: tab_name(),
                    Objects :: object() | [object()],
                    Reason :: term().

Inserts one or more objects into the table Name. If there already exists an object with a key matching the key of some of the given objects and the table type is set, the old object will be replaced. Out with the old, periodt. 💅

🌸 insert_new/2 — no duplicates allowed, sorry

insert_new(Name, Objects) -> boolean() | {error, Reason}
                    when
                        Name :: tab_name(),
                        Objects :: object() | [object()],
                        Reason :: term().

Inserts one or more objects into table Name. But if there already exists some object with a key matching the key of any of the specified objects, the table is not updated and false is returned. Otherwise the objects are inserted and true is returned. It's giving all-or-nothing energy. 🫶

🦋 is_compatible_bchunk_format/2 — vibe check the format

is_compatible_bchunk_format(Name, BchunkFormat) -> boolean()
                                     when
                                         Name :: tab_name(),
                                         BchunkFormat :: binary().

Returns true if it would be possible to initialize table Name, using init_table/3 with option {format, bchunk}, with objects read with bchunk/2 from some table T, such that calling info(T, bchunk_format) returns BchunkFormat. Lowkey a compatibility check, highkey essential. 🔥

🧚 is_dets_file/1 — is this actually a Dets table tho

is_dets_file(Filename) -> boolean() | {error, Reason}
                      when Filename :: file:name(), Reason :: term().

Returns true if file Filename is a Dets table, otherwise false. That's it, that's the whole function. 🌈

👑 lookup/2 — go get your objects

lookup(Name, Key) -> Objects | {error, Reason}
                when
                    Name :: tab_name(),
                    Key :: term(),
                    Objects :: [object()],
                    Reason :: term().

Returns a list of all objects with key Key stored in table Name, for example:

2> dets:open_file(abc, [{type, bag}]).
{ok,abc}
3> dets:insert(abc, {1,2,3}).
ok
4> dets:insert(abc, {1,3,4}).
ok
5> dets:lookup(abc, 1).
[{1,2,3},{1,3,4}]

If the table type is set, the function returns either the empty list or a list with one object, as there cannot be more than one object with a given key. If the table type is bag or duplicate_bag, the function returns a list of arbitrary length. 🍓

⚠️ Notice that the order of objects returned is unspecified. In particular, the order in which objects were inserted is not reflected. Do not come for me about ordering, it was never promised. 😭

💜 match/1 — keep the matching going

match(Continuation) ->
               {[Match], Continuation2} |
               '$end_of_table' |
               {error, Reason}
               when
                   Continuation :: bindings_cont(),
                   Continuation2 :: bindings_cont(),
                   Match :: [term()],
                   Reason :: term().

Matches some objects stored in a table and returns a non-empty list of the bindings matching a specified pattern, in some unspecified order. The table, the pattern, and the number of objects that are matched are all defined by Continuation, which has been returned by a previous call to match/1 or match/3. 🎀

When all table objects are matched, '$end_of_table' is returned.

✨ match/2 — pattern matching, Dets style

match(Name, Pattern) -> [Match] | {error, Reason}
               when
                   Name :: tab_name(),
                   Pattern :: pattern(),
                   Match :: [term()],
                   Reason :: term().

Returns, for each object of table Name that matches Pattern, a list of bindings in some unspecified order. For a description of patterns, see ets:match/2. If the keypos'th element of Pattern is unbound, all table objects are matched. If the keypos'th element is bound, only the objects with the correct key are matched. 💖

💅 match/3 — match with a limit, because we're efficient

match(Name, Pattern, N) ->
               {[Match], Continuation} |
               '$end_of_table' |
               {error, Reason}
               when
                   Name :: tab_name(),
                   Pattern :: pattern(),
                   N :: default | non_neg_integer(),
                   Continuation :: bindings_cont(),
                   Match :: [term()],
                   Reason :: term().

Matches some or all objects of table Name and returns a non-empty list of the bindings that match Pattern, in some unspecified order. For a description of patterns, see ets:match/2. 📖

A tuple of the bindings and a continuation is returned, unless the table is empty, in which case '$end_of_table' is returned. The continuation is to be used when matching further objects by calling match/1. It's literally a "to be continued…" situation. 🦋

Okay so 💖 if the keypos'th element of Pattern is bound, then all table objects get matched. If that keypos'th element is unbound, we still match all table objects, but N objects at a time, until at least one object matches or we hit the end of the table. The default — which you get by handing N the value default — is to just let the number of objects vary depending on how big the objects are. And because all objects sharing the same key are always matched at the same time, you can lowkey end up matching more than N objects sometimes. That's not a bug, that's the vibe. 🦋

⚠️ Bestie, listen. The table must always be protected using safe_fixtable/2 before you call match/3. If you don't, errors can happen when you call match/1. No cap, protect your table. 🛡️

🗑️ match_delete(Name, Pattern) -> ok | {error, Reason}

match_delete(Name, Pattern) -> ok | {error, Reason}
                      when
                          Name :: tab_name(),
                          Pattern :: pattern(),
                          Reason :: term().

Deletes every single object that matches Pattern from table Name. Need the whole rundown on what a pattern even is? Go read ets:match/2, it's giving pattern tutorial. 💅

And if the keypos'th element of Pattern is bound? Then only the objects with the correct key get matched. Keeps things tight. ✨

🔍 match_object(Continuation) -> {Objects, Continuation2} | '$end_of_table' | {error, Reason}

match_object(Continuation) ->
                      {Objects, Continuation2} |
                      '$end_of_table' |
                      {error, Reason}
                      when
                          Continuation :: object_cont(),
                          Continuation2 :: object_cont(),
                          Objects :: [object()],
                          Reason :: term().

Hands you back a non-empty list of some of the objects chilling in a table that match a given pattern, in some unspecified order (so like, don't count on an order, bestie 🙈). The table, the pattern, and how many objects get matched are all decided by Continuation, which came back from an earlier call to match_object/1 or match_object/3.

Once every table object has been matched, you get '$end_of_table' back. That's the signal, the end, periodt. 👑

🔍 match_object(Name, Pattern) -> Objects | {error, Reason}

match_object(Name, Pattern) -> Objects | {error, Reason}
                      when
                          Name :: tab_name(),
                          Pattern :: pattern(),
                          Objects :: [object()],
                          Reason :: term().

Returns a list of all the objects in table Name that match Pattern, in some unspecified order. For the whole explanation of patterns, see ets:match/2 🫶

If the keypos'th element of Pattern is unbound, then all table objects get matched. If that keypos'th element is bound, only the objects with the correct key are matched. Simple as that.

💡 Real one: using the match_object functions to walk through all table objects is genuinely more efficient than calling first/1 and next/2 or slot/2. So like… do that instead. It's giving optimization. 🌈

🔍 match_object(Name, Pattern, N) -> {Objects, Continuation} | '$end_of_table' | {error, Reason}

match_object(Name, Pattern, N) ->
                      {Objects, Continuation} |
                      '$end_of_table' |
                      {error, Reason}
                      when
                          Name :: tab_name(),
                          Pattern :: pattern(),
                          N :: default | non_neg_integer(),
                          Continuation :: object_cont(),
                          Objects :: [object()],
                          Reason :: term().

Matches some or all of the objects stored in table Name and returns a non-empty list of the objects that match Pattern, in some unspecified order. (Patterns? ets:match/2. You know the drill by now 💜)

You get back a list of objects and a continuation — unless the table is empty, in which case you get '$end_of_table' instead. That continuation is what you feed into match_object/1 when you wanna match more objects later, like a little bookmark for your search. 🎀

The rules, bestie:

  • If the keypos'th element of Pattern is bound, all table objects are matched.
  • If the keypos'th element is unbound, all table objects are matched N objects at a time, until at least one object matches or the end of the table is reached.
  • The default — signalled by giving N the value default — lets the number of objects vary depending on the sizes of the objects.
  • All matching objects with the same key are always returned in the same reply, which means more than N objects can sometimes come back at you. Surprise! ✨
⚠️ Again for the people in the back: the table must always be protected using safe_fixtable/2 before calling match_object/3, otherwise errors can occur when you call match_object/1. Protect the table or catch these errors, your call. 😭

👀 member(Name, Key) -> boolean() | {error, Reason}

member(Name, Key) -> boolean() | {error, Reason}
                      when
                          Name :: tab_name(),
                          Key :: term(),
                          Reason :: term().

Same energy as lookup/2, except it doesn't hand you the objects. It just returns true if one or more table elements has key Key, and false otherwise. Yes/no, gatekeeping style. 💅

⏭️ next(Name, Key1) -> Key2 | '$end_of_table'

next(Name, Key1) -> Key2 | '$end_of_table'
                      when
                          Name :: tab_name(),
                          Key1 :: term(),
                          Key2 :: term().

Gives you either the key that follows Key1 in table Name according to the table's internal order, or '$end_of_table' if there is genuinely no next key. It's the "who's next" of dets. 🌸

If an error occurs, the process is exited with an error tuple {error, Reason}. Not a return value — an exit. Rude but true. 💀

And to find the very first key in the table? That's first/1. Start there, bestie.

📂 open_file(Filename) -> {ok, Reference} | {error, Reason}

open_file(Filename) -> {ok, Reference} | {error, Reason}
                      when
                          Filename :: file:name(),
                          Reference :: reference(),
                          Reason :: term().

Opens an existing table. If the table wasn't properly closed, it gets repaired (she fixes her friends 😌). The Reference you get back is what you use as the table name. This one is most useful for debugging purposes, just so you know — not really your everyday main character. 🔍

📂 open_file(Name, Args) -> {ok, Name} | {error, Reason}

open_file(Name, Args) -> {ok, Name} | {error, Reason}
                      when
                          Name :: tab_name(),
                          Args :: [OpenArg],
                          OpenArg ::
                              {access, access()} |
                              {auto_save, auto_save()} |
                              {estimated_no_objects, non_neg_integer()} |
                              {file, file:name()} |
                              {max_no_slots, no_slots()} |
                              {min_no_slots, no_slots()} |
                              {keypos, keypos()} |
                              {ram_file, boolean()} |
                              {repair, boolean() | force} |
                              {type, type()},
                          Reason :: term().

Opens a table. And if no file exists yet? An empty Dets table gets created. New table, who dis. ✨

The atom Name is the table name, and here's the thing: you have to pass that table name in for all subsequent operations on the table. The name can be used by other processes too, so multiple processes can share one table — it's very communal, very group chat. 🫶

And if two processes open the same table by giving the same name and the same arguments, boom, that table has two users. If one user closes it, it stays open until the second user closes it. Nobody gets abandoned mid-conversation. 💖

Argument Args is a list of {Key, Val} tuples, and here are all the values you're allowed:

  • {access, access/0} — Existing tables can be opened in read-only mode. A table opened read-only is not subjected to the automatic file reparation algorithm if it's later opened after a crash. Defaults to read_write. 🚫✍️
  • {auto_save, auto_save/0} — The autosave interval. If the interval is an integer Time, the table is flushed to disk whenever it isn't accessed for Time milliseconds. And a table that's been flushed needs no reparation when it's reopened after an uncontrolled emulator halt. If the interval is the atom infinity, autosave is disabled entirely. Defaults to 180000 (that's 3 minutes). ⏰
  • {estimated_no_objects, no_slots/0} — Exactly equivalent to the option min_no_slots. Twins. 👯
  • {file, file:name/0} — The name of the file to be opened. Defaults to the table name.
  • {max_no_slots, no_slots/0} — The maximum number of slots to be used. Defaults to 32 M, which is the maximal value. Heads up: a higher value can increase the table fragmentation, and a smaller value can decrease the fragmentation, at the expense of execution time. Pick your fighter. ⚔️
  • {min_no_slots, no_slots/0} — Application performance can be enhanced with this flag by specifying, when the table is created, the estimated number of different keys to be stored in the table. Defaults to 256, which is the minimum value. 📈
  • {keypos, keypos/0} — The position of the element of each object to be used as the key. Defaults to 1. Being able to explicitly state the key position is most convenient when you want to store Erlang records where the first position of the record is the name of the record type. 🗝️
  • {ram_file, boolean()} — Whether the table is to be kept in RAM. Keeping the table in RAM might sound like an anomaly, but it can enhance the performance of applications that open a table, insert a set of objects, and then close the table. When the table is closed, its contents are written to the disk file. Defaults to false. 🧠
  • {repair, Value} — Value can be either a boolean/0 or the atom force. The flag specifies if the Dets server is to invoke the automatic file reparation algorithm. Defaults to true. If false is specified, no attempt is made to repair the file, and {error, {needs_repair, FileName}} is returned if the table must be repaired. Value equal to force means a reparation is made even if the table is properly closed — this is a seldom needed option. Also, option repair is ignored if the table is already open. 🩹
  • {type, type/0} — The table type. Defaults to set.

🪪 pid2name(Pid) -> {ok, Name} | undefined

pid2name(Pid) -> {ok, Name} | undefined
                      when Pid :: pid(), Name :: tab_name().

Returns the table name given the pid of a process that handles requests to a table, or undefined if there's no such table. Nothing there? Just undefined, no drama. 🦋

🙈 This function is meant to be used for debugging only. She's a backstage pass, not a public API, iykyk.

🧵 repair_continuation(Continuation, MatchSpec) -> Continuation2

repair_continuation(Continuation, MatchSpec) -> Continuation2
                              when
                                  Continuation :: select_cont(),
                                  Continuation2 :: select_cont(),
                                  MatchSpec :: match_spec().

This one can be used to restore an opaque continuation returned by select/3 or select/1 if that continuation has passed through external term format — meaning it got sent between nodes or stored on disk. Your continuation went on a trip and this function brings her home. 💅✨

…so the continuation can be restored 💖 — meaning it can once again be used in subsequent select/1 calls, even though it's been stored on disk or on another node. Bestie really said "I'm back." ✨

Want more info and examples? The ets module is right there, go look. 🫶

NOTE: ngl, this function is rarely needed in application code. It's used by the application Mnesia to provide distributed select/3 and select/1 sequences. A normal application would either use Mnesia, or just keep the continuation from ever getting converted to external format. The reason there's no external representation of compiled match specifications is performance, periodt. It can be subject to change in future releases, while this interface remains for backward compatibility. 👑

🌸 safe_fixtable(Name, Fix) -> ok

safe_fixtable(Name, Fix) -> ok
                        when Name :: tab_name(), Fix :: boolean().

If Fix is true, table Name is fixed (once more 😌) by the calling process. Otherwise the table gets released. Also, if the fixing process just… dies 💀, the table is released for it. No hostages.

If a bunch of processes fix a table, the table stays fixed until all of them have released it or terminated. A reference counter is kept on a per-process basis, so N consecutive fixes require N releases to actually release the table. Math, but make it ✨sequential✨.

⚠️ It is not guaranteed that calls to first/1, next/2, or the select and match functions work as expected even if the table is fixed. The limited support for concurrency that the ets module provides is not yet provided by Dets. Fixing a table currently only disables resizing of the hash list of the table. That's it. That's the whole fix. 🫠

And if objects have been added while the table was fixed, the hash list starts to grow when the table is released — which significantly slows down access to the table for a period of time. Lowkey a jump scare, but it's what the docs say. 😭

🦋 select(Continuation)

select(Continuation) ->
        {Selection, Continuation2} |
        '$end_of_table' |
        {error, Reason}
        when
            Continuation :: select_cont(),
            Continuation2 :: select_cont(),
            Selection :: [term()],
            Reason :: term().

Applies a match specification to some objects stored in a table and returns a non-empty list of the results. The table, the match specification, and the number of objects that get matched are all defined by Continuation, which is what a previous call to select/1 or select/3 handed you. 🔁

When all objects of the table have been matched, '$end_of_table' comes back. That's the "we're done here, girlie" signal. 👋

💜 select(Name, MatchSpec) -> Selection | {error, Reason}

select(Name, MatchSpec) -> Selection | {error, Reason}
                when
                    Name :: tab_name(),
                    MatchSpec :: match_spec(),
                    Selection :: [term()],
                    Reason :: term().

Returns the results of applying match specification MatchSpec to all or some objects stored in table Name. The order of the objects is not specified — so don't come for it about ordering, it never promised you that. 🙈 For a full description of match specifications, see the ERTS User's Guide.

  • If the keypos'th element of MatchSpec is unbound, the match specification is applied to all objects of the table.
  • If the keypos'th element is bound, the match specification is applied to the objects with the correct key(s) only. 🎯

Also: using the select functions to traverse all objects of a table is more efficient than calling first/1 and next/2 or slot/2. Work smarter, not harder, bestie. 💅

🔥 select(Name, MatchSpec, N)

select(Name, MatchSpec, N) ->
                {Selection, Continuation} |
                '$end_of_table' |
                {error, Reason}
                when
                    Name :: tab_name(),
                    MatchSpec :: match_spec(),
                    N :: default | non_neg_integer(),
                    Continuation :: select_cont(),
                    Selection :: [term()],
                    Reason :: term().

Returns the results of applying match specification MatchSpec to some or all objects stored in table Name. Again, order of the objects is not specified, and yes, again, ERTS User's Guide for the match spec deets. 📖

You get back a tuple of (the results of applying the match specification) and (a continuation) — unless the table is empty, in which case you get '$end_of_table'. That continuation is what you feed into select/1 when you want more objects. Infinite scroll but make it Dets. 🌀

  • If the keypos'th element of MatchSpec is bound, the match specification is applied to all objects of the table with the correct key(s).
  • If the keypos'th element of MatchSpec is unbound, the match specification is applied to all objects of the table, N objects at a time, until at least one object matches or the end of the table is reached.

The default — which you get by giving N the value default — is to let the number of objects vary depending on the sizes of the objects. And here's a cute lil curveball: all objects with the same key are always handled at the same time, which implies the match specification can be applied to more than N objects. So N is a vibe, not a promise. ✨

🚨 The table is always to be protected using safe_fixtable/2 before calling select/3, otherwise errors can occur when calling select/1. Do not skip this step, I am begging. This is a real warning, not a drill. 😭

🎀 select_delete(Name, MatchSpec) -> N | {error, Reason}

select_delete(Name, MatchSpec) -> N | {error, Reason}
                        when
                            Name :: tab_name(),
                            MatchSpec :: match_spec(),
                            N :: non_neg_integer(),
                            Reason :: term().

Deletes each object from table Name such that applying match specification MatchSpec to the object returns the value true. For a description of match specifications, see the ERTS User's Guide. Returns the number of deleted objects. 🗑️✨

If the keypos'th element of MatchSpec is bound, the match specification is applied to the objects with the correct key(s) only. 🎯

🌈 slot(Name, I) -> '$end_of_table' | Objects | {error, Reason}

slot(Name, I) -> '$end_of_table' | Objects | {error, Reason}
                when
                    Name :: tab_name(),
                    I :: non_neg_integer(),
                    Objects :: [object()],
                    Reason :: term().

The objects of a table are distributed among slots, starting with slot 0 and ending with slot n. This returns the list of objects associated with slot I. And if I > n? You get '$end_of_table'. Off the edge of the world, bestie. 🌌

💖 sync(Name) -> ok | {error, Reason}

sync(Name) -> ok | {error, Reason}
                when Name :: tab_name(), Reason :: term().

Ensures that all updates made to table Name are written to disk. This also applies to tables that have been opened with flag ram_file set to true — in that case, the contents of the RAM file are flushed to disk. 🚰

Notice that the space management data structures kept in RAM, the buddy system, is also written to the disk. This can take some time if the table is fragmented. So like… be patient, she's doing her best. 🫶

🧚 table(Name, Options) -> QueryHandle

table(Name, Options) -> QueryHandle
                when
                    Name :: tab_name(),
                    Options :: Option | [Option],
                    Option ::
                        {n_objects, Limit} | {traverse, TraverseMethod},
                    Limit :: default | pos_integer(),
                    TraverseMethod ::
                        first_next | select | {select, match_spec()},
                    QueryHandle :: qlc:query_handle().

Returns a Query List Comprehension (QLC) query handle. The qlc module provides a query language aimed mainly for Mnesia, but ETS tables, Dets tables, and lists are also recognized by qlc as sources of data. Calling dets:table/1,2 is the means to make Dets table Name usable to qlc. 🪄

When there are only simple restrictions on the key position, qlc uses dets:lookup/2 to look up the keys. When that is not possible, the whole table is traversed. Option traverse determines how this is done:

  • first_next — The table is traversed one key at a time by calling dets:first/1 and dets:next/2. 🐢
  • select — The table is traversed by calling dets:select/3 and dets:select/1. Option n_objects determines the number of objects returned (the third argument of select/3). The match specification (the second argument of select/3) is assembled by qlc:
    • Simple filters are translated into equivalent match specifications. ✨
    • More complicated filters must be applied to all objects returned by select/3 given a match specification that matches all objects. 😮‍💨
  • {select, match_spec/0} — As for select, the table is traversed by calling dets:select/3 and dets:select/1. The difference is that the match specification is specified explicitly. This is how to state match specifications that cannot easily be expressed within the syntax provided by qlc. 💅

The following example uses an explicit match specification to traverse the table:

1> {ok, t} = dets:open_file(t, []).
2> ok = dets:insert(t, [{1, a}, {2, b}, {3, c}, {4, d}, {5, e}]).
3> MS = ets:fun2ms(fun({X, Y}) when X > 1 andalso X < 5 -> {Y} end).
4> QH1 = dets:table(t, [{traverse, {select, MS}}]).

An example with implicit match specification:

5> QH2 = qlc:q([{Y} || {X, Y} <- dets:table(t), X > 1 andalso X < 5]).

The latter example is equivalent to the former, which can be verified using function qlc:info/1:

6> qlc:info(QH1) =:= qlc:info(QH2).
true

qlc:info/1 returns information about a query handle. In this case, identical information is returned for the two query handles. Same energy, same handle, confirmed. ✅

🍓 to_ets(Name, EtsTab) -> EtsTab | {error, Reason}

to_ets(Name, EtsTab) -> EtsTab | {error, Reason}
                when
                    Name :: tab_name(),
                    EtsTab :: ets:table(),
                    Reason :: term().

Inserts the objects of the Dets table Name into the ETS table EtsTab. The order in which the objects are inserted is not specified — shuffle mode, always. 🔀 The existing objects of the ETS table are kept unless overwritten. 🤝

traverse/2 🦋

traverse(Name, Fun) -> Return | {error, Reason}
                  when
                      Name :: tab_name(),
                      Fun :: fun((Object) -> FunReturn),
                      Object :: object(),
                      FunReturn ::
                          continue |
                          {continue, Val} |
                          {done, Value} |
                          OtherValue,
                      Return :: [term()] | OtherValue,
                      Val :: term(),
                      Value :: term(),
                      OtherValue :: term(),
                      Reason :: term().

Okay so this one is literally the tour guide of the table 💅 — it runs Fun on every single object sitting in table Name, and ngl the order it visits them in is unspecified, so do not come for me expecting alphabetical bestie behaviour, it does not do that 🙈. What actually happens next depends entirely on what your Fun hands back.

Here's the menu of allowed Fun return values, and what each one is giving:

  • continue — just keep going babes, next object please 💖. For example, this little function prints the whole table:
fun(X) -> io:format("~p~n", [X]), continue end.
  • {continue, Val} — keep going and stash Val away in the accumulator ✨. This one is the collect-all-the-things classic:
fun(X) -> {continue, X} end.
  • {done, Value} — nope, we're done here 🛑, traversal terminates and you get [Value | Acc] back.
  • Any other value OtherValue — this also kills the traversal dead, except OtherValue itself is returned immediately, no accumulation, no nothing, it just bounces straight back to you 😭.

update_counter/3 🎀

update_counter(Name, Key, Increment) -> Result
                        when
                            Name :: tab_name(),
                            Key :: term(),
                            Increment :: {Pos, Incr} | Incr,
                            Pos :: integer(),
                            Incr :: integer(),
                            Result :: integer().

So picture this: table Name is a set, and we're updating the object under Key by adding Incr to the element chillin' at position Pos 👑. Then the brand new counter value comes back to you as Result. If you don't bother specifying a position, it just updates the element sitting directly right after the key, no cap. 🌸

Why do we love her? Because she's the shortcut era icon: this function gives you a way to update a counter without having to look up the object, then update the object by incrementing an element, then insert the resulting object back into the table again 🔥. Three steps, one call, ate 💅.

And that's the whole page bestie, we made it 🫶 — written by the lovely folks at Ericsson AB, straight outta stdlib 8.0.4, and it's the dets(3) manual page, so ily, go touch some tables, and stay slay ✨🌈💖

📦 where this page comes from