disarray(4rheolef)

πŸ“¦
version 7.2 Β· 4 versions
, in debian trixie, ubuntu noble, ubuntu questing

πŸ¦‹ NAME πŸ¦‹

disarray β€” distributed container (rheolef-7.2) πŸ’–

And yes bestie, she's a section-4 page, so her full government name is disarray(4rheolef) β€” write it in your notes, it's giving citation energy. πŸ‘‘


✨ DESCRIPTION ✨

Okay so picture a std::vector container but make it distributed memory model πŸ’… β€” that's disarray, no cap.

The interface is lowkey just like the std::vector ones, except there's a whole extra layer of communication features bolted on for the distributed memory model. That's the plot twist. 🫢


🌸 EXAMPLE 🌸

Here's the vibe, in one lil' main:

int main (int argc, char**argv) {
  environment distributed(argc, argv);
  distributor ownership (100);
  disarray<double> x (ownership, 3.14);
  dout << x << endl;
}

Iconic. She's constructed, she's filled with 3.14, she's printed. Ate. ✨


πŸ’… NOTATION πŸ’…

So there are two kinds of indexes, and you HAVE to know the difference or you will be confused, and confusion is not the vibe:

  • dis_i β€” this index refers to the complete array. It is valid on all processes. 🌈 So yeah, she's global, she's everywhere, main character.
  • i β€” this index refers to the subset of the array that is owned by the current process. It is valid only on the local process. 🏠 Very local. Very homebody.

Read and write accessors to the subset of the array that is owned by the current process are still denoted by square brackets, e.g. value = x[i] and x[i] = value, respectively. Same old brackets, same old love. πŸŽ€

Read and write accessors to the complete array β€” including the array subsets that are owned by some other processors β€” got introduced with the help of some brand new member functions:

  • x.dis_entry(dis_i) = value β€” write access at any location ✍️
  • value = x.dis_at(dis_i) β€” read access at any location πŸ‘€

And listen: in order to optimize communications, they should be grouped. Do not be out here doing them one by one, that's a crime against bandwidth. 🚫


πŸŽ€ GROUPED WRITES πŸŽ€

Step one: loop on any dis_i and write your value:

for (...) {
  x.dis_entry (dis_i) = value;
}

Finally, perform all communications in one pass:

x.dis_entry_assembly();

After this command, each value is stored in x, at its right place, depending on dis_i and its ownership. 🏑

And note the cute little optimization: when dis_i is owned by the current process, the value is directly written as x[i] = value and no communication are generated. Zero. Zilch. No packets were harmed. πŸ’€


🧚 GROUPED READS 🧚

First, define the set of indexes that you want to access:

std::set<size_t> dis_i_set;

Then, loop on dis_i all these indexes and append it to this set:

for (...) {
  dis_i_set.insert (dis_i);
}

Next, perform communications:

x.set_dis_indexes (dis_i_set);

After this command, each values associated to the dis_i index, and that belongs to the index set, is also available also on the current processor:

for (...) {
  value = x.dis_at (dis_i);
}

And same deal as before: when dis_i is owned by the current process, the value is directly read as value = x[i] and no communication are generated. πŸ¦‹ Efficiency queen.


⭐ TEMPLATE PARAMETERS ⭐

Quick reminder bestie: the std::class takes two template parameters β€” a T one for the stored elements and an A one for the memory allocator. Our girl disarray takes three. She's built different, fr fr:

  • T: the stored object type πŸ“
  • M: the memory model, i.e. sequential or distributed, by default default_memory_model, and that's defined at the configuration stage
  • A: the memory allocator, by default std::allocator 🧺

πŸ”₯ CONSTRUCTOR πŸ”₯

In the sequential case, the class interface provides a simplified constructor:

int local_size = 100;
disarray<double,sequential> x (local_size, init_val);

This declaration is similar to those of a std::vector one: no communications are possible. None. Nada. It's a solo act. πŸ™ˆ

In order to enable communication, you have to replace the local_size by information on how the array is distributed in memory:

distributor ownership (100);
disarray<double> x (ownership, 3.14);

And the distributor(4) class does this job. She's the plug. πŸ”Œ


πŸ‘‘ IMPLEMENTATION πŸ‘‘

This documentation has been generated from file linalg/lib/disarray.h β€” so like, the receipts are right there if you wanna go look. πŸ’œ

template <class T, class A>
class disarray<T,distributed,A> : public smart_pointer<disarray_rep<T,distributed,A> > {
public:

// typedefs:

    typedef disarray_rep<T,distributed,A> rep;
    typedef smart_pointer<rep>            base;

    typedef distributed                   memory_type;
    typedef typename rep::size_type       size_type;
    typedef typename rep::difference_type difference_type;
    typedef typename rep::value_type      value_type;
    typedef typename rep::reference       reference;
    typedef typename rep::dis_reference   dis_reference;
    typedef typename rep::iterator        iterator;
    typedef typename rep::const_reference const_reference;
    typedef typename rep::const_iterator  const_iterator;
    typedef typename rep::scatter_map_type scatter_map_type;

// allocators:

    disarray       (const distributor& ownership = distributor(), const T& init_val = T(), const A& alloc = A());
    void resize (const distributor& ownership = distributor(), const T& init_val = T());

// local accessors & modifiers:

    A get_allocator() const              { return base::data().get_allocator(); }
    size_type     size () const          { return base::data().size(); }
    size_type dis_size () const          { return base::data().dis_size(); }
    const distributor& ownership() const { return base::data().ownership(); }
    const communicator& comm() const     { return base::data().comm(); }

    reference       operator[] (size_type i)       { return base::data().operator[] (i); }
    const_reference operator[] (size_type i) const { return base::data().operator[] (i); }
    reference       operator() (size_type i)       { return base::data().operator[] (i); }
    const_reference operator() (size_type i) const { return base::data().operator[] (i); }

          iterator begin()       { return base::data().begin(); }
    const_iterator begin() const { return base::data().begin(); }
          iterator end()         { return base::data().end(); }
    const_iterator end() const   { return base::data().end(); }

// global accessor:

    template<class Set, class Map>
    void append_dis_entry (const Set& ext_idx_set, Map& ext_idx_map) const { base::data().append_dis_entry (ext_idx_set, ext_idx_map); }

    template<class Set, class Map>
    void get_dis_entry    (const Set& ext_idx_set, Map& ext_idx_map) const { base::data().get_dis_entry (ext_idx_set, ext_idx_map); }

    template<class Set>
    void append_dis_indexes (const Set& ext_idx_set) const { base::data().append_dis_indexes (ext_idx_set); }
    void reset_dis_indexes() const { base::data().reset_dis_indexes(); }
    void get_dis_indexes (std::set<size_type>& ext_idx_set) const { base::data().get_dis_indexes (ext_idx_set); }

    template<class Set>
    void set_dis_indexes    (const Set& ext_idx_set) const { base::data().set_dis_indexes (ext_idx_set); }

    const T& dis_at (size_type dis_i) const { return base::data().dis_at (dis_i); }

    // get all external pairs (dis_i, values):
    const scatter_map_type& get_dis_map_entries() const { return base::data().get_dis_map_entries(); }

// global modifiers (for compatibility with distributed interface):

    dis_reference dis_entry (size_type dis_i) { return base::data().dis_entry(dis_i); }

    template<class SetOp = typename details::default_set_op_traits<T>::type>
    void dis_entry_assembly_begin (SetOp my_set_op = SetOp()) { base::data().dis_entry_assembly_begin (my_set_op); }
    template<class SetOp = typename details::default_set_op_traits<T>::type>
    void dis_entry_assembly_end   (SetOp my_set_op = SetOp()) { base::data().dis_entry_assembly_end   (my_set_op); }
    template<class SetOp = typename details::default_set_op_traits<T>::type>
    void dis_entry_assembly       (SetOp my_set_op = SetOp()) { base::data().dis_entry_assembly       (my_set_op); }

    void dis_entry_assembly_begin() { base::data().template dis_entry_assembly_begin<typename details::default_set_op_traits<T>::type>(); }
    void dis_entry_assembly_end()   { base::data().template dis_entry_assembly_end<typename details::default_set_op_traits<T>::type>(); }
    void dis_entry_assembly()       { dis_entry_assembly_begin(); dis_entry_assembly_end(); }

// apply a partition:

    template<class RepSize>
    void repartition (                              // old_numbering for *this
        const RepSize&        partition,            // old_ownership
        disarray<T,distributed>& new_disarray,            // new_ownership (created)
        RepSize&              old_numbering,        // new_ownership
        RepSize&              new_numbering) const  // old_ownership
        { return base::data().repartition (partition.data(), new_disarray.data(), old_numbering.data(), new_numbering.data()); }

    template<class RepSize>
    void permutation_apply (                       // old_numbering for *this
        const RepSize&          new_numbering,     // old_ownership
        disarray<T,distributed,A>& new_disarray) const   // new_ownership (already allocated)
        { base::data().permutation_apply (new_numbering.data(), new_disarray.data()); }

    void reverse_permutation (                                 // old_ownership for *this=iold2dis_inew
        disarray<size_type,distributed,A>& inew2dis_iold) const   // new_ownership
        { base::data().reverse_permutation (inew2dis_iold.data()); }

// i/o:

    odiststream& put_values (odiststream& ops) const { return base::data().put_values(ops); }
    idiststream& get_values (idiststream& ips)       { return base::data().get_values(ips); }
    void dump (std::string name) const      { return base::data().dump(name); }

    template <class GetFunction>
    idiststream& get_values (idiststream& ips, GetFunction get_element)       { return base::data().get_values(ips, get_element); }
    template <class PutFunction>
    odiststream& put_values (odiststream& ops, PutFunction put_element) const { return base::data().put_values(ops, put_element); }
    template <class PutFunction, class A2> odiststream& permuted_put_values (
                odiststream& ops, const disarray<size_type,distributed,A2>& perm, PutFunction put_element) const
                                                                     { return base::data().permuted_put_values (ops, perm.data(), put_element); }
};

πŸ’œ AUTHOR πŸ’œ

Pierre Saramito <Pierre.Saramito@imag.fr> β€” yes that's the real email, copy it exactly bestie, don't be out here typo-ing someone's address. πŸ“§


Copyright (C) 2000-2018 Pierre Saramito <Pierre.Saramito@imag.fr>

GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.

This is free software: you are free to change and redistribute it.

⚠️ There is NO WARRANTY, to the extent permitted by law. That's a big all-caps NO, so like... respect it. 😭

And that's the whole page, babes β€” this was disarray(4rheolef) from rheolef Version 7.2, brought to you by the rheolef crew. Go forth and distribute your arrays responsibly. 🫢✨

πŸ“¦ where this page comes from