A region is an arbitrarily sized, contiguous area of memory identified
by a unique region identifier. A region identifier is a portable
and stable name for a region; in order to access a region, a processor
node must know the region's region identifier. New regions can be
created dynamically (using
rgn_create) and can later be removed
when they are no longer needed. Blocks of memory comprising distinct
regions (those with different region identifiers) are non-overlapping.
In order to access the memory associated with a region, it must be
mapped into the local address space. To map a region, one calls
rgn_map with the region identifier of the
region to be mapped; the function returns a pointer to the base of the
region's data area.
CRL also provides operations for sharing regions. Specifically, CRL provides library calls that can be used to delimit operations---portions of the code where regions will be read from or written to. The library calls initiating and terminating operations are required to maintain the consistency of shared data. The coherence model provided by CRL considers entire operations on regions as indivisible units. From this perspective, CRL provides sequential consistency for read and write operations in the same sense that a sequentially-consistent hardware-based DSM does for individual loads and stores.
In order to use CRL functionality, application source files should
include the file crl.h. When compiling, users of the TCP/UNIX
implementation should make sure that the symbol
before including crl.h (e.g., before the
#include or via a
compiler directive) to ensure that slight differences in the CRL
interface (see Section 6) necessary for the TCP/UNIX
version are reflected in the include file. After compilation,
application object files should be linked with the CRL object file
(libcrl.o) to produce an executable.
This section provides a brief description of the functionality provided by CRL 1.0.
crl_initshould be called on all nodes simultaneously.
crl_initshould be called before any CRL functionality is utilized. (The TCP/UNIX version of CRL 1.0 uses a slightly different interface to
crl_init; see Section 6 for details.)
crl_num_nodes-1; this index can be used to allow programs to perform different tasks on different nodes. ( This may be removed or altered in future releases.)
typedef unsigned rid_t
rid_tis a scalar type that is used to represent region identifiers; region identifiers are portable, unique `names' for regions. In this implementation, region identifiers are represented using (32-bit) unsigned integers.
rid_t rgn_create(unsigned size)
void rgn_destroy(rid_t rgn_id)
rgn_id. ( In the current implementation,
rid_t rgn_rid(void *rgn)
rgn_map(see subsection 3.1.3).
unsigned rgn_size(void *rgn)
rgn_map(see subsection 3.1.3).
void rgn_flush(void *rgn)
rgn_map(see subsection 3.1.3). When sensible to do so,
rgn_flushboth writes back any local changes to a region's home node and sends a protocol message relinquishing read ownership.
void *rgn_map(rid_t rgn_id)
rgn_idinto the address space of the local node. Return a pointer to the base of the region's data area; the pointer is guaranteed to be doubleword aligned. This pointer is also used to name a mapped region when initiating and terminating operations or unmapping the region. A region can be mapped multiple times on the same node.
The address at which a region is mapped into the local address space
is only guaranteed to remain fixed for the duration of a mapping;
subsequent mappings of the same region may yield different
addresses. Further, two mappings of the same region on different nodes
are not guaranteed to be at the same address. Because of this, the
only globally portable and stable name for a region is the
corresponding region identifier. Thus, applications that need to store
a ``pointer'' to shared data (e.g., in another region as part of a
distributed, shared data structure) must store the corresponding
region's unique identifier (as returned by
not the address at which the region is currently mapped. Subsequent
references to the data referenced by the region identifier must be
preceded by calls to
rgn_map (to obtain the address at which
the region is mapped) and followed by calls to
clean up the mapping).
void rgn_unmap(void *rgn)
rgn_map. If a region is mapped multiple times on the calling node,
rgn_unmaponly terminates one `reference' to the mapped region.
void rgn_start_read(void *rgn)
rgn_map. The read operation is considered to be in progress from the time
rgn_start_readreturns until a corresponding call to
rgn_end_read; during this time, the contents of the region's data area may be safely read.
If any write operations are in-progress or pending on a region,
rgn_start_read will block (not return) until they have been
completed. Multiple read operations on the same region may be in
void rgn_end_read(void *rgn)
rgn_map. If multiple read operations are in progress on the calling node,
rgn_end_readonly terminates one of them.
void rgn_start_write(void *rgn)
rgn_map. The write operation is considered to be in progress from the time
rgn_start_writereturns until a corresponding call to
rgn_end_write; during this time, the contents of the region's data area may be safely read or written.
If any read or write operations are in-progress or pending on a
rgn_start_write will block (not return) until they have
been completed. Because write operations are serialized with respect
to all other operations on the same region, no other operations can be
in progress during a write operation (including subsequent read or
write operations initiated at the node with a write operation already
void rgn_end_write(void *rgn)
rgn_barrierdoes not return on any node until it has been called on all nodes.
void rgn_bcast_send(int len, void *buf)
rgn_bcast_recvwith the same len argument and a pointer to an appropriately-sized buffer.
void rgn_bcast_recv(int len, void *buf)
double rgn_reduce_dadd(double arg)
rgn_reduce_dadddoes not return on any node until it has been called on all nodes. Returns the sum of the arg values provided on each node.
double rgn_reduce_dmin(double arg)
rgn_reduce_dmindoes not return on any node until it has been called on all nodes. Returns the minimum of the arg values provided on each node.
double rgn_reduce_dmax(double arg)
rgn_reduce_dmaxdoes not return on any node until it has been called on all nodes. Returns the maximum of the arg values provided on each node.
CRL 1.0 provides replacement functions for malloc and free. This is necessary because CRL protocol message handlers use malloc to dynamically allocate memory (when necessary). Since message handlers can interrupt the ``normal'' computation running on a node and standard implementations of malloc and other memory allocation functions are not reentrant, this can lead to serious problems if care isn't taken to ensure that calls to malloc and free are executed atomically.
void *safe_malloc(unsigned nbytes)
safe_malloc. Calls to other malloc-like functions (e.g., calloc, realloc) should either be replaced with calls to
safe_mallocor by implementing new
safe_mallocas a template.