ATOMIC_ADD | (ATOM,VALUE,STAT) |
ATOMIC_AND | (ATOM,VALUE,STAT) |
ATOMIC_CAS | (ATOM,OLD,COMPARE,NEW,STAT) |
ATOMIC_FETCH_ADD | (ATOM,VALUE,OLD,STAT) |
ATOMIC_FETCH_AND | (ATOM,VALUE,OLD,STAT) |
ATOMIC_FETCH_OR | (ATOM,VALUE,OLD,STAT) |
ATOMIC_FETCH_XOR | (ATOM,VALUE,OLD,STAT) |
ATOMIC_OR | (ATOM,VALUE,STAT) |
ATOMIC_XOR | (ATOM,VALUE,STAT) |
The arguments ATOM, COMPARE, NEW and OLD are all INTEGER(ATOMIC_INT_KIND). The ATOM argument is the one that is updated, and must be a coarray or a coindexed variable. The OLD argument is INTENT(OUT), and receives the value of ATOM before the operation. The STAT argument is optional, and must be a non-coindexed variable of type INTEGER and at least 16 bits in size.
The VALUE argument must be INTEGER but can be of any kind; however, both VALUE and the result of the operation must be representable in INTEGER(ATOMIC_INT_KIND).
The *_ADD operation is addition, the *_AND operation is bitwise and (like IAND), the *_OR operation is bitwise or (like IOR) and the *_XOR operation is bitwise exclusive or (like IEOR).
ATOMIC_CAS is an atomic compare-and-swap operation. If ATOM is equal to COMPARE, it is assigned the value NEW; otherwise, it remains unchanged. In either case, the value before the operation is assigned to OLD. Note that both COMPARE and NEW must also be INTEGER(ATOMIC_INT_KIND).
If the ATOM is a coindexed variable, and is located on a failed image, the operation fails and an error condition is raised; the OLD argument becomes undefined, and if STAT is present, it is assigned the value STAT_FAILED_IMAGE; if STAT is not present, the program is terminated. If no error occurs and STAT is present, it is assigned the value zero.
COSHAPE( COARRAY [, KIND ] )
COARRAY : coarray of any type; if it is ALLOCATABLE, it must be allocated; if it is a structure component, the rightmost component must be a coarray component;
KIND (optional) : scalar Integer constant expression that is a valid Integer kind number;
Result : vector of type Integer, or Integer(KIND)} if KIND is present; the size of the result is equal to the co-rank of COARRAY.
For example, if a coarray is declared
REAL x[5,*]
and there are eight images in the current team, COSHAPE(x) will be equal to [ 5,2 ].IMAGE_STATUS( IMAGE [, TEAM ] )
IMAGE : positive integer that is a valid image number;
TEAM (optional) : scalar TEAM_TYPE value that identifies the current or an ancestor team;
Result : default Integer.
The value of the result is STAT_FAILED_IMAGE if the image has failed, STAT_STOPPED_IMAGE if the image has stopped, and zero otherwise. The optional TEAM argument specifies which team the image number applies to; if it is not specified, the current team is used.
STOPPED_IMAGES( [ TEAM, KIND ] )
TEAM (optional) : scalar TEAM_TYPE value that identifies the current or an ancestor team;
KIND (optional) : scalar INTEGER constant expression that is a valid Integer kind number;
Result : vector of type Integer, or Integer(KIND) if KIND is present.
The elements of the result are the stopped image numbers in ascending order.
Like type LOCK_TYPE, entities of type EVENT_TYPE are required to be variables or components, variables of type EVENT_TYPE are required to be coarrays, and variables with noncoarray subcomponents of type LOCK_TYPE are required to be coarrays. Such variables are called event variables. An event variable is not permitted to appear in a variable definition context (i.e. any context where it might be modified), except in an EVENT POST or EVENT WAIT statement, or as an actual argument where the dummy argument is INTENT(INOUT).
An event variable on an image may have an event “posted” to it by means of the image control statement EVENT POST, which has the form
EVENT POST ( event-variable [, sync-stat ]... )where the optional sync-stats may be a single STAT=stat-variable specifier and/or a single ERRSMG=errmsg-variable specifier; stat-variable must be a scalar integer variable that can hold values up to 9999, and errmsg-variable must be a scalar default character variable. Posting an event increments the variable's “outstanding event count” (this count is initially zero). The event-variable in this statement will usually be a coindexed variable, as it is rarely useful for an image to post an event to itself.
If STAT= appears and the post is successful, zero is assigned to the stat-variable. If the image on which the event-variable is located has stopped, STAT_STOPPED_IMAGE is assigned to the stat-variable; if the image has failed, STAT_FAILED_IMAGE is assigned, and if any other error occurs, some other positive value is assigned. If ERRMSG= appears and any error occurs, an explanatory message is assigned to the errmsg-variable. Note that if STAT= does not appear and an error occurs, the program will be error-terminated, so having ERRMSG= without STAT= is useless.
Events are received by the image control statement EVENT WAIT, which has the form
EVENT WAIT ( event-variable [, event-wait-spec-list ] )where the optional event-wait-spec-list is a comma-separated list that may contain a single STAT=stat-variable specifier, a single ERRSMG=errmsg-variable specifier, and/or a single UNTIL_COUNT=scalar-integer-expr specifier. Waiting on an event waits until its “outstanding event count” is greater than or equal to the UNTIL_COUNT= specifier value, or greater than zero if UNTIL_COUNT= does not appear. If the value specified in UNTIL_COUNT= is less than one, it is treated as if it were equal to one.
The event-variable in this statement is not permitted to be coindexed; that is, an image can only wait for events posted to its own event variables. There is a partial synchronisation between the waiting image and the images that contributed to the “outstanding event count”; the segment following execution of the EVENT WAIT statement follows the segments before the EVENT POST statement executions. The synchronisation does not operate in reverse, that is, there is no implication that execution of any segment in a posting image follows any segment in the waiting image.
The STAT= and ERRMSG= operate similarly to the EVENT POST statement, except of course that STAT_FAILED_IMAGE and STAT_STOPPED_IMAGE are impossible.
Finally, the intrinsic function EVENT_QUERY can be used to interrogate an event variable without waiting for it. It has the form
EVENT_QUERY ( EVENT, COUNT [, STAT ] )where EVENT is an event variable, COUNT is an integer variable at least as big as default integer, and the optional STAT is an integer variable that can hold values up to 9999. EVENT is not permitted to be a coindexed variable; that is, only the image where the event variable is located is permitted to query its count. COUNT is assigned the current “outstanding event count” of the event variable. If STAT is present, it is assigned the value zero on successful execution, and a positive value if any error occurs. If any error occurs and STAT is not present, the program is error-terminated.
Note that event posts in unordered segments might not be included in the value assigned to count; that is, it might take some (communication) time for an event post to reach the variable, and it is only guaranteed to have reached the variable if the images have already synchronised. Use of EVENT_QUERY does not by itself imply any synchronisation.
Unlike EVENT_TYPE and LOCK_TYPE, functions that return TEAM_TYPE are permitted. Furthermore, a variable of type TEAM_TYPE is forbidden from being a coarray, and assigning a TEAM_TYPE value from another image (e.g. as a component of a derived type assignment) makes the variable undefined; this is because the TEAM_TYPE value might contain information specific to a particular image, e.g. routing information to the other images. Variables of type TEAM_TYPE are called team variables.
Creating teams | The set of all the images in the program is called the initial team.
At any time, a particular image will be executing in a particular team,
the current team.
A set of subteams of the current team can be created at any time by using
the FORM TEAM statement, which has the form
FORM TEAM ( team-number , team-variable [, form-team-spec ]... )where team-number is a scalar integer expression that evaluates to a positive value, team-variable is a team variable, and each form-team-spec is STAT=, ERRMSG=, and NEW_INDEX=index-value specifier. At most one of each kind of form-team-spec may appear in a FORM TEAM statement. All active images of the current team must execute the same FORM TEAM statement. If NEW_INDEX= appears, index-value must be a positive scalar integer (see below). The STAT= and ERRMSG= specifiers have their usual form and semantics. The number of subteams that execution of FORM TEAM produces is equal to the number of unique team-number values in that execution; each unique team-number value identifies a subteam in the set, and each image belongs to the subteam whose team number it specified. If NEW_INDEX= appears, it specifies the image number that the image will have in its new subteam, and therefore must be in the range 1 to N, where N is the number of images in that subteam, and must be unique. If NEW_INDEX= does not appear, it is processor-dependent what the image number in the new subteam will be. For example, TYPE(TEAM_TYPE) oddeven myteamnumber = 111*(MOD(THIS_IMAGE(),2) + 1) FORM TEAM ( myteamnumber, oddeven )will create a set of two subteams, one with team number 111, the other with team number 222. Team 111 will contain the images with even image numbers in the current team, and team 222 will contain the images with odd image numbers in the current team. On each image, the variable oddeven identifies the subteam to which that image belongs. Note that the team numbers are completely arbitrary (being chosen by the program), and only have meaning within that set of subteams, which are called “sibling” teams. |
Changing to a subteam | The current team is changed by executing a CHANGE TEAM construct, which has the basic form:
CHANGE TEAM ( team-value [, sync-stat-list ] ) statements END TEAM [ ( [ sync-stat-list ] ) ]where team-value is a value of type TEAM_TYPE, and the optional sync-stat-list is a comma-separated list containing at most one STAT= and ERRMSG= specifier; the STAT= and ERRMSG= specifiers have their usual form and semantics. Execution of the statements within the construct are with the current team set to the team identified by team-value; this must be a subteam of the current team outside the construct. The setting of the current team remains in effect during procedure calls, so any procedure referenced by the construct will also be executed with the new team current. Transfer of control out of the construct, e.g. by a RETURN or GOTO statement is prohibited. The construct may be exited by executing its END TEAM statement, or by executing an EXIT statement that belongs to the construct; the latter is only possible if the construct is given a name (this is not shown in the form above, but consists of “construct-name:” prefix to the CHANGE TEAM statement, and and a “construct-name” suffix to the END TEAM statement). While executing a CHANGE TEAM construct, image selectors operate using the new team's image indices, the intrinsic functions NUM_IMAGES and THIS_IMAGES return the data for the new team, and SYNC ALL synchronises the new team only. There is an implicit synchronisation of all images of the new team both on the CHANGE TEAM statement, and on the END TEAM statement, and all active images must execute the same statement at this time. |
Synchronising parent or ancestor teams | While executing within a CHANGE TEAM construct, the effects of SYNC ALL and
SYNC IMAGES only apply to images within the current team.
For SYNC ALL to synchronise the parent team, it would be necessary to first exit the
construct.
This may be inconvenient when the computation following the synchronisation would be
within the team.
For this purpose, the SYNC TEAM statement has been added, with the form SYNC TEAM ( team-value [, sync-stat-list ] )where team-value identifies the current team or an ancestor thereof, and sync-stat-list is the usual comma-separated list containing at most one STAT= specifier and at most one ERRMSG= specifier (these have their usual semantics and so are not further described here). The effect is to synchronise all images in the specified team. |
Team-related intrinsic functions |
|
Information about sibling and ancestor teams | The intrinsic functions NUM_IMAGES and THIS_IMAGE normally return information
relevant to the current team, but they can return information for an ancestor team by using the
optional TEAM argument, which takes a TEAM_TYPE value that identifies the current
team or an ancestor.
Similarly, the NUM_IMAGES intrinsic can return information for a sibling team by using the optional TEAM_NUMBER
argument, which takes an integer value that is equal to the team number of the current or a sibling
team. (Note that because the executing image is never a member of a sibling team, THIS_IMAGE
does not accept a TEAM_NUMBER argument.)
The intrinsic function NUM_IMAGES thus has two additional forms as follows:
NUM_IMAGES( TEAM ) NUM_IMAGES( TEAM_NUMBER ) For THIS_IMAGE, the revised forms it may take are as follows: THIS_IMAGE( [ TEAM ] ) THIS_IMAGE( COARRAY [, TEAM ] ) THIS_IMAGE( COARRAY, DIM [, TEAM ] )The meanings of the COARRAY and DIM arguments is unchanged. The optional TEAM argument specifies the team for which to return the information. |
Establishing coarrays | A coarray is not allowed to be used within a team unless it is established in that team
or an ancestor thereof.
The basic rules for establishment are as follows:
|
Allocating and deallocating coarrays in teams | If a coarray with the ALLOCATABLE attribute is already allocated when a CHANGE TEAM
statement is executed, it is not allowed to DEALLOCATE it within that construct (or within
a procedure called from that construct).
If a coarray with the ALLOCATABLE attribute is unallocated when a CHANGE TEAM statement is executed, it may be allocated (using ALLOCATE) within that construct (or within a procedure called from that construct), and may be subsequently deallocated as well. If such a coarray remains allocated when the END TEAM statement is executed, it is automatically deallocated at that time. This means that when using teams, allocatable coarrays may be allocated on some images (within the team), but unallocated on other images (outside the team), or allocated with a different shape or type parameters on other images (also outside the team). However, when executing in a team, the coarray is either unallocated on all images of the team, or allocated with the same type parameters and shape on all images of the team. |
Accessing coarrays in sibling teams | Access to a coarray outside the current team, but in a sibling team, is possible using
the TEAM_NUMBER= specifier in an image selector.
This uses the extended syntax for image selectors:
[ cosubscript-list [, image-selector-spec-list ] ]where cosubscript-list is the usual list of cosubscripts, and image-selector-spec-list contains a TEAM_NUMBER=team-number specifier, where team-number is the positive integer value that identifies a sibling team. The image-selector-spec-list may also contain a STAT= specifier (this is described later, under Fault tolerance). When the TEAM_NUMBER= specifier is used the cosubscripts are treated as cosubscripts in the sibling team. Note that access in this way is quite risky, and will typically require synchronisation, possibly of the whole parent team. The coarray in question must be established in the parent team. |
Accessing coarrays in ancestor teams | Access to a coarray in the parent or more distant ancestor team is possible using
the TEAM= specifier in an image selector.
This uses the extended syntax for image selectors:
[ cosubscript-list [, image-selector-spec-list ] ]where cosubscript-list is the usual list of cosubscripts, and image-selector-spec-list contains a TEAM=team-value specifier, where team-value is a value of type TEAM_TYPE that identifies the current team or an ancestor. The image-selector-spec-list may also contain a STAT= specifier (this is described later, under Fault tolerance). When the TEAM= specifier is used the cosubscripts are treated as cosubscripts in the specified ancestor team, and the image thus specified may lie within or outside the current team. If the access is to an image that is outside the current team, care should be taken that the images are appropriately synchronised; such synchronisation cannot be obtained by SYNC ALL or SYNC IMAGES, as they operate within a team, but may be obtained by SYNC TEAM specifying an ancestor team, or by using locks or events. The coarray in question must be established in the specified (current or ancestor) team. |
Coarray association in CHANGE TEAM | It is possible to associate a local coarray-name in a CHANGE TEAM construct
with a named coarray outside the construct, changing the codimension and/or coextents in the
process.
This acts like a limited kind of argument association; the local coarray-name has the
type, parameters, rank and array shape of the outside coarray, but does not have the
ALLOCATABLE attribute.
The syntax of the CHANGE TEAM construct with one or more such associations is as follows:
CHANGE TEAM ( team-value , coarray-association-list [, sync-stat-list ] )where coarray-association-list is a comma-separated list of local-coarray-name [ explicit-coshape-spec ] => outer-coarray-nameand explicit-coshape-spec is [ [ lower-cobound : ] upper-cobound , ]... [ lower-cobound : ] *(The notation [ something ]... means something occurring zero or more times.) The cobounds expressions are evaluated on execution of the CHANGE TEAM statement. Use of this feature is not encouraged, as it is less powerful and more confusing than argument association. |
The form of the FAIL IMAGE statement is simply
FAIL IMAGEand execution of this statement will cause the current image to “fail”, that is, cease to participate in program execution. This is the only way that an image can fail in NAG Fortran 7.0.
If all images have failed or stopped, program execution will terminate. NAG Fortran will display a warning message if any images have failed.
An image selector has an optional list of specifiers, the revised syntax of an image selector being (where the normal square brackets are literally square brackets, and the italic square brackets indicate optionality):
[ cosubscript-list [, image-selector-spec-list ] ]where cosubscript-list is a comma-separated list of cosubscripts, one scalar integer per codimension of the variable, and image-selector-spec-list is a comma-separated containing at most one STAT=stat-variable specifier, and at most one TEAM= or TEAM_NUMBER= specifier (these were described earlier). If the coindexed object being accessed lies on a failed image, the value STAT_FAILED_IMAGE is assigned to the stat-variable, and otherwise the value zero is assigned.
The intrinsic function FAILED_IMAGES returns an array of images that are known to have failed (it is possible that an image might fail and no other image realise until it tries to synchronise with it). Its syntax is as follows.
FAILED_IMAGES( [ TEAM, KIND ] )
TEAM (optional) : scalar TEAM_TYPE value that identifies the current or an ancestor team;
KIND (optional) : scalar Integer constant expression that is a valid Integer kind number;
Result : vector of type Integer, or Integer(KIND) if KIND is present.
The elements of the result are the failed image numbers in ascending order.
In order to be able to handle failed images, the following semantics apply:
The fault tolerance features are in principle intended to permit recovery from hardware failure, with the FAIL IMAGE statement allowing some testing of recovery scenarios. The NAG Fortran Compiler does not support recovery from hardware failure (at Release 7.1).
All of these subroutines have optional STAT and ERRMSG arguments. On successful execution, the STAT argument is assigned the value zero and the ERRMSG argument is left unchanged. If an error occurs, a positive value is assigned to STAT and an explanatory message is assigned to ERRMSG. Only the errors STAT_FAILED_IMAGE and STAT_STOPPED_IMAGE are likely to be able to be caught in this way. Because there is not full synchronisation (see below), different images may receive different errors, or none at all. If an error occurs and STAT is not present, execution is terminated. Note that if the actual arguments for STAT or ERRMSG are optional dummy arguments, they must be present on all images or absent on all images.
A reference (CALL) to one of these subroutines is not an image control statement, does not end the current segment, and does not imply synchronisation (though some partial synchronisation will occur during the computation). However, such calls are only permitted where an image control statement is permitted.
Each image in a team must execute the same sequence of CALL statements to collective subroutines as the other images in the team. There must be no synchronisation between the images at the time of the call; the invocations must come from unordered segments.
All collective subroutines have the first argument “A”, which is INTENT(INOUT), and must not be a coindexed object. This argument contains the data for the calculation, and must have the same type, type parameters, and shape on all images in the current team. If it is a coarray that is a dummy argument, it must have the same ultimate argument on all images.
SUBROUTINE CO_BROADCAST ( A, SOURCE_IMAGE [, STAT, ERRMSG ] )
A : variable of any type; it must not be a coindexed object, and must have the same type, type parameters and shape on all images in the current team; if A is a coarray that is a dummy argument, it must have the same ultimate argument on each image;
SOURCE_IMAGE : integer scalar, in the range one to NUM_IMAGES(), this argument must have the same value on all images in the current team;
STAT (optional) : integer scalar variable, not coindexed;
ERRMSG (optional) : character scalar variable of default kind, not coindexed.
The value of argument A on image SOURCE_IMAGE is assigned to the argument A on all the other images.
SUBROUTINE CO_MAX ( A [, RESULT_IMAGE, STAT, ERRMSG ] )A: variable of type Integer, Real, or Character; it must not be a coindexed object, and must have the same type, type parameters and shape on all images in the current team; if A is a coarray that is a dummy argument, it must have the same ultimate argument on each image;
RESULT_IMAGE (optional) : integer scalar, in the range one to NUM_IMAGES(), this argument must be either present on all images or absent on all images; if present, it must have the same value on all images in the current team;
STAT (optional) : integer scalar variable, not coindexed;
ERRMSG (optional) : character scalar variable of default kind, not coindexed.
This subroutine computes the maximum value of A across all images; if A is an array, the value is computed elementally. If RESULT_IMAGE is present, the result is assigned to argument A on that image, otherwise it is assigned to argument A on all images.
SUBROUTINE CO_MIN ( A [, RESULT_IMAGE, STAT, ERRMSG ] )A: variable of type Integer, Real, or Character; it must not be a coindexed object, and must have the same type, type parameters and shape on all images in the current team; if A is a coarray that is a dummy argument, it must have the same ultimate argument on each image;
RESULT_IMAGE (optional) : integer scalar, in the range one to NUM_IMAGES(), this argument must be either present on all images or absent on all images; if present, it must have the same value on all images in the current team;
STAT (optional) : integer scalar variable, not coindexed;
ERRMSG (optional) : character scalar variable of default kind, not coindexed.
This subroutine computes the minimum value of A across all images; if A is an array, the value is computed elementally. If RESULT_IMAGE is present, the result is assigned to argument A on that image, otherwise it is assigned to argument A on all images.
SUBROUTINE CO_REDUCE ( A, OPERATION [, RESULT_IMAGE, STAT, ERRMSG ] )A: non-polymorphic variable of any type; it must not be a coindexed object, and must have the same type, type parameters and shape on all images in the current team; if A is a coarray that is a dummy argument, it must have the same ultimate argument on each image;
OPERATION : pure function with exactly two arguments; the dummy arguments of OPERATION must be non-allocatable, non-optional, non-pointer, non-polymorphic dummy variables, and each argument and the result of the function must be scalar with the same type and type parameters as A;
RESULT_IMAGE (optional) : integer scalar, in the range one to NUM_IMAGES(), this argument must be either present on all images or absent on all images; if present, it must have the same value on all images in the current team;
STAT (optional) : integer scalar variable, not coindexed;
ERRMSG (optional) : character scalar variable of default kind, not coindexed.
This subroutine computes an arbitrary reduction of A across all images; if A is an array, the value is computed elementally. The reduction is computed starting with the set of corresponding values of A on all images; this is an iterative process, taking two values from the set and converting them to a single value by applying the OPERATION function; the process continues until the set contains only a single value — that value is the result. If RESULT_IMAGE is present, the result is assigned to argument A on that image, otherwise it is assigned to argument A on all images.
SUBROUTINE CO_SUM ( A [, RESULT_IMAGE, STAT, ERRMSG ] )A: variable of type Integer, Real, or Complex; it must not be a coindexed object, and must have the same type, type parameters and shape on all images in the current team; if A is a coarray that is a dummy argument, it must have the same ultimate argument on each image;
RESULT_IMAGE (optional) : integer scalar, in the range one to NUM_IMAGES(), this argument must be either present on all images or absent on all images; if present, it must have the same value on all images in the current team;
STAT (optional) : integer scalar variable, not coindexed;
ERRMSG (optional) : character scalar variable of default kind, not coindexed.
This subroutine computes the sum of A across all images; if A is an array, the value is computed elementally. If RESULT_IMAGE is present, the result is assigned to argument A on that image, otherwise it is assigned to argument A on all images.