Document revision date: 30 March 2001
[Compaq] [Go to the documentation home page] [How to order documentation] [Help on this site] [How to contact us]
[OpenVMS documentation]
OpenVMS VAX RTL Mathematics (MTH$) Manual

OpenVMS VAX RTL Mathematics (MTH$) Manual

Order Number: AA--PVXJD--TE

April 2001

This manual documents the mathematics routines contained in the MTH$ facility of the OpenVMS Run-Time Library.

Revision/Update Information: This manual supersedes the OpenVMS VAX RTL Mathematics (MTH$) Manual, Version 7.1.

Software Version: OpenVMS VAX Version 7.3

Compaq Computer Corporation
Houston, Texas

© 2001 Compaq Computer Corporation

Compaq, VAX, VMS, and the Compaq logo Registered in U.S. Patent and Trademark Office.

Compaq Ada and OpenVMS are trademarks of Compaq Information Technologies Group, L.P. in the United States and other countries.

The following are third-party trademarks:

BASIC is a registered trademark of the Trustees of Dartmouth College, D.B.A. Dartmouth College.

All other product names mentioned herein may be the trademarks or registered trademarks of their respective companies.

Confidential computer software. Valid license from Compaq required for possession, use, or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license.

Compaq shall not be liable for technical or editorial errors or omissions contained herein.

The information in this document is provided "as is" without warranty of any kind and is subject to change without notice. The warranties for Compaq products are set forth in the express limited waranty statements accompanying such products. Nothing herein should be construed as constituting an additional warranty.


The Compaq OpenVMS documentation set is available on CD-ROM.

Contents Index


This manual provides users of the Compaq OpenVMS operating system with detailed usage and reference information on mathematics routines supplied in the MTH$ facility of the Run-Time Library.

Run-Time Library routines can be used only in programs written in languages that produce native code for the VAX hardware. At present, these languages include VAX MACRO and the following compiled high-level languages:

Compaq Ada
Compaq BASIC for OpenVMS VAX Systems
Compaq C for OpenVMS VAX
Compaq COBOL for OpenVMS VAX
Compaq Pascal for OpenVMS VAX Systems
Compaq Fortran for OpenVMS VAX Systems

Interpreted languages that can also access Run-Time Library routines include VAX DSM and Compaq Datatrieve.

Intended Audience

This manual is intended for system and application programmers who write programs that call MTH$ Run-Time Library routines.

Document Structure

This manual contains two tutorial chapters, two reference sections, and two appendixes:

Related Documents

The Run-Time Library routines are documented in a series of reference manuals. A description of how the Run-Time Library routines are accessed and of how OpenVMS features and functionality are available through calls to the MTH$ Run-Time Library appears in OpenVMS Programming Concepts Manual. Descriptions of the other RTL facilities and their corresponding routines are presented in the following books:

Application programmers using any language can refer to the Guide to Creating OpenVMS Modular Procedures for writing modular and reentrant code.

High-level language programmers will find additional information on calling Run-Time Library routines in their language reference manuals. Additional information may also be found in the language user's guide provided with your OpenVMS language software.

For a complete list and description of the manuals in the OpenVMS documentation set, see the OpenVMS Version 7.3 New Features and Documentation Overview.

For additional information about Compaq OpenVMS products and services, access the Compaq website at the following location: 


1 This manual has been archived but is available on the OpenVMS documentation CD-ROM.

Reader's Comments

Compaq welcomes your comments on this manual. Please send comments to either of the following addresses:
Mail Compaq Computer Corporation
OSSG Documentation Group, ZKO3-4/U08
110 Spit Brook Rd.
Nashua, NH 03062-2698

How to Order Additional Documentation

Use the following World Wide Web address to order additional documentation: 

If you need help deciding which documentation best meets your needs, call 800-282-6672.


The following conventions are used in this manual:
Ctrl/ x A sequence such as Ctrl/ x indicates that you must hold down the key labeled Ctrl while you press another key or a pointing device button.
PF1 x A sequence such as PF1 x indicates that you must first press and release the key labeled PF1 and then press and release another key or a pointing device button.
[Return] In examples, a key name enclosed in a box indicates that you press a key on the keyboard. (In text, a key name is not enclosed in a box.)

In the HTML version of this document, this convention appears as brackets, rather than a box.

... A horizontal ellipsis in examples indicates one of the following possibilities:
  • Additional optional arguments in a statement have been omitted.
  • The preceding item or items can be repeated one or more times.
  • Additional parameters, values, or other information can be entered.
A vertical ellipsis indicates the omission of items from a code example or command format; the items are omitted because they are not important to the topic being discussed.
( ) In command format descriptions, parentheses indicate that you must enclose choices in parentheses if you specify more than one.
[ ] In command format descriptions, brackets indicate optional choices. You can choose one or more items or no items. Do not type the brackets on the command line. However, you must include the brackets in the syntax for OpenVMS directory specifications and for a substring specification in an assignment statement.
{ } In command format descriptions, braces indicate required choices; you must choose at least one of the items listed. Do not type the braces on the command line.
bold text This typeface represents the introduction of a new term. It also represents the name of an argument, an attribute, or a reason.
italic text Italic text indicates important information, complete titles of manuals, or variables. Variables include information that varies in system output (Internal error number), in command lines (/PRODUCER= name), and in command parameters in text (where dd represents the predefined code for the device type).
UPPERCASE TEXT Uppercase text indicates a command, the name of a routine, the name of a file, or the abbreviation for a system privilege.
Monospace text

Monospace type indicates code examples and interactive screen displays.

In the C programming language, monospace type in text identifies the following elements: keywords, the names of independently compiled external functions and files, syntax summaries, and references to variables or identifiers introduced in an example.

- A hyphen at the end of a command format description, command line, or code line indicates that the command or statement continues on the following line.
numbers All numbers in text are assumed to be decimal unless otherwise noted. Nondecimal radixes---binary, octal, or hexadecimal---are explicitly indicated.

Part 1
MTH$ Tutorial Section

This part of the OpenVMS VAX RTL Mathematics (MTH$) Manual contains tutorial information about the OpenVMS RTL MTH$ facility, and is structured as follows:

Chapter 1
OpenVMS Run-Time Library Mathematics (MTH$) Facility

The OpenVMS Run-Time Library Mathematics (MTH$) facility contains routines to perform a wide variety of computations including the following:

The OTS$ facility provides additional language-independent arithmetic support routines (see the OpenVMS RTL General Purpose (OTS$) Manual).

This chapter contains an introduction to the MTH$ facility and includes examples of how to call mathematics routines from BASIC, COBOL, Fortran, MACRO, Pascal, PL/I, and Ada.

Chapter 2 contains an overview of the vector routines available on VAX processors.

The Scalar MTH$ Reference Section describes the MTH$ scalar routines.

The Vector MTH$ Reference Section describes the MTH$ vector routines.

1.1 Entry Point Names

The names of the mathematics routines are formed by adding the MTH$ prefix to the function names.

When function arguments and returned values are of the same data type, the first letter of the name indicates this data type. When function arguments and returned values are of different data types, the first letter indicates the data type of the returned value, and the second letter indicates the data type of the arguments.

The letters used as data type prefixes are listed below.
Letter Data Type
I Word
J Longword
D D_floating
G G_floating
H H_floating
C F_floating complex
CD D_floating complex
CG G_floating complex

Generally, F-floating data types have no letter designation. For example, MTH$SIN returns an F-floating value of the sine of an F-floating argument and MTH$DSIN returns a D-floating value of the sine of a D-floating argument. However, in some of the miscellaneous functions, F-floating data types are referenced by the letter designation A.

1.2 Calling Conventions

For calling conventions specific to the MTH$ vector routines, refer to Chapter 2.

All calls to mathematics routines, as described in the Format section of each routine, accept arguments passed by reference. JSB entry points accept arguments passed by value.

All mathematics routines return values in R0 or R0/R1 except those routines for which the values cannot fit in 64 bits. D-floating complex, G-floating complex, and H-floating values are data structures which are larger than 64 bits. Routines returning values that cannot fit in registers R0/R1 return their function values into the first argument in the argument list.

The notation JSB MTH$NAME_Rn, where n is the highest register number referenced, indicates that an equivalent JSB entry point is available. Registers R0:Rn are not preserved.

Routines with JSB entry points accept a single argument in R0:Rm, where m, which is defined in the following table, is dependent on the data type.
Data Type m
F_floating 0
D_floating 1
G_floating 1
H_floating 3

A routine returning one value returns it to registers R0:Rm.

When a routine returns two values (for example, MTH$SINCOS), the first value is returned in R0:Rm and the second value is returned in (R<m+1>:R<2*m+1>).

Note that for routines returning a single value, n>=m. For routines returning two values, n>=2*m + 1.

In general, CALL entry points for mathematics routines do the following:

JSB entry points execute in the context of the caller with the enable operations as set by the caller. Since the routines do not cause arithmetic traps or faults, their operation is not affected by the setting of the arithmetic trap enables, except as noted.

For more detailed information on CALL and JSB entry points, refer to the OpenVMS Programming Concepts Manual.

1.3 Algorithms

For those mathematics routines having corresponding algorithms, the complete algorithm can be found in the Description section of the routine description appearing in the Scalar MTH$ Reference Section of this manual.

1.4 Condition Handling

Error conditions are indicated by using the VAX signaling mechanism. The VAX signaling mechanism signals all conditions in mathematics routines as SEVERE by calling LIB$SIGNAL. When a SEVERE error is signaled, the default handler causes the image to exit after printing an error message. A user-established condition handler can be written to cause execution to continue at the point of the error by returning SS$_CONTINUE. A mathematics routine returns to its caller after the contents of R0/R1 have been restored from the mechanism argument vector CHF$L_MCH_SAVR0/R1. Thus, the user-established handler should correct CHF$L_MCH_SAVR0/R1 to the desired function value to be returned to the caller of the mathematics routine.

D-floating complex, G-floating complex, and H-floating values cannot be corrected with a user-established condition handler, because R2/R3 is not available in the mechanism argument vector.

Note that it is more reliable to correct R0 and R1 to resemble R0 and R1 of a double-precision floating-point value. A double-precision floating-point value correction works for both single- and double-precision values.

If the correction is not performed, the floating-point reserved operand --0.0 is returned. A floating-point reserved operand is a floating-point datum with a sign bit of 1 and a biased exponent of 0. Accessing the floating-point reserved operand will cause a reserved operand fault. See the OpenVMS RTL Library (LIB$) Manual for a complete description of how to write user condition handlers for SEVERE errors.

A few mathematics routines signal floating underflow if the calling program (JSB or CALL) has enabled floating underflow faults or traps.

All mathematics routines access input arguments and the real and imaginary parts of complex numbers using floating-point instructions. Therefore, a reserved operand fault can occur in any mathematics routine.

1.5 Complex Numbers

A complex number y is defined as an ordered pair of real numbers r and i, where r is the real part and i is the imaginary part of the complex number.


OpenVMS supports three floating-point complex types: F-floating complex, D-floating complex, and G-floating complex. There is no H-floating complex data type.

Run-Time Library mathematics routines that use complex arguments require a pointer to a structure containing two x-floating values to be passed by reference for each argument. The first x-floating value contains r, the real part of the complex number. The second x-floating value contains i, the imaginary part of the complex number. Similarly, Run-Time Library mathematics routines that return complex function values return two x-floating values. Some Language Independent Support (OTS$) routines also calculate complex functions.

Note that complex functions have no JSB entry points.

1.6 Mathematics Routines Not Documented in the MTH$ Reference Section

The mathematics routines in Table 1-1 are not found in the reference section of this manual. Instead, their entry points and argument information are listed in Appendix A of this manual.

A reserved operand fault can occur for any floating-point input argument in any mathematics routine. Other condition values signaled by each mathematics routine are indicated in the footnotes.

Table 1-1 Additional Mathematics Routines
Entry Point Function
Absolute Value Routines
MTH$ABS F-floating absolute value
MTH$DABS D-floating absolute value
MTH$GABS G-floating absolute value
MTH$HABS H-floating absolute value 1
MTH$IIABS Word absolute value 2
MTH$JIABS Longword absolute value 2
Bitwise AND Operator Routines
MTH$IIAND Bitwise AND of two word parameters
MTH$JIAND Bitwise AND of two longword parameters
F-Floating Conversion Routines
MTH$DBLE Convert F-floating to D-floating (exact)
MTH$GDBLE Convert F-floating to G-floating (exact)
MTH$IIFIX Convert F-floating to word (truncated) 2
MTH$JIFIX Convert F-floating to longword (truncated) 2
Floating-Point Positive Difference Routines
MTH$DIM Positive difference of two F-floating parameters 3
MTH$DDIM Positive difference of two D-floating parameters 3
MTH$GDIM Positive difference of two G-floating parameters 3
MTH$HDIM Positive difference of two H-floating parameters 1,3
MTH$IIDIM Positive difference of two word parameters 2
MTH$JIDIM Positive difference of two longword parameters 2
Bitwise Exclusive OR Operator Routines
MTH$IIEOR Bitwise exclusive OR of two word parameters
MTH$JIEOR Bitwise exclusive OR of two longword parameters
Integer to Floating-Point Conversion Routines
MTH$FLOATI Convert word to F-floating (exact)
MTH$DFLOTI Convert word to D-floating (exact)
MTH$GFLOTI Convert word to G-floating (exact)
MTH$FLOATJ Convert longword to F-floating (rounded)
MTH$DFLOTJ Convert longword to D-floating (exact)
MTH$GFLOTJ Convert longword to G-floating (exact)
Conversion to Greatest Floating-Point Integer Routines
MTH$FLOOR Convert F-floating to greatest F-floating integer
MTH$DFLOOR Convert D-floating to greatest D-floating integer
MTH$GFLOOR Convert G-floating to greatest G-floating integer
MTH$HFLOOR Convert H-floating to greatest H-floating integer 1
Floating-Point Truncation Routines
MTH$AINT Convert F-floating to truncated F-floating
MTH$IINT Convert F-floating to truncated word 2
MTH$JINT Convert F-floating to truncated longword 2
MTH$DINT Convert D-floating to truncated D-floating
MTH$IIDINT Convert D-floating to truncated word 2
MTH$JIDINT Convert D-floating to truncated longword 2
MTH$GINT Convert G-floating to truncated G-floating
MTH$IIGINT Convert G-floating to truncated word 2
MTH$JIGINT Convert G-floating to truncated longword 2
MTH$HINT Convert H-floating to truncated H-floating 1
MTH$IIHINT Convert H-floating to truncated word 2
MTH$JIHINT Convert H-floating to truncated longword 2
Bitwise Inclusive OR Operator Routines
MTH$IIOR Bitwise inclusive OR of two word parameters
MTH$JIOR Bitwise inclusive OR of two longword parameters
Maximum Value Routines
MTH$AIMAX0 F-floating maximum of n word parameters
MTH$AJMAX0 F-floating maximum of n longword parameters
MTH$IMAX0 Word maximum of n word parameters
MTH$JMAX0 Longword maximum of n longword parameters
MTH$AMAX1 F-floating maximum of n F-floating parameters
MTH$DMAX1 D-floating maximum of n D-floating parameters
MTH$GMAX1 G-floating maximum of n G-floating parameters
MTH$HMAX1 H-floating maximum of n H-floating parameters 1
MTH$IMAX1 Word maximum of n F-floating parameters 2
MTH$JMAX1 Longword maximum of n F-floating parameters 2
Minimum Value Routines
MTH$AIMIN0 F-floating minimum of n word parameters
MTH$AJMIN0 F-floating minimum of n longword parameters
MTH$IMIN0 Word minimum of n word parameters
MTH$JMIN0 Longword minimum of n longword parameters
MTH$AMIN1 F-floating minimum of n F-floating parameters
MTH$DMIN1 D-floating minimum of n D-floating parameters
MTH$GMIN1 G-floating minimum of n G-floating parameters
MTH$HMIN1 H-floating minimum of n H-floating parameters 1
MTH$IMIN1 Word minimum of n F-floating parameters 2
MTH$JMIN1 Longword minimum of n F-floating parameters 2
Remainder Routines
MTH$AMOD Remainder of two F-floating parameters, arg1/arg2 3,5
MTH$DMOD Remainder of two D-floating parameters, arg1/arg2 3,5
MTH$GMOD Remainder of two G-floating parameters, arg1/arg2 3
MTH$HMOD Remainder of two H-floating parameters, arg1/arg2 1,3
MTH$IMOD Remainder of two word parameters, arg1/arg2 4
MTH$JMOD Remainder of two longword parameters, arg1/arg2 4
Floating-Point Conversion to Nearest Value Routines
MTH$ANINT Convert F-floating to nearest F-floating integer
MTH$ININT Convert F-floating to nearest word integer 2
MTH$JNINT Convert F-floating to nearest longword integer 2
MTH$DNINT Convert D-floating to nearest D-floating integer
MTH$IIDNNT Convert D-floating to nearest word integer 2
MTH$JIDNNT Convert D-floating to nearest longword integer 2
MTH$GNINT Convert G-floating to nearest G-floating integer
MTH$IIGNNT Convert G-floating to nearest word integer 2
MTH$JIGNNT Convert G-floating to nearest longword integer 2
MTH$HNINT Convert H-floating to nearest H-floating integer 1
MTH$IIHNNT Convert H-floating to nearest word integer 2
MTH$JIHNNT Convert H-floating to nearest longword integer 2
Bitwise Complement Operator Routines
MTH$INOT Bitwise complement of word parameter
MTH$JNOT Bitwise complement of longword parameter
Floating-Point Multiplication Routines
MTH$DPROD D-floating product of two F-floating parameters 3
MTH$GPROD G-floating product of two F-floating parameters
Bitwise Shift Operator Routines
MTH$IISHFT Bitwise shift of word
MTH$JISHFT Bitwise shift of longword
Floating-Point Sign Function Routines
MTH$SGN F- or D-floating sign function
MTH$SIGN F-floating transfer of sign of y to sign of x
MTH$DSIGN D-floating transfer of sign of y to sign of x
MTH$GSIGN G-floating transfer of sign of y to sign of x
MTH$HSIGN H-floating transfer of sign of y to sign of x 1
MTH$IISIGN Word transfer of sign of y to sign of x
MTH$JISIGN Longword transfer of sign of y to sign of x
Conversion of Double to Single Floating-Point Routines
MTH$SNGL Convert D-floating to F-floating (rounded) 3
MTH$SNGLG Convert G-floating to F-floating (rounded) 3,6

1Returns value to the first argument; value exceeds 64 bits.
2Integer overflow exceptions can occur.
3Floating-point overflow exceptions can occur.
4Divide-by-zero exceptions can occur.
5Floating-point underflow exceptions are signaled.
6Floating-point underflow exceptions can occur.

Next Contents Index

  [Go to the documentation home page] [How to order documentation] [Help on this site] [How to contact us]  
  privacy and legal statement