Fortran

Fortran#

In this section, examples are presented using the SmartRedis Fortran API to interact with the RedisAI tensor, model, and script data types. Additionally, an example of utilizing the SmartRedis DataSet API is also provided.

Note

The Fortran API examples rely on the SSDB environment variable being set to the address and port of the Redis database.

Note

The Fortran API examples are written to connect to a clustered database or clustered SmartSim Orchestrator. Update the Client constructor cluster flag to .false. to connect to a single shard (single compute host) database.

Error handling#

The core of the SmartRedis library is written in C++ which utilizes the exception handling features of the language to catch errors. This same functionality does not exist in Fortran, so instead most SmartRedis methods are functions that return error codes that can be checked. This also has the added benefit that Fortran programs can incorporate SmartRedis calls within their own error handling methods. A full list of return codes for Fortran can be found in enum_fortran.inc. Additionally, the errors module has get_last_error and print_last_error to retrieve the text of the error message emitted within the C++ code.

Tensors#

The SmartRedis Fortran client is used to communicate between a Fortran client and the Redis database. In this example, the client will be used to send an array to the database and then unpack the data into another Fortran array.

This example will go step-by-step through the program and then present the entirety of the example code at the end.

Importing and declaring the SmartRedis client

The SmartRedis client must be declared as the derived type client_type imported from the smartredis_client module.

program example
  use smartredis_client, only : client_type

  type(client_type) :: client
end program example

Initializing the SmartRedis client

The SmartRedis client needs to be initialized before it can be used to interact with the database. Within Fortran this is done by calling the type-bound procedure initialize with the input argument .true. if using a clustered database or .false. otherwise.

program example
  use smartredis_client, only : client_type

  type(client_type) :: client
  integer :: return_code

  return_code = client%initialize(.false.) ! Change .false. to true if using a clustered database
  if (return_code .ne. SRNoError) stop 'Error in initializing client'
end program example

Putting a Fortran array into the database

After the SmartRedis client has been initialized, a Fortran array of any dimension and shape and with a type of either 8, 16, 32, 64 bit integer or 32 or 64-bit real can be put into the database using the type-bound procedure put_tensor. In this example, as a proxy for model-generated data, the array send_array_real_64 will be filled with random numbers and stored in the database using put_tensor. This subroutine requires the user to specify a string used as the ‘key’ (here: send_array) identifying the tensor in the database, the array to be stored, and the shape of the array.

1  call random_number(send_array_real_64)
2
3  ! Initialize a client
4  result = client%initialize("smartredis_put_get_3D")
5  if (result .ne. SRNoError) error stop 'client%initialize failed'
6
7  ! Send a tensor to the database via the client and verify that we can retrieve it
8  result = client%put_tensor("send_array", send_array_real_64, shape(send_array_real_64))
9  if (result .ne. SRNoError) error stop 'client%put_tensor failed'

Unpacking an array stored in the database

‘Unpacking’ an array in SmartRedis refers to filling a Fortran array with the values of a tensor stored in the database. The dimensions and type of data of the incoming array and the pre-declared array are checked within the client to ensure that they match. Unpacking requires declaring an array and using the unpack_tensor procedure. This example generates an array of random numbers, puts that into the database, and retrieves the values from the database into a different array.

 1! BSD 2-Clause License
 2!
 3! Copyright (c) 2021-2024, Hewlett Packard Enterprise
 4! All rights reserved.
 5!
 6! Redistribution and use in source and binary forms, with or without
 7! modification, are permitted provided that the following conditions are met:
 8!
 9! 1. Redistributions of source code must retain the above copyright notice, this
10!    list of conditions and the following disclaimer.
11!
12! 2. Redistributions in binary form must reproduce the above copyright notice,
13!    this list of conditions and the following disclaimer in the documentation
14!    and/or other materials provided with the distribution.
15!
16! THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
17! AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
18! IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
19! DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
20! FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
21! DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
22! SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
23! CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
24! OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
25! OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
26
27program main
28
29  use iso_c_binding
30  use smartredis_client, only : client_type
31
32  implicit none
33
34#include "enum_fortran.inc"
35
36  integer, parameter :: dim1 = 10
37  integer, parameter :: dim2 = 20
38  integer, parameter :: dim3 = 30
39
40  real(kind=8),    dimension(dim1, dim2, dim3) :: recv_array_real_64
41  real(kind=c_double),    dimension(dim1, dim2, dim3) :: send_array_real_64
42
43  integer :: i, j, k, result
44  type(client_type) :: client
45
46  call random_number(send_array_real_64)
47
48  ! Initialize a client
49  result = client%initialize("smartredis_put_get_3D")
50  if (result .ne. SRNoError) error stop 'client%initialize failed'
51
52  ! Send a tensor to the database via the client and verify that we can retrieve it
53  result = client%put_tensor("send_array", send_array_real_64, shape(send_array_real_64))
54  if (result .ne. SRNoError) error stop 'client%put_tensor failed'
55  result = client%unpack_tensor("send_array", recv_array_real_64, shape(recv_array_real_64))
56  if (result .ne. SRNoError) error stop 'client%unpack_tensor failed'
57
58  ! Done
59  call exit()
60
61end program main

Datasets#

The following code snippet shows how to use the Fortran Client to store and retrieve dataset tensors and dataset metadata scalars.

 1! BSD 2-Clause License
 2!
 3! Copyright (c) 2021-2024, Hewlett Packard Enterprise
 4! All rights reserved.
 5!
 6! Redistribution and use in source and binary forms, with or without
 7! modification, are permitted provided that the following conditions are met:
 8!
 9! 1. Redistributions of source code must retain the above copyright notice, this
10!    list of conditions and the following disclaimer.
11!
12! 2. Redistributions in binary form must reproduce the above copyright notice,
13!    this list of conditions and the following disclaimer in the documentation
14!    and/or other materials provided with the distribution.
15!
16! THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
17! AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
18! IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
19! DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
20! FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
21! DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
22! SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
23! CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
24! OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
25! OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
26
27program main
28
29  use iso_c_binding
30  use smartredis_dataset, only : dataset_type
31  use smartredis_client, only : client_type
32
33  implicit none
34
35#include "enum_fortran.inc"
36
37  integer, parameter :: dim1 = 10
38  integer, parameter :: dim2 = 20
39  integer, parameter :: dim3 = 30
40
41  real(kind=c_float),      dimension(dim1, dim2, dim3) :: recv_array_real_32
42
43  real(kind=c_float),      dimension(dim1, dim2, dim3) :: true_array_real_32
44
45  character(len=16) :: meta_float = 'meta_float'
46
47  real(kind=c_float),  dimension(dim1) :: meta_flt_vec
48  real(kind=c_float), dimension(:), pointer :: meta_flt_recv
49
50  integer :: i, result
51  type(dataset_type) :: dataset
52  type(client_type) :: client
53
54  ! Fill array
55  call random_number(true_array_real_32)
56
57  ! Initialize a dataset
58  result = dataset%initialize("example_fortran_dataset")
59  if (result .ne. SRNoError) error stop 'dataset initialization failed'
60
61  ! Add a tensor to the dataset and verify that we can retrieve it
62  result = dataset%add_tensor("true_array_real_32", true_array_real_32, shape(true_array_real_32))
63  if (result .ne. SRNoError) error stop 'dataset%add_tensor failed'
64  result = dataset%unpack_dataset_tensor("true_array_real_32", recv_array_real_32, shape(recv_array_real_32))
65  if (result .ne. SRNoError) error stop 'dataset%unpack_dataset_tensor failed'
66
67  call random_number(meta_flt_vec)
68
69  ! Add metascalars to the dataset and verify that we can retrieve them
70  do i=1,dim1
71    result = dataset%add_meta_scalar(meta_float, meta_flt_vec(i))
72    if (result .ne. SRNoError) error stop 'dataset%add_meta_scalar failed'
73  enddo
74  result = dataset%get_meta_scalars(meta_float, meta_flt_recv)
75  if (result .ne. SRNoError) error stop 'dataset%get_meta_scalars failed'
76
77  ! Initialize a client
78  result = client%initialize("smartredis_dataset")
79  if (result .ne. SRNoError) error stop 'client%initialize failed'
80
81  ! Send the dataset to the database via the client
82  result = client%put_dataset(dataset)
83  if (result .ne. SRNoError) error stop 'client%put_dataset failed'
84
85  ! Done
86  call exit()
87
88end program main

Models#

For an example of placing a model in the database and executing the model using a stored tensor, see the Parallel (MPI) execution example. The aforementioned example is customized to show how key collisions can be avoided in parallel applications, but the Client API calls pertaining to model actions are identical to non-parallel applications.

Scripts#

For an example of placing a PyTorch script in the database and executing the script using a stored tensor, see the Parallel (MPI) execution example. The aforementioned example is customized to show how key collisions can be avoided in parallel applications, but the Client API calls pertaining to script actions are identical to non-parallel applications.

Parallel (MPI) execution#

In this example, an MPI program that sets a model, sets a script, executes a script, executes a model, sends a tensor, and receives a tensor is shown. This example illustrates how keys can be prefixed to prevent key collisions across MPI ranks. Note that only one model and script are set, which is shared across all ranks. It is important to note that the Client API calls made in this program are equally applicable to non-MPI programs.

This example will go step-by-step through the program and then present the entirety of the example code at the end.

The MNIST dataset and model typically take images of digits and quantifies how likely that number is to be 0, 1, 2, etc.. For simplicity here, this example instead generates random numbers to represent an image.

Initialization

At the top of the program, the SmartRedis Fortran client (which is coded as a Fortran module) is imported using

use smartredis_client, only : client_type

where client_type is a Fortran derived-type containing the methods used to communicate with the RedisAI database. A particular instance is declared via

type(client_type) :: client

An initializer routine, implemented as a type-bound procedure, must be called before any of the other methods are used:

return_code = client%initialize(.true.)
if (return_code .ne. SRNoError) stop 'Error in initializing client'

The only optional argument to the initialize routine is to determine whether the RedisAI database is clustered (i.e. spread over a number of nodes, .true.) or exists as a single instance.

If an individual rank is expected to send only its local data, a separate client must be initialized on every MPI task Furthermore, to avoid the collision of key names when running on multiple MPI tasks, we store the rank of the MPI process which will be used as the suffix for all keys in this example.

On the root MPI task, two additional client methods (set_model_from_file and set_script_from_file) are called. set_model_from_file loads a saved PyTorch model and stores it in the database using the key mnist_model. Similarly, set_script_from_file loads a script that can be used to process data on the database cluster.

if (pe_id == 0) then
  return_code = client%set_model_from_file(model_key, model_file, "TORCH", "CPU")
  if (return_code .ne. SRNoError) stop 'Error in setting model'
  return_code = client%set_script_from_file(script_key, "CPU", script_file)
  if (return_code .ne. SRNoError) stop 'Error in setting script'
endif

This only needs to be done on the root MPI task because this example assumes that every rank is using the same model. If the model is intended to be rank-specific, a unique identifier (like the MPI rank) must be used.

At this point the initialization of the program is complete: each rank has its own SmartRedis client, initialized a PyTorch model has been loaded and stored into the database with its own identifying key, and a preprocessing script has also been loaded and stored in the database

Performing inference on Fortran data

The run_mnist subroutine coordinates the inference cycle of generating data (i.e. the synthetic MNIST image) from the application and then the use of the client to run a preprocessing script on data within the database and to perform an inference from the AI model. The local variables are declared at the top of the subroutine and are instructive to communicate the expected shape of the inputs to the various client methods.

integer, parameter :: mnist_dim1 = 28
integer, parameter :: mnist_dim2 = 28
integer, parameter :: result_dim1 = 10

The first two integers mnist_dim1 and mnist_dim2 specify the shape of the input data. In the case of the MNIST dataset, it expects a 4D tensor describing a ‘picture’ of a number with dimensions [1,1,28,28] representing a batch size (of one) and a three dimensional array. result_dim1 specifies what the size of the resulting inference will be. In this case, it is a vector of length 10, where each element represents the probability that the data represents a number from 0-9.

The next declaration declares the strings that will be used to define objects representing inputs/outputs from the scripts and inference models.

character(len=255) :: in_key
character(len=255) :: script_out_key
character(len=255) :: out_key

Note that these are standard Fortran strings. However, because the model and scripts may require the use of multiple inputs/outputs, these will need to be converted into a vector of strings.

character(len=255), dimension(1) :: inputs
character(len=255), dimension(1) :: outputs

In this case, only one input and output are expected the vector of strings only need to be one element long. In the case of multiple inputs/outputs, change the dimension attribute of the inputs and outputs accordingly, e.g. for two inputs this code would be character(len=255), dimension(2) :: inputs.

Next, the input and output keys for the model and script are now constructed

in_key = "mnist_input_rank"//trim(key_suffix)
script_out_key = "mnist_processed_input_rank"//trim(key_suffix)
out_key = "mnist_processed_input_rank"//trim(key_suffix)

As mentioned previously, unique identifying keys are constructed by including a suffix based on MPI tasks.

The subroutine, in place of an actual simulation, next generates an array of random numbers and puts this array into the Redis database.

call random_number(array)
return_code = client%put_tensor(in_key, array, shape(array))
if (return_code .ne. SRNoError) stop 'Error putting tensor in the database'

The Redis database can now be called to run preprocessing scripts on these data.

inputs(1) = in_key
outputs(1) = script_out_key
return_code = client%run_script(script_name, "pre_process", inputs, outputs)
if (return_code .ne. SRNoError) stop 'Error running script'

The call to client%run_script specifies the key used to identify the script loaded during initialization, pre_process is the name of the function to run that is defined in that script, and the inputs/outputs are the vector of keys described previously. In this case, the call to run_script will trigger the RedisAI database to execute pre_process on the generated data (stored using the key mnist_input_rank_XX where XX represents the MPI rank) and storing the result of pre_process in the database as mnist_processed_input_rank_XX. One key aspect to emphasize, is that the calculations are done within the database, not on the application side and the results are not immediately available to the application. The retrieval of data from the database is demonstrated next.

The data have been processed and now we can run the inference model. The setup of the inputs/outputs is the same as before, with the exception that the input to the inference model, is stored using the key mnist_processed_input_rank_XX and the output will stored using the same key.

inputs(1) = script_out_key
outputs(1) = out_key
return_code = client%run_model(model_name, inputs, outputs)
if (return_code .ne. SRNoError) stop 'Error running model'

As before the results of running the inference are stored within the database and are not available to the application immediately. However, we can ‘retrieve’ the tensor from the database by using the unpack_tensor method.

return_code = client%unpack_tensor(out_key, result, shape(result))
if (return_code .ne. SRNoError) stop 'Error retrieving the tensor'

The result array now contains the outcome of the inference. It is a 10-element array representing the likelihood that the ‘image’ (generated using the random numbers) is one of the numbers [0-9].

Key points

The script, models, and data used here represent the coordination of different software stacks (PyTorch, RedisAI, and Fortran) however the application code is all written in standard Fortran. Any operations that need to be done to communicate with the database and exchange data are opaque to the application.

Source Code

Fortran program:

  1! BSD 2-Clause License
  2!
  3! Copyright (c) 2021-2024, Hewlett Packard Enterprise
  4! All rights reserved.
  5!
  6! Redistribution and use in source and binary forms, with or without
  7! modification, are permitted provided that the following conditions are met:
  8!
  9! 1. Redistributions of source code must retain the above copyright notice, this
 10!    list of conditions and the following disclaimer.
 11!
 12! 2. Redistributions in binary form must reproduce the above copyright notice,
 13!    this list of conditions and the following disclaimer in the documentation
 14!    and/or other materials provided with the distribution.
 15!
 16! THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
 17! AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 18! IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 19! DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
 20! FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
 21! DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
 22! SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
 23! CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
 24! OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 25! OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 26
 27program mnist_example
 28
 29  use mpi
 30  use iso_c_binding
 31  use smartredis_client, only : client_type
 32
 33  implicit none
 34
 35#include "enum_fortran.inc"
 36
 37  character(len=*), parameter :: model_key = "mnist_model"
 38  character(len=*), parameter :: model_file = "mnist_data/mnist_cnn.pt"
 39  character(len=*), parameter :: script_key = "mnist_script"
 40  character(len=*), parameter :: script_file = "mnist_data/data_processing_script.txt"
 41
 42  type(client_type) :: client
 43  integer :: err_code, pe_id, result
 44  character(len=2) :: key_suffix
 45
 46  ! Initialize MPI and get the rank of the processor
 47  call MPI_init(err_code)
 48  call MPI_comm_rank( MPI_COMM_WORLD, pe_id, err_code)
 49
 50  ! Format the suffix for a key as a zero-padded version of the rank
 51  write(key_suffix, "(A,I1.1)") "_",pe_id
 52
 53  ! Initialize a client
 54  result = client%initialize("smartredis_mnist")
 55  if (result .ne. SRNoError) error stop 'client%initialize failed'
 56
 57  ! Set up model and script for the computation
 58  if (pe_id == 0) then
 59    result = client%set_model_from_file(model_key, model_file, "TORCH", "CPU")
 60    if (result .ne. SRNoError) error stop 'client%set_model_from_file failed'
 61    result = client%set_script_from_file(script_key, "CPU", script_file)
 62    if (result .ne. SRNoError) error stop 'client%set_script_from_file failed'
 63  endif
 64
 65  ! Get all PEs lined up
 66  call MPI_barrier(MPI_COMM_WORLD, err_code)
 67
 68  ! Run the main computation
 69  call run_mnist(client, key_suffix, model_key, script_key)
 70
 71  ! Shut down MPI
 72  call MPI_finalize(err_code)
 73
 74  ! Check final result
 75  if (pe_id == 0) then
 76    print *, "SmartRedis Fortran MPI MNIST example finished without errors."
 77  endif
 78
 79  ! Done
 80  call exit()
 81
 82contains
 83
 84subroutine run_mnist( client, key_suffix, model_name, script_name )
 85  type(client_type), intent(in) :: client
 86  character(len=*),  intent(in) :: key_suffix
 87  character(len=*),  intent(in) :: model_name
 88  character(len=*),  intent(in) :: script_name
 89
 90  integer, parameter :: mnist_dim1 = 28
 91  integer, parameter :: mnist_dim2 = 28
 92  integer, parameter :: result_dim1 = 10
 93
 94  real, dimension(1,1,mnist_dim1,mnist_dim2) :: array
 95  real, dimension(1,result_dim1) :: output_result
 96
 97  character(len=255) :: in_key
 98  character(len=255) :: script_out_key
 99  character(len=255) :: out_key
100
101  character(len=255), dimension(1) :: inputs
102  character(len=255), dimension(1) :: outputs
103
104  ! Construct the keys used for the specifiying inputs and outputs
105  in_key = "mnist_input_rank"//trim(key_suffix)
106  script_out_key = "mnist_processed_input_rank"//trim(key_suffix)
107  out_key = "mnist_processed_input_rank"//trim(key_suffix)
108
109  ! Generate some fake data for inference and send it to the database
110  call random_number(array)
111  result = client%put_tensor(in_key, array, shape(array))
112  if (result .ne. SRNoError) error stop 'client%put_tensor failed'
113
114  ! Prepare the script inputs and outputs
115  inputs(1) = in_key
116  outputs(1) = script_out_key
117  result = client%run_script(script_name, "pre_process", inputs, outputs)
118  if (result .ne. SRNoError) error stop 'client%run_script failed'
119  inputs(1) = script_out_key
120  outputs(1) = out_key
121  result = client%run_model(model_name, inputs, outputs)
122  if (result .ne. SRNoError) error stop 'client%run_model failed'
123  output_result(:,:) = 0.
124  result = client%unpack_tensor(out_key, output_result, shape(output_result))
125  if (result .ne. SRNoError) error stop 'client%unpack_tensor failed'
126
127end subroutine run_mnist
128
129end program mnist_example

Python Pre-Processing:

1def pre_process(inp):
2    mean = torch.zeros(1).float().to(inp.device)
3    mean[0] = 2.0
4    temp = inp.float() * mean
5    return temp
Contents