1 Introduction

This document briefly explains the steps to install and run Virtual Reactor application on a Virtual Machine deployed in Azure Cloud Platform and presents the performance results.

This recipe document covers the following topics: -

• Deploy & connect a required virtual machine on Azure platform.
• Install required NVIDIA drivers in virtual machine.
• Install Application on virtual machine.
• Install license manager for current application
• Performance results of current application on azure virtual machine
• Azure cost
• Summary

2 Deploy Virtual Machine on Azure Cloud Platform

2.1 Azure Cloud Architecture for Application

The below Architecture explains the virtual machine running on an Azure Platform

2.2 Azure Virtual Machine (VM)

A VM is a virtualized instance of a computer that can perform almost all the same functions as a computer, including running applications and operating systems.

An Azure VM gives the flexibility of virtualization without having to buy and maintain the physical hardware that runs it. However, the user still needs to maintain the VM by performing tasks, such as configuring, patching, and installing the software that runs on it.

Things to be considered before deploying a VM,

• The names of the application resources
• The location where the resources are stored
• The size of the VM
• The operating system that the VM runs
• The configuration of the VM
• The related resources that the VM needs

There are different sizes and options available for the Azure virtual machines and user can use to run apps and workloads. Depending on the workload user must choose the appropriate VM size. For complete list, check this https://docs.microsoft.com/en-us/azure/virtual-machines/sizes/.

To test the performance of Virtual Reactor on Azure Platform, NDv4 A100, NCv3-series& NCas_T4-v3-series virtual machines are deployed.

1. Standard_ND96asr_v4: This VMs is powered by NVIDIA Ampere A100 Tensor Core GPUs and 96 physical 2nd-generation AMD Epyc™ CPU cores (2.44GHz). The “ND96asr_v4” VM has 8 GPUs with 40 GB of memory each and supported by 96 AMD processor cores with a total memory of 921GB. Each GPU features NVLINK 3.0 connectivity for communication within the VM.

2. Standard NC24s_v3: This VM belongs to Ncv3-series This VMs are powered by NVIDIA Tesla V100 GPUs. These GPUs can provide 1.5x the computational performance of the NCv2-series.

3. Standard NC64as_T4_v3: This VM Belongs to NCasT4_v3-series.These Series VMs are powered by Nvidia T4 GPUs and AMD EPYC 7V12(Rome) CPUs. The “NC64as_T4_v3” VM has 4 NVIDIA T4 GPUs with 16 GB of memory each, up to 64 non-multithreaded AMD processor cores with a total memory of 448GB.

2.3 Create a Virtual Machine on Azure Platform

Sign into Azure

Sign into the Azure portal by using https://portal.azure.com/

Free Trial subscriptions are not eligible for limit or quota increases.

After the successful sign in or sign up, one has to upgrade the Azure subscription to Pay-As-You-Go to deploy a Virtual machine. For deployment, the user must have regional vCPU quota, which can be obtained by raising a request.

The step-by-step procedure to increase the vCPU quota is given below,

https://docs.microsoft.com/en-us/azure/azure-portal/supportability/per-vm-quota-requests

The step-by-step procedure to deploy Virtual machine is given below,

1. Type virtual machines in the marketplace search.

2. Under Services, select Virtual machines.

3. In the Virtual machines page, select Create Then Virtual machine.

4. In the Basics tab, under Project details, make sure the correct subscription is selected and then choose to Create new resource group. Type Azure-PerformanceTest (user choice) for the name.

5. Under Instance details, type Azure-VM (user choice) for the Virtual machine name and choose West Europe for Region. Choose CentOS, Version 8.1 HPC-Gen2 for the Image and Standard_ND96asr_v4 for the Size (user choice). Leave the other defaults.

Note:

• Region must be decided based on where the Virtual machine is going to be deployed. To avoid the network latency, the region should be near to the location where the VM is to be deployed.
• Image selection is of user choice based on the application user can choose the image (Windows 10, Linux based OS and Windows Server)

6. Under Administrator account, provide a username, such as Azureuser and a password. The password must be at least 12 characters long and meet the defined complexity requirements.

7. Under Inbound port rules, choose Allow selected ports and then select SSH (22) from the drop-down.

8. In Advanced tab, under the Extensions click Select an extension to install, select NVIDIA GPU Driver Extension (user choice)

Note: This Step 8, will installs the older version of Nvidia and Cuda driver hence the user can skip this step and proceed the VM deployment without extension. Follow manual installation as per chapter 2.5.

9. Leave the remaining defaults and then select the Review + create button at the bottom of the page.

10. After validation runs, select the Create button at the bottom of the page.

11. After deployment is complete, select Go to resource.

2.4 Connect to virtual machine

Create a SSH connection to the virtual machine.

1. On the overview page of a virtual machine, select the Connect button then select SSH and copy the public IP address.

2. Open any SSH client tool of user choice, in this case PUTTY is used.

3. Enter the IP address and port as 22 and click Open

4. Enter username in Login as, enter the password, and click Enter.

5. Now user connected to the VM in Console

2.5 Install Nvidia Drivers

To take advantage of the GPU capabilities of Azure N-series VMs backed by NVIDIA GPUs, the user must install NVIDIA GPU drivers.

If the machine has a NVLink or NVSwitch connecting the GPUs like in the case of NDv4 virtual machine, one should also install the associated version of the Nvidia fabric manager.

There are two ways to install the drivers,

• The NVIDIA GPU Driver Extension installs appropriate NVIDIA CUDA or GRID drivers on an N-series VM. Install or manage the extension using the Azure portal (Ref 8)
• Install NVIDIA GPU drivers manually. (For the manual installation follow the procedure given below)

Nvidia Data Center Driver installation guide for CentOS 8

The step-by-step instruction is given below,

1. In some cases, user may need to install some additional dependencies that are required

For installing the NVIDIA drivers.

# sudo dnf install -y tar bzip2 make automake gcc gcc-c++ pciutils elfutils-libelf-devel libglvnd-devel

2. Enable the PowerTools repo and set up the external dependency on EPEL for DKMS.

# sudo dnf config-manager --set-enabled PowerTools

# sudo dnf install -y epel-release

3. Install the CUDA repository public GPG key

# Distribution=rhel8

4. Setup the CUDA network repository

# ARCH=$( /bin/arch)

# sudo dnf config-manager --add-repo http://developer.download.nvidia.com/compute/cuda/repos/$distribution/${ARCH}/cuda-$distribution.repo

5. For CentOS 8, ensure that the system has the correct Linux kernel sources from the CentOS repositories:

# sudo dnf install -y kernel-devel-$(uname -r) kernel-headers-$(uname -r)

6. Note that in some cases, user may have to update the running kernel as a pre-requisite:

# sudo dnf install -y kernel kernel-core kernel-modules

7. Update the repository cache and install the driver using the nvidia-driver meta-package

# sudo dnf clean all

# sudo dnf -y module install nvidia-driver:latest-dkms

8. Install fabric manager to activate the NVswitch (For NDv4 instance)

# sudo dnf module install nvidia-driver:latest-dkms/fm

9. Enable FM Service to Auto Start at Boot

# sudo systemctl enable nvidia-fabricmanager

Reference: https://docs.nvidia.com/datacenter/tesla/tesla-installation-notes/index.html#centos8/

3 Install Barracuda Virtual Reactor Application on Virtual Machine

3.1 Download product

Instructions on how to download Virtual Reactor products from CPFD Marketplace:

Before beginning, open CPFD Marketplace in a web browser and log on.

1. Type Virtual Reactor in the search box and press Enter.

2. Click on CPFD software: Home

3. Click on Log on to CPFD account at upper right corner.

4. For log in user need to create CPFD account.

5. After log in user have dashboard panel

6. Dashboard panel contain two downloadable packages.

7. Click the Download For Linux icon.

8. That package contains one installation user guide and one run file.

3.2 Install Barracuda Virtual Reactor application on Linux

Installation instructions for Barracuda Virtual Reactor can be found in below path:

https://cpfd-software.com/user-manual/installation.html

4 Performance results of Barracuda Virtual Reactor on Azure virtual machine

4.1 Barracuda Virtual Reactor Overview

Barracuda Virtual Reactor is a computational fluid dynamics (CFD) software package developed specifically for engineering design, problem solving, and optimization of industrial-scale fluidized bed reactors (FBRs). Employing an easy-to-use Graphical User Interface (GUI), and enhanced with visualization software from Tecplot, Inc. of Bellevue, Washington, U.S.A. Virtual Reactor TM helps engineers improve the reliability, capacity, and profitability of FBRs – through efficient modelling of both non-reacting and reacting gas-particle dynamics. Leveraging low-cost, high-performance computing, Barracuda physics-based simulation technology delivers key process performance information supporting critical design and operational decisions – providing competitive advantage through improved equipment performance, lower warranty risk and rework, higher yields, lower operating costs and longer up-time

4.2 Barracuda Virtual Reactor on Azure Platform

To carry out the CFD simulations on Virtual Reactor software, right hardware with latest CPUs and GPUs are required. Microsoft partnered with Nvidia provides the required and suitable Infrastructure and hardware on Azure cloud platform. It provides the latest and fastest compute capabilities for both CPU and GPU intensive workloads.

4.3 Barracuda Virtual Reactor Performance Results

Model Details:

Particle-based fluid dynamics simulations are carried out on the following Virtual machine configurations.

Azure Virtual machines tested: ND96asr_v4, NC24s_v3 and NC64as_T4_v3

The simulations are carried out to see the performance of GPU and CPU on the chosen Azure Virtual Machine Configuration. Refer chapter 2.2 for the VM specifications.

The results are presented below,

Elapsed time for NDV4 Virtual Machine - (“Seconds”)

Elapsed time for NCV3 Virtual Machine - (“Seconds”)

Elapsed time for NCas-T4 Virtual Machine - (“Seconds”)

Simulation speed-up for NDV4 Virtual Machine - (“Seconds/Day”)

Simulation speed-up for NCV3 Virtual Machine - (“Seconds/Day”)

Simulation speed-up for NCas-T4 Virtual Machine - (“Seconds/Day”)

Simulation Speed-up Charts for NDV4 Virtual Machine

The below chart shows the performance of two similar models with difference in particle count,

 

 

Simulation Speedup charts for NCV3 Virtual Machine

The below chart shows the performance of two similar models with difference in particle count,

 

 

Simulation Speedup Charts for NCas-T4 Virtual Machine

The below chart shows the performance of two similar models with difference in particle count,

 

 

5 Azure Cost

For the below cost reports, the application installation time is not considered and only wall clock time is considered for the cost calculation. The Hourly rates reported are subject to change. For the current rate please refer the link “https://azure.microsoft.com/en-in/pricing/calculator/”.

Note: The Elapsed time for full production-level test cases will be on the higher side than the results presented here and hence, the associated cost will increase.

6 Summary

1. Virtual Reactor is successfully tested on NDV4, NCv3 and NCas-T4 Virtual Machines on Azure Cloud Platform

2. For NDv4 Virtual Machine, we can see that the application is scaling well up to 4 GPUs for the models with larger particle counts and for the models with lesser particle counts, it is scaling only up to 2 GPUs.

3. For NCv3 Virtual Machine, we can see performance improvement up to 3 GPUs for the models with larger particle counts and for models with smaller particle count, only 1 GPU configuration is recommended

4. For NCas-T4 Virtual Machine, the application is speed-up well with all the 4 GPUs for models with larger particle count and for the models with lesser particle count, it is scaling well only with 1 GPU configuration.

5. For simulations with large numbers of particles and cells, a single 16 GB GPU may not be sufficient to run well due to the memory requirements. In these cases, running the simulation with two GPUs is critical. This is seen in test problems 480, 481, and 482 on the NCV3 and NCAS T4 machines

6. For smaller simulations where there are performance penalties when utilizing all GPUs on an instance, concurrent simulations can be run using the other GPUs. In this scenario, multiple points in the simulation parameter space could be explored more quickly.