Below are some brief guides for the FYP undergraduate students and new members of our research group in National University of Singapore to build up the OpenFOAM or other numerical combustion platforms in the super-computing clusters from NSCC (National Supercomputing Centre Singapore).
1, NSCC Account Applications
For new users, please go to the homepage of NSCC (https://www.nscc.sg/) and click “Enrollments”, and then register in the page: https://user.nscc.sg/saml/. You have to use your own NUS ID and for the identity provider, please choose “NUS”. Once you have this account, then you can compile your codes and submit your jobs to NSCC. More information can be found from the NSCC homepage, regarding the quick guide or architecture information about their facilities.
If you meet any problems about NSCC, please write to help@nscc.sg, or submit a service request through: https://servicedesk.nscc.sg/.
User Guide from NSCC: Getting Started with NSCC Supercomputing on ASPIRE 1
2，Remote Access and Data Transfer between Local Computer and NSCC Cluster
If the system of your own computer is Windows, please install the software PuTTY to connect yourself and NSCC. When loggin in, please key in “nus.nscc.sg” if you are in NUS campus or “aspire.nscc.sg” if off campus, then input your own NSCC account and password.
Also, please download the data transfer software FileZilla and install it in your own computer. When you log in, the Host is “nus.nscc.sg” if in campus or “aspire.nscc.sg”, the User Name is “your NSCC Account”, the Password is the one you set, and Port is “22”.
For those of you are beyond the NUS campus, you need to set up your VPN, please follow the procedures given by https://help.nscc.sg/user-guide/, before you use the host “aspire.nscc.sg”.
3, OpenFOAM Compilation in NSCC （Following procedures also apply for OpenFOAM-v2006 and OpenFOAM-8）
1- Download the OpenFOAM 6.0 source package to your local computer, and note that there are two files: OpenFOAM-6-version-6.tar.gz and ThirdParty-6-version-6.tar.gz
2- In your NSCC account, create an OpenFOAM folder in your home directory through “mkdir OpenFOAM”
3- Transfer the above two compressed packages into the OpenFOAM folder in NSCC through FileZilla
4- Decompress the two files (in NSCC Linux environment from this step and afterwards) through “tar -zxvf OpenFOAM-6-version-6.tar.gz” and “tar -zxvf ThirdParty-6-version-6.tar.gz”
5- Rename the two files through “mv OpenFOAM-6-version-6 OpenFOAM-6” and “mv ThirdParty-6-version-6 ThirdParty-6”
6- Set the environment variables through adding the following two separate lines in $HOME/.bashrc: “module load hdf5/1.8.16/openmpi” and “source $HOME/OpenFOAM/OpenFOAM-6/etc/bashrc (Open $HOME/.bashrc” using “vi” and add this line in it.)
7- Enter the folder ThirdParty-6 and run “./Allwmake” in order to compile some third-party software packages (e.g. Scotch and PT-Scotch, Paraview)
8- With the Third Party software installed, go into the OpenFOAM-6 directory and compile OpenFOAM through “./Allwmake”. The compiling process may take 3-4 hours, and therefore we recommend you to submit the compilation task into the serial queue in NSCC using the following submission PBS script:
#PBS -q normal
#PBS -l select=1:ncpus=1:mem=4G
#PBS -l walltime=24:00:00
#PBS -N OpenFOAM
#PBS -P Personal
#PBS -m bea
#PBS -M xxxxxx@nus.edu.sg
source $HOME/OpenFOAM/OpenFOAM-6/etc/bashrc
cd $PBS_O_WORKDIR
./Allwmake > out.txt 2> err.txt
(Please copy the above script into a created file named as “pbs.srl”, leave it in the folder where you want to compile and then submit it using qsub pbs.srl. You can see all the outputs in out.txt using “vi out.txt”, or view them interactively using “tail -f out.txt“.)
For those who use two (or more) different versions of OpenFOAM in NSCC account, you need to differentiate their corresponding environments. A recommended way to do this is to use “alias” (this is not an OpenFOAM jargon, it is from Linux) defined in $HOME/.bashrc. In Step 3.6, you have already added the line “source $HOME/OpenFOAM/OpenFOAM-6/etc/bashrc”. This means that the environment for OpenFOAM 6.0 will be automatically loaded for your use in this terminal. However, for dual or multiple versions of OF, please DECOMMENT this line. Instead, add (as two lines) “alias of50=’source $HOME/OpenFOAM/OpenFOAM-5.0/etc/bashrc’ ” and “alias of60=’source $HOME/OpenFOAM/OpenFOAM-6.0/etc/bashrc’ “. In this case, when you would like to use OpenFOAM 5.0, run “of50” in the terminal, then in it the corresponding environment variables will be loaded. Same for OpenFOAM 6.0. Note that when you use this way to handle the version shift, it will be safe for you to explicitly specify the version in your submission script through adding: “source $HOME/OpenFOAM/OpenFOAM-6/etc/bashrc” (see the sample script above). This means that before the job starts to run, the NSCC environment for OpenFOAM 6.0 will be loaded, and therefore no chance to have the errors related to version mixing.
There are two documents for the beginners to read:
– OpenFOAM User’s Guide
– OpenFOAM Programmer’s Guide
There are also some online forums related to OpenFOAM:
– CFD Online
For fundamentals of Linux commands:
– Basic commands, OpenFOAM Linux Guide
– Advanced commands: Please ask Google!
4, Data Visualization
Please install the Data Visualization Software Paraview to post-process the numerical simulation results from OpenFOAM in your own computer.
For those working on two-phase flow problems based on Lagrangian method, you need to use the Paraview in OpenFOAM as third-party software (termed as “paraFoam”) to visualize the droplets or particles, instead of your own Paraview. This is because in recent versions of OpenFOAM, they have considerably changed the data format the dispersed phases (using Barycentric tracking). To achieve this, please compile an OpenFOAM in your own computer. For Windows 10 OS, please use the windows-version OpenFOAM (downloadable from here).
If you would like to extract some experimental data from the published papers, please use GetData Graph Digitizer to digitize the data in the figures of interests. However, when you use the data in your reports or papers, please provide the reference information for them.
******Below is a schematic guide (provided by Dr Junhong Wang from NUS Information Technology) to use ANSYS Fluent in NUS-HPC clusters for FYP students or any other NUS members who want to use it (click it for a clearer review):******
******Below are some notes for new (or newly explored from our research activities) features or useful summaries related to OpenFOAM package.******
1, Adaptive Mesh Refinement (AMR): can be used for capturing the fronts with considerable gradients (e.g. deflagrative wave, shock wave or detonation wave) or material interfaces (e.g. between two non-miscible fluids). The explanations for the parameters in the dynamicRefineFvMesh dictionary are given here. Below are two figures respectively for cell level in AMR (base mesh: level=0; finest mesh: level=3) and mesh clustering for the interface between water and air.
