Abaqus is available via WSE IT. ABAQUS is not forward compatible means ABAQUS 2021 won't be able to open the .CAE and .ODB files generated by ABAQUS 2022 or later versions. But the newer version can open files from an older version. So select an appropriate version that you can use for a long time (check what version MARCC/ Rockfish offers as well). You can download the software from https://software.wse.jhu.edu/ and install it by yourself by obtaining the license information from WSE IT (or ask one of your labmates who uses ABAQUS). Please make sure to select ABAQUS/ Standard and Explicit solver with CAE package and CAA API components during installation.

If you have multiple versions of Abaqus installed, then you will need to configure the abaqus.bat file located in C:\SIMULIA\Commands to call the version of your choice.

It does not matter what order is followed for installing ABAQUS but Visual Studio and Intel oneAPI have to be installed in that particular order.

To be able to leverage user-defined features of Abaqus, Microsoft Visual Studio and Intel Fortran Compiler are required to be installed in that order. Installing in different order will not work. Intel Fortran (and C++) compiler is a part of Intel oneAPI Toolkit which is available for free. Download the latest stable version of Intel oneAPI, both Base Toolkit and HPC Toolkit. The Base Toolkit has the compiler and oneMKL package and HPC Toolkit has the parallel computing packages. Please make sure to check which version of Microsoft Visual Studio is compatible with that particular Intel oneAPI version. Download the compatible version of the community edition of Visual Studio. Both these software are freely available.

1. Start the installation procedure with Visual Studio. Make sure to select Desktop development with C++ on the installation page while other settings could be default. The installation process is trivial.
   
   
2. Then install the Intel oneAPI Base Toolkit and Intel oneAPI HPC Toolkit, respectively. Default installation should work and both of these software will recognize existing Visual Studio and link itself. This process might take 30-40 minutes.
   
   
3. Upon successful installation, Intel oneAPI packages will be installed in C:\Program Files (x86)\Intel\oneAPI directory on Windows. Navigate further into the directory and check the folder C:\Program Files (x86)\Intel\oneAPI\compiler\2021.1.1\windows\bin\intel64 (version could be different) for the ifort executable. Copy the path to a text editor for later use.
   
   
4. Now navigate to the C:\Program Files (x86)\Intel\oneAPI\compiler\2021.1.1\env directory (again, the version could be different) and locate the vars.bat file. If your directory names are arranged differently for Intel oneAPI, you can search for the vars.bat through Windows Explorer and locate the one under the compiler directory. There will be other vars.bat files for different oneAPI packages, so, please make sure you find the right one. Copy the path to a text editor for later use.
   
   
5. Now please locate the abqXXXX.bat file (XXXX for version) in C:\SIMULIA\Commands. Open the abqXXXX.bat file with a text editor. You will need to save the file later as an administrator. To do so, open Notepad from the program menu, run it as an administrator, and then open the file or use the Notepad++ text editor to open the file.
   

   SET PATH=%PATH%;C:\Program Files (x86)\Intel\oneAPI\compiler\2021.1.1\windows\bin\intel64;
   call “C:\Program Files (x86)\Intel\oneAPI\compiler\2021.1.1\env\vars.bat” intel64
   

   The first line here is the path directory for the ifort compiler and the second line calls the batch file (or script) to initialize the environment variable for the ifort compiler when the ABAQUS command is invoked.
   
6. Now, open either Abaqus Commands, cmd, or Powershell from the Windows start menu and type the following to check and verify the installation. This should return all the system information, including the linker and compiler on the terminal.

   abaqus info=system 

7. The following commands can be used to verify installation and linking as well. For successful installation and linking, it will show PASS on the terminal.

   abaqus verify -user_std    #Abaqus/Standard user subroutine feature
   abaqus verify -user_exp    #Abaqus/Explicit user subroutine feature
   abaqus verify -all         #All Abaqus feature (this will take time) 

1. Intel oneMKL (Math Kernel Library) is also part of the Intel oneAPI distribution. Intel oneMKL has Intel's implementation of BLAS, LAPACK, FFTW, and other math libraries which can be used in the user subroutine feature of ABAQUS. Navigate back to C:\SIMULIA\Commands directory and open the abqXXX.bat file again. Add the following line below the previously added lines for the ifort compiler. This will initialize the environment variables for oneMKL when ABAQUS is called.

   call "C:\Program Files (x86)\Intel\oneAPI\mkl\2021.1.1\env\vars.bat" intel64 

2. Now navigate to C:\SIMULIA\EstProducts\2020\win_b64\SMA\site (or similar directory) based on your ABAQUS installation and locate win86_64.env file and open it as an administrator (same as before). Find the following line and add /Qmkl:sequential in between other compiler flag options. The file should now look as below for that line.

   compile_fortran=['ifort','/Qmkl:sequential', ... ...
   

   You should be able to use the Intel oneMKL library now with ABAQUS. Apparently, by using /Qmkl:parallel, you may be able to use the parallel version of oneMKL, however, it has never been tested nor found to be necessary for the ABAQUS user subroutine feature so far.

Open Abaqus Commands, cmd, or Powershell, and use the following command to run Abaqus analysis.

abaqus interactive analysis double ask_delete=off job=job_name input=input_file.inp user=user_subroutine.for cpus=no_of_processors

interactive processes run sequentially, double is machine precision, and ask_delete=off will delete and overwrite the old job with the same name without asking (if that's what you want). You can create a PowerShell script on Windows or a bash script on Linux to run the job efficiently and/or even run multiple jobs at the same time.

Open job_name.dat and job_name.msg file in any text editor to see the details of the simulation. To check the convergence log during simulation, open a new pane or tab on the terminal and write:

tail -f job_name.sta                #On MARCC or other Linux bash terminal
Get-Content -wait job_name.sta      #On Windows PowerShell terminal

As of Spring 2021, MARCC has a few different versions of Abaqus and Intel Fortran compiler installed. Abaqus lacks backward compatibility, so load the appropriate version of Abaqus, i.e., the same or older than the version installed on the local workstation. You won't be able to open the .odb files on local computer if you run the simulations on MARCC with the newer Abaqus version. Please note, that MARCC is not configured to run the Abaqus job on more than a single node. Make sure you're not running the job on the login node. Please use the following bash script template to run Abaqus on MARCC. Make necessary changes to the parameters, based on the MARCC website.

#!/bin/bash

# BEGIN PARAMETERS
WORKDIR=/scratch/groups/tnguy108/yourdirectory
JOBNAME=abaqus_job_name                     # abaqus job name -- same as input file name
INPUTFILE=abaqus_input_file.inp             # abaqus input file name -- job name with .inp extension
USERFILE=user_subroutine.for                # user defined subroutine

# check out the MARCC website while changing the following parameters
PARTITION=unlimited                         # marcc partition
NUM_NODES=1                                 # number of nodes to be used
TASKS_PER_NODE=24                           # number of simulations running per node
CPUS_PER_TASK=1                             # number of core/ threads to be used
# TASKS_PER_NODE*CPUS_PER_TASK = 24 (available core on each node)
# total number of cpu cores to be used
TOTAL_CPUS=$((NUM_NODES*TASKS_PER_NODE*CPUS_PER_TASK))
MAX_TIME=168:00:00                          # maximum simulation time before exiting: 7 days
[email protected]                    # email for notification
EMAIL_TYPES=BEGIN,FAIL,END                  # type of notifications to be received
# END PARAMETERS

#SBATCH --job-name=$JOBNAME
#SBATCH --partition=$PARTITION
#SBATCH --nodes=$NUM_NODES
#SBATCH --ntasks-per-node=$TASKS_PER_NODE
#SBATCH --cpus-per-task=$CPUS_PER_TASK
#SBATCH --time=$MAX_TIME
#SBATCH --mail-type=$EMAIL_TYPES
#SBATCH --mail-user=$EMAIL

echo "All parameters are set"

# if sbatch script is in a different folder then uncomment the following lines
# cd $WORKDIR
# echo "Working directory is: $WORKDIR"

module load abaqus

echo "Abaqus loaded"

abaqus double interactive analysis ask_delete=off job=$JOBNAME input=$INPUTFILE cpus=$TOTAL_CPUS user=$USERFILE

echo "Abaqus JOB $JOBNAME with SLURM JOB ID $SLURM_JOB_ID is completed"