Tahoe

Tahoe is a research-oriented platform for the development of numerical methods, finite element models, and material models. It is divided into 2 projects, the public Tahoe project (Tahoe v 2.1) hosted by SourceForge and the private development project hosted by Codesion. The open Tahoe project includes methods and models that have been published and implementations that have been verified. The development project contains unpublished methods models and implementations that have not been verified and thus not yet ready for public release. Go to the tahoe project page for a user guide (incomplete), discussion forum, and developer's wiki http://tahoe.sourceforge.net/

TAHOE User Manual

An incomplete but useful user manual for Tahoe is available here: tahoe.user.pdf

TAHOE on GitHub

Tahoe is now available on GitHub: https://github.com/NguyenLabJHU/Tahoe-FEM. Follow the instructions on GitHub to install Tahoe on macOS and Linux (Ubuntu). Installation instructions on GitHub are based on the following instructions and may need some tweaks. Please contact Bibek if you would like to install Tahoe.

Installation of TAHOE

Below are the directions to download and install both the open and development modules and associated libraries and to update both projects using CVS. Try it first. If you encounter errors, read the log files to figure out the error. If you can't, then email me the log files so that I can help you. Please do not just email me and say that you encountered an error. I need to know the specific error to help you debug.

Nguyen Lab: Please don't just do nothing or copy someone else's executable if you encounter an error. Past experience has shown that doing the latter will only create more problems and will severely hinder my ability to help you with programming, debugging, etc. Ask google, your lab mates, and me for help. This is our primary computing tool and if you can't compile it you won't be getting any work done.

Old instructions for installing TAHOE on macOS and Linux are archived here. You might find it useful in specific circumstances.

Installing on macOS:

TAHOE was installed and tested on macOS Catalina 10.15 with open-mpi 4.1.3 and gcc 11.2.0. Sourceforge doesn’t have the current version tahoe-manager available yet but will be updated soon.

Install xcode command line tools to enable basic development libraries and features. On macOS terminal:
```
$ xcode-select --install 
```
If you don’t have an X11 client on macOS, download and install XQuartz. Visit https://www.xquartz.org. If you are accessing macOS via the terminal and have no GUI, you can install XQuartz using homebrew later.
Apple changed the default shell from bash to zsh. In case you don’t have an existing zsh profile, open one. You can skip the step if you have an existing zsh profile.
```
$ open ~/.zshrc 
```
Save and close the zshrc file. We will use this file later to save the environment variables and path related to TAHOE installation.

Download and install homebrew package manager:

$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Change the read and write permission for homebrew:
```
$ sudo chown -R $(whoami) $(brew --prefix)/* 
```
Download and install XQuartz via homebrew, if you haven’t done it before:
```
$ brew install --cask xquartz 
```

Install the following packages using homebrew:

$ brew install wget
$ brew install cmake
$ brew install automake
$ brew install open-mpi 
$ brew options subversion
$ brew install subversion

The current stable version of open-mpi available via homebrew is 4.1.3 and depends on gcc 11.2.0. Please check the installed versions of gcc, g++, and gfortran from the following directory before proceeding to next step.
```
$ cd /usr/local/bin 
```
Create a symlink for gcc, g++, and gfortran compilers using (Please make sure to use the correct version of the compilers):
```
$ ln -s gcc-11 gcc
$ ln -s g++-11 g++
$ ln -s gfortran-11 gfortran 
```
Log out of zsh terminal to activate the changes and log back in.
In your home directory, create a directory for downloading and installing Tahoe and its dependencies:
```
$ mkdir tahoe && cd tahoe 
```

Clone the ACCESS library using git. We will use exodus format. Install the Third-Party Libraries (TPL), build and install SEACAS using the following:

$ git clone https://github.com/gsjaardema/seacas.git
$ cd seacas
$ ./install-tpl.sh
$ mkdir build && cd build
$ ../cmake-exodus
$ make && make install

Save the following environment variables to your .zshrc file. Go back to the Tahoe directory using - cd .. commands. Please make sure to use the correct directory.

$ TAHOE_MAIN=$PWD
$ echo export TAHOE_MAIN=$TAHOE_MAIN >> ~/.zshrc
$ echo export ACCESS=$TAHOE_MAIN/seacas >> ~/.zshrc
$ echo export LD_LIBRARY_PATH=$ACCESS/lib >> ~/.zshrc
$ echo export CVS_RSH=ssh >> ~/.zshrc
$ source ~/.zshrc

Create symlinks for exodus libraries.

$ cd $ACCESS
$ ln -s include/ inc
$ cd lib
$ ln -s libexodus.dylib libexoIIv2c.dylib
$ ln -s libexodus.a libexoIIv2c.a

Log out of zsh terminal to activate the changes and log back in.
Download the latest version of TAHOE-Installer (tahoe-manager, installation macros, and scripts) to /Users/username/Downloads/TAHOE-Installer.
Create a backup directory for downloading all the default and optional modules for TAHOE.
```
$ cd $TAHOE_MAIN
$ mkdir tahoe-backup && cd tahoe-backup 
```
Set the following variables for ease of usage (“username” should be changed to your own username):
```
$ TAHOE_DOWNLOAD=/Users/username/Downloads/TAHOE-Installer
$ TAHOE_BACKUP=$TAHOE_MAIN/tahoe-backup 
```

Copy tahoe-manager to the tahoe-backup directory.

$ cp -i $TAHOE_DOWNLOAD/tahoe-manager $TAHOE_BACKUP

If the tahoe-manager isn’t already an executable, convert it to an executable file. If tahoe-manager is already an executable, you can skip this step below (it should be under “$TAHOE_BACKUP” directory).
```
$ sudo chmod 755 ./tahoe-manager 
```
Run tahoe-manager to download the macros, default modules, and optional modules:
```
$ ./tahoe-manager 
```
Select connection type: svn, and enter SourceForge user name. It will start downloading the macros. Once the macros are downloaded, enter any architecture type to continue the download.
For optional modules, select 0 (CBLAS), 6 (ACCESS), 8 (benchmark_XML), 10 (contrib), 11 (development), 12 (development_benchmark_xml), 13 (f2c), 14 (metis), 15 (spooles), 16 (spoolesMPI).
Copy the new macro files, GNU-GCC-MPI-9.3.macros and GNU-GCC9.3.macros, to the downloaded marcos directory. This is a temporary step until the newer macros are available on SourceForge.
```
$ cp -i $TAHOE_DOWNLOAD/*.macros $TAHOE_BACKUP/macros 
```
Edit the .tahoe-config file to change the architecture to either GNU-GCC-MPI-9.3 or GNU-GCC9.3.
Use the libf2c_.a file here to replace the one under tahoe-install/f2c/lib directory.
Make a copy of the backup directory for installation and rename it.
```
$ cd $TAHOE_MAIN
$ cp -r tahoe-backup tahoe-install 
```
Go to the installation directory and run tahoe-manager to update the installation settings with the correct architecture.
```
$ cd tahoe-install
$ ./tahoe-manager update 
```
Select a connection type as before, enter a user name, and select an architecture. For the parallel version of TAHOE, select GNU-GCC-MPI-9.3 or for serial version of TAHOE, select GNU-GCC-9.3. Select the optional modules as before. If the serial version of TAHOE is being installed, skip optional module 16 (spoolesMPI).
Build and compile TAHOE:
```
$ ./tahoe-manager build 
```

Once TAHOE is compiled and built, add the following environment variables and path to your .zshrc file:

$ echo export TAHOE_MOD=$TAHOE_MAIN/tahoe-install >> ~/.zshrc
$ echo export TAHOE_DIR=$TAHOE_MOD/tahoe >> ~/.zshrc
$ echo export PATH=$PATH:$TAHOE_MOD/bin >> ~/.zshrc
$ source ~/.zshrc

If you are using macOS with Apple Silicon processors, then you may have to add the following to your .bashrc file. We have not found a workaround for it yet. Make sure to choose your username.
```
install_name_tool -add_rpath “$ACCESS/lib” /Users/beijunshen/tahoe/tahoe-install/tahoe/tahoe 
```

Installing on Ubuntu:

The parallel version of TAHOE was installed and tested on Ubuntu 20.04 via Windows Subsystem for Linux (WSL). Download the latest version of tahoe-manager, installation macros, and scripts from Nguyen Lab OneDrive. Sourceforge doesn’t have this version and will be updated soon.

Download the latest version of TAHOE-Installer (tahoe-manager, installation macros and scripts) from Nguyen Lab OneDrive to /mnt/c/Users/username/Downloads.
Go to your home directory, and create a directory for downloading and installing Tahoe modules and their dependencies:
```
$ cd
$ mkdir tahoe && cd tahoe
$ mkdir tahoe-backup && cd tahoe-backup 
```

Set the following variables for ease of usage. Make sure to use the correct directory.

$ TAHOE_DOWNLOAD=/mnt/c/Users/username/Downloads/TAHOE-Installer
$ TAHOE_MAIN=/home/username/tahoe
$ TAHOE_BACKUP=/home/username/tahoe/tahoe-backup

Copy install_pre.sh and tahoe-manager to the tahoe-backup and tahoe directory:

$ cp -i $TAHOE_DOWNLOAD/install_pre.sh $TAHOE_MAIN
$ cp -i $TAHOE_DOWNLOAD/tahoe-manager $TAHOE_BACKUP

Open install_pre.sh using a text editor (for example, VS Code) and change the installation directory variable, TAHOE_MAIN, within the file. Save and close the file.
```
$ code install_pre.sh 
```
Change the permission for install_pre.sh to make it an executable. Run the script to download and install pre-requisite packages for TAHOE.
```
$ chmod a+x install_pre.sh
$ ./install_pre.sh
```
Log out of the bash shell to activate the change and log back in.

Set the following variables for ease of usage:

$ TAHOE_DOWNLOAD=/mnt/c/Users/username/Downloads/TAHOE-Installer
$ TAHOE_BACKUP=/home/bibek/TAHOE/tahoe-backup

Go to tahoe-backup directory. Run tahoe-manager to download the macros, default modules, and optional modules:
```
$ cd $TAHOE_BACKUP
$ ./tahoe-manager 
```
Select connection type: svn, and enter the SourceForge user name. It will start downloading the macros. Once the macros are downloaded, enter any architecture type to continue downloading.
For optional modules, select 0 (CBLAS), 6 (ACCESS), 8 (benchmark_XML), 10 (contrib), 11 (development), 12 (development_benchmark_xml), 13 (f2c), 14 (metis), 15 (spooles), 16 (spoolesMPI).
Copy the new macro files, GNU-GCC-MPI-9.3.macros and GNU-GCC-9.3.macros, to the downloaded marcos directory. This is a temporary step until the newer macros are available on SourceForge.
```
$ cp -i $TAHOE_DOWNLOAD/*.macros $TAHOE_BACKUP/macros 
```
Clean all the settings before making a copy of the backup directory for installation purposes.
```
$ ./tahoe-manager clean 
```
Make a copy of the backup directory for installation and rename it.
```
$ cd $TAHOE_MAIN
$ cp -r tahoe-backup tahoe-install 
```
Go to the installation directory and run tahoe-manager to update the installation settings with the correct architecture.
```
$ cd tahoe-install
$ ./tahoe-manager update 
```
Select a connection type, enter a user name, and select an architecture. For the parallel version of TAHOE, select GNU-GCC-MPI-9.3 or for serial version of TAHOE, select GNU-GCC-9.3. Select the optional modules as before. If the serial version of TAHOE is being installed, skip optional module 16 (spoolesMPI).
Build and compile TAHOE:
```
$ ./tahoe-manager build 
```

Once TAHOE is compiled and built, add the following environment variables and path to your .bashrc file:

$ echo export TAHOE_MOD=$TAHOE_MAIN/Tahoe-Install >> ~/.bashrc
$ echo export TAHOE_DIR=$TAHOE_MOD/tahoe >> ~/.bashrc
$ echo export PATH="$TAHOE_MOD/bin:$PATH" >> ~/.bashrc
$ source ~/.bashrc

Installing Tahoe for MARCC

Assuming what is needed in Tahoe serials compiling has been established.

1) load the module required for Tahoe parallel compile on MARCC (GCC, MARCC, CMake)

  module load MARCC

  module load gcc

  module load cmake

These can be added to the .bashrc (for bash shell) file

2) download the tahoe-install directory

  cvs -d username@tahoe.cvs.sourceforge.net:/cvsroot/tahoe checkout tahoe-install

3) configure tahoe for build:

  cd tahoe-install

 ./tahoe-manager config
 enter Sourceforge "username"
 enter Cloudforge "tahoe.username"
 enter your password for Sourceforge or Cloudforge when prompted (if you don't have ssh key pairs setup)
 default build architecture number: ?   choose 112 (for openmpi on linux)
                                                            * you chose : RedHat.OpenMPI
 do want this to be the default architecture for all modules (y/n) [y]: 
                                                            * default modules : macros expat toolbox tahoe
 do want to add optional modules (y/n) [n]: y
 Choice or modules to toggle (applied left to right): 0 6 7 10 11 13 14 15 16 (turn on CBLAS access aztec contrib development f2c metis spooles spoolesMPI)

4) Build Tahoe

./tahoe-manager build

After installation, If you get the following error when running Tahoe: error while loading shared libraries: libnetcdf.so.11: cannot open shared object file: No such file or directory. Then add “export LD_LIBRARY_PATH=/XXX/lib” (replace XXX with the directory of ACCESS, e.g. LD_LIBRARY_PATH=/home-2/hdong6@jhu.edu/seacas/lib) into the .bashrc file and restart the bash.

Tips: 1. When running parallel, make sure to use the command: $ mpirun -np number_of_cores tahoe -f. Using tahoe -f will run program in serial.

2. When running parallel, make sure that you choose SPOOLES under solver

3. MARCC changed compilers, if your Tahoe does not compile, connect Zheliang Wang (wzhelia1@jhu.edu) for an updated zip file of Tahoe.

4. For more information, see tahoe user manual.

Building Tahoe using Make

The Tahoe project is built using Make. Each module contains a makefile containing the build options and configurations for each module is.

1) To create a makefile for a module (e.g. tahoe), cd to the directory associated with the module (e.g. full_path_to_tahoe-install/tahoe) and cp makefile.template to makefile. Add the HOME_DIR for the module (e.g., full_path_to_tahoe-install/tahoe), and the build architecture (e.g., DARWIN-GCC4.X). Uncomment modules that you want to link to in the build options (e.g. access).

2) Type make help to get all targets. For example, the targets for the Tahoe module are:

init - (re-)initialize headers and file dependencies
build - (re-)build binaries
objects - (re-)compile object file but don't link
clean - remove object files and build temporaries
help - display this list
additional targets for configuration:

clean_config - remove config files in src/config
update_config - update config files in src/config
additional targets for (un-)installing optional modules:

touch_spooles - sparse, direct solver library
touch_SuperLU - sparse, direct solver library
touch_Aztec - sparse, iterative solver library
touch_f2c - f2c support
touch_SEACAS - Sandia tools
touch_MPI - support for parallel execution
touch_METIS - domain decomposition using METIS
touch_qhull - computational geometry using qhull
touch_NOX - NOX nonlinear solver library
touch_fossum - geomaterial development module
touch_expat - XML parsing
touch_SIMOD - Shared Interface MODels libraries

3) Each module can be built individually with make init build -s. Initialize and build the toolbox and libraries before the tahoe directory.

4)Resources/instructions for Make

GNU Make

Free Software Foundation

Using SVN

SVN Subversion is an open-source version control system used to manage the collaborative development of Tahoe. Browse through the tahoe svn repositories to see previous versions and track the difference between versions.

The svn repository for the public tahoe project: https://sourceforge.net/p/tahoe/codesvn/HEAD/tree/

The svn repository for the development project: https://sourceforge.net/p/tahoe/codesvn_dev/HEAD/tree/

The svn repository for the development project benchmark: https://sourceforge.net/p/tahoe/codesvn_dev_bench/HEAD/tree/

A tutorial to svn: http://svnbook.red-bean.com/

key commands: commit, update, checkout

Creating Tahoe input files

1) Tahoe uses XML files as input files. There are many XML editors available for purchase and free download from the web. A common one used in the lab is the no-frills GUI XAmple editor.

Installing Xample XML editor on macOS

Xample is probably the best open-source XML editor. It allows you to easily load custom XML schemas (.xsd files), informs you of all the possible inputs for lists and sublists, and allows for easy adding and removing of nodes. Unfortunately, Xample is no longer supported by its original creator, which means it will not run on the latest Java version. The last Java version compatible that Xample ran on was version 8 (1.8.0_91). If you do not have version 8, you must install it. The easiest way to do this is to use homebrew.

$ brew install cask
$ brew cask install java8

check the paths of the versions installed:

$ /usr/libexec java_home -V

You should see something like this:

9.0.1, x86_64: “Java SE 9.0.1” /Library/Java/JavaVirtualMachines/jdk-9.0.1.jdk/Contents/Home
1.8.0_91, x86_64: “Java SE 8” /Library/Java/JavaVirtualMachines/jdk1.8.0_91.jdk/Contents/Home

Alternatively, download the following disk image for v8 and try installing it yourself without brew

jre-8u151-macosx-x64.dmg

Download the following distribution of Xample

xample.tar

untar: tar -xvf xample.tar

If you're using csh or tcsh, you will execute Xample using the run-xample.csh script, but first you will need to set a few variables. open the the run-xample.csh script and add the following after “#!/bin/csh -f ”

1) setenv JAVA_HOME /Library/Java/JavaVirtualMachines/jdk1.8.0_91.jdk/Contents/Home

2) set JAVA = /usr/bin/java

3) set xampledir = [Full path to Xample]/XAmple-distr You can run Xample by running the script in the X-window.

Next, create the xml schema by typing tahoe -xsd (to create file tahoe.xsd). 3) Open tahoe.xsd in Xample. 4) At this point, you can create an .xml input file from scratch or open and edit an existing .xml input file (for example using a benchmark).

Install jedit

Another good open-source xml editor is jedit. I allows you to load custom .xsd or .dtd xml schema and has a tree view of the inputs as well as a table of allowable inputs for each sublist. It's still not as convenient as Xample, but it is an actively supported application that runs on the latest version of java.

To install jedit, download the installation package for your platform from: http://www.jedit.org/index.php?page=download Install according to the directions on the website. For mac, you may have to run the application from the command line using the latest version of java.

% java -jar [full path to jEdit]/jEdit.app/Contents/Java jedit.jar

Next, install the XML plug-in. You can do this in the application's pull-down menu Plugins → Plugin Manager and check the boxes for XML, XercesPlugin, and SideKick. To download and install the plug-ins manually, follow the instructions at http://plugins.jedit.org/ .

When you open an existing tahoe xml file or create a new one, you have to associate either the tahoe.xsd or tahoe.dtd schema with it. To do so got to Plugins → Set Schema Type and choose between XSD or DTD. Next, go to Plugins → Set Schema. The sidekick window will now show you a tree view of all of your lists and sublists of the input file. You can dock the sidekick window by clicking on the down arrow at the top of the window. To insert a new field, scroll to the line that you want to insert something in, then go to Plugins → XML Insert. This will create another window of all the input fields allowed for that list item. You can also dock this window in the main window by clicking on the down arrow at the top of the menu.

Benchmark examples

All benchmark examples reside in tahoe-install/benchmark_XML and are organized as follows:

level.0 - element formulations (e.g., elastodynamic, elastostatic, diffusion, adhesion, etc..)
level.1 - material formulations for solid and cohesive elements; stress smoothing algorithms
level.2 - boundary controllers and conditions, contact
level.3 - parallel execution

Running Tahoe

1) Run interactively by typing tahoe at the command line. You'll be prompted to enter the input file path or option.

2) Run tahoe with command line options (tahoe [-option]):

f [input_file.xml] executes tahoe with the specified input file.
xsd generates the xml template tahoe.xsd file.
dtd generates the xml template tahoe.dtd file
verbose executes tahoe with verbose message logs
decomp decomposes geometry file for parallel execution
join joins output files for parallel execution

3) Run tahoe batch processes and create a text file with @ in the first line. List the input files in subsequent lines. Download example:

run.batch

Adding users and permissions Add user to sourceforge and cloudforge repositories. For sourceforge:

In the tahoe directory, checkout CVSROOT

cd CVSROOT
vi avail
Add the username to the avail file
cvs commit

Table of Contents