# Installing prerequisites
The instructions can be found [here](https://aiida.readthedocs.io/projects/aiida-core/en/latest/intro/install_system.html#intro-get-started-system-wide-install). I haven't tried them on linux but I don't expect issues since `apt-get` et al. tend to have good versioning. This is relevant because, at the time of this writting, `aiida-core` does not work with `rabbitmq` versions 3.8.5 or later. For a workaround if you can't downgrade, see [here](https://aiida.readthedocs.io/projects/aiida-core/en/latest/intro/troubleshooting.html#intro-troubleshooting-installation-rabbitmq).
## When MacOS turns into a nightmare
At the time of this writing, `brew` has only 1 version of `rabbitmq`: 3.10.7. You cannot downgrade unless you write your own formula. This normally shouldn't be a problem since you can just use the workaround linked above, but with the introduction of apple silicon (M series processors, which use the `arm64` architecture compared to `x86_64` on older Intel Macs), things are a little more complicated.
If you use a `rosetta` (i.e., intel) terminal to install `brew`, or if you cloned your M1 machine from an Intel Mac that already had brew installed (as I did), brew will install `x86_64` formulas. Normally, this is fine, almost all of my formulas are installed this way. But not `rabbitmq`, you will immediately get a `segfault`.
The only way around this is to install TWO version of `brew`, one for each architecture.
Here are the steps:
1. Check which version of brew you have (assuming standard installation)
```bash
$ which brew
> /opt/homebrew/bin/brew # Probably arm64, skip to step 5
$ which brew
> /usr/local/bin/brew # Probably x86_64, proceed.
```
2. Install native (`arm64`), defaults to `/opt/homebrew/
```bash
$ arch --arm64 bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
```
3. Configure your `.zshrc` (or equivalent), don't forget `source ~/.zshrc` afterwards. Every time you spawn a new shell, this will check the architecture and use the correct brew.
```
if [ "$(arch)" = "arm64" ]; then
eval "$(/opt/homebrew/bin/brew shellenv)"
else
eval "$(/usr/local/bin/brew shellenv)"
fi
```
4. Make sure everything works
```bash
$ arch --arm64 zsh # spawns new shell
$ brew --prefix
> /opt/homebrew/bin/brew
$ arch --x86_64 zsh # spawns new shell
$ brew --prefix
> /opt/homebrew
```
5. Install `rabbitmq` and `postgresql` with the `arm64` brew
```bash
$ arch --arm64 zsh
$ brew install pkg-config rabbitmq postgresql@14
$ brew services start rabbitmq
$ brew services start postgresql@14
$ exit
```
6. Check services are running
```bash
$ brew services list
postgresql@14 started ashour ~/Library/LaunchAgents/
[email protected]
rabbitmq started ashour ~/Library/LaunchAgents/homebrew.mxcl.rabbitmq.plist
```
See [here](https://dansketcher.com/2021/03/30/shmmax-error-on-big-sur/) if you have kernel memory issues (I sure did). Also make sure to follow the timeout modification process for `rabbitmq` discussed [here](https://aiida.readthedocs.io/projects/aiida-core/en/latest/intro/troubleshooting.html#intro-troubleshooting-installation-rabbitmq)
You can also alias your `arm64` `brew` in `.zshrc` or equivalent
```bash
alias brewarm='/opt/homebrew/bin/brew'
```
# AiiDA-Core Installation
You can just follow the guide [here](https://aiida.readthedocs.io/projects/aiida-core/en/latest/intro/install_system.html#intro-get-started-system-wide-install) for the most up to date instructions, but as a quick summary:
```bash
$ python --version # Need 3.8+ for AiiDA 2.0.4
$ python -m venv ~/venvs/aiida # or wherever you want
$ pip install aiida-core
```
> [!WARNING]
> I haven't been able to get `aiida-core` 2.0.4 to work with `python` 3.10, there seems to be an issue with `circus` when running python versions with a 2 digit minor version. I used 3.9.
We set up a profile:
```bash
$ verdi quicksetup
Report: enter ? for help.
Report: enter ! to ignore the default and set no value.
Profile name [quicksetup]: spike
Email Address (for sharing data) [()]:
[email protected]
First name [()]: Spike
Last name [()]: Spiegel
Institution [()]: Bebop Bounty Hunters
Success: created new profile `spike`.
Report: initialising the profile storage.
Report: initialising empty storage schema
Success: storage initialisation completed.
```
```bash
$ verdi daemon start 2
```
And check everything works:
```
$ verdi status
✔ version: AiiDA v2.0.4
✔ config: /path/to/.aiida
✔ profile: spike
✔ storage: Storage for 'spike' [open] <something>
Warning: RabbitMQ v3.10.7 is not supported and will cause unexpected problems!
Warning: It can cause long-running workflows to crash and jobs to be submitted multiple times.
Warning: See https://github.com/aiidateam/aiida-core/wiki/RabbitMQ-version-to-use for details.
⏺ rabbitmq: Incompatible RabbitMQ version detected! Connected to RabbitMQ v3.10.7 as amqp://guest:
[email protected]:5672?heartbeat=600
✔ daemon: Daemon is running as PID 21161 since 2022-09-24 13:36:08
```
Since we have already modified the `rabbitmq` configuration, we can just suppress that warning:
```
verdi config set warnings.rabbitmq_version False
```
>[!HINT]
> I ran into a ton of issues during this process and had to do it multiple times. To get rid of the artifacts of previous attempts, you can run:
> ```bash
> $ brew services stop postgresql
> $ verdi profile delete $PROFILE # delete your profile
> $ rm -rf /path/to/.aiida # Usually in $HOME
> /opt/homebrew/bin/psql
> # You are now in the psql shell
> \l # This lists all the databases
> DROP DATABASE db_name; # pick aiida DBs
> \q # quit
> $ brew services start postgresql
> ```
# AiiDA-VASP Installation
We need to install the development version of AiiDA-VASP for compatibility with `aiida-core`>=2.0.1, but make sure to check their [github](https://github.com/aiida-vasp/aiida-vasp) for up to date instructions.
```bash
$ pip install git+https://github.com/aiida-vasp/aiida-vasp.git@develop#egg=aiida-vasp
```
We should check that all the groups, workflows and calculations are registered properly:
```bash
$ verdi plugin list aiida.groups
$ verdi plugin list aiida.workflows
$ verdi plugin list aiida.calculations
```
All these commands should include some `vasp.*` in their output.
# Adding a computer: NERSC
The instructions can be found [here](https://aiida.readthedocs.io/projects/aiida-core/en/latest/howto/run_codes.html).
Below are configuration options for connecting to NERSC's Cori. I'm assuming you're already configured with SSH keys generated using NERSC's `sshproxy.sh`. If not, see [here](https://docs.nersc.gov/connect/mfa/#sshproxy) and [here](https://github.com/oashour/NERSC-SSH-1Password) for a little script I wrote.
In this example, I will be configuring Cori's Haswell partition. The process should be quite similar for Cori KNL and Perlmutter CPU. Not sure about Perlmutter GPU since you need to explicitly specify the number of GPUs.
After doing this process for a few times (apparently you can't edit computer after you create them), I figured out you can use `.yaml` files.
Using this file:
```
---
label: "cori-hsw"
hostname: "cori.nersc.gov"
description: "NERSC's Cori cluster, Haswell partition"
transport: "core.ssh"
scheduler: "core.slurm"
mpiprocs_per_machine: 32
default_memory_per_machine: 12800000
work_dir: "/global/cscratch1/sd/{username}/aiida/"
mpirun_command: "srun -n {tot_num_mpiprocs} -c 2"
prepend_text: |
### computer prepend_text start ###
#SBATCH -C haswell
# OMP Settings
export OMP_PLACES=threads
export OMP_PROC_BIND=true
export OMP_NUM_THREADS=1
### computer prepend_text end ###
```
We can set up Cori Haswell like this
```bash
$ verdi computer setup --config cori-hsw.yaml
```
Note that I prepended `export OMP_NUM_CORES=1` to disable OpenMP, which I never use on Haswell. I'm sure you can do the whole OpenMP thing by prepending at the individual calculation level. You probably want to do that if you set up Cori KNL or another Xeon Phi cluster.
Now we need to configure SSH access. Using the file below:
```
---
username: "your_nersc_username"
```
then running
```
verdi computer configure core.ssh cori-hsw --config cori-ssh.yaml
```
I just used the default for everything else here, but make sure `verdi` knows the correct path to your SSH key.
Finally, we need to test the connection
```
verdi computer test cori-hsw
```
All five tests should pass.
# Setting up a code
Now we have to set up a VASP code. If you use multiple versions of VASP, set up a code for each of them. I will be using 6.3.0 here. You will also need different codes for `vasp_std` and `vasp_ncl`.
Since Cori offers different modules for Haswell and KNL, you will also want to set up different AiiDA codes for each of them (you have to do that anyway since each code is connected to one computer).
```
verdi code setup --config vasp-6.3.0-hsw-std.yaml
```
# Adding the Potentials
We still need to add the VASP pseudopotentials to the AiiDA database. Download them from the VASP Portal (or from NERSC) and extract them somewhere (technically `aiida-vasp` can import `.tar` and `.tar.gz` but due to a bug in `pymatgen`, you sometimes get issues).
```bash
verdi data vasp-potcar uploadfamily --path=$HOME/potpaw_PBE.54 --name=PBE.54 --description="PBE potentials for version 5.4.4"
```
Executing this gave me a mountain of errors (`pymatgen.2022.x.x` again), but then
```bash
verdi data vasp-potcar listfamilies
```
lists 327 potentials in the `PBE.54` family. So at least it works.
If you want to use another pseudopotential family, such as PBE.52 or LDA, you can add them the same way.
# Testing AiiDA
We can just follow the example from `aiida-vasp` shown [here](https://aiida-vasp.readthedocs.io/en/latest/getting_started/test_run.html). After downloading the script, you will need to modify `OPTIONS.qos`, which is mandatory on Cori. To run in the debug queue, just change line 177 to
``` python
OPTIONS.qos = 'debug'
```
Everything else can stay the same. Run the script
```
python run_vasp_lean.py
```
and check the status of the calculations
```
verdi process list -a
```
They should finish fairly quickly. With the above configuration files, this worked for me. You can now follow standard AiiDA tutorials.
# Conclusion
With a running AiiDA installation, the world is your oyster. Unfortunately, after spending 5 hours installing and debugging it, I decided it's just not for me considering I never launch too many calculations simultaneously. I need better time management.