The logging on a FortiGate firewall is very scarse, making it difficult to troubleshoot issues. This can especially be a problem when setting up a site-to-site IPSEC VPN tunnel. Although the web interface doesn't provide much information for troubleshooting and debugging, the console does when debugging is enabled.

On most (if not, all) FortiGate appliances, you can access the console through the web interface. It usually can be found on the Dashboard (> Status).

As it says, click on the console to activate it.

IPSEC VPN debugging

Enabling debugging for all IPSEC VPNs means we enable debug mode on "IKE". This is done by the following series of commands.

If any debugging is already in progress, it needs to be stopped first:

diagnose debug disable

If you previously did VPN debugging, you also have to clear any filters applied to the logging output:

diagnose vpn ike log-filter clear

To filter on a specific tunnel, we can set the logging output filter to only show us debug logging for a specific host. Usually, you will want to set it to filter on the remote WAN IP. So, if the other side has a WAN address of 1.2.3.4, you will set the filter like so:

diagnose vpn ike log-filter dst-addr4 1.2.3.4

Finally, enable debug mode on the IKE (handshaking) process, and re-enable global debugging to output the debug logging to the console:

diagnose debug app ike 255
diagnose debug enable

If the connection is not already started, go in the web interface and "Bring up" the VPN. You can find this at VPN > Monitor, and then click Bring Up on the according VPN tunnel.

During debug logging, a lot of output will continue to appear in the console, making it difficult to troubleshoot. You can suspend debug logging by disabling debug mode:

diagnose debug disable

You can then use the scrollback functionality of the console to view past debug messages, and try to troubleshoot the VPN tunnel.

When troubleshooting, it is often recommended to do the same on the remote firewall. With both logs, you will be able to quickly deduce whatever goes wrong when the tunnel is set up.

Flush/reset a VPN tunnel

Sometimes, it may not be clear as to why a tunnel goes down and doesn't come up. In that case, it may be necessary to reset a VPN tunnel so the SA sessions will be cleared (in case they are stuck):

You can "flush" a tunnel so the SAs can be re-established:

diagnose vpn tunnel flush my-phase1-name

Replace my-phase1-name with the name of the Phase1 part of your VPN tunnel. If you do not specify a name, all tunnels will be "flushed".

If flushing the tunnel does not help, you can perform a complete reset of the VPN tunnel, resulting in a complete re-negotiation of the specified IPSEC VPN tunnel:

diagnose vpn tunnel reset my-phase1-name

Replace my-phase1-name with the name of the phase1 part of your tunnel. Like with the "flush" command, not specifying a tunnel name will reset all tunnels.

Restart a process

If flushing/resetting a tunnel does not help, you can also try to restart the entire VPN process.

Look up the PIDs of the VPN processes:

get system performance top

This will regularly output a list of current top resource using processes.

The second column shows the PID. You'll need the PID for the process called iked. Send it a SIGNAL 11 to force a restart of the process.

diagnose system kill 11 the-pid-i-got-earlier

Replace the-pid-i-got-earlier with the one you retrieved from the output of the previous command.
E.g. if the PID was 123, the command would become: diagnose system kill 11 123

If you ran the get system performance top command again, you would notice the process iked would then have another PID than before.

Note that you can basically do this with any process running on the FortiGate, although be careful with which process you kill, as killing the wrong one will result in your FortiGate becoming unresponsive, and requiring a power cycle to become unstuck again.