Raccoon??? My God, it even used somewhere? Where he keeps configs in Linux - I don't know, although FreeBSD with them and worked. To find the configs via
rpm-qa ipsec-tools | less
view the contents of files in /etc (most likely they will be there).
To see the iptables rules, including the ipsec, ike, esp Protocol, ports 500 and 4500 (ipsec itself uses the esp Protocol, ports 500 and 4500 uses ike, which negotiates parameters of the tunnels)