- In certain specific scenarios, the debugging commands included in this document may need to be supplemented by more advanced debugging procedures.
- Advanced procedures should be executed in conjunction with the Check Point Escalation engineers.
- Debugging should only be performed when the described issue can be captured.
FireWall Common debugging
Kernel debugging
Usage
% fw ctl debug -buf [buffer size]
% fw ctl debug [-x] [-m <module>] [+|-] <options | all | 0>
% fw ctl kdebug –f > <output file>
To disable the Kernel debugging, execute:
% fw ctl debug –buf 0
% fw ctl debug x
Common Syntax
% fw ctl debug –buf 12288
% fw ctl debug –m fw conn drop ld packet if
% fw ctl kdebug –f > <output file>
The ld option may cause high CPU usage. It is advised to use it for short
session debugging
only.
To execute the kernel you can also use fw
ctl zdebug to allocate the buffer (where the
buffer can only be 1024).
% fw ctl zdebug
% fw ctl kdebug -f > <output file>
User Mode Processes debugging
Usage
% fw debug <process name> <on/off> TDERROR_ALL_ALL=<value
1-5>
CPD is treated differently from the other User Mode processes and will be
executed
differently,
Debugging CPD
CPD is a high in the hierarchichal chain and helps to execute many
services, such as Secure
Internal Communcation (SIC), Licensing and status report.
For CPD debug, execute: % cpd_admin
debug on TDERROR_ALL_ALL=5
The debug file is located under $CPDIR/log/cpd.elg
To stop the CPD debug, execute: %
cpd_admin debug off TDERROR_ALL_ALL=1
Debugging FWM
The FWM process is responsible for the execution of the database activities
of the
SmartCenter server. It is; therefore, responsible for Policy installation,
Management High
Availability (HA) Synchronization, saving the Policy, Database Read/Write
action, Log
Display, etc.
For FWM debug, execute:
% fw debug fwm on TDERROR_ALL_ALL=5
% fw debug fwm on
OPSEC_DEBUG_LEVEL=9
The debug file is located under $FWDIR/log/fwm.elg
To stop the FWM debug, execute:
% fw debug fwm off TDERROR_ALL_ALL=1
% fw debug fwm off
OPSEC_DEBUG_LEVEL=1
Debugging FWD
The FWD process is responsible for logging. It is executed in relation to
logging, Security
Servers and communication with OPSEC applications.
For FWD debug, execute: % fw debug
fwd debug on TDERROR_ALL_ALL=5
The debug file is located under $FWDIR/log/fwd.elg
To stop the FWD debug, execute: % fw
debug fwd off TDERROR_ALL_ALL=1
FireWall Monitor Network Capturing
The FireWall Monitor is responsible for packet flow analysis.
To execute: % fw monitor –e
“accept;” –o <output file>
ClusterXL debugging
For ClusterXL debugging for Clustering, Synchronization, High Availability,
Fail-over,
execute:
% cphaprob state
% cphaprob -ia list
% cphaprob -a if
% fw ctl pstat
Kernel debug for packet filter analysis
% fw ctl debug –buf 12288
% fw ctl debug –m fw conn drop packet if sync
% fw ctl debug –m cluster all
% fw ctl kdebug –f > <output file>
Disable SecureXL for capturing proper logs:
- If you have SecureXL enabled, it fast switches the packets and won’t show you all the detailed logs that you would like to see in your captures. You can disable SecureXL temporarily, if you want to inspect packets at that granularity (i.e ‘I’ and ‘o’). In my experience, disabling SecureXL hasn’t been a problem* and I haven’t seen any performance impact as such.
Right before the capture, turn off SecureXL on the gateway:
fwaccel off
Immediately after the capture, turn on SecureXL on the gateway:
fwaccel on
To turn on/off SecureXL on multiple VSXs at once:
fwaccel on -a fwaccel off -a
- Ensure that you re-enable after you are done capturing the logs.
- Be aware that SecureXL does affect how the traffic passes through the firewall and only disable it if you are 100% sure of what you’re doing. Do it at your own risk, I do not take any responsibility for anything going wrong with this as it does affect the live traffic immediately. Read the blog’s disclaimer. :)
Display all 8 inspection points (iIoO) for a complete two-way traffic on the console:
For specific flow:
fw monitor -e 'accept (src=10.1.1.1 and dst=20.2.2.2) or (src=20.2.2.2 and dst=10.1.1.1);' -m iIoO
OR
For specific IP:
fw monitor -e 'accept (src=10.1.1.1 or dst=10.1.1.1);' -m iIoO
Display 4 inspection points (iO) for a complete two-way traffic on the console:
fw monitor -e 'accept (src=10.1.1.1 and dst=20.2.2.2) or (src=20.2.2.2 and dst=10.1.1.1);' -m iO
- Note that we are only grepping iO, that means you are not looking for what is going on at the post-inbound (I) and pre-outbound (o) inspection points and just trying to get fewer output to see if it actually leaves the egress interface or not.
- If you suspect something missing, then run -m iIoO and see at which inspection point it’s getting stuck and use the previous post for troubleshooting tips.
Saving fw monitor logs to a .pcap file to analyse in wireshark:
fw monitor -e 'accept (src=10.1.1.1 and dst=20.2.2.2) or (src=20.2.2.2 and dst=10.1.1.1);' -m iIoO -o wireshark.pcap
- By default, the capture file is saved to the /home/admin directory.
- Use WinSCP to access the Security Gateway and copy the file to your local drive to analyze it in Wireshark.
Saving a TCP dump in a .pcap file:
tcpdump -w capture.pcap -i eth-s1p2c0 host 10.1.1.1 and host 20.2.2.2
tcpdump -nni any host 10.1.1.1 -w capture.pcap
tcpdump -nni any host 10.1.1.1 and host 20.2.2.2 -w capture.pcap
- Replace the interface name and capture name as required.
Switches:
| Ctrl+C | This is the break sequence |
| -e | Custom expression. Sets the filter for fw monitor. |
| -o | Writes the raw packet data output to a file. |
| -ci -co | Captures only the number of inbound (ci) and outbound (co) packets. This is useful if you have many packets going between a source and destination and the break sequence can take time to stop those captures.-vSpecifies the virtual device where the fw monitor needs to run |
