Tuesday, July 5, 2016

Chapter 2 Chain Module

  1. Check Point kernel is the bridge between the hardware and the OS.
  2. The inspection operations in the kernel are divided into modules, and the modules are divided into chains.
  3. The number of chains on every Security Gateway is different. It depends on which blades/features are enabled on the Security Gateway.
  4. Each direction has its own modules and order of inspection.
  5. Handlers (INSPECT code) decide which modules will inspect the packet.
  6. The kernel consist of inbound (ingress) and outbound (egress) Chain Modules.
  7. It inspects packets in both directions.
  8. Each packet passes through a list of Inbound and Outbound chain modules which may    modify pass or drop packets.
  9. They are responsible for decryption, encryption, rule matching, and policy verification.
  10. Chain Positions is an absolute number and never change
  11. Each chain module is independent and redundant
  12. They execute their own unique inspection.
  13. Inspection is performed on virtually fragmented packets.
  14. In (VPN) firewall kernel, each kernel chain is associated with a key. The key specify the type of traffic applicable to this chain.

fw monitor inserts its own modules in this module chain and is capturing packets there. By default this is not the first and last position in the chain.

To see all active chains in the Security Gateway, run:

fw ctl chain 

Security Gateway

User Mode   (CPD/FWD/WPND/FWSSD/CPWD)
OS IP Stack
Firewall-1 Kernel  (Inbound/Outbound Chain)
NIC Hardware Layer NIC

Use "fw ctl chain" command to see this list of Chain Modules

Module Location in the chain  (Serial Number)
Chain Position
Function Pointer
Mode in which the chain applies:
1. - Stateful mode
2. - Wired mode
3. - All Packets
fff - All packets  (identical to 3)


[Expert@bostestint-fwa:0]# fw ctl chain
in chain (11):
        0: -7f800000 (ffffffff8882b820) (ffffffff) IP Options Strip (in) (ipopt_strip)
        1: - 1fffff8 (ffffffff8882d050) (00000001) Stateless verifications (in) (asm)
        2: - 1fffff7 (ffffffff8886eb00) (00000001) fw multik misc proto forwarding
        3: - 1000000 (ffffffff88911d10) (00000003) SecureXL conn sync (secxl_sync)
        4:         0 (ffffffff887ceee0) (00000001) fw VM inbound  (fw)
        5:        10 (ffffffff887e3690) (00000001) fw accounting inbound (acct)
        6:  10000000 (ffffffff88910330) (00000003) SecureXL inbound (secxl)
        7:  7f600000 (ffffffff88820bf0) (00000001) fw SCV inbound (scv)
        8:  7f730000 (ffffffff88a35fb0) (00000001) passive streaming (in) (pass_str)
        9:  7f750000 (ffffffff88c53260) (00000001) TCP streaming (in) (cpas)
        10:  7f800000 (ffffffff8882bbe0) (ffffffff) IP Options Restore (in) (ipopt_res)
out chain (10):
        0: -7f800000 (ffffffff8882b820) (ffffffff) IP Options Strip (out) (ipopt_strip)
        1: - 1fffff0 (ffffffff88c534a0) (00000001) TCP streaming (out) (cpas)
        2: - 1ffff50 (ffffffff88a35fb0) (00000001) passive streaming (out) (pass_str)
        3: - 1f00000 (ffffffff8882d050) (00000001) Stateless verifications (out) (asm)
        4: -     1ff (ffffffff88e6eb80) (00000001) NAC Packet Outbound (nac_tag)
        5:         0 (ffffffff887ceee0) (00000001) fw VM outbound (fw)
        6:  10000000 (ffffffff88910330) (00000003) SecureXL outbound (secxl)
        7:  7f000000 (ffffffff887e3690) (00000001) fw accounting outbound (acct)
        8:  7f700000 (ffffffff88c53690) (00000001) TCP streaming post VM (cpas)
        9:  7f800000 (ffffffff8882bbe0) (ffffffff) IP Options Restore (out) (ipopt_res)
[Expert@bostestint-fwa:0]#


Chain Insertion Points

FireWall-1 kernel module between Layer 2 NIC driver and Layer 3 IP stack
fw monitor uses its own kernel module to capture packets
it capture packets in 4 positions

Capture position fw monitor mask value
pre-inbound i (lowercase i)
post-inbound I (uppercase i)
pre-outbound o (lowercase o)
post-outbound O (uppercase o)

[Expert@mytestint-fwa:0]# fw ctl chain
in chain (11):
        0: -7f800000 (ffffffff8882b820) (ffffffff) IP Options Strip (in) (ipopt_strip)
        1: - 1fffff8 (ffffffff8882d050) (00000001) Stateless verifications (in) (asm)
        2: - 1fffff7 (ffffffff8886eb00) (00000001) fw multik misc proto forwarding
        3: - 1000000 (ffffffff88911d10) (00000003) SecureXL conn sync (secxl_sync)
        4:         0 (ffffffff887ceee0) (00000001) fw VM inbound  (fw)
        5:        10 (ffffffff887e3690) (00000001) fw accounting inbound (acct)
        6:  10000000 (ffffffff88910330) (00000003) SecureXL inbound (secxl)
        7:  7f600000 (ffffffff88820bf0) (00000001) fw SCV inbound (scv)
        8:  7f730000 (ffffffff88a35fb0) (00000001) passive streaming (in) (pass_str)
        9:  7f750000 (ffffffff88c53260) (00000001) TCP streaming (in) (cpas)
        10:  7f800000 (ffffffff8882bbe0) (ffffffff) IP Options Restore (in) (ipopt_res)
out chain (10):
        0: -7f800000 (ffffffff8882b820) (ffffffff) IP Options Strip (out) (ipopt_strip)
        1: - 1fffff0 (ffffffff88c534a0) (00000001) TCP streaming (out) (cpas)
        2: - 1ffff50 (ffffffff88a35fb0) (00000001) passive streaming (out) (pass_str)
        3: - 1f00000 (ffffffff8882d050) (00000001) Stateless verifications (out) (asm)
        4: -     1ff (ffffffff88e6eb80) (00000001) NAC Packet Outbound (nac_tag)
        5:         0 (ffffffff887ceee0) (00000001) fw VM outbound (fw)
        6:  10000000 (ffffffff88910330) (00000003) SecureXL outbound (secxl)
        7:  7f000000 (ffffffff887e3690) (00000001) fw accounting outbound (acct)
        8:  7f700000 (ffffffff88c53690) (00000001) TCP streaming post VM (cpas)
        9:  7f800000 (ffffffff8882bbe0) (ffffffff) IP Options Restore (out) (ipopt_res)
[Expert@mytestint-fwa:0]#
   
[Expert@mytestint-fwa:0]# fw ctl chain
in chain (13):
        0: -7f800000 (ffffffff8882b820) (ffffffff) IP Options Strip (in) (ipopt_strip)
        1: -70000000 (ffffffff88804d70) (ffffffff) fwmonitor (i/f side)
        2: - 1fffff8 (ffffffff8882d050) (00000001) Stateless verifications (in) (asm)
        3: - 1fffff7 (ffffffff8886eb00) (00000001) fw multik misc proto forwarding
        4: - 1000000 (ffffffff88911d10) (00000003) SecureXL conn sync (secxl_sync)
        5:         0 (ffffffff887ceee0) (00000001) fw VM inbound  (fw)
        6:        10 (ffffffff887e3690) (00000001) fw accounting inbound (acct)
        7:  10000000 (ffffffff88910330) (00000003) SecureXL inbound (secxl)
        8:  70000000 (ffffffff88804d70) (ffffffff) fwmonitor (IP  side)
        9:  7f600000 (ffffffff88820bf0) (00000001) fw SCV inbound (scv)
        10:  7f730000 (ffffffff88a35fb0) (00000001) passive streaming (in) (pass_str)
        11:  7f750000 (ffffffff88c53260) (00000001) TCP streaming (in) (cpas)
        12:  7f800000 (ffffffff8882bbe0) (ffffffff) IP Options Restore (in) (ipopt_res)
out chain (12):
        0: -7f800000 (ffffffff8882b820) (ffffffff) IP Options Strip (out) (ipopt_strip)
        1: -70000000 (ffffffff88804d70) (ffffffff) fwmonitor (i/f side)
        2: - 1fffff0 (ffffffff88c534a0) (00000001) TCP streaming (out) (cpas)
        3: - 1ffff50 (ffffffff88a35fb0) (00000001) passive streaming (out) (pass_str)
        4: - 1f00000 (ffffffff8882d050) (00000001) Stateless verifications (out) (asm)
        5: -     1ff (ffffffff88e6eb80) (00000001) NAC Packet Outbound (nac_tag)
        6:         0 (ffffffff887ceee0) (00000001) fw VM outbound (fw)
        7:  10000000 (ffffffff88910330) (00000003) SecureXL outbound (secxl)
        8:  70000000 (ffffffff88804d70) (ffffffff) fwmonitor (IP side)
        9:  7f000000 (ffffffff887e3690) (00000001) fw accounting outbound (acct)
        10:  7f700000 (ffffffff88c53690) (00000001) TCP streaming post VM (cpas)
        11:  7f800000 (ffffffff8882bbe0) (ffffffff) IP Options Restore (out) (ipopt_res)
[Expert@mytestint-fwa:0]# 


Insert fw monitor into the chain to report on all packets. It may be inserted into the chain at the following four positions.

inbound minus 0x70000000
Inbound 0x70000000
outbound minus 0x70000000
Outbound 0x70000000

To put the monitor in a certain place use the -p parameter and the number of item that will be after fw monitor.

To put the monitor in the end of the chain, use -pI 999 or -pO 999. This -p parameter has the following syntax:

fw monitor - p [i|I|o|O] absolute pos| relative pos| [+ |-} alias]

fw ctl chain with FW Monitor inserted

fw monior captures packets as they enter or leave the firewall kernel and when the packet enters and leaves the inbound and outbound chain.

With fw monitor running  once fw monitor is executed, a specified INSPECT filter is compiled an loaded into kernel.  Any parameters following "accept" in the fw monitor command will be displayed by the fw monitor.  The same filter is applied to all interface in all direction.

fw monitor output uses specific expressions to explain the location of the packet as it moves through the firewall. There are four inspection points as a packet passes through the kernel (or Virtual Machine)

i   Before the virtual machine, in the inbound direction (pre-inbound)
  After virtual machine in the inbound direction (post -inbound)
o  Before the virtual machine, in the outbound direction (pre-outbound)
After virtual machine in the outbound direction (post -outbound)

In client to server scenario, 
'i' represent the packet as it left the client.
'I' represent the packet already checked against the tables and rule base.
in case of static-nat, the destination address will be changed.
'o' means the packet is before the oubound kernel (same as 'I')  and
'O' means the packet is in the outbound kernel chain. as it appears at the webserver.

in the case of hide-nat the source IP address will be different.

The general syntax:
fw monitor -e "accept <expression>;"


[Expert@bostestint-fwa:0]# fw ctl chain
in chain (13):
        0: -7f800000 (ffffffff8882b820) (ffffffff) IP Options Strip (in) (ipopt_strip)
        1: -70000000 (ffffffff88804d70) (ffffffff) fwmonitor (i/f side)
        2: - 1fffff8 (ffffffff8882d050) (00000001) Stateless verifications (in) (asm)
        3: - 1fffff7 (ffffffff8886eb00) (00000001) fw multik misc proto forwarding
        4: - 1000000 (ffffffff88911d10) (00000003) SecureXL conn sync (secxl_sync)
        5:         0 (ffffffff887ceee0) (00000001) fw VM inbound  (fw)
        6:        10 (ffffffff887e3690) (00000001) fw accounting inbound (acct)
        7:  10000000 (ffffffff88910330) (00000003) SecureXL inbound (secxl)
        8:  70000000 (ffffffff88804d70) (ffffffff) fwmonitor (IP  side)
        9:  7f600000 (ffffffff88820bf0) (00000001) fw SCV inbound (scv)
        10:  7f730000 (ffffffff88a35fb0) (00000001) passive streaming (in) (pass_str)
        11:  7f750000 (ffffffff88c53260) (00000001) TCP streaming (in) (cpas)
        12:  7f800000 (ffffffff8882bbe0) (ffffffff) IP Options Restore (in) (ipopt_res)
out chain (12):
        0: -7f800000 (ffffffff8882b820) (ffffffff) IP Options Strip (out) (ipopt_strip)
        1: -70000000 (ffffffff88804d70) (ffffffff) fwmonitor (i/f side)
        2: - 1fffff0 (ffffffff88c534a0) (00000001) TCP streaming (out) (cpas)
        3: - 1ffff50 (ffffffff88a35fb0) (00000001) passive streaming (out) (pass_str)
        4: - 1f00000 (ffffffff8882d050) (00000001) Stateless verifications (out) (asm)
        5: -     1ff (ffffffff88e6eb80) (00000001) NAC Packet Outbound (nac_tag)
        6:         0 (ffffffff887ceee0) (00000001) fw VM outbound (fw)
        7:  10000000 (ffffffff88910330) (00000003) SecureXL outbound (secxl)
        8:  70000000 (ffffffff88804d70) (ffffffff) fwmonitor (IP side)
        9:  7f000000 (ffffffff887e3690) (00000001) fw accounting outbound (acct)
        10:  7f700000 (ffffffff88c53690) (00000001) TCP streaming post VM (cpas)
        11:  7f800000 (ffffffff8882bbe0) (ffffffff) IP Options Restore (out) (ipopt_res)
[Expert@bostestint-fwa:0]#



[Expert@myfwl-int01:0]# fw ctl chain
in chain (12):
        0: -7f800000 (ffffffff8882e8e0) (ffffffff) IP Options Strip (in) (ipopt_strip)
        1: - 1fffff8 (ffffffff88830110) (00000001) Stateless verifications (in) (asm)
        2: - 1fffff7 (ffffffff88871ce0) (00000001) fw multik misc proto forwarding
        3: - 1000000 (ffffffff88914f00) (00000003) SecureXL conn sync (secxl_sync)
        4:         0 (ffffffff887d1f00) (00000001) fw VM inbound  (fw)
        5:        10 (ffffffff887e66d0) (00000001) fw accounting inbound (acct)
        6:  10000000 (ffffffff88913520) (00000003) SecureXL inbound (secxl)
        7:  7f600000 (ffffffff88823cb0) (00000001) fw SCV inbound (scv)
        8:  7f730000 (ffffffff88a39b10) (00000001) passive streaming (in) (pass_str)
        9:  7f750000 (ffffffff88c574f0) (00000001) TCP streaming (in) (cpas)
        10:  7f800000 (ffffffff8882eca0) (ffffffff) IP Options Restore (in) (ipopt_res)
        11:  7fb00000 (ffffffff890210c0) (00000001) HA Forwarding (ha_for)
out chain (10):
        0: -7f800000 (ffffffff8882e8e0) (ffffffff) IP Options Strip (out) (ipopt_strip)
        1: - 1fffff0 (ffffffff88c57730) (00000001) TCP streaming (out) (cpas)
        2: - 1ffff50 (ffffffff88a39b10) (00000001) passive streaming (out) (pass_str)
        3: - 1f00000 (ffffffff88830110) (00000001) Stateless verifications (out) (asm)
        4: -     1ff (ffffffff88e76ae0) (00000001) NAC Packet Outbound (nac_tag)
        5:         0 (ffffffff887d1f00) (00000001) fw VM outbound (fw)
        6:  10000000 (ffffffff88913520) (00000003) SecureXL outbound (secxl)
        7:  7f000000 (ffffffff887e66d0) (00000001) fw accounting outbound (acct)
        8:  7f700000 (ffffffff88c57920) (00000001) TCP streaming post VM (cpas)
        9:  7f800000 (ffffffff8882eca0) (ffffffff) IP Options Restore (out) (ipopt_res)

[Expert@myfwl-int01:0]#


Stateful Features
  1. Streaming based application (web security etc)
  2. Sequence verification and translation
  3. Hide NAT (when server to client packets returning to firewall might not match the rule base, need to add explicit entry to connection table)
  4. Logging, Accounting, monitoring, etc
  5. Client and server identication
  6. Data connections.

fw tab -s
fw tab -t
[Expert@mytestint-fwa:0]# fw tab -t connections -s
HOST                  NAME                                ID #VALS #PEAK #SLINKS
localhost             connections                       8158    47  1223      49
[Expert@mytestint-fwa:0]#


[Expert@myvpn-fwb:0]# fw ctl chain

in chain (20):
        0: -7ffffff0 (ffffffff8ab199f0) (00000001) tcpt inbound (tcp_tun)
        1: -7f800000 (ffffffff88c17f30) (ffffffff) IP Options Strip (in) (ipopt_strip)
        2: -7d000000 (ffffffff8ab22f90) (00000003) vpn multik forward in
        3: - 2000000 (ffffffff8ab06690) (00000003) vpn decrypt (vpn)
        4: - 1fffff8 (ffffffff8ab121a0) (00000001) l2tp inbound (l2tp)
        5: - 1fffff6 (ffffffff88c19760) (00000001) Stateless verifications (in) (asm)
        6: - 1fffff5 (ffffffff88c51a60) (00000001) fw multik misc proto forwarding
        7: - 1fffff2 (ffffffff8ab30a60) (00000003) vpn tagging inbound (tagging)
        8: - 1fffff0 (ffffffff8ab05290) (00000003) vpn decrypt verify (vpn_ver)
        9: - 1000000 (ffffffff88c9c440) (00000003) SecureXL conn sync (secxl_sync)
        10:         0 (ffffffff88bc0650) (00000001) fw VM inbound  (fw)
        11:         1 (ffffffff88c3ad10) (00000002) wire VM inbound  (wire_vm)
        12:        10 (ffffffff88bd4400) (00000001) fw accounting inbound (acct)
        13:   2000000 (ffffffff8ab04900) (00000003) vpn policy inbound (vpn_pol)
        14:  10000000 (ffffffff88c9acd0) (00000003) SecureXL inbound (secxl)
        15:  7f600000 (ffffffff88c0d3f0) (00000001) fw SCV inbound (scv)
        16:  7f730000 (ffffffff88db3050) (00000001) passive streaming (in) (pass_str)
        17:  7f750000 (ffffffff88fbfbd0) (00000001) TCP streaming (in) (cpas)
        18:  7f800000 (ffffffff88c182f0) (ffffffff) IP Options Restore (in) (ipopt_res)
        19:  7fb00000 (ffffffff8936dca0) (00000001) HA Forwarding (ha_for)

out chain (18):
        0: -7f800000 (ffffffff88c17f30) (ffffffff) IP Options Strip (out) (ipopt_strip)
        1: -78000000 (ffffffff8ab22f70) (00000003) vpn multik forward out
        2: - 1ffffff (ffffffff8ab04140) (00000003) vpn nat outbound (vpn_nat)
        3: - 1fffff0 (ffffffff88fbfa50) (00000001) TCP streaming (out) (cpas)
        4: - 1ffff50 (ffffffff88db3050) (00000001) passive streaming (out) (pass_str)
        5: - 1ff0000 (ffffffff8ab30a60) (00000003) vpn tagging outbound (tagging)
        6: - 1f00000 (ffffffff88c19760) (00000001) Stateless verifications (out) (asm)
        7: -     1ff (ffffffff891d42e0) (00000001) NAC Packet Outbound (nac_tag)
        8:         0 (ffffffff88bc0650) (00000001) fw VM outbound (fw)
        9:         1 (ffffffff88c3ad10) (00000002) wire VM outbound  (wire_vm)
        10:   2000000 (ffffffff8ab04150) (00000003) vpn policy outbound (vpn_pol)
        11:  10000000 (ffffffff88c9acd0) (00000003) SecureXL outbound (secxl)
        12:  1ffffff0 (ffffffff8ab12ee0) (00000001) l2tp outbound (l2tp)
        13:  20000000 (ffffffff8ab07120) (00000003) vpn encrypt (vpn)
        14:  60000000 (ffffffff8ab1a130) (00000001) tcpt outbound (tcp_tun)
        15:  7f000000 (ffffffff88bd4400) (00000001) fw accounting outbound (acct)
        16:  7f700000 (ffffffff88fc2070) (00000001) TCP streaming post VM (cpas)
        17:  7f800000 (ffffffff88c182f0) (ffffffff) IP Options Restore (out) (ipopt_res)

[Expert@myvpn-fwb:0]#


Inbound
-------
NIC
Wireside Acct/Virtual Reass    IP Options Strip (in) (ipopt_strip)
VPN Dec
VPN verify
VM/NAT
Accounting
VPN Policy
FG Policy
IQ Engine
RTM/E2E
TCP/IP



Outbound
-------
TCP/IP
Virtual Reass/Wireside Acct
VM/NAT
VPN Policy
FG Policy
VPN Enc
IQ Engine
Accounting
RTM/E2E
NIC


Relative Positions

To Insert fw monitor after TCP streaming (out) for the outbound chain, execute the following command
fw monitor -po -0x1ffffe0


[Expert@mytestint-fwa:0]# fw ctl chain
in chain (11):
        0: -7f800000 (ffffffff8882b820) (ffffffff) IP Options Strip (in) (ipopt_strip)
        1: - 1fffff8 (ffffffff8882d050) (00000001) Stateless verifications (in) (asm)
        2: - 1fffff7 (ffffffff8886eb00) (00000001) fw multik misc proto forwarding
        3: - 1000000 (ffffffff88911d10) (00000003) SecureXL conn sync (secxl_sync)
        4:         0 (ffffffff887ceee0) (00000001) fw VM inbound  (fw)
        5:        10 (ffffffff887e3690) (00000001) fw accounting inbound (acct)
        6:  10000000 (ffffffff88910330) (00000003) SecureXL inbound (secxl)
        7:  7f600000 (ffffffff88820bf0) (00000001) fw SCV inbound (scv)
        8:  7f730000 (ffffffff88a35fb0) (00000001) passive streaming (in) (pass_str)
        9:  7f750000 (ffffffff88c53260) (00000001) TCP streaming (in) (cpas)
        10:  7f800000 (ffffffff8882bbe0) (ffffffff) IP Options Restore (in) (ipopt_res)
out chain (10):
        0: -7f800000 (ffffffff8882b820) (ffffffff) IP Options Strip (out) (ipopt_strip)
        1: - 1fffff0 (ffffffff88c534a0) (00000001) TCP streaming (out) (cpas)
        2: - 1ffff50 (ffffffff88a35fb0) (00000001) passive streaming (out) (pass_str)
        3: - 1f00000 (ffffffff8882d050) (00000001) Stateless verifications (out) (asm)
        4: -     1ff (ffffffff88e6eb80) (00000001) NAC Packet Outbound (nac_tag)
        5:         0 (ffffffff887ceee0) (00000001) fw VM outbound (fw)
        6:  10000000 (ffffffff88910330) (00000003) SecureXL outbound (secxl)
        7:  7f000000 (ffffffff887e3690) (00000001) fw accounting outbound (acct)
        8:  7f700000 (ffffffff88c53690) (00000001) TCP streaming post VM (cpas)
        9:  7f800000 (ffffffff8882bbe0) (ffffffff) IP Options Restore (out) (ipopt_res)
[Expert@mytestint-fwa:0]#
[Expert@mytestint-fwa:0]#
[Expert@mytestint-fwa:0]#

[Expert@mytestint-fwa:0]#  fw monitor -po -0x1ffffe0


[Expert@mytestint-fwa:0]# fw ctl chain
in chain (13):
        0: -7f800000 (ffffffff8882b820) (ffffffff) IP Options Strip (in) (ipopt_strip)
        1: -70000000 (ffffffff88804d70) (ffffffff) fwmonitor (i/f side)
        2: - 1fffff8 (ffffffff8882d050) (00000001) Stateless verifications (in) (asm)
        3: - 1fffff7 (ffffffff8886eb00) (00000001) fw multik misc proto forwarding
        4: - 1000000 (ffffffff88911d10) (00000003) SecureXL conn sync (secxl_sync)
        5:         0 (ffffffff887ceee0) (00000001) fw VM inbound  (fw)
        6:        10 (ffffffff887e3690) (00000001) fw accounting inbound (acct)
        7:  10000000 (ffffffff88910330) (00000003) SecureXL inbound (secxl)
        8:  70000000 (ffffffff88804d70) (ffffffff) fwmonitor (IP  side)
        9:  7f600000 (ffffffff88820bf0) (00000001) fw SCV inbound (scv)
        10:  7f730000 (ffffffff88a35fb0) (00000001) passive streaming (in) (pass_str)
        11:  7f750000 (ffffffff88c53260) (00000001) TCP streaming (in) (cpas)
        12:  7f800000 (ffffffff8882bbe0) (ffffffff) IP Options Restore (in) (ipopt_res)
out chain (12):
        0: -7f800000 (ffffffff8882b820) (ffffffff) IP Options Strip (out) (ipopt_strip)
        1: - 1fffff0 (ffffffff88c534a0) (00000001) TCP streaming (out) (cpas)
        2: - 1ffffe0 (ffffffff88804d70) (ffffffff) fwmonitor (i/f side)
        3: - 1ffff50 (ffffffff88a35fb0) (00000001) passive streaming (out) (pass_str)
        4: - 1f00000 (ffffffff8882d050) (00000001) Stateless verifications (out) (asm)
        5: -     1ff (ffffffff88e6eb80) (00000001) NAC Packet Outbound (nac_tag)
        6:         0 (ffffffff887ceee0) (00000001) fw VM outbound (fw)
        7:  10000000 (ffffffff88910330) (00000003) SecureXL outbound (secxl)
        8:  70000000 (ffffffff88804d70) (ffffffff) fwmonitor (IP side)
        9:  7f000000 (ffffffff887e3690) (00000001) fw accounting outbound (acct)
        10:  7f700000 (ffffffff88c53690) (00000001) TCP streaming post VM (cpas)
        11:  7f800000 (ffffffff8882bbe0) (ffffffff) IP Options Restore (out) (ipopt_res)
[Expert@mytestint-fwa:0]#


Here is an example where fw monitor is inserted as a chain module at pre-inbound  position 4

[Expert@mytestint-fwa:0]# fw monitor -pi 4

[Expert@mytestint-fwa:0]# fw ctl chain
in chain (13):
        0: -7f800000 (ffffffff8882b820) (ffffffff) IP Options Strip (in) (ipopt_strip)
        1: - 1fffff8 (ffffffff8882d050) (00000001) Stateless verifications (in) (asm)
        2: - 1fffff7 (ffffffff8886eb00) (00000001) fw multik misc proto forwarding
        3: - 1000000 (ffffffff88911d10) (00000003) SecureXL conn sync (secxl_sync)
        4: -       1 (ffffffff88804d70) (ffffffff) fwmonitor (i/f side)
        5:         0 (ffffffff887ceee0) (00000001) fw VM inbound  (fw)
        6:        10 (ffffffff887e3690) (00000001) fw accounting inbound (acct)
        7:  10000000 (ffffffff88910330) (00000003) SecureXL inbound (secxl)
        8:  70000000 (ffffffff88804d70) (ffffffff) fwmonitor (IP  side)
        9:  7f600000 (ffffffff88820bf0) (00000001) fw SCV inbound (scv)
        10:  7f730000 (ffffffff88a35fb0) (00000001) passive streaming (in) (pass_str)
        11:  7f750000 (ffffffff88c53260) (00000001) TCP streaming (in) (cpas)
        12:  7f800000 (ffffffff8882bbe0) (ffffffff) IP Options Restore (in) (ipopt_res)
out chain (12):
        0: -7f800000 (ffffffff8882b820) (ffffffff) IP Options Strip (out) (ipopt_strip)
        1: -70000000 (ffffffff88804d70) (ffffffff) fwmonitor (i/f side)
        2: - 1fffff0 (ffffffff88c534a0) (00000001) TCP streaming (out) (cpas)
        3: - 1ffff50 (ffffffff88a35fb0) (00000001) passive streaming (out) (pass_str)
        4: - 1f00000 (ffffffff8882d050) (00000001) Stateless verifications (out) (asm)
        5: -     1ff (ffffffff88e6eb80) (00000001) NAC Packet Outbound (nac_tag)
        6:         0 (ffffffff887ceee0) (00000001) fw VM outbound (fw)
        7:  10000000 (ffffffff88910330) (00000003) SecureXL outbound (secxl)
        8:  70000000 (ffffffff88804d70) (ffffffff) fwmonitor (IP side)
        9:  7f000000 (ffffffff887e3690) (00000001) fw accounting outbound (acct)
        10:  7f700000 (ffffffff88c53690) (00000001) TCP streaming post VM (cpas)
        11:  7f800000 (ffffffff8882bbe0) (ffffffff) IP Options Restore (out) (ipopt_res)
[Expert@bostestint-fwa:0]#



fw debug fwm   - view fem.elg to find issues as SIC, Misconfigured rules, GUI Client problems and improper information. 


Use the command fw ctl chain to study chain module behavior.  Observe how policy changes impact the chain
Use the command fw debug fwm on  and review the file fwm.elg  to find such issues as SIC, mis configured rules, GUI client connectivity problems and improperly entered information.

Studying Chain Module behavior
Inbound and outbound
Independent and redundant
Unique inspection
fw ctl chain


fw monitor  inspection points
inbound  minus   - coming in
Inbound      - about to leave inbound
outbound  minus  - coming in
Outbound - about to leave outbound

Fw monitor –pi  -vpn_ver –o monitor




Process of the Inspection Engine


Start -> NIC - Decision New Connection - No  - to TCP/IPStack
Yes  ->  Decision Packet Match Rule  - Yes - > Process  Log/Alert  -> Decision Pass Packet -Yes -to TCP/IPStack
No  -> Decision Packet Rejected by Rule - Yes -> Process Send NAC - STOP
No - Drop the packet

Decision Packet Match Rule  - NO - > Process is there another rule?   -> NO -> Drop the packet ->Stop

Decision Packet Match Rule  -  Yes - > Process  Packet Match Rule - Yes/NO




Packet Flow Through Checkpoint Firewall

Checkpoint process the packet in the ingress and the egress using two CHAINS. 


Basic:

Physical layer - ingress interface
Data Link Layer/Ethernet
Inspect Driver [inspect Engine]
Network Layer/IP Routing
Inspect Driver
Data Link Layer/Ethernet
Physical layer - egress interface


Advance:

1. NIC hardware
-The network card receives electrical signalling from the link partner.

2. NIC driver
-Sanity checks
-The NIC hardware decodes the signal and passes it to the operating system's NIC driver via the PCI bus
-The frame is converted to an mbuf entry and the frame headers are stored for later use.
-NIC driver hands off the data to the operating system's mbuf memory space

3. Operating system IP protocol stack
-The OS performs sanity checks on the packet
-Hand off to SXL if enabled, or to Firewall Kernel if not

4. SecureXL (if enabled)
-SXL lookup is performed, if it matches, bypass the firewall kernel and proceed with (Operating system IP protocol stack, outbound side)

5. Firewall Kernel (inbound processing)
-FW Monitor starts here (so, perhaps you need to disable secureXL [fwaccel offCAUTION... )
-Connection state lookups, some protocol inspection, rulebase processing, antispoofing lookups etc
-Processing order can be seen via fw ctl chain
-Bypass complex inspection if not needed

6. Complex protocol inspection (AV is an example)
-Leave the kernel and process under userland.
-Enters back at this same stage if the traffic passed

(inbound processing stops here)

(outbound processing starts here)

7. Firewall Kernel 
-Route lookup
-Check Point sanity checks etc
-FW Monitor ends here
-Pass to operating system

8. Operating system IP protocol stack
-The OS performs sanity checks on the packet
-Pass the mbuf to the NIC driver for the appropriate outbound interface

9. NIC driver
-Tag the packet as an ethernet frame by adding MAC addresses for source and destination

10. NIC hardware
-The NIC hardware encodes the signal and transmits it via wire




                                                       Fig. Ctl Chain

TIP: 
1. DST NAT can happen between i & I (when client side NAT enabled [DEFAULT]) or between o & O (server side NAT) 
2. SRC NAT will happen immediately after o (after routing) regardless the client side NAT enabled or not.

  • Static NAT - One to one translation
  • Hide/Dynamic NAT - Allows you to NAT multiple IPs behind one IP/Interface
  • Automatic NAT - Quick basic address NAT translation.  
  • Manual NAT - Allows greater flexibility over automatic NAT; it is preferred over Automatic NAT. If you are using manual rules, you must configure proxy ARPs to associate the translated IP address with the MAC address of the Security Gateway interface that is on the same network as the translated addresses.Configuration depends of  OS.
  • ['Global Properties' - go to 'NAT' - go to 'Automatic NAT rules'
  • Server Side NAT - destination is NAT`d by the outbound kernel
  • Client Side NAT  - destination is NAT`d by the inbound kernel 
  • ['Global Properties' - go to 'NAT' - go to 'Manual NAT rules'


Between all the steps there are queues. These queues accumulate packets and on intervals flush them to the next step. All of this happens very very quickly in small CPU time slices.

The INSPECT engine itself is more to do specifically with protocol inspection rather than all of the other steps. INSPECT runs traffic against definitions, if the definitions match it usually means that it hit a protection and the appropriate action is to (drop, log) the traffic.


VM is used for referring firewall Virtual Machine (VM, and it is CP terminology): 
i (PREIN) – inbound direction before firewall VM 
I (POSTIN) – inbound direction after firewall VM.
o (PREOUT) – outbound direction before firewall VM,
O (POSTOUT) – outbound direction after firewall VM.



                                                     Fig. fw monitor in the Chain. 


TIP: Source NAT occurs after outbound inspection (o), not between (I) and (o), so if we are doing src nat we won't see (O), the same will happen if the traffic flows via a vpn.




(00000001) new processed flows
(00000002) previous processed flows
(00000003) ciphered traffic
(ffffffff) Everything

[Expert@firewall]# fw ctl chain
in chain (19):
0: -7f800000 (f27f9c20) (ffffffff) IP Options Strip (in) (ipopt_strip)
1: -7d000000 (f1ee6000) (00000003) vpn multik forward in
2: - 2000000 (f1ecad30) (00000003) vpn decrypt (vpn)
3: - 1fffff8 (f1ed7550) (00000001) l2tp inbound (l2tp)
4: - 1fffff6 (f27faff0) (00000001) Stateless verifications (in) (asm)
5: - 1fffff5 (f2c6b240) (00000001) fw multik VoIP Forwarding
6: - 1fffff2 (f1ef30b0) (00000003) vpn tagging inbound (tagging)
7: - 1fffff0 (f1ecbb20) (00000003) vpn decrypt verify (vpn_ver)
8: - 1000000 (f2849190) (00000003) SecureXL conn sync (secxl_sync)
9: 0 (f27af6b0) (00000001) fw VM inbound (fw)
10: 1 (f2812680) (00000002) wire VM inbound (wire_vm)
11: 2000000 (f1ecdff0) (00000003) vpn policy inbound (vpn_pol)
12: 10000000 (f284e9e0) (00000003) SecureXL inbound (secxl)
13: 21500000 (f3b2c940) (00000001) RTM packet in (rtm)
14: 7f600000 (f27f0910) (00000001) fw SCV inbound (scv)
15: 7f730000 (f2928620) (00000001) passive streaming (in) (pass_str)
16: 7f750000 (f2a3bbf0) (00000001) TCP streaming (in) (cpas)
17: 7f800000 (f27f9fb0) (ffffffff) IP Options Restore (in) (ipopt_res)
18: 7fb00000 (f2a07540) (00000001) HA Forwarding (ha_for)
out chain (16):
0: -7f800000 (f27f9c20) (ffffffff) IP Options Strip (out) (ipopt_strip)
1: -78000000 (f1ee5fe0) (00000003) vpn multik forward out
2: - 1ffffff (f1eccbd0) (00000003) vpn nat outbound (vpn_nat)
3: - 1fffff0 (f2a3ba70) (00000001) TCP streaming (out) (cpas)
4: - 1ffff50 (f2928620) (00000001) passive streaming (out) (pass_str)
5: - 1ff0000 (f1ef30b0) (00000003) vpn tagging outbound (tagging)
6: - 1f00000 (f27faff0) (00000001) Stateless verifications (out) (asm)
7: 0 (f27af6b0) (00000001) fw VM outbound (fw)
8: 1 (f2812680) (00000002) wire VM outbound (wire_vm)
9: 2000000 (f1ecdaa0) (00000003) vpn policy outbound (vpn_pol)
10: 10000000 (f284e9e0) (00000003) SecureXL outbound (secxl)
11: 1ffffff0 (f1ed71e0) (00000001) l2tp outbound (l2tp)
12: 20000000 (f1ecce40) (00000003) vpn encrypt (vpn)
13: 24000000 (f3b2c940) (00000001) RTM packet out (rtm)
14: 7f700000 (f2a3b810) (00000001) TCP streaming post VM (cpas)
15: 7f800000 (f27f9fb0) (ffffffff) IP Options Restore (out) (ipopt_res)


[Expert@HostName]# fw ctl chain
Example:
in chain (17):
0: -7f800000 (f206df90) (ffffffff) IP Options Strip (in) (ipopt_strip)
1: - 2000000 (f149dd70) (00000003) vpn decrypt (vpn)
2: - 1fffff8 (f14a8b20) (00000001) l2tp inbound (l2tp)
3: - 1fffff6 (f206f290) (00000001) Stateless verifications (in) (asm)
4: - 1fffff2 (f14c4940) (00000003) vpn tagging inbound (tagging)
5: - 1fffff0 (f149bc10) (00000003) vpn decrypt verify (vpn_ver)
6: - 1000000 (f20c4980) (00000003) SecureXL conn sync (secxl_sync)
7: 0 (f201df50) (00000001) fw VM inbound (fw)
8: 1 (f2087ed0) (00000002) wire VM inbound (wire_vm)
9: 10 (f202f610) (00000001) fw accounting inbound (acct)
10: 2000000 (f149eaf0) (00000003) vpn policy inbound (vpn_pol)
11: 10000000 (f20ca740) (00000003) SecureXL inbound (secxl)
12: 7f600000 (f20646b0) (00000001) fw SCV inbound (scv)
13: 7f730000 (f21b11f0) (00000001) passive streaming (in) (pass_str)
14: 7f750000 (f231c540) (00000001) TCP streaming (in) (cpas)
15: 7f800000 (f206e320) (ffffffff) IP Options Restore (in) (ipopt_res)
16: 7fb00000 (f22de3b0) (00000001) HA Forwarding (ha_for)
out chain (15):
0: -7f800000 (f206df90) (ffffffff) IP Options Strip (out) (ipopt_strip)
1: - 1ffffff (f149cea0) (00000003) vpn nat outbound (vpn_nat)
2: - 1fffff0 (f231c3c0) (00000001) TCP streaming (out) (cpas)
3: - 1ffff50 (f21b11f0) (00000001) passive streaming (out) (pass_str)
4: - 1ff0000 (f14c4940) (00000003) vpn tagging outbound (tagging)
5: - 1f00000 (f206f290) (00000001) Stateless verifications (out) (asm)
6: 0 (f201df50) (00000001) fw VM outbound (fw)
7: 1 (f2087ed0) (00000002) wire VM outbound (wire_vm)
8: 2000000 (f149e5a0) (00000003) vpn policy outbound (vpn_pol)
9: 10000000 (f20ca740) (00000003) SecureXL outbound (secxl)
10: 1ffffff0 (f14a87b0) (00000001) l2tp outbound (l2tp)
11: 20000000 (f149d110) (00000003) vpn encrypt (vpn)
12: 7f000000 (f202f610) (00000001) fw accounting outbound (acct)
13: 7f700000 (f231e5b0) (00000001) TCP streaming post VM (cpas)
14: 7f800000 (f206e320) (ffffffff) IP Options Restore (out) (ipopt_res)

ModuleExplanation
KissKernel I/s 
Kissflow Kernel I/s communication with other modules and policy enforcement 
fw Firewall, the most common module
h323 H323 VoIP traffic module 
Multik CoreXL module 
uc Usercheck module 
dlpkData Loss Prevention module 
cluster ClusterXL module 
CPAS Check Point Active Streaming (unlike PSL) 
cmi_loader Signature loading for security blades 
NRB Next Rule Base, security blades rulebase 
SGENSecurity Gateway Enforcement (stateful)
RAD_KERNEL Very important I/S, DNS resolving, string matching, md5sum check against database and 3rd party, resource categorization 
WS Web Security module - HTTP inspection by all blades
APPI Application Control module
CI Content Inspection (Anti-Virus, Anti-Bot )
RTMReal time monitoring 
VPN VPN module
SFT Secure File Type 
Common traffic flags in fw module



ModuleExplanation
drop the connection that was dropped, the function in the firewall that dropped it and the reason 
conn information from the connections table (everything that can be seen with this command #fw tab -t 8158) . Direction of packets in connections, source/destination of IP and ports, service type, handler, timeout, the rule that this packet is matched on, ETC 
ld this flag refers to the kernel dynamic tables infrastructure. It reads and write everything that is written to any kernel table. Adding this flag to the debug syntax may cause the machine high CPU and can even cause the machine to hang. Check load before performing this debug on production environment. 
nat basic NAT information 
xlate the firewall NAT infrastructure - basic NAT information + NAT cache 
xltrc NAT additional information, print NAT rulebase information and more information from the NAT kernel table (fwx_alloc) 
packetinformation on actions performed on packet - like accept, drop, fragment, inspection 
packval advanced information of the packet's header. stateless verifications - sequences, fragments, translations and other header changes and verifications. 
 vmshows actions of all the virtual chains that can be seen with the command: # fw ctl chain on the traffic that go through the chain handle function fw_filter_chain 
tcpstr TCP streaming mechanism, organize TCP packets by order and checks protocol messages. debug the PSL I/S. (passive streaming layer) 
chainchain cookie information, chain modules 
chainfwd chain forwarding - related to fwha_perform_chain_forwarding global kernel variable (clusterXL only) 
cookie the kernel saves information and holds connections for inspection, this flag will show virtual de-fragmentation and cookie issues (cookies in the data structure holding the packets) 
hold holding mechanism and all packets being held / released 


ModuleExplanation
aspiiAccelerated Stateful Protocol Inspection Infrastructure (INPSECT streaming) 
spii Stateful Protocol Inspection Infrastructure 
cmi Context Management Infrastructure - IPS signature manager
advp IPS and Application Control signatures, contexts and patterns 
synatk IPS protection "SYN Attack" (SYNDefender)


Kernel debug command line usage
Defaulting all kernel debug properties:
[Expert@HostName]# fw ctl debug 0
Define a kernel debug buffer:
[Expert@HostName]# fw ctl debug -buf <buffer_size>
If the machine has enough memory, it is recommended to set the maximum buffer of 32768.
The fact that the buffer size is 8000 or 32768 will not affect the file size.
The kernel writes the debug messages to the buffer. A user mode process named 'kdebug' writes the messages from the buffer to the file.
Check which debug flags are enabled:
[Expert@HostName]# fw ctl debug -m
Check which debug flags are on for specific module:
[Expert@HostName]# fw ctl debug -m <module_name>
Set the debugging flags for the required module:
[Expert@HostName]# fw ctl debug -m <module_name> + flag1 flag2 flag3
UnSet the debugging flags from the module:
[Expert@HostName]# fw ctl debug -m <module_name> - flag1 flag2 flag3
Add Timestamp (necessary to match the traffic capture to kernel debug):
[Expert@HostName]# fw ctl kdebug -T -f > /var/log/debug.txt
Dividing the debug output file into multiple file limited to certain size:
[Expert@HostName]# fw ctl kdebug -f -o <file_name> -m <number> -s <size>

Kernel tables
The kernel writes all of its actions to relevant tables. Every kernel table has an assigned name and assigned kernel ID, and it holds specific information.
The forwarding, handling and decisions will be done based on informatin that is stored in different kernel tables.

Kernel tables command line usage
To view the kernel tables, run:
[Expert@HostName]# fw tab
To view a certain table, run:
[Expert@HostName]# fw tab -t <Table_Name | Table_ID>
To view all entries (unlimited number) in the table:
[Expert@HostName]# fw tab -t <Table_Name | Table_ID> -u
To view the table entries translated from HEX to DEC format:
[Expert@HostName]# fw tab -t <Table_Name | Table_ID> -u -f
To export the table contents to a file:
[Expert@HostName]# fw tab -t <Table_Name | Table_ID> -u > /var/log/file_name.txt
To delete all entries from a certain kernel table (all involved connections will be immediately lost and disconnected):
[Expert@HostName]# fw tab -t <Table_Name | Table_ID> -x -y
Important Note: This operation deletes all information from the table. Do NOT use this command in production environment, as it might cause a traffic outage.
To delete specific entry:
[Expert@HostName]# fw tab -t <Table_Name | Table_ID> -x -e <Entry_ID | Tuple>
Display a summary about entries in certain table:
[Expert@HostName]# fw tab -t <Table_Name | Table_ID> -s
Display a summary about entries in all tables:
[Expert@HostName]# fw tab -s

Connections Table
The most used kernel table is the Connections Table. Every connection that passed the firewall is written to the Connections Table. Timeouts, rule numbers, symbolic links, state decisions, all are written and decided by this table.
Real entry contains the following:
<DIRECTION,6-tuple-key;r_ctype,r_cflags,rule,service_id,handler,uuid1,uuid2,uuid3,uuid4,ifncin,ifncout,ifnsin,ifnsout,bits1,bits2,connection_module_kbufs@ttl/timeout>
Tuple is the basic information on the connection that is written to the connections table.
<direction,src_ip,src_port,dst_ip,dst_port,protocol>

Example 1
:
<00000001, C21D2B59, 0000AF6B, C21D24D9, 00000035, 00000011> - This is the first connection that was written to the Connections Table. The culmination of all six values named "six tuple" or - connections table EntryID
00000001 - first value of the tuple indicates the direction (00000001 - outbound)
C21D2B59 - Source IP - in our example, Security Gateway (in Dec = 194.29.43.89)
0000AF6B - Source port (in Dec = 44907)
C21D24D9 - Destination IP - in our example, DNS Server (in Dec = 194.29.36.217)
00000053 - Destination port (in Dec = 53)
00000017 - Protocol (in Dec = 11 = UDP)

Example 2:
The reply of the DNS request connection will be:
<00000000, C21D24D9, 00000035, C21D2B59, 0000AF6B, 00000011>
00000000 - first value of the tuple indicates the direction (00000000 - inbound)
C21D24D9 - Source IP- in our example, DNS Server (in Dec = 194.29.36.217)
00000053 - Source port (in Dec = 53)
C21D2B59 - Destination IP - in our example, Security Gateway (in Dec = 194.29.43.89)
0000AF6B - Destination port (in Dec = 44907)

00000017 - Protocol (in Dec = 11 = UDP)


Kernel debug analysis
In terms of analysis for the debug, the ability to analyze the debug depends on how familiar you are with the expected behavior.
Here is a basic flow of accepted inbound handling of a connection:

  1. fwconn_lookup: connection lookup, the Security Gateway wants to see if there is a match for the connection that just arrived via the interface

    [-- Stateful VM inbound: Entering (1244447195) --];

    Before VM:

    <192.168.121.1:4461 -> 10.0.121.13:23 IPP 6> (len-48) TCP flags-0x2 (SYN), seq-55a92b89, ack-0, data end-55a92b8a (ifn-0) (first seen) (looked up)
  2. Since the connection is "not found in connections table", the next step is to record it

    fwconn_record_conn: record conn
  3. Stateful inspection - SYN:

    fwconn_record_conn: SYN packet. Turn on SYN_SENT flag;

    Since we have enabled the syn_sent flag, only RST or SYN-ACK are allowed. In the phase for example, if any other packet besides RST or SYN-ACK, it will be dropped on "out of state".
  4. Getting further information from the Security Gateway connections table:

    fwconn_get_service_timeout: Getting default protocol
  5. Setting partial_handle bit: loading further code handling.

    fwconn_record_conn: setting PARTIAL_SETUP flag
  6. Creating REAL entry

    fwconn_record_conn: created real

    <dir 0, 192.168.121.1:4461 -> 10.0.121.13:23 IPP 6>
  7. After we checked that stateful inspection is ok, rulebase matching:

    fw_handle_first_packet: Rulebase returned
  8. NAT rulebase match:

    fw_xlate_match conn-<192.168.121.1:4461 -> 10.0.121.13:23 IPP 6>;
    fw_xlate_find_all_matches_old: No match for conn
  9. Creating symbolic links (after NAT decision)

    fwconn_init_links: Creating links

    fwconn_init_links: Creating links (inbound). One way links-0, Replies from any-0;

    ;fwconn_set_links_inbound: create link cls_o <dir 1, 10.0.121.13:23 -> 192.168.121.1:4461 IPP 6> -> <dir 0, 192.168.121.1:4461 -> 10.0.121.13:23 IPP 6>(0x5);

    fwconn_set_links_inbound: create link srs_o <dir 1, 192.168.121.1:4461 -> 10.0.121.13:23 IPP 6> -> <dir 0, 192.168.121.1:4461 -> 10.0.121.13:23 IPP 6>(0x2);
  10. Handling inbound is finished:

    fwconn_init_links: connection completed, unset PARTIAL_SETUP bit;
  11. fw_tcp_state_update: first SYN
  12. VM Final action-ACCEPT;
  13. ----- Stateful VM inbound Completed -----
  14. fwconn_lookup (outbound.......)

    <10001,44000,1,1ae,0,4a2cc1db,0,4779a8c0,ffff,0,ffffffff,ffffffff,ffffffff,2000000,0,0,0,0,0,0,0,0,0,0,0,0,7c9c0800,0,0>

    found in connections table dir-1;

    After this phase, the packet is entering the outbound chains and the handling will continue in accordance with the chain forwarding.
  15. [-- Stateful VM outbound: Entering (1244447195) --]; 
  16. This point represents the chains between I and o.

    Before VM:

    <192.168.121.1:4461 -> 10.0.121.13:23 IPP 6> (len-48) TCP flags-0x2 (SYN), seq-55a92b89, ack-0, data end-55a92b8a (ifn-1) (looked up) ;

    fwconn_chain_update_anti_spoofing_cache: updating anti-spoofing interface info. [cin-0, cout--1, sin--1, sout--1], dir-1, cdir-1, new ifn-1, spoof offset-3, previous ifn--1;
  17. NAT rulebase match, cache and Slinks, after we have routing info.

    fw_xlate_match: conn-<192.168.121.1:4461 -> 10.0.121.13:23 IPP 6>; ; 8Jun2009 10:46:35.420110;fw_xlate_match: cache hit!;
  18. fw_conn_post_inspect: Setting SRS_OUTBOUND_SEEN flag;
  19. Syn accepted, written, inspected by stateful inspection, rulebase and anti-spoofing.

    fw_filter_chain: Final switch, action-ACCEPT;

    After VM:

    <192.168.121.1:4461 -> 10.0.121.13:23 IPP 6> (len-48) TCP flags-0x2 (SYN), seq-55a92b89, ack-0, data end-55a92b8a

    VM Final action-ACCEPT
    ----- Stateful VM outbound Completed -----

When troubleshooting and trying to understand which chain is causing a problem on the Security Gateway, use the following command:
[Expert@HostName]# fw monitor -e "accept;" -p all
The '-p all' flag will show all the chains, through which the traffic passed.

To see it in kernel debug, use module fw with the flags monitorallvmchain and chainfwd
Note: These flags generate large number of messages, hence will cause high load on CPU.
To see all active modules on the Security Gateway, run:
[Expert@HostName]# fw ctl debug -m

Debugging Modules

When debugging the kernel, it is important to first understand the flow of the packets and connections and understand the failure point; it is important to understand which module you need to debug.
Every module has its own flags that were designed to print specific debug information. It is important to know the common flags to be more accurate when debugging the kernel.


Common IPS flags in fw module

Kernel debug command line usage
Defaulting all kernel debug properties:
Define a kernel debug buffer:
If the machine has enough memory, it is recommended to set the maximum buffer of 32768.
The fact that the buffer size is 8000 or 32768 will not affect the file size.
The kernel writes the debug messages to the buffer. A user mode process named 'kdebug' writes the messages from the buffer to the file.
Check which debug flags are enabled:
[Expert@HostName]# fw ctl debug -m
Check which debug flags are on for specific module:
[Expert@HostName]# fw ctl debug -m <module_name>
Set the debugging flags for the required module:
[Expert@HostName]# fw ctl debug -m <module_name> + flag1 flag2 flag3
UnSet the debugging flags from the module:
[Expert@HostName]# fw ctl debug -m <module_name> - flag1 flag2 flag3
Add Timestamp (necessary to match the traffic capture to kernel debug):
[Expert@HostName]# fw ctl kdebug -T -f > /var/log/debug.txt
Dividing the debug output file into multiple file limited to certain size:
[Expert@HostName]# fw ctl kdebug -f -o <file_name> -m <number> -s <size>

Kernel tables
The kernel writes all of its actions to relevant tables. Every kernel table has an assigned name and assigned kernel ID, and it holds specific information.

Kernel tables command line usage
To view the kernel tables, run:
[Expert@HostName]# fw tab
To view a certain table, run:
[Expert@HostName]# fw tab -t <Table_Name | Table_ID>
To view all entries (unlimited number) in the table:
[Expert@HostName]# fw tab -t <Table_Name | Table_ID> -u
To view the table entries translated from HEX to DEC format:
[Expert@HostName]# fw tab -t <Table_Name | Table_ID> -u -f
To export the table contents to a file:
[Expert@HostName]# fw tab -t <Table_Name | Table_ID> -u > /var/log/file_name.txt
To delete all entries from a certain kernel table (all involved connections will be immediately lost and disconnected):
[Expert@HostName]# fw tab -t <Table_Name | Table_ID> -x -y
To delete specific entry:
[Expert@HostName]# fw tab -t <Table_Name | Table_ID> -x -e <Entry_ID | Tuple>
Display a summary about entries in certain table:
[Expert@HostName]# fw tab -t <Table_Name | Table_ID> -s
Display a summary about entries in all tables:
[Expert@HostName]# fw tab -s

Connections Table
The most used kernel table is the Connections Table. Every connection that passed the firewall is written to the Connections Table. Timeouts, rule numbers, symbolic links, state decisions, all are written and decided by this table.
Real entry contains the following:
<DIRECTION,6-tuple-key;r_ctype,r_cflags,rule,service_id,handler,uuid1,uuid2,uuid3,uuid4,ifncin,ifncout,ifnsin,ifnsout,bits1,bits2,connection_module_kbufs@ttl/timeout>
Tuple is the basic information on the connection that is written to the connections table.
<direction,src_ip,src_port,dst_ip,dst_port,protocol>

<00000001, C21D2B59, 0000AF6B, C21D24D9, 00000035, 00000011> - This is the first connection that was written to the Connections Table. The culmination of all six values named "six tuple" or - connections table EntryID
00000001 - first value of the tuple indicates the direction (00000001 - outbound)
C21D2B59 - Source IP - in our example, Security Gateway (in Dec = 194.29.43.89)
0000AF6B - Source port (in Dec = 44907)
C21D24D9 - Destination IP - in our example, DNS Server (in Dec = 194.29.36.217)
00000053 - Destination port (in Dec = 53)
00000017 - Protocol (in Dec = 11 = UDP)

Example 2:
The reply of the DNS request connection will be:
<00000000, C21D24D9, 00000035, C21D2B59, 0000AF6B, 00000011>
00000000 - first value of the tuple indicates the direction (00000000 - inbound)
C21D24D9 - Source IP- in our example, DNS Server (in Dec = 194.29.36.217)
00000053 - Source port (in Dec = 53)
C21D2B59 - Destination IP - in our example, Security Gateway (in Dec = 194.29.43.89)
0000AF6B - Destination port (in Dec = 44907)
00000017 - Protocol (in Dec = 11 = UDP)

Kernel debug analysis
In terms of analysis for the debug, the ability to analyze the debug depends on how familiar you are with the expected behavior.
Here is a basic flow of accepted inbound handling of a connection:

Currently as we see, the timeout is "25", as the handshake was not completed.
The SYN-ACK will go through the same inspection process, however the timeout will be also be "25". It will change only after the 3-way handshake will be completed.
In terms of stateful inspection, the only allowed flags according to the current state (SYN_SENT) are SYN-ACK and RST.

Every other packet will be dropped for "out of state".


Acceleration

Performance Pack is a software acceleration product installed on Security Gateways. Performance Pack uses SecureXL technology and other innovative network acceleration techniques to deliver wire-speed performance for Security Gateways.

In a SecureXL-enabled gateway, the firewall first uses the SecureXL API to query the SecureXL device and discover its capabilities. The firewall then implements a policy that determines which parts of what sessions are to be handled by the firewall, and which should be offloaded to the SecureXL device. When new sessions attempt to get established across the gateway, the first packet of each new session is inspected by the firewall to ensure that the connection is allowed by the security policy. As the packet is inspected, the firewall determines the required behavior for the session, and based on its policy it may then offload some or all of the session handling to the SecureXL device. Thereafter, the appropriate packets belonging to that session are inspected directly by the SecureXL device. The SecureXL device implements the security logic required for further analysis and handling of the traffic. If it identifies anomalies it then consults back with the firewall software and IPS engine. In addition, SecureXL provides a mode that allows for connection setup to be done entirely in the SecureXL device, thus providing extremely high session rate.



                                                                     Fig. FW Paths and core processing  

Medium Path is known as PXL (SXL + PSL [PSL passive Streaming Library IPS related])
Slow Path (Firewall Path) is known as F2F (Forwarded 2 Firewall)


SecureXL is implemented either in software (Core), or in hardware (SAM cards on Check Point 21000 appliances; ADP cards on IP Series appliances with CPUs inside). The Dispatcher + Performance Pack combination is known as SND (Secure Network Dispatch).

Fwaccell (Firewall Acceleation feature) is used to check/manage the SecureXL Acceleration Device 
SIM (SecureXL implementation Device) Affinity (Association with a Core) can be  managed automatically by checkpoint (each 60 seconds) or statically configured.
TIP: To achieve the best performance, pairs of interfaces carrying significant data flows (based on network topology) should be assigned to pairs of CPU cores on the same physical processor.

  • Active Streaming (CPAS) - Technology that sends streams of data to be inspected in the kernel, since more than a single packet at a time is needed in order to understand the application that is running (such as HTTP data). Active Streaming is Read- and Write-enabled, and works as a transparent proxy. Connections that pass through Active Streaming can not be accelerated by SecureXL.
  • Passive Streaming - Technology that sends streams of data to be inspected in the kernel, since more than a single packet at a time is needed in order to understand the application that is running (such as HTTP data). Passive Streaming is Read-only and it cannot hold packets, but the connections are accelerated by SecureXL.
  • Passive Streaming Library (PSL) - IPS infrastructure, which transparently listens to TCP traffic as network packets, and rebuilds the TCP stream out of these packets. Passive Streaming can listen to all TCP traffic, but process only the data packets, which belong to a previously registered connection. For more details, refer to sk95193 (ATRG: IPS).
  • PXL - Technology name for combination of SecureXL and PSL.
  • QXL - Technology name for combination of SecureXL and QoS (R77.10 and above).
  • F2F / F2Fed - Packets that can not be accelerated by SecureXL (refer to sk32578 (SecureXL Mechanism)) are Forwarded to Firewall.
  • F2P - Forward to PSL/Applications. Feature that allows to perform the PSL processing on the CPU cores, which are dedicated to the Firewall.


HyperThreading 

SMT: New in R77. Minimices Context change inside a physical core and it may improve performance. (or, may not f.e memory used is increased and connection table size can de reduced)

It is not supported in Open Servers.

It is strongly recommended to disable Hyper-Threading in BIOS when CoreXL is enabled (on Check Point appliances this is disabled, by default). Applies to Intel processors prior to "Intel Nehalem (Core i7)", where this technology was improved (called Simultaneous Multi-Threading, or Intel® Hyper-Threading)



Multicore 

Multi Core processing in Checkpoint is known as CoreXL. Thre are some limitations when CoreXL is enabled.


The following features/settings are not supported in CoreXL:
  1. Check Point QoS (Quality of Service)(1)
  2. 'Traffic View' in SmartView Monitor(2) (all other views are available)
  3. Route-based VPN
  4. IP Pool NAT(3) (refer to sk76800)
  5. IPv6(4)
  6. Firewall-1 GX
  7. Overlapping NAT
  8. SMTP Resource(3)
  9. VPN Traditional Mode (refer to VPN Administration Guide - Appendix B for converting a Traditional policy to a Community-Based policy)
If any of the above features/settings is enabled/configured in SmartDashboard, then CoreXL acceleration will be automatically disabled on the Gateway (while CoreXL is still enabled). In order to preserve consistent configuration, before enabling one of the unsupported features, deactivate CoreXL via 'cpconfig' menu and reboot the Gateway (in cluster setup, CoreXL should be deactivated on all members).
Notes:
  • (1) - supported on R77.10 and above (refer to sk98229)
  • (2) - supported on R75.30 and above
  • (3) - supported on R75.40 and above
  • (4) - supported on R75.40 and above on SecurePlatform/Gaia/Linux only


There are two main roles for CPUs applicable to SecureXL and CoreXL:
  • SecureXL and CoreXL dispatcher CPU (the SND - Secure Network Distributor)
    You can manually configure this using the sim affinity -s command. (Exception: cpmq if multiqueue)
  • CoreXL firewall instance CPU
    You can manually configure this using the fw ctl affinity command 

Traffic is processed by the CoreXL FW instances only when the traffic is not accelerated by SecureXL (if SecureXL is installed and enabled). So, if all the traffic is accelerated, we can have several fw instances idle.

  • CoreXL will improve performance with almost linear scalability in the following scenarios:
    • Much of the traffic can not be accelerated by SecureXL
    • Many IPS features enabled, which disable SecureXL functionality
    • Large rulebase
    • NAT rules
  • CoreXL will not improve performance in the following scenarios:
    • SecureXL accelerates much of the traffic
    • Traffic mostly consists of VPN (VPN traffic inspection occurs only in CoreXL FW instance #0)
    • Traffic mostly consists of VoIP (VoIP control connections are processed in only in CoreXLFW instance #0)

In some cases it may be advisable to change the distribution of kernel instances, the SND, and other processes, among the processing cores. This configuration change is known asPerformance Tuning. This is done by changing the affinities of different NICs (interfaces) and/or processes. However, to ensure CoreXL's efficiency, all interface traffic must be directed to cores not running kernel instances. Therefore, if you change affinities of interfaces or other processes, you will need to accordingly set the number of kernel instances and ensure that the instances run on other processing cores.


Automatic Mode — (default) Affinity is determined by analysis of the load on each NIC. If a NIC is not activated, Affinity is not set. NIC load is analyzed every 60 seconds.



Manual Mode — Configure Affinity settings for each interface: the processor numbers (separated by space) that handle this interface, or all. In Manual Mode, periodic NIC analysis is disabled

The default affinity setting for all interfaces is Automatic. Automatic affinity means that if Performance Pack is running, the affinity for each interface is automatically reset every 60 seconds, and balanced between available cores. If Performance Pack is not running, the default affinities of all interfaces are with one available core. In both cases, any processing core running a kernel instance, or defined as the affinity for another process, is considered unavailable and will not be set as the affinity for any interface. Poor decisions maybe done with automatic affinity.





                                                    Figure. Possible affinity setting. 






                                                  Figure. MultiCore Processing and packet flow paths.




SecureXL and CoreXL connection info exchage:

  • Connection offload - Firewall kernel passes the relevant information about the connection from Firewall Connections Table to SecureXL Connections Table.
    Note: In ClusterXL High Availability, the connections are not offloaded to SecureXL on Standby member.

  • Connection notification - SecureXL passes the relevant information about the accelerated connection from SecureXL Connections Table to Firewall Connections Table.

  • Partial connection - Connection that exists in the Firewall Connections Table, but not in the SecureXL Connections Table (versions R70 and above).
    • In Cluster HA - partial connections are offloaded when member becomes Active
    • In Cluster LS - partial connections are offloaded upon post-sync (only for NAT / VPN connections)
    Such connections must be offloaded to SecureXL, since packets in these connections must not be dropped.
    If a packet matched a partial connection in the outbound, then it should be dropped.

  • Delayed connection - Connection created from SecureXL Connection Templates without notifying the Firewall for a predefined period of time. The notified connections are deleted by the Firewall.