Training - VPN Troubleshooting

Locate Source of Encryption Failures


[Expert@a-gw:0]# vpn debug
 Usage: vpn debug < on [ DEBUG_TOPIC=level ] | off | ikeon [ -s size(Mb) ]| ikeoff | trunc [ DEBUG_TOPIC=level ] | truncon [ DEBUG_TOPIC=level ] | truncoff | timeon [ SECONDS ] | timeoff | ikefail [ -s size(Mb) ]| mon | moff | say [ string ] >
[Expert@a-gw:0]#

vpnd.elg

A few years ago I compiled a list of VPN debugs, error messages, and common gotchas. This information is relevant for Check Point NGX firewall, but is not a complete VPN Debugging Guide.

DEBUGGING INSTRUCTIONS:

From the command line ( if cluster, active member )

• vpn debug on
• vpn debug ikeon
• vpn tu
• select the option to delete IPSEC+IKE SAs for a given peer (gw)
• Try the traffic to bring up the tunnel
• vpn debug ikeoff
• vpn debug off

Log Files are

• $FWDIR/log/ike.elg
• $FWDIR/log/vpnd.elg

COMMON MESSAGES:

According to the Policy the Packet should not have been decrypted

• The networks are not defined properly or have a typo
• Make sure VPN domains under gateway A are all local to gateway A
• Make sure VPN domains under gateway B are all local to gateway B

Wrong Remote Address

Failed to match proposal

• sk21636 – cisco side not configured for compression

No response from peer

• check encryption domains.
• remote end needs a decrypt rule
• remote firewall not setup for encryption
• somethign is blocking communication between VPN endpoints
• Check UDP 500 and protocol 50

No Valid SA

• both ends need the same definition for the encrytpion domain.
• sk19243 – (LAST OPTION) use debedit objects_5_0.c, then add subnets/hosts in users.def
• likely phase2 settings
• cisco might say ‘no proxy id allowed”
• Disable NAT inside VPN community
• Support Key exchange for subnets is properly configured
• Make sure firewall external interface is in public IP in general properties

No Proposal chosen

• sk19243 – usually cuased when a peer does not agree to VPN Domain or subnet mask
• make sure that encryption and hash match as well in Phase 2 settings

Cannot Identify Peer (to encryption connection)

• sk22102 – rules refer to an object that is not part of the local firewalls encryption domain
• may have overlapping encryption domains
• 2 peers in the same domain
• sk18972 – explains overlapping

Invalid ID

• sk25893 – Gateway: VPN-> VPN Advanced, Clear “Support key exhcnage for subnets”, Install policy

Authentication Failure

Payload Malformed

• check pre shared secrets

RESPONDER-LIFETIME

• As seen in ike debugs, make sure they match on both ends

Invalid Certificate

• sk17106 – Remote side peer object is incorrectly configured
• sk23586 – nat rules are needed
• sk18805 – multiple issues, define a static nat, add a rule, check time
• sk25262 – port 18264 has problems
• sk32648 – port 18264 problems v2
• sk15037 – make sure gateway can communicate with management

No Valid CRL

• sk32721 – CRL has expired, and module can’t get a new valid CRL

AddNegotiation

• FW-1 is handling more than 200 key negotiations at once
• vSet maximum concurrent IKE connections

Could not get SAs from packet

FW MONITOR NOTES

• packet comes back i I o O
• packet will be ESP between o and O

BASIC STUFF TO CHECK IN THE CONFIGURATION:

Accept FW-1 Control Connections

VPN domains

• setup in the topology of that item
• using topology is recommended, but you must define
• looking for overlap, or missing networks.
• Check remote and local objects.

Encryption Domains

• your firewall contains your networks
• their firewall contains their networks

Rule Setup

• you need a rule for the originator.
• Reply rule is only required for 2 way tunnel

Preshared secret or certificate

• Make sure times are accurate

Security rulebase

• make sure there are rules to allow the traffic

Address Translation

• be aware that this will effect the Phase 2 negotiations
• most people disable NAT in the community

Community Properties

• Tunnel management, Phase1 Phase2 encrypt settings.

Link selection

Routing

• make sure that the destination is routed across the interface that you want it to encrypt on
• you need IP proto 50 and 51 fo IPSEC related traffic
• you need port 500 UDP for IKE
• netstat -rn and look for a single valid default route

Smartview Tracker Logs

• purple = encrypted
• red = dropped
• green = no encryption

TRADITIONAL MODE NOTES

• can’t VPN Route
• encryption happens when you hit explicit rule
• rules must be created

SIMPLIFIED MODE NOTES

• VPN Communities
• Encryption happens at rule 0
• rules are implied

CHECKLIST

• Define encryption domains for each site
• Define firewall workstation objects for each site
• Configure the gateway objects for the correct encryption domain
• Configure the extranet community with the appropriate gateways and objects
• Create the necessary encryption rules.
• Configure the encryption properties for each encryption rule.
• Install the security Policy

IKE PACKET MODE QUICK REFERENCE

• – > outgoing
• < – incoming

PHASE 1 (MAIN MODE)

• 1 > Pre shared Secrets, Encryption & hash Algorithims, Auth method, inititor cookie (clear text)
• 2 < agree on one encryption & hash, responder cookie (clear text)
• 3 > random numbers sent to prove identity (if it fails here, reinstall)
• 4 < random numbers sent to prove identity (if it fails here, reinstall)
• 5 > authentication between peers, peers ip address, certificates exchange, shared secrets, expired certs, time offsets
• 6 < peer has agreed to the proposal and has authenticated initiator, expired certs, time offsets

PHASE 2 (QUICK MODE)

• 1 > Use a subnet or a host ID, Encryption, hash, ID data
• 2 < agrees with it’s own subnet or host ID and encryption and hash
• 3 > completes IKE negotiation

GOOD SKS to KNOW

• sk31221 – The NGX Advanced Troubleshooting Reference Guide (ATRG)
• sk26362 – Troubleshooting MTU related issues
• sk30509 – Configuring VPN-1/FireWall-1
• sk31567 – What is ike.elg?
• sk20277 – “Tunnel failure, cannot find IPSec methods of the community (VPN Error code 01)” appears
• sk31279 – Files copied over encrypted tunnel displaying error: “network path is too deep”
• sk32648 – Site-to-site VPN using certificates issued by the ICA (Internal Certificate Authority) fails
• sk19243 – largest possible subnet even when the largest_possible_subnet option is set to false
• sk31619 – VPN tunnel is down troubleshooting
• sk19599 – how to edit user.def for largest possible subnets & host only

Understanding CoreXL

Understanding Check Point CoreXL Part 1

Understanding Checkpoint CoreXL Part 2

CoreXL

1. Multi-Core systems
2. Need for CoreXL
3. coreXL components
4. overview of CoreXL
5. CoreXL Performance
6. Affinity

The QUEST for Performance 

Improving Performance
- Hardware Vendors
- Reduce Cost

Performance via frequency Increase
-CPU clock Speed
-Physical Limitations

Multi CPU era
- Multiple processor solution
- New software needed  - the creating of CoreXL code


Checkpoint Code
- R65  first firewall code to support multi-core (Add support for multi-core)

- Linux
  Kernel 2.6
  Firest linux kernel to introduce multi-core


How firewall process packet before on  CoreXL

FW CODE
fw_filter() {
 fw_lock();
 fwchain_do();
 fw_unlock();
}

CPU1
Flow --- Packet comes in to CPU - CPU Sends it to FW Code - does a FW Lock - the a FWChain process packet  then do a fwunlock and forward packet

Firewall Kernel Instance

NIC - Dispatcher - CPU - FW1 Code

If you have more that 1 core ..

FW-0 CODE
fw_filter() {
 fw_lock();
 fwchain_do();
 fw_unlock();
}

CPU0


FW-1 CODE
fw_filter() {
 fw_lock();
 fwchain_do();
 fw_unlock();
}

CPU1


CPU 2  - Dispatcher  (acts like a load balance)

Packet  from NIC to dispatcher CPU2 (on same CPU) - make a decision where to send packet to CPU0 or CPU1  if rule allow packet then FW-0 or FW-1 code for processing

1st kernel instance


Dispatcher (dispatching table) - [fw0 Queue - fw0 ]- connection table and update dispatching table   == > IP STACT
parllel processing
independent inspection Kernals
Packet is sticky
Data locality  - data is local

1. CPU Affinity -  Unbound fw processes and rebound Dispatcher from 1 CPU to another 
2. SIM Affinity -  assign different Interfaces to specific CPU to process
3. Multi-queue  - IRQ assign to specific CPU -  2 queue bound to on port 

NIC -> Global Dispatcher Table -> Secure Network Dispatcher ->FW0 -4 firewall workers




3 Components

SecureXL
- Software/hardware Acceleration
- Packet rate and connection rate
- Certain traffic cannot be accelerated
 (fwaccel happens in Kernel and cannot do deep packet inspection )
SSL Inspection traffic will not be acceleration 
fwaccel  [on | off  | ver  | stats | conns | templates]
SND core handles acceleration

CoreXL
- Parallel security gateway kernels
- Leverage modern processor architectures
- Suited to medium path

ClusterXL
- Implement load-sharing (active/active) Cluster to increase throughput
- delivers high-availability/ security gateway

base.def  - add application ports etc
/opt/CPsuite-R77/fw1/lib/base.def

The CoreXL software architecture includes the Secure Network Distributor (SND). The SND is responsible for:

• Processing incoming traffic from the network interfaces
• Securely accelerating authorized packets (if Performance Pack is running)
• Distributing non-accelerated packets among kernel instances.

Checkpoint_Management_Server

Troubleshooting Firewall Management Problems 

Client   - [Management] - [Firewall Enforcement]
Management Server 3 Important Database Files

1. $FWDIR/conf/objects_5_0_C    (objects - network, host, firewall object)
2. $FWDIR/conf/fwauth.NDB       (Users, administrators)  if fwauth.NDB  gets corrupted .. delete it and stop and start firewall it will automatically recreate it from fwauthsav.NDB bakup
3. $FWDIR/conf/rulebase_5_0.fws  (Rules, NAT, )  and rulebase_5_0.fws.backup is its backup

cpstop;cpstart
GuiDBEdit  - edit the database

                CPMI
Client SmartDashboard  <== CPMI protocol==>  FWM (Process) Database (user/objects/rulebase)
SmartView Tracker
SmartView Monitor

Database Editing
1. SmartDashboard
2. VI  (limited/experience)
3. GUIDbedit  (hidden tool GUI client install) (must close smartdashboard)
4. dbedit (command line) advance tool (must close smartdashboard)

database structure   [Table  => object = Field Name]

Management Roles
- Policy configuration  - Rules - object
- Status Monitoring  - status CPU/memory
- Certicate Authority  - ICA (internal Certifcate /SSL vpn etc, sic
- Log Server - logs users/resources

Firewall Management Primary Role
- Policy Configuration
- Policy Push

Certificate Authority
-ICA
-VPN
-SIC

Log Server
-Logs
-Alerts

Status Monitoring
-Monitoring (CPU/Mem)
-Status (IPS/Clustering/SecureXL)
-Performance (Graphs/Metrics top service/destination etc)


Main Process

1. fwm  (Listen 18190)- firewall management  (serving the different GUI Clients) (manages Databases)(collecting status)(Policy Complication)
2. fwd  - (listen 257) firewall daemon  (received logs fw.log from firewall endforce )
3. cpd   - Checkpoint daemon - build secure channel (SIC establishment) (Loading policy) (Status Collection)
4. cpca (listen 18264 /18265 ) - Checkpoint Certificate authority  (Child deamon on fwd process) (ICA Management tool)
5. cpwd (listen - Checkpoint watchdog .. monitor all other firewall processes (FWM FWD CPD CPCA etc)

# cpwd_admin list (Checkpoint monitoring processes
Key
APP Application names
Status E (executing)  T (Terminating)

CPMI (Checkpoint Point Management Interface)  listen on 18190

FWM Debug

Management Station CPD secure channel <== SIC SSL Session ==> CPD Firewall Enforcement
Certificate

CPD passes information to FWD on endpoint Enforcement to kernel

Firewall listen on port 18211 for management to firewall for SIC
CPD - 18192  CDP - cpu/mem/
AMON Application Monitor


https://www.youtube.com/watch?v=zOG7Empolyg

Understanding Check Point Management Station Part 1

Understanding Check Point Management Station Part 2

Troubleshooting Firewall Management Problems 

Installation failure on MGNT server occurs in verification or complication stage of the installation.  To troubleshoot this failure, follow these steps:

              

1.      Install policy from the CLI

fwm –d load $FWDIR/conf/PolicyName.W <target>

2.      Install the policy from the SmartDashboard

a.      Clean the old log files

cd $FWDIR/log

rm fwm.elg

echo  ‘ ‘> fwm.elg

                             

enable the fwm debug

fw debug fwm on TDERROR_ALL_ALL=5

fw debug fwm on OPSEC_DEBUG_LEVEL=9

                                                                                     .

Replicate the problem by installing the policy from the Dashboard GUI

                              Stop the fwm debug on the FWMGMT  server

fw debug fwm off TDERROR_ALL_ALL=1

fw debug fwm off OPSEC_DEBUG_LEVEL=1

Debug output are stored in fwm.elg, located under $FWDIR/log.

Policy Installation failure could be result of SIC, Misconfigured rules, GUI client connectivity problems or improper entered information, expired license. Evaluate output file fwm.elg to determine with of these may have cause the failure.

Lab   Troubleshoot SIC   - Policy Installation failure

From A-SMS (FW Management Server)

1.      cpca_client lscert    (use this command to validate certificate)

2.      fw debug fwm on Establish

3.      from Client GUI DashBoard, attempt to restablish SIC

4.      from A-SMS     fw debug fwm off

5.      review debug log output        less $FWDIR/log/fwm.elg  

6.      WinSCP to transfer fwm.elg file  and view in Test Editor to identity source of problem from debug file

Lab   Troubleshoot MisConfigured Rules  - Policy Installation Failure

From A-SMS (FW Management Server)

1.      In Expert mode

cd  $FWDIR/log

echo >fwm.elg

fw debug fwm on

2.      fw debug fwm on

TDERROR_ALL_ALL

fwm –d load $FWDIR/conf/standard.W A-GW

Note: you will see the output on the screen and fwm.elg. you can see additional info if you do a TDERROR_ALL_ALL=5

Stop the debugs

fw debug fwm off

unset TDERROR_ALL_ALL

               view fwm.elg     less $FWDIR/conf/fwm.elg

Lab   Troubleshoot GUI Client Connectivity Problem 

Look at less $FWDIR/con/gui-client file for allowed IP address

From A-SMS (FW Management Server)

cpconfig command

At the prompt type 3 and enter

Type Y and Enter , Type D and Enter  (delete your IP to test failure)

Enter 10.20.59.250   (your IP address)

Exit cpconfig option 9 enter

From the command prompt type

fw debug fwm on TDERROR_ALL_ALL=5

from your Client 10.20.59.250  Dashboard, launch the GUI

from the FWM stop the debug and view logs of the GUI client IP

fw debug fwm off

grep 10.20.59.250 $FWDIR/log/fwm.elg | less

How to uninstall and install another policy on firewall

ISSUE

There may be a time where you install the wrong policy onto a Check Point Firewall. This can block your connections, and screw which traffic is allowed through the firewall.

RESOLUTION

These steps will show you how to remove and reinstall the correct policy via the CLI on the manager (SCS),

  1. First of all we look at the policy history, so we can find out the name of the policy we need to reinstall.

fw stat -l [firewall ip]

  2. Next we remove the security policy from the firewall.

fwm unload [fwname]

  3. Finally we install the correct policy back onto the Firewall.

Note : Note how we add the .W to the policy name as it has yet to be be compiled into a .cf file (which is what is installed onto the Firewall/Gateway)

[PolicyName].W is  uncompelled policy

[PolicyName].CF is compiled Policy

fwm load [PolicyName].W [fwname]

fwm load RuleBaseName.W TargetGatewayName

[Expert@MYFWM01]# fwm load Standard samplegw

Installing policy on R77 compatible targets:

Standard.W: Security Policy Script generated into CustomerPolicy.pf

Standard:

Compiled OK.

Installing Security Gateway policy on: examplegw . ..

Security Gateway policy installed successfully on examplegw. ..

Security Gateway policy installation complete

Security Gateway policy installation succeeded for: examplegw

Installation failed; reason – load on module failed, failed to load security policy

Recently I tried a policy installation on a Security Gateway appliance which failed with the message: Installation failed; reason - load on module failed, failed to load security policy

From my experience I know that a cpstop ; cpstart normaly solves this problem.

But since I was dealing with a remote site gateway which also was stand-alone installation, issuing cpstop was no option since it would interrupt the service.

So I utilized the cpwd_admin command for stopping and restarting FWM and CPD.

cpwd_admin stop -name FWM -path "$FWDIR/bin/fw" -command "fw kill fwm"

cpwd_admin start -name FWM -path "$FWDIR/bin/fwm" -command "fwm"

cpwd_admin stop -name CPD -path "$CPDIR/bin/cpd_admin" -command "cpd_admin stop"

cpwd_admin start -name CPD -path "$CPDIR/bin/cpd" -command "cpd"

Then I checked the status of all my services with cpwd_admin list.

All services were up and I tried policy installation again, which worked as expected.

I was able to solve the issue using the following steps i stumbled on while surfing the net:

1.            tellpm process:monitord

2.            ps aux | grep cpd

3.            kill -15

4.            tellpm process:monitord t

5.            cpwd_admin start -name CPD -path “$CPDIR/bin/cpd” -command “cpd”

Here is the expected output:

[Expert@CP-DMZ:0]# tellpm process:monitord

[Expert@CP-DMZ:0]#

Message from syslogd@ at Tue Sep 16 10:27:42 2014 …

CP-M-DMZ monitord[4129]: monitord got killed

[Expert@CP-DMZ:0]#

[Expert@CP-DMZ:0]# ps aux | grep cpd

admin 4461 0.0 0.3 212412 3196 ? Dsl Aug06 42:47 cpd

admin 6905 0.0 0.0 1816 492 pts/2 S+ 10:27 0:00 grep cpd

[Expert@CP-DMZ:0]# kill -15 4461

[Expert@CP-DMZ:0]#

[Expert@CP-DMZ:0]# tellpm process:monitord t

[Expert@CP-DMZ:0]#

[Expert@CP-DMZ:0]# cpwd_admin start -name CPD -path “$CPDIR/bin/cpd” -command “cpd”

cpwd_admin:

Process CPD started successfully (pid=7030)

[Expert@CP-DMZ:0]#

So the problem was solved without service interruption.

List of Check Point Firewall Ports

Common List Ports that you will need to open on a typical Check Point Firewall. Note: don’t open all of these ports in the list, instead – use this list of ports as a reference for your Check Point firewall configuration.

PORT	TYPE	SERVICE DESCRIPTION
21	TCP	ftp File transfer Protocol (control)
21	UDP	ftp File transfer Protocol (control)
22	Both	ssh SSH remote login
25	both	SMTP Simple Mail transfer Protocol
50		Encryption IP protocols esp – IPSEC Encapsulation Security Payload
51		Encryption IP protocols ah – IPSEC Authentication Header Protocol
53	Both	Domain Name Server
69	Both	TFTP Trivial File Transfer Protocol
94	TCP	Encryption IP protocols fwz_encapsulation (FW1_Eencapsulation)
137	Both	Netbios-ns NETBIOS Name Service
138	Both	netbios-dgm NETBIOS Datagram
139	Both	netbios-ssn NETBIOS Session
256	TCP	FW1 (fwd) policy install port FWD_SVC_PORT
257	TCP	FW1_log FW1_log FWD_LOG_PORT
258	TCP	FW1_mgmt FWM_SSVVC_PORT
259	TCP	FW1_clientauth_telnet
259	UDP	RDP Reliable Datagram Protocol
260	TCP	sync
260	UDP	FW1_snmp FWD_SNMP_PORT
261	TCP	FW1_snauth Session Authentication Daemon
262	TCP	MDQ – mail dequer
263	TCP	dbs
264	TCP	FW1_topop Check Point SecureClient Topology Requests
265	TCP	FW1_key Check Point VPN-1 Public key transfer protocol
389	Both	LDAP Secure Client connecting to LDAP without SSL
443		SNX VPN can use 443 too
444	TCP	SNX VPN SNX VPN tunnel in connectra only
500	UDP	IPSEC IKE Protocol (formerly ISAKMP/Oakley)
500	TCP	IKE over TCP
500	UDP	ISAKMPD_SPORT & ISAKMPD_DPORT
514	UDP	Syslog Syslog
636		LDAP Secure Client connecting to LDAP with SSL
900	TCP	FW1_clntauth_http Client Authentication Daemon
981		Management https on the edge
1247
1494	TCP	Winframe Citrix
1645	TCP	Radius
1719	UDP	VOIP
1720	TCP	VOIP
2040	TCP	MIP meta Ip admin server
2746	UDP	UDP encapsualtion for SR VPN1_IPSEC_encapsulation VPN1_IPSEC encapsulation
2746	TCP	CPUDPENCap
4000		Policy Server Port (Redmond)
4433	TCP	Connectra Admin HTTPS Connectra admin port
4500	UDP	NAT-T NAT Traversal
4532	TCP	SNDAEMON_PORT sn_auth_trap: sn_auth daemon Sec.Serv comm,
5001	TCP	Meta IP Web Connection, MIP
5002	TCP	Meta IP DHCP Failover
5004	TCP	Meta IP UAM
5005	TCP	Meta IP SMC
6969	UDP	KP_PORT KeyProt
8116	UDP	Check Point HA SyncMode= CPHAP (new sync mode)
8116	UDP	Connection table synchronization between firewalls
8989	TCP	CPIS Messaging MSG_DEFAULT_PORT
8998	TCP	MDS_SERVER_PORT
9000		Command Line Port for Secure Client
10001	TCP	Default CPRSM listener port for coms with RealSecure Console
18181	TCP	FW1_cvp Check Point OPSEC Content Vectoring Protocol
18182	TCP	FW1_ufp Check Point OPSEC URL Filtering Protocol
18183	TCP	FW1_sam Check Point OPSEC Suspicious Activity monitoring Proto (SAM API)
18184	TCP	FW1_lea Check Point OPSEC Log Export API
18185	TCP	FW1_omi Check Point OPSEC Objects Management Interface
18186	TCP	FW1_omi-sic Check Point OPSEC Objects management Interface with Secure Internal Communication
18187	TCP	FW1_ela Check Point OPSEC Event Loging API
18190	TCP	CPMI Check Point Management Interface
18191	TCP	CPD Check Point Daemon Proto NG
18192	TCP	CPD_amon Check Point Internal Application Monitoring NG
18193	TCP	FW1_amon Check Point OPSEC Appication Monitoring NG
18201	TCP	FGD_SVC_PORT
18202	TCP	CP_rtm Check Point Real time Monitoring
18203	TCP	FGD_RTMP_PORT
18204	TCP	CE communication
18205	TCP	CP_reporting Check Point Reporting Client Protocol
18207	TCP	FW1_pslogon Check Point Policy Server logon Protocol
18208	TCP	FW1_CPRID (SmartUpdate) Check Point remote Installation Protocol
18209	TCP	FWM CA for establishing SIC communication
18210	TCP	FW1_ica_pull Check Point Internal CA Pull Certificate Service
18211	TCP	FW1_ica_pull Check Point Internal CA Push Certificate Service
18212	UDP	Connect Control – Load Agent port
18213	TCP	cpinp: inp (admin server)
18214	TCP	cpsmc: SMC
18214	UDP	cpsmc: SMC Connectionless
18221	TCP	CP_redundant Check Point Redundant Management Protocol NG
18231	TCP	FW1_pslogon_NG Check Point NG Policy Server Logon Protocol
18231	TCP	NG listens on this port by default dtps.exe
18232	TCP	FW1_sds_logon Check Point SecuRemote Distribution Server Protocol
18233	UDP	Check Point SecureClient Verification Keepalive Protocol FW1_scv_keep_alive
18241	UDP	e2ecp
18262	TCP	CP_Exnet_PK Check Point Public Key Resolution
18263	TCP	CP_Exnet_resolve Check Point Extranet remote objects resolution
18264	TCP	FW1_ica_services Check Point Internal CA Fetch CRL and User Registration Services
19190	TCP	FW1_netso Check Point OPSEC User Authority Simple Protocol
19191	TCP	FW1_uaa Check point OPSEC User Authority API
65524		FW1_sds_logon_NG Secure Client Distribution Server Protocol (VC and Higher)

Check Point General Common Ports

PORT	TYPE	SERVICE DESCRIPTION
257	tcp	FireWall-1 log transfer
18208	tcp	CPRID (SmartUpdate)
18190	tcp	SmartDashboard to SCS
18191	tcp	SCS to FW-1 gateway for policy install
18192	tcp	SCS monitoring of firewalls (SmartView Status)

Check Point SIC Ports

PORT	TYPE	SERVICE DESCRIPTION
18209	tcp	NGX Gateways <> ICAs (status, issue, or revoke).
18210	tcp	Pulls Certificates from an ICA.
18211	tcp	Used by the cpd daemon (on the gateway) to receive Certificates.

Check Point Authentication Ports

PORT	TYPE	SERVICE DESCRIPTION
259	tcp	Client Authentication (Telnet)
900	tcp	Client Authentication (HTTP)

Tags: Check Point, Firewall, ports

Checkpoint_Technology

A Quick Recap

CheckPoint security revolves around 3 key components

• GUI Clients – This is is mainly SmartConsole and it’s contained applications such as Update, Tracker and Dashboard. All release candidates for this are a separate track to the main product suite and HFAs for this set are independent of the main software release. These are effectively the software configuration and interaction between the user and the SMS/Gateways/Reporting
• Security Management – Responsible for all management of the CheckPoint system. It interacts in User-Mode Processes (Explained later) in order to configure all product suites and relies on the following (most significant) processes
  • FWM
  • FWD
  • CPD
  • FWSSD
  • CPWD
• Gateway – The gateway is the device that enforces your security policy. It is really just a computer running software including an OS as normal (IPSO, SPLAT,GAIA etc.) and as such is technically vulnerable. To get around this the firewall runs in the Kernel of the system, separating the hardware from the OS and protecting itself. The above listed processes run as User-Mode processes in the OS. Traffic flow is therefore NIC -> Kernel -> OS (Including IP stack) -> User-Mode features

User and Kernel Mode Processes

So what’s the difference between the two? The Kernel is in effect the binder for the Hardware to Software traffic flow. It intercepts all traffic and provides the interaction between the Modules and Drivers. Modules here (in the Firewall-1 Kernel) would included processes for NAT, Security Enforcement, Encryption and Decryption. Pretty much the things we think of that make it a firewall. Every bit of data (or packet) that arrives at the firewall will hit the Kernel and be inspected. the inspection decides what happens to it, whether it be dropped, forwarded or some other operation applied such as crypto functions.

On top of this (almost literally!) resides the User-Mode. These processes allow the Gateway to utilise functions of the underlying operating system in a protected environment. An example of these are the processes listed earlier such as FWM and FWD.

Communication is handled from the Kernel to User-Mode using traps (Think SNMP) and from User-Mode to Kernel using IOctl, allowing the entity to call a function in the Kernel and write to it.

This entire segregation here is necessary and understanding it allows you to debug effectively. For example, if your log server is no longer receiving logs, why? The Security Policy (In the Kernel) looks fine and you have enough disk space. Start debugging the right User-Mode process and it will usually direct you where to look – FWD. This can save a lot of time.

By Process of Elimination

The core process we find on every CheckPoint product is CPD. This is the life of the devices and handles many things. The primary features we may see it handling are:

• SIC
• Status (AMON status pulled from GW/SMS and sent to SmartEvent)
• Message handling between Firewall-1 processes
• Policy installation on Gateways

I suppose you could think of it as a conductor, it calls and controls how other parts of the system interact.

FWM is a management process that exists on all management products such as the SMS. Some of the functions it does are:

• Talking between the GUI and SMS (SmartConsole)
• Database Editing; adding objects, users rules etc.
• Security Policy Compilation
• Log display
• Management HA Sync between SMSs

Logging to internal and external servers including the management server is handled by FWD(TCP/257). It also handles calling some child processes such as VPND and interacts with policy installation. FW commands in the shell such as fw ctl chain are also handled here

FWSSD is spawned by FWD as a child process and runs the Security Servers. There are multiples of these that run, one for each service and are labelled as in.xxx or out.xxx where xxx = service (such as DLP or SMTP). Some examples are:

• in.asmtpd = SMTP Security Server
• in.ahttpd = HTTP Security Server
• in.emaild.smtp
• in.emaild.pop3

You can view these by entering Expert mode and running

cat $FWDIR/conf/fwauthd.conf

THE EBB AND FLOW (PLUS CHAINS)

One of Checkpoints’ inventions as they like to tout (and rightly so) is the idea of Stateful Inspection, which we have looked at before. The overview here is that it allows a packet to be identified not only by it’s IP and Port, but also by it’s state, flow and extensibility. This allows us to allow our firewall to add in permit statements as and when needed based upon the conversation it is having – allow traffic to the Internet from your LAN and return traffic is allowed by default, but only once verified and by conversation. The other thing to note is that passing traffic based upon a state table is MUCH faster than sequentially querying a rule base each time a packet comes in (providing it has already been identified). We will cover this more later.

Packet Flow

A firewall has two main directions that traffic can flow, and most of the time it will abide by both. Simply this is “Inbound” and “Outbound”. Sounds simple, and really it is – a packet heading to the firewall is inbound and traffic exiting the firewall is outbound. This is an important consideration when reflected upon as follows. We receive a reply packet that has traversed over a VPN from a partner company. What do we do with it? Common sense says “decrypt the packet and apply to to a rule base”. Common sense would mostly be right, but let’s say the packet was NATd originally on our firewall, what then? Looking at this logically allows you to mostly guess what happens, but the real order of operation comes from what is known as an FW CTL Chain, or Firewall Control Chain. These are similar between gateways but can vary depending on the blades you’re running. Some basic facts around these

• The Kernel makes no assumptions – just because another part of the kernel may have inspected it before, it takes no chances and will do so again
• Some functionality occurs both inbound and outbound
• Each direction has its own chain modules and they will vary slightly
• The modules known as Handlers, each decide whether to pass or drop the traffic based upon their inspection role
• Inspection is performed on virtually defragged packets

The gateway will assume however that if something didn’t come IN an interface (or chain) then it (the gateway) generated the packet. Likewise, if it came through an inbound module then it didn’t send it.

Some example chain modules are:

• IP Options (strip in)
• IP Options (strip out)
• VPN Decrypt (Inbound)
• IP Options Restore
• HA forwarding
• VPN Encrypt (Outbound)
• TCP Streaming

Expert@CPGW-01]# fw ctl chain
in chain (20):
0: -7ffffff0 (f163cf00) (00000001) tcpt inbound (tcp_tun)
1: -7f800000 (f21eadc0) (ffffffff) IP Options Strip (in) (ipopt_strip)
2: -7d000000 (f1644440) (00000003) vpn multik forward in
3: – 2000000 (f162ab50) (00000003) vpn decrypt (vpn)
4: – 1fffff8 (f16358e0) (00000001) l2tp inbound (l2tp)
5: – 1fffff6 (f21ec0c0) (00000001) Stateless verifications (in) (asm)
6: – 1fffff5 (f2788530) (00000001) fw multik VoIP Forwarding
7: – 1fffff4 (f229f6f0) (00000001) fw early SIP NAT (sipnat)
8: – 1fffff2 (f16515d0) (00000003) vpn tagging inbound (tagging)
9: – 1fffff0 (f1628bd0) (00000003) vpn decrypt verify (vpn_ver)
10: – 1000000 (f2241650) (00000003) SecureXL conn sync (secxl_sync)
11: 0 (f219ae00) (00000001) fw VM inbound (fw)
12: 1 (f2204d00) (00000002) wire VM inbound (wire_vm)
13: 2000000 (f162b8b0) (00000003) vpn policy inbound (vpn_pol)
14: 10000000 (f2247410) (00000003) SecureXL inbound (secxl)
15: 7f600000 (f21e14e0) (00000001) fw SCV inbound (scv)
16: 7f730000 (f232e6b0) (00000001) passive streaming (in) (pass_str)
17: 7f750000 (f2497090) (00000001) TCP streaming (in) (cpas)
18: 7f800000 (f21eb150) (ffffffff) IP Options Restore (in) (ipopt_res)
19: 7fb00000 (f2459170) (00000001) HA Forwarding (ha_for)
out chain (17):
0: -7f800000 (f21eadc0) (ffffffff) IP Options Strip (out) (ipopt_strip)
1: -78000000 (f1644420) (00000003) vpn multik forward out
2: – 1ffffff (f1629c80) (00000003) vpn nat outbound (vpn_nat)
3: – 1fffff0 (f2496f10) (00000001) TCP streaming (out) (cpas)
4: – 1ffff50 (f232e6b0) (00000001) passive streaming (out) (pass_str)
5: – 1ff0000 (f16515d0) (00000003) vpn tagging outbound (tagging)
6: – 1f00000 (f21ec0c0) (00000001) Stateless verifications (out) (asm)
7: 0 (f219ae00) (00000001) fw VM outbound (fw)
8: 1 (f2204d00) (00000002) wire VM outbound (wire_vm)
9: 2000000 (f162b360) (00000003) vpn policy outbound (vpn_pol)
10: 10000000 (f2247410) (00000003) SecureXL outbound (secxl)
11: 18000000 (f22cf850) (00000001) fw record data outbound
12: 1ffffff0 (f1635570) (00000001) l2tp outbound (l2tp)
13: 20000000 (f1629ef0) (00000003) vpn encrypt (vpn)
14: 60000000 (f163d160) (00000001) tcpt outbound (tcp_tun)
15: 7f700000 (f2499100) (00000001) TCP streaming post VM (cpas)
16: 7f800000 (f21eb150) (ffffffff) IP Options Restore (out) (ipopt_res)

Above is an example of the chains

X – Module location in chain (Serial Number)

X – Chain Position

X – Function Pointer

X – Mode: Stateful Mode

X – Mode: Wired Mode

X – Mode All Packets (00000003 & ffffffff are identical)

X – Chain Name

The location of modules within the chains are relative and can differ between gateways depending on the modules installed. the Chain Position is an absolute number and never changes between gateways

INSPECT AND INSTALL PROCESS

Some of the sections we go over are going to be partial recaps on others. A lot of the course does this, especially considering how closely they are tied together within processes (not to mention my less than professional writing ability). After looking at the inspection areas, lets have a look at the “complete” packet flow.

1. Packets arrive at the NIC on the appliance inbound, for the sake of assumption it’s from an internal client accessing the web
2. The firewall kernel will process inspection on the packet
3. Once a packet is matched against the rulebase a log is sent to the Gateways’ user-mode FWD process
4. The GW FWD process sends a log to the SMS FWD process using the CPD daemon
5. FWM on the SMS logs to SmartConsole (SmartviewTracker)
6. Lastly the packet is routed (usually, unless it’s VPN for example) using the OS to the relevant outbound NIC where the kernel intercepts and does its outbound work

The policy installation process is am ore interesting one, simply because compared to certain other firewalls, there is rule computation, compilation and composition based upon a number of internal files within the units. This is also an extremely good reason (There are many) to have a distributed deployment as opposed to standalone. Have your SMS sitting on a secured internal VM where it doesn’t matter what happens to the gateways – you can’t recover as easily from a failure by just copying and pasting config like you can on an ASA or NetScreen. Here’s an overview of policy installation:

1. Create your policy within Smart Dashboard – let’s assume it’s just some outbound rules and a few manual NATs with a policy called SUPERSECURE
2. When you save you create a new file called SUPERSECURE.W within $FWDIR/conf. All these aresaved within rulebases_5_0.fws
3. The fwm_gen process then compiles your selected policy (SUPERSECURE.W) into a .pf file within the same location.

The .pf files themselves are composites from a number of files: *.W and objects.C and are stored in machine language within $FWDIR/conf. Objects.C is also a file specific to the policy installation only, it exists in $FWDIR/database and is relevant to that gateway only. Objects_5_0.C contains all objects defined, whether in use within that policy or not and exists on the SMS under $FWDIR/conf

One reason for this is as such: if you have an SMS, say a Smart-1 that is managing multiple nodes, they may share the same objects. Each gateway may have VPNs between them so will need to know the CheckPoint Gateway objects, their subnets, ports etc. The objects_5_0C file means that you only need to have them specified once, why duplicate? Each gateway then borrows the relevant parts from this file and has its own objects.C file created for it.

Actually installing the policy goes through an umber of steps:

1. Initiation – You click your install button within SmartDashboard to start the process and provide information it needs (such as the gateway it needs to install on, does it matter if it fails one one, DB revision?)
2. Verification – This is where the SMS goes over your policy to ensure it meets a number of rules before continuing. An example here might be flagging up one rule specifically hiding another and it will fail (or pass if it’s fine)
3. Conversion – The databases have their relevant information taken and converted to separate formats needed by later processes
4. Compilation & generation – Here the policy is translated to the INSPECT language and then passed to the INSPECT compiler to be…compiled!
5. CPTA – CheckPoint Transfer Agent uses SIC to send the relevant policy to the necessary gateways
6. Commit – The gateway loads up its new policy

The processes themselves handle the process as follows:

1. SmartConsole CPMI sends the policy installation command to FWM on the SMS
2. FWM then does its verification and compiling of the policy elements. This is then sent to CPD
3. CPD generates the necessary code  and compiles it before initiating CPTA to forward the policies to the relevant gateways for installation
4. CPD on the gateway receives the compiled policy and verifies it
5. FWD then updates the relevant processes – VPND and FWSSD for example, with the new policy elements CPD then starts the work of replacing the kernel
6. As the new policy is getting ready for replacing, the GW starts to queue all incoming traffic and stops forwarding
7. The Atomic load (What a cool name) then starts where by the kernel is replaced
8. Queued traffic is now forwarded through the new policy

KERNEL TABLES (BITS TO CHECK)

Although we have mentioned the  Kernel a fair bit and identified that certain processes run in User Mode or Kernel Mode, we haven’t touched on a massively practical application of knowing this (other than using them to identify the possible cause of  an issue). Checkpoint units themselves have a lot of tables – you can see this list easily enough using the command

fw tab -s | more

That gives you a summary list similar to:

Fair bit of info, but what we really want to interrogate is the contents of some of the tables. Lets look at the connections table

[Expert@CPGW-01]# fw tab -t connections -f | more
Using cptfmt
Formatting table’s data – this might take a while…

13:06:25 xx.xx.xx.xx > : ———————————–(+); Direction: 0; Source: 172.23.117.111; SPort: 53874; Dest: 192.168.10.202; DPort: 161; Protocol: udp; CPTFMT_sep_1: ->; Direction_1: 0; Source_1: 172.23.117.111; SPort_1: 53874; Dest_1: 172.20.140.202; DPort_1: 161; Protocol_1: udp; FW_symval: 17; product: VPN-1 & FireWall-1;

13:06:25 xx.xx.xx.xx > : ———————————–(+); Direction: 1; Source: yy.yy.yy.yy; SPort: 80; Dest: 172.19.201.16; DPort: 3555; Protocol: tcp; CPTFMT_sep_1: ->; Direction_1: 0; Source_1: 172.19.201.16; SPort_1: 3555; Dest_1: yy.yy.yy.yy; DPort_1: 80; Protocol_1: tcp; FW_symval: 5; product: VPN-1 & FireWall-1;

These are great. Lets digest what we can see.

The examples shows us:

• Direction: O, so we know it is inbound from an external source initially.
• We can see the source and destination IPs and port numbers
• We can see the protocol type too
• Direction: 1, this is outbound
• Notice the difference between the way the UDP and TCP connections are listed. I don’t know why this is but think it’s to do with the connection-oriented/connectionless style of the protocols and linking to their previous packet.

We can’t see anything relating to the information on it’s interface or sequence number. I actually struggled to find anything solid about how to interpret this table, there just doesn’t seem much out there, even on other blogs or the CP support knowledge base. I have my own assumptions but as I can;t prove them, I won’t pass information around.

The connections table itself achieves some very important goals for firewall security and performance:

• It’s faster to pass traffic based upon an entry in the state table than it is querying the firewall policy as part of the INSPECT module
• Replies are allowed back – this means we don’t need to add specific reply entries in to our rule base
• Stateful items such as TCP sequencing, AAA, NAT etc. can all be handled within the table