Tuesday, February 27, 2018

Buzzwords


Swag
competing Priorities
Collaborate

Here’s my top 25 buzzwords and phrases that get bandied about. I’m sure you’ll add your own in the comments…
  1. Capturing hearts and minds: Aaaaghhhhh. Stop it. Stop it now. This is too clichéd. And may be way over the top for what you need to do.
  2. Agile: Note the big “A” — are you using it as an adjective (small a) as opposed to a large A (the methodology
  3. Benefits realisation: jargon for does your change pay off. Never use it outside of a business case!
  4. Rightsizing: Shudder. DO NOT USE. There is nothing right about laying people off, because you have made poor decisions.
  5. Paradigm shift: Thought it went out in the 80’s? Apparently not…
  6. Air-gap. A new one doing the rounds — it’s an IT network computing term, which means one computer is isolated from the others. It’s being used to describe teams that won’t talk to each other.
  7. Take them on a journey. Please stop it. All I can think of is National Lampoons Vacation.
  8. Disruption. One man’s disruption is another man’s discomfort. Be very careful about using this one…if its new for you it may not be disruptive for others.
  9. Time box: A project planning term which means thinking about deliverables in a fixed period of time. It should not be used as a verb. That’s just sloppy.
  10. Purpose driven: Nothing makes a change more noble than making sure it’s purpose driven…
  11. User adoption: Only to be used in IT change — otherwise you are being horrible to your employees.
  12. Collaborate: So so misused. Collaboration has impacts that are bigger than simply working together. It doesn’t mean play nicely.
  13. Grab the low hanging fruit: Or just find things that are easy to fix. Don’t forget low hanging fruit are exposed to the elements. Can rot quicker.
  14. Future Ready: A brand that every corporate change agenda uses… but here’s the thing. The future never arrives…
  15. Pivot: Once applied to strategic changes in the lean start up world, now seems to mean “oops, doing something different”
  16. Herding cats: I think we are doing cats a disservice here. Put some food down, they line up fast.
  17. Socialise the document: You mean share or send some-one a document for comment? Pass me the G&T…
  18. Change ready: Yep, in your dreams…
  19. Touch base and dialogue: God no, just no! You’re doing it wrong!
  20. Change champion: Only if they get to wear a sheriff’s badge…and stand on a dias.
  21. Drinking from the firehose: You mean you are overwhelmed? Then say that. Unless you have firemen as your change champions. TOTALLY different story.
  22. Synergistic: You mean it all fits together? Well pretty sure that was the intent.
  23. Hyperconnected: Ooh…so your internal organisation is connected to your external stakeholders. As it should be. Not too frenetic that one. Try uber connected.
  24. Holistic: Make sure it really is greater than the sum of all its parts. Holistic is more than organisational wide.
  25. Transform: ONLY to be used if something will truly change to something that can’t be imagined. Other wise it’s a step change. Oh wait… another buzzword bingo sheet?

Friday, February 23, 2018

R80.10 API - Troubleshooting


On FWM
1. WebCLI Create an account tufincli wiht admin role and cli.sh shell
2. Admin GUI  create and account with read/write privileges checkpoint password
3. Install Database on FWM 


login as: tufincli
This system is for authorized use only.
tufincli@hin0301fwmtest's password:
Last login: Fri Feb 23 11:55:21 2018 from dkhem01063322.bcbsma.com
hin0301fwmtest> expert
Enter expert password:

Warning! All configurations should be done through clish
You are in expert mode now.

[Expert@hin0301fwmtest:0]#

[Expert@hin0301fwmtest:0]# mgmt_cli show-version
Username: tufinapi
Password:
code: "generic_internal_error"
message: "Internal error. For more info search for incident [44bdcbf1-b640-4b19-                                       9330-e3811111b8e9] in log file"

[Expert@hin0301fwmtest:0]#
[Expert@hin0301fwmtest:0]# api restart
2018-Feb-23 12:12:45 - Stopping API...
2018-Feb-23 12:12:48 - API stopped successfully.
2018-Feb-23 12:12:48 - Starting API...
. . . . . . . . . . . . . . . . .
2018-Feb-23 12:14:03 - API started successfully.
[Expert@hin0301fwmtest:0]#
[Expert@hin0301fwmtest:0]#
[Expert@hin0301fwmtest:0]#
[Expert@hin0301fwmtest:0]#
[Expert@hin0301fwmtest:0]#
[Expert@hin0301fwmtest:0]# api status

API Settings:
---------------------
Accessibility:                      Require ip 127.0.0.1
Automatic Start:                    Enabled

Processes:

Name      State     PID       More Information
-------------------------------------------------
API       Started   10190
CPM       Started   4398      Check Point Security Management Server is running and ready
FWM       Started   3910

Port Details:
-------------------
JETTY Internal Port:      50277
APACHE Gaia Port:         443
                          Apache port retrieved from: httpd-ssl.conf


--------------------------------------------
Overall API Status: Started
--------------------------------------------

API readiness test SUCCESSFUL. The server is up and ready to receive connections

Notes:
------------
To collect troubleshooting data, please run 'api status -s <comment>'

[Expert@hin0301fwmtest:0]#
[Expert@hin0301fwmtest:0]# mgmt_cli show-version
Username: tufinapi
Password:
code: "generic_internal_error"
message: "Internal error. For more info search for incident [c6c26b63-9283-4534-9a87-fe6c8109da84] in log fi           le"

[Expert@hin0301fwmtest:0]# api status -s

API Settings:
---------------------
Accessibility:                      Require ip 127.0.0.1
Automatic Start:                    Enabled

Processes:

Name      State     PID       More Information
-------------------------------------------------
API       Started   10190
CPM       Started   4398      Check Point Security Management Server is running and ready
FWM       Started   3910

Port Details:
-------------------
JETTY Internal Port:      50277
APACHE Gaia Port:         443
                          Apache port retrieved from: httpd-ssl.conf


--------------------------------------------
Overall API Status: Started
--------------------------------------------

API readiness test SUCCESSFUL. The server is up and ready to receive connections

Collecting and compressing diagnostic data... Please wait...
Adding api.elg
Adding api_sh.elg
Adding api.json
Adding api.csv
Adding cpm.elg
Adding fwm.elg
Adding httpd_access_log
Adding httpd2.conf
Adding extra/httpd2-webapi.conf
Adding httpd2_access_log
Adding httpd2_error_log
Adding memory.elg
Adding disk_space.elg
Adding ifconfig.elg
Adding cpwd_admin_list.elg
File /home/tufincli/2018.02.23_12-14-42_api_data_.tgz has been created

[Expert@hin0301fwmtest:0]# sftp sftp@ott.checkpoint.com
Connecting to ott.checkpoint.com...
[Expert@hin0301fwmtest:0]#

[Expert@hin0301fwmtest:0]# sftp bcbsma@sftp.ott.checkpoint.com
Connecting to sftp.ott.checkpoint.com...
The authenticity of host 'sftp.ott.checkpoint.com (67.210.167.35)' can't be established.
RSA key fingerprint is 4b:e3:22:02:14:ff:92:6b:22:e0:a8:fb:16:86:36:2a.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'sftp.ott.checkpoint.com,67.210.167.35' (RSA) to the list of known hosts.
Check Point FTP ServerEnter password:
sftp>
sftp> cd incoming/api-test
sftp>
sftp> put /home/tufincli/2018.02.23_12-14-42_api_data_.tgz
Uploading /home/tufincli/2018.02.23_12-14-42_api_data_.tgz to /incoming/api-test/2018.02.23_12-14-42_api_dat           a_.tgz
/home/tufincli/2018.02.23_12-14-42_api_data_.tgz                                     100%   25MB   1.0MB/s   00:24
sftp>




pert@hin0301fwmtest:0]# $FWDIR/scripts/cpm_status.sh
Check Point Security Management Server is during initialization
[Expert@hin0301fwmtest:0]#
[Expert@hin0301fwmtest:0]#
[Expert@hin0301fwmtest:0]#
[Expert@hin0301fwmtest:0]#
[Expert@hin0301fwmtest:0]# watch -d -n 1.0 !!
watch -d -n 1.0 $FWDIR/scripts/cpm_status.sh
Every 1.0s: /opt/CPsuite-R80/fw1/scripts/cpm_status.sh                                         Fri Feb 23 12:20:24 2018

Check Point Security Management Server is during initialization


  1. cpstop; cpstart on your production environment
  2. Validate that we can execute the api call successfully
sftp> quit
[Expert@hin0301fwmtest:0]# cpstop; cpstart
cpwd_admin:
Process DASERVICE terminated
UEPM: Endpoint Security Management isn't activated
Management Portal: Stopping CPWMD
cpwd_admin:
Process CPWMD isn't monitored by cpWatchDog. Stop request aborts
Management Portal: CPWMD failed to stop
Management Portal: Stopping CPHTTPD
cpwd_admin:
Process CPHTTPD isn't monitored by cpWatchDog. Stop request aborts
Management Portal: CPHTTPD failed to stop
Stop Search Infrastructure...
Stopping RFL ...
cpwd_admin:
successful Detach operation
Stopping Solr ...
cpwd_admin:
successful Detach operation
Stop SmartView ...
Stopping SmartView ...
cpwd_admin:
successful Detach operation
Stop Log Indexer...
cpwd_admin:
Process INDEXER (pid=4263) stopped with command "kill 4263". Exit code 0.
Stop SmartLog Server...
cpwd_admin:
Process SMARTLOG_SERVER terminated
dbsync is not running
evstop: Stopping product - SmartEvent Server
evstop: Stopping product - SmartEvent Correlation Unit
Check Point SmartEvent Correlation Unit is not running
SmartView Monitor: Management stopped
FireWall-1: cpm stopped
FireWall-1: fwm stopped
VPN-1/FW-1 stopped
Stopping Critical Alerts Sensor
SVN Foundation: cpd stopped
Stopping cpviewd
SVN Foundation: cpWatchDog stopped
SVN Foundation stopped


cpstart: Power-Up self tests passed successfully

cpstart: Starting product - SVN Foundation

SVN Foundation: Starting cpWatchDog
Starting cpviewd
Starting Critical Alerts Sensor...
SVN Foundation: Starting cpd
SVN Foundation started

cpstart: Starting product - VPN-1

 Local host is not a FireWall-1 module
FireWall-1: Starting fwd
FireWall-1: Starting cpm. Please wait...
[1] 12186
FireWall-1: Finished starting cpm successfully
FireWall-1: Starting fwm (SmartCenter Server)

FireWall-1: This is a SmartCenter server. No security policy will be loaded
FireWall-1 started

cpstart: Starting product - SmartView Monitor

SmartView Monitor: Not active

cpstart: Starting product - Eventia Suite

Start Search Infrastructure...
index mode was set to true
cpwd_admin:
Process SOLR started successfully (pid=12475)
Starting RFL ...
cpwd_admin:
Process RFL started successfully (pid=12503)
Starting SmartView ...
cpwd_admin:
Process SMARTVIEW started successfully (pid=12530)
Start Log Indexer...
cpwd_admin:
Process INDEXER started successfully (pid=12550)
Start SmartLog Server...
cpwd_admin:
Process SMARTLOG_SERVER started successfully (pid=12595)


cpstart: Starting product - Management Portal

Management Portal: Starting CPWMD
Management Portal: CPWMD failed to start
Management Portal: Starting CPHTTPD
Management Portal: CPHTTPD failed to start

cpstart: Starting product - UEPM

UEPM: Endpoint Security Management isn't activated and will not be started

cpstart: Starting product - Deployment Agent

cpwd_admin:
Process DASERVICE started successfully (pid=12793)
[Expert@hin0301fwmtest:0]# $FWDIR/scripts/cpm_status.sh
Check Point Security Management Server is during initialization
[Expert@hin0301fwmtest:0]#
[Expert@hin0301fwmtest:0]# watch -d -n 1.0 !!
watch -d -n 1.0 $FWDIR/scripts/cpm_status.sh
[Expert@hin0301fwmtest:0]#
[Expert@hin0301fwmtest:0]# mgmt_cli show-version
Username: tufinapi
Password:
code: "generic_internal_error"
message: "Internal error. For more info search for incident [267d3016-0fe0-4145-98cc-717c5a149572] in log file"

[Expert@hin0301fwmtest:0]# api restart
2018-Feb-23 12:23:06 - Stopping API...
2018-Feb-23 12:23:08 - API stopped successfully.
2018-Feb-23 12:23:08 - Starting API...
. . . . . . . . . . . . . . . . .
2018-Feb-23 12:24:23 - API started successfully.
[Expert@hin0301fwmtest:0]#
[Expert@hin0301fwmtest:0]# clish -c "lock database override"
CLICMD0201  Config lock is already turned on.
[Expert@hin0301fwmtest:0]#
[Expert@hin0301fwmtest:0]# mgmt_cli show-version
Username: tufinapi
Password:
product-version: "Check Point Gaia R80.10"
os-build: "421"
os-kernel-version: "2.6.18-92cpx86_64"
os-edition: "64-bit"

[Expert@hin0301fwmtest:0]#


STEP 1
Created 2 NEW Users to be utilized on our test.

User:
tufincli
Pass:
vpn123
Purpose:
For Tufin Command Line Access
Where:
Gaia WebUI
Rights:
Admin-Role
Authentication:
clish

User:
tufinapi
Pass:
vpn123
Purpose:
For Tufin API access
Where:
R80.10 SmartDashboard
Rights:
Super User
Authentication:
Check Point Password


STEP 2
Applied the changes by performing the following
  1. Installed database through the console
  2. Restarted API (#api restart)

STEP 3
Executed the API call with ERROR


[Expert@hin0301fwmtest:0]# mgmt_cli show-version
Username: tufinapi
Password:
code: "generic_internal_error"
message: "Internal error. For more info search for incident [c6c26b63-9283-4534-9a87-fe6c8109da84] in log file"



STEP 4
Restarted Check Point Process and Services


[Expert@hin0301fwmtest:0]# cpstop; cpstart



STEP 5
Restarted API


[Expert@hin0301fwmtest:0]# api restart
2018-Feb-23 12:23:06 - Stopping API...
2018-Feb-23 12:23:08 - API stopped successfully.
2018-Feb-23 12:23:08 - Starting API...
. . . . . . . . . . . . . . . . .
2018-Feb-23 12:24:23 - API started successfully.



STEP 6
Executed the API call with SUCCESS


[Expert@hin0301fwmtest:0]# mgmt_cli show-version
Username: tufinapi
Password:
product-version: "Check Point Gaia R80.10"
os-build: "421"
os-kernel-version: "2.6.18-92cpx86_64"
os-edition: "64-bit"










Checkpoint Health Checks

----------------------------------------------
Checkpoint Special Config Files
----------------------------------------------
1. fwkern.conf - $FWDIR/boot/modules/fwkern.conf Magic Mac
2. local.arp   - $FWDIR/conf/local.arp   GAiA manual ARP
3. sdconf.rec  -  /var/ace  RAS authentication
4. rc.local    -  /etc/rc.d/rc.local
5. netconf.C      (/etc/sysconfig) Network interfaces/Routes
6. external.if    (/etc/sysconfig)
7. ifcfg-eth1      (/etc/sysconfig/network-scripts/)
----------------------------------------------
checkpoint scripts:
----------------------------------------------
1. checkup.sh
2. cpsizeme
3. nohup mpstat -P ALL 1 86400 > mpstat.out &
----------------------------------------------
Checkpoint Health Checks -Commands
----------------------------------------------
uptime
ver
fw ver
cpinfo -y all
cplic print
expert
df -h
reboot
shutdown
fwunload local

----------------------------------------------
Firewall Performance
----------------------------------------------
top
ps auxwww
fw tab -t connections -s
fw ctl pstat
fwaccel stats
tcpdump -i eth0 src 100.25.240.57 and dst 216.230.64.82


----------------------------------------------
Verfication:
----------------------------------------------
cat /etc/sysconfig/ntp
vmstat 1 10
cat /proc/meminfo
cpstat os -f cpu
cpstat os -f memory

Interface Configurations
------------------------
netstat -i
ifconig -a
netstat -rn
ethtool –i eth0
ethtool -S eth1-01
ethtool -S eth1-02


cpview
fw tab -t userc_key -s
fw tab -t userc_users -s
fw tab -t om_assigned_ips -s  (verify # of Seed license)

Cluster XL (High Avaiablility)
------------------------------
cpstop
cpstart
cphastop
cphastart
clusterXL_admin up/down
cphaprob –a if
cphaprob list
cphaprob stat
cpstat ha -f all
cphaprob syncstat
cphaprob list

cpconfig

--------------------------------------------------------------------------------
Performance -cpconfig utility enable/disable Checkpoint SecureXL
--------------------------------------------------------------------------------
fwaccel stats  (Usage: fwaccel on | off | ver | stat | conns | dbg <...> | help
fwaccel conns
fwaccel conns -s
fw ctl multik stat
fw ctl affinity -l -a -v
gaia> fwaccel conns -s  (checks SecureXL Connections)
gaia> fw ctl get int fwx_max_conns (checks maximum connectins 0 means it set to auto/unlimited

[Expert@myfwe-int02:0]# fw ctl multik stat  (connection to Core Distribution)
ID | Active  | CPU    | Connections | Peak
----------------------------------------------
 0 | Yes     | 11     |         178 |      303
 1 | Yes     | 10     |         203 |      380
 2 | Yes     | 9      |         168 |      262
 3 | Yes     | 8      |         179 |      188
 4 | Yes     | 7      |         149 |      278
 5 | Yes     | 6      |         113 |      194
 6 | Yes     | 5      |         128 |      221
 7 | Yes     | 4      |         282 |      387
 8 | Yes     | 3      |         186 |      292
 9 | Yes     | 2      |         296 |      439
[Expert@myfwe-int02:0]#


[Expert@myfwe-int02:0]# fw ctl affinity -l -a -v    (check CPU core to NIC Mapping -can be change in $FWDIR/conf/fwaffinity/conf)
Interface eth1-05 (irq 218): CPU 1
Interface Sync (irq 124): CPU 1
Interface eth1-01 (irq 107): CPU 1
Interface eth1-02 (irq 123): CPU 0
Interface eth3-01 (irq 171): CPU 0
Kernel fw_0: CPU 11
Kernel fw_1: CPU 10
Kernel fw_2: CPU 9
Kernel fw_3: CPU 8
Kernel fw_4: CPU 7
Kernel fw_5: CPU 6
Kernel fw_6: CPU 5
Kernel fw_7: CPU 4
Kernel fw_8: CPU 3
Kernel fw_9: CPU 2
Daemon in.geod: CPU all
Daemon vpnd: CPU all
Daemon in.acapd: CPU all
Daemon in.asessiond: CPU all
Daemon in.msd: CPU all
Daemon rad: CPU all
Daemon rtmd: CPU all
Daemon fwd: CPU all
Daemon mpdaemon: CPU all
Daemon usrchkd: CPU all
Daemon cpd: CPU all
Daemon cprid: CPU all
[Expert@hinfwe-int02:0]#

[Expert@myfwe-int02:0]# fwaccel stat
Accelerator Status : on
Accept Templates   : disabled by Firewall
                     disabled from rule #35
Drop Templates     : disabled
NAT Templates      : disabled by user

Accelerator Features : Accounting, NAT, Cryptography, Routing,
                       HasClock, Templates, Synchronous, IdleDetection,
                       Sequencing, TcpStateDetect, AutoExpire,
                       DelayedNotif, TcpStateDetectV2, CPLS, McastRouting,
                       WireMode, DropTemplates, NatTemplates,
                       Streaming, MultiFW, AntiSpoofing, ViolationStats,
                       Nac, AsychronicNotif, ERDOS, McastRoutingV2
Cryptography Features : Tunnel, UDPEncapsulation, MD5, SHA1, NULL,
                        3DES, DES, CAST, CAST-40, AES-128, AES-256,
                        ESP, LinkSelection, DynamicVPN, NatTraversal,
                        EncRouting, AES-XCBC, SHA256
[Expert@hinfwe-int02:0]#



[Expert@myfwe-int02:0]# fwaccel conns  |grep  216.231.83.228 | more
Source          SPort Destination     DPort PR Flags       C2S i/f S2C i/f Inst Identity
--------------- ----- --------------- ----- -- ----------- ------- ------- ---- --------
 216.231.83.228    53   74.94.152.161  1580 17 F..A...S... 7/8     8/7      7        0
   66.189.0.104 21318  216.231.83.228    53 17 ...A...S... 7/8     8/7      7        0
 216.231.83.228    53    50.204.98.98 39412 17 F..A...S... 7/8     8/7      9        0
 216.231.83.228    53    68.87.71.237 22618 17 F..A...S... 7/8     8/7      2        0
   71.243.0.148 21446  216.231.83.228    53 17 ...A...S... 7/8     8/7      5        0
  74.125.19.215 36506  216.231.83.228    53 17 F..A...S... 7/8     8/7      4        0
 216.231.83.228    53   216.19.226.66 18445 17 ...A...S... 7/8     8/7      8        0
 216.231.83.228    53    65.55.238.47 62154 17 F..A...S... 7/8     8/7      5        0
  216.231.65.79   467  216.231.83.228     0  1 F.......... 10/8    8/10     4        0

Usage: fwaccel on | off | ver | stat | cfg <...> | conns | dbg <...> | help
Options:
on-turns acceleration on
off-turns acceleration off
ver-show acceleration/FW version
stat-show acceleration status
cfg-configure acceleration parameters
stats-print the acceleration statistics
conns-print the accelerator's connection table
templates-print the accelerator's templates table
dbg-set debug flags
help-this help messages
diag-policy diagnostics


----------------------------------------------
Troubleshooting
----------------------------------------------
fw monitor | grep 10.210.7.250
fw ctl zdebug + drop > text.drops
fw ctl zdebug + drop | grep 204.105.57.69
tcpdump -ni eth8 src 172.30.25.132
tcpdump -i eth1 port 1089 and dst 216.118.184.254
netstat -rn |grep 204.105

RE: Traffic failing between internet  Clusters
Run a packet capture and a kernel debug on the firewall so I can get a packet-level look at what is happening to the traffic.
From expert mode on the Active Firewall:
1. # fwaccel off  (Turn off SecureXL, if enabled)
2. # df -h (Check your disk space to make sure you have sufficient space to run a capture and debug_
3. # fw monitor -o /var/log/fwmon.cap   (In one session: Run the capture.)
4. # fw ctl zdebug drop > /var/log/drop.txt  (In another session: Run the kernel debug for drops.)
5. # tcpdump -nnei any -w /var/log/tcp.cap (In a third session: Run a tcpdump capture.)
6. Re-create the problem.
7. Control-C   (End the fw monitor, tcpdump and the kernel debug with the following:)
8. # fwaccel on  (Turn on SecureXL, if you disabled it)


----------------------------------------------
log files
----------------------------------------------
$FWDIR/log/vpnd.elg
/var/log/messages
dmesg
/var/log/routed.log


----------------------------------------------
How to Set Specific
----------------------------------------------
clusterXL_admin down/up
ethtool -s eth1  autoneg on


--------------------------------------
/etc/resolv.conf    # What's the name resolution config -- sometimes performance is adversely influenced by bad DNS settings
/etc/ntpd.conf      # Time config
/etc/ntp.conf
/etc/modprobe.conf  # Any NIC or kernel tweaks?
/etc/sysctl.conf    # Any kernel tweaks?
/etc/ssh/sshd_config # Any hacks to sshd?
/etc/issue           # console banner file
/etc/issue.net       # network banner file
/etc/motd            # message of the day file
/etc/grub.conf       # Grub config -- important to see vmalloc
/etc/gated.ami       # gated config file
/etc/gated_xl.ami    # gated config file
/etc/rc.d/rc.local   # local RC files -- any changes here (such as kernel tweaks)?
$FWDIR/boot/boot.conf              # Firewall boot params
$FWDIR/boot/modules/fwkern.conf    # Any firewall kernel tweaks?
$PPKDIR/boot/modules/simkern.conf  # Any SIM tweaks?
$FWDIR/conf/discntd.if             # ClusterXL Disconnected Interfaces
$FWDIR/conf/local.arp              # SPLAT / GAiA manual ARP
$FWDIR/conf/vsaffinity_exception.conf      # Relevant to R75.40VS and later Virtual systems only
$MDSDIR/conf/external.if                   # Relevant to P1 / MDSM only


----------------------------------------------------------------------------------------------
ARPING
-----------------------------------------------------------------------------------------------
[myinet-fwa]# fw ctl arp
 (26.18.190.123) at 00-1c-7f-3f-6c-fd
 (26.18.190.100) at 00-1c-7f-3f-6c-fd


[myinet-fwa]# arping -I eth3-04 216.118.190.88 (check for Arping on specific interface for an IP)
ARPING 216.118.190.88 from 216.118.190.7 eth3-04
Sent 7 probes (7 broadcast(s))
Received 0 response(s)

[myinet-fwa]# arping -I eth3-04 26.18.190.87
ARPING 216.118.190.87 from 26.18.190.7 eth3-04
Sent 9 probes (9 broadcast(s))
Received 0 response(s)

[myinet-fwa]# arping -I eth3-04 26.18.190.89
ARPING 216.118.190.89 from 26.18.190.7 eth3-04
Sent 14 probes (14 broadcast(s))
Received 0 response(s)
[myinet-fwa]


-------------------------------------------------------------------------------------------
ClusterXL Troubleshooting
-------------------------------------------------------------------------------------------
Cluster XL (High Avaiablility)
cpstop
cpstart
cphastop
cphastart
clusterXL_admin up/down
cphaprob –a if
cphaprob list
cphaprob stat
cpwd_admin list

[Expert@mydev-fwa]# cphaprob stat

Cluster Mode:   New High Availability (Active Up)

Number     Unique Address  Assigned Load   State

1 (local)  192.168.42.1    100%            Active
2          192.168.42.2    0%              Standby

[Expert@mydev-fwa]#


[Expert@mydev-fwa]# cphaprob -a if

Required interfaces: 6
Required secured interfaces: 1

eth0       UP                    non sync(non secured), multicast
eth1       UP                    non sync(non secured), multicast
eth2       UP                    non sync(non secured), multicast
eth3       UP                    non sync(non secured), multicast
eth4       UP                    non sync(non secured), multicast
eth5       UP                    sync(secured), multicast

Virtual cluster interfaces: 5

eth0            172.30.25.54
eth1            10.125.240.4
eth2            10.125.242.4
eth3            10.125.244.4
eth4            10.125.246.4

[Expert@mydev-fwa]#


[Expert@mydev-fwa]# cphaprob list

Built-in Devices:

Device Name: Interface Active Check
Current state: OK

Registered Devices:

Device Name: Synchronization
Registration number: 0
Timeout: none
Current state: OK
Time since last report: 388872 sec

Device Name: Filter
Registration number: 1
Timeout: none
Current state: OK
Time since last report: 388866 sec

Device Name: cphad
Registration number: 2
Timeout: 2 sec
Current state: OK
Time since last report: 0.9 sec

Device Name: fwd
Registration number: 3
Timeout: 2 sec
Current state: OK
Time since last report: 0.9 sec

Expert@mydev-fwa]#

[Expert@mydev-fwa]# cpwd_admin list
cpwd_admin:
APP        PID    STAT  #START  START_TIME             COMMAND              MON
CPD        3449   E     1       [20:24:21] 7/6/2013    cpd                  Y
CI_CLEANUP 3534   E     1       [20:24:35] 7/6/2013    avi_del_tmp_files    N
CIHS       3546   E     1       [20:24:35] 7/6/2013    ci_http_server -j -f /opt/CPsuite-R71/fw1/conf/cihs.conf N
FWD        3548   E     1       [20:24:36] 7/6/2013    fwd                  N
RTMD       4051   E     1       [20:24:59] 7/6/2013    rtmd                 N
[Expert@mydev-fwa]#

cphaprob list. There reported issues with fib with problem state. Next ran cpwd_admin list and noticed that FIBMGR and DROUTER restarted 2x's.


[Expert@myvpn-fwb]# clusterXL_admin down
Setting member to administratively down state ...
Member current state is Down
[Expert@myvpn-fwb]# cphaprob stat

Cluster Mode:   New High Availability (Active Up)


Number     Unique Address  Assigned Load   State

1          192.168.25.241  100%            Active
2 (local)  192.168.25.242  0%              Down

[Expert@myvpn-fwb]# clusterXL_admin up
Setting member to normal operation ...
Member current state is Standby
[Expert@myvpn-fwb]#


Implement IPv6

Lab 11   Implementing IPv6 

IPv6 -  does not work on
Checkpoint Gaia .. kernel is old .. IPv6  is not running properly
Clustering issue


Lab  12 Routed VPN and Domain Based VPN 
Advanced VPN

Routed based VPN  (go away?)

  • unreliable internet connectivity
  • direct connection between peer security gateways using VTI 
  • VTI (virtual Tunnel Interface) uses as the security gateway to encryption domain pf a peer secuirty gateway

Cofiguring VTI for Route based VPN gateway



Domain Based VPN
Control how VPN traffic is routed between Security gateways and Remote Access Clients.

A-GW  -> IPSec_VPN -> Link Selection
Link Selection - Gateway A should talk to Gateway B via a specific Interface


IPSec VPN  -  Meshed Community property  Advanced Settings Wired Mode
Wire Mode (a VPN that does not inspection (Stateful inspection or blade or performance)
Improves connectivity
firewall can be bypassed for VPN connections
Configured in 2 places:
 community Properties
 Security Gateway Property





Admin Tool

Check Point Firewall Administrator's Tools



Check Point Backup Procedures
SSH
SCP
Vi
tar/gzip
Virtual CloneDrive
Create and Maintain Your Own Check Point Software Repository


Intermediate
cpinfo/InfoView
Scripts and Tools
VMware
Check Point Disaster Recovery


Advanced
Check Point Firewall Administrator's Tools
fw monitor
tcpdump
Wireshark


When you have to expand an ARP cache  not segmented/ large network eg Class B
ethtool NIC






#!/bin/bash

# Warning:
#
#     * Scripting is not a supported feature. The user
#       should implement scripts with care.  This is
#       only a demo of how sample code might work.
#
#  The script should be something like, overtime.sh and
#
# first, make sure that it's executable:
# chmod u+x overtime.sh
#
# then, run it:
# ./overtime.sh
#
# You'll get a file that has date time stamps in it.
#
# use common sense so that scripts do not run forever
# don't let a script fill your hard drive.  /var usually
# has the most space available for running scripts like this
#
# If you are getting timed out, run from a cron job without
# the while loop, or increase/remove idle time
#
# It should contain the following:
#

while true; do
  # adjust the date output to something like: 200707071200
  DATE=`/bin/date +%Y%m%d%H%M`

  # do your commands.  Note > overwrites, while >> appends
  echo $DATE >> SR-NUMBER.debug

  echo '------------------------------------' >> SR-NUMBER.debug
  vmstat -n 3 5 >> SR-NUMBER.debug

  echo '------------vmstat------------------' >> SR-NUMBER.debug
  cat /proc/meminfo >> SR-NUMBER.debug

  echo '-------procmeminfo------------------' >> SR-NUMBER.debug
  fw tab -t connections -s >> SR-NUMBER.debug

  echo '-------------fwtab------------------' >> SR-NUMBER.debug
  top -n 1 >> SR-NUMBER.debug

  echo '--------------top-------------------' >> SR-NUMBER.debug
  fw ctl pstat >> SR-NUMBER.debug

  echo '--------------free------------------' >> SR-NUMBER.debug
  free >> SR-NUMBER.debug

  echo '------------------------------------' >> SR-NUMBER.debug

  # sleep is measured in seconds, 1200 = 10 minutes.
  sleep 2400

done

FW Monitor Reference


A quick debugging reference sheet of all usable options for the fw monitor tool.
By default the fw monitor sniffing driver is inserted into the 4 locations on the Firewall kernel chain .

Here they are:
 i (PREIN) – inbound direction before firewall Virtual Machine (VM, and it is CP terminology) .

Most important fact to know about that is that this packet capturing location shows packets BEFORE any security rule in the policy is applied. That is, no matter what rules say a packet should at
least be seen here, this would prove that packets actually reach the firewall at all.

 I (POSTIN) – inbound direction after firewall VM.
 o (PREOUT) – outbound direction before firewall VM,
 O (POSTOUT) – outbound direction after firewall VM.

You can change point of insertion within the fw chain with :

# fw monitor –p<i|I|O|o> <where to insert>

Easiest way to specify where to insert is to first see the chain:

# fw ctl chain 
then give relative to any module you see there <+|->module_name
Now the usage itself:

# fw monitor
Usage: fw monitor [- u|s] [-i] [-d] [-T] <{-e
expression}+|-f <filter-file|->> [-l len] [-m mask] [-x offset[,len]]
[-o <file>] <[-pi pos] [-pI pos] [-po pos] [-pO pos] | -p all [-a
]> [-ci count] [-co count]

Round up of options:
-m mask , which point of capture is to be displayed, possible: i,I,o,O
-d/-D debug output from fw monitor itself, not very useful IMO.
-u|s print also connection/session Universal ID
– i after writing each packet flush stdout
-T add timestamp, not interesting
-e expr expression to filter the packets (in detail later)
-f filter_file the same as above but read expression from file
-l <len> packet length to capture
Expressions
On the very low level fw monitor understands byte offsets from the header
start. So to specify for example 20th byte of the IP packet (that is source IP)
you can just use:



# fw monitor -e 'accept [12,b]=8.8.8.8;'

Where:
12 – offset in bytes from the beginning of the packet
b – mandatory, means big endian order.
4 – not seen here but size (in bytes) of how many bytes to look for from the starting offset (default is 4 )
To look for source port 53 (UDP/TCP) in raw packet:
# fw monitor -m i -e 'accept [20:2,b]=53;'

Here I say to fw monitor to look at 2 bytes at offset 20.
While this way of looking at packets is the most general and therefore includes all cases, you rarely have the need for such a granular looking glass. In 99% of the cases you will be doing alright with a limited known set of expressions.

Just for that Checkpoint defined and kindly provided us in every Splat installation with definition files that give meaningful synonyms to the most used patterns. There are few definition files but they circularly reference each other providing multiple synonyms for the same pattern.
I put all those predefined patterns in the list below for the easy to use reference.


Summary table of possible expressions to be fed to the fw
monitor
Specifying
Hosts
 host(IP_address) to or from this host
 src=IP_address where source ip = IP_address
 dst=IP_address where destination ip = IP_address
 net(network_address,netmask) to or from this network
 to_net(network_address,netmask) to this network
 from_net(network_address,netmask) from this network
 Specifying ports
 port(port_number) having this source or destination port
 sport=port_number having this source port
 dport=port_number having this destination port
 tcpport(port_number) having this source or destination port that is also TCP
 udpport(port_number) having this source or destination port that is also UDP
 Specifying protocols
 ip_p=<protocol_number_as_per_IANA> this way you can specifiy any known protocol by its registered
number in IANAFor detailed list of protocol numbers see www.iana.org/assignments/protocol-numbers
 icmp what it says , icmp protocol
 tcp TCP
 udp UDP
 Protocol specific oprions
 IP
 ip_tos = <value> TOS field of the IP packet
 ip_len = <length_in_bytes> Length of the IP packet in bytes
 ip_src/ ip_dst = <IP_address> Source or destination IP address of the packet
 ip_p =<protocol_number_as_per_IANA> See above
 ICMP
  echo_reply ICMP reply packets
  echo_req Echo requests
  ping Echo requests and echo replies
  icmp_error ICMP error messages (Redirect,Unreachables,Time exceeded,Source
quench,Parameter problem)
  traceroute Traceroute as implemented in Unix (UDP packets to high ports)
  tracert Traceroute as implemented in Windows (ICMP packets , TTL
<30)
  icmp_type = <ICMP types as per RFC> catch packets of certain type
  icmp_code = <ICMP type as per RFC> catch packets of certain code
  ICMP types and where applicable respective codes:ICMP_ECHOREPLY
ICMP_UNREACH
      ICMP_UNREACH_NET
      ICMP_UNREACH_HOST
      ICMP_UNREACH_PROTOCOL
      ICMP_UNREACH_PORT
      ICMP_UNREACH_NEEDFRAG
      ICMP_UNREACH_SRCFAIL
ICMP_SOURCEQUENCH
ICMP_REDIRECT
      ICMP_REDIRECT_NET
      ICMP_REDIRECT_HOST
      ICMP_REDIRECT_TOSNET
      ICMP_REDIRECT_TOSHOST
ICMP_ECHO
ICMP_ROUTERADVERT
ICMP_ROUTERSOLICIT
ICMP_TIMXCEED
      ICMP_TIMXCEED_INTRANS
      ICMP_TIMXCEED_REASS
ICMP_PARAMPROB
ICMP_TSTAMP
ICMP_TSTAMPREPLY
ICMP_IREQ
ICMP_IREQREPLY
ICMP_MASKREQ
ICMP_MASKREPLY
  icmp_ip_len = <length> Length of ICMP packet
 icmp_ip_ttl = <TTL> TTL of ICMP packet, use with icmp protocol otherwise will catch ANY
packet with TTL given
 < cut here—-bunch of other icmp-related fields
like ID ,sequence I don’t see any value in bringing here–>
 TCP
 syn SYN flag set
 fin FIN flag set
 rst RST flag set
 ack ACK flag set
 first first packet (means SYN is set but ACK is not)
 not_first not first packet (SYN is not set)
 established established connection (means ACK is set but SYN is not)
 last last packet in stream (ACK and FIN are set)
 tcpdone RST or FIN are set
 th_flags – more general way to match the flags inside
TCP packets
 th_flags = TH_PUSH Push flag set
 th_flags = TH_URG Urgent flag set
 UDP
 uh_ulen = <length_in_bytes> Length of the UDP header (doesnt include IP header)
  
And the last thing to remember before we move to examples – expressions support logical operators and numerical values support relative operators:
and – logical AND
or – logical OR
not – logical NOT
> MORE than
< LESS than
>= MORE than or EQUAL to
<= LESS than or EQUAL to

You can combine logical expressions and influence order by using ()

Below is laundry list of examples to showcase the reference table above.

# fw monitor -m i -e 'accept host(208.44.108.136) ;'
# fw monitor -e 'accept src=216.12.145.20 ;'  packets where source ip = 216.12.145.20
# fw monitor -e 'accept src=216.12.145.20 or dst= 216.12.145.20;'  packets where source or destination ip = 216.12.145.20
# fw monitor -e 'accept port(25) ;'  packets where destination or source port = 25
# fw monitor -e 'accept dport=80 ;'  packets where destination port = 80
# fw monitor -e 'accept sport>22 and dport>22 ; '  packets with source and destination ports greater than 22
# fw monitor -e 'accept ip_len = 1477;'  packets where their length equals exactly 1477 bytes
# fw monitor -e 'accept icmp_type=ICMP_UNREACH;'  ICMP packets of Unreachable type
# fw monitor -e 'accept from_net(216.163.137.68,24);'  packets having source IP in the network 216.163.137.0/24
# fw monitor -e 'accept from_net(216.163.137.68,24) and port(25) and dst=8.8.8.8 ;'  packets coming from network 216.163.137.0/24 that are destined to the host 8.8.8.8 and hving source or destination port = 25

# fw monitor -m i -x 40,450 -e 'accept port(80);'  incoming packets before any rules are applied also
display contents of the packet starting at 40th byte of 450 bytes length

# fw monitor -m i -pi -ipopt_strip -e 'accept host(66.240.206.90);'  incoming packets from/to host 66.240.206.90 , insert sniffer before module named ipopt_strip
# fw monitor -D -m i -pi -ipopt_strip -e 'accept host(66.240.206.90);'  same as above but add debug info


There is something I didn’t include in the previous post fw monitor command reference about fw monitor as I think it is rather optional and you can do well without it . I talk about tables in defining filter expressions. INSPECT – proprietary scripting language by the Checkpoint on which filtering expressions are based allows creating tables.
I won’t delve into INSPECT syntax (for today) but will list the following examples you can easily modify to suit your needs.
Legend:
{} – delimit the table
<,> – specify range of values inside (e.g. <22,25> means from 22 up to 25 inclusive)
ifid – interface identifier
#fw monitor -e "bad_ports = static {22,25,443}; accept dport in bad_ports;”   packets with destination port bein" equal to 22,25 or 443
#fw monitor -e " bad_ports = static {<22,25>} ; accept dport in bad_ports;"  packets with destination ports being equal to 22,23,24 or 25
# fw monitor -e " bad_ports = static {<22,25>,<80,443>} ; accept dport in bad_ports;"  packets with destination ports being in ranges 22-25 or 80-443
#fw monitor -e "bad_nets = static {<194.1.0.0,194.1.255.255>} ;accept src in bad_nets;"  packets originated in range of networks 194.1.0.0 – 194.1.255.255
#fw ctl iflist   Here I see what are the index values of each interface card
0 : Internal
1 : External
#fw monitor -e "bad_nets = static {<194.1.0.0,194.1.255.255>} ;accept src in bad_nets and ifid=0;"  packets originated in range of networks 194.1.0.0 – 194.1.255.255 and captured on interface eth3 only








[Expert@bostestint-fwa:0]# fw ctl chain
in chain (11):
        0: -7f800000 (ffffffff8882b820) (ffffffff) IP Options Strip (in) (ipopt_strip)
        1: - 1fffff8 (ffffffff8882d050) (00000001) Stateless verifications (in) (asm)
        2: - 1fffff7 (ffffffff8886eb00) (00000001) fw multik misc proto forwarding
        3: - 1000000 (ffffffff88911d10) (00000003) SecureXL conn sync (secxl_sync)
        4:         0 (ffffffff887ceee0) (00000001) fw VM inbound  (fw)
        5:        10 (ffffffff887e3690) (00000001) fw accounting inbound (acct)
        6:  10000000 (ffffffff88910330) (00000003) SecureXL inbound (secxl)
        7:  7f600000 (ffffffff88820bf0) (00000001) fw SCV inbound (scv)
        8:  7f730000 (ffffffff88a35fb0) (00000001) passive streaming (in) (pass_str)
        9:  7f750000 (ffffffff88c53260) (00000001) TCP streaming (in) (cpas)
        10:  7f800000 (ffffffff8882bbe0) (ffffffff) IP Options Restore (in) (ipopt_res)
out chain (10):
        0: -7f800000 (ffffffff8882b820) (ffffffff) IP Options Strip (out) (ipopt_strip)
        1: - 1fffff0 (ffffffff88c534a0) (00000001) TCP streaming (out) (cpas)
        2: - 1ffff50 (ffffffff88a35fb0) (00000001) passive streaming (out) (pass_str)
        3: - 1f00000 (ffffffff8882d050) (00000001) Stateless verifications (out) (asm)
        4: -     1ff (ffffffff88e6eb80) (00000001) NAC Packet Outbound (nac_tag)
        5:         0 (ffffffff887ceee0) (00000001) fw VM outbound (fw)
        6:  10000000 (ffffffff88910330) (00000003) SecureXL outbound (secxl)
        7:  7f000000 (ffffffff887e3690) (00000001) fw accounting outbound (acct)
        8:  7f700000 (ffffffff88c53690) (00000001) TCP streaming post VM (cpas)
        9:  7f800000 (ffffffff8882bbe0) (ffffffff) IP Options Restore (out) (ipopt_res)
[Expert@bostestint-fwa:0]#



with FW Monitor running

[Expert@bostestint-fwa:0]# fw ctl chain
in chain (13):
        0: -7f800000 (ffffffff8882b820) (ffffffff) IP Options Strip (in) (ipopt_strip)
        1: -70000000 (ffffffff88804d70) (ffffffff) fwmonitor (i/f side)
        2: - 1fffff8 (ffffffff8882d050) (00000001) Stateless verifications (in) (asm)
        3: - 1fffff7 (ffffffff8886eb00) (00000001) fw multik misc proto forwarding
        4: - 1000000 (ffffffff88911d10) (00000003) SecureXL conn sync (secxl_sync)
        5:         0 (ffffffff887ceee0) (00000001) fw VM inbound  (fw)
        6:        10 (ffffffff887e3690) (00000001) fw accounting inbound (acct)
        7:  10000000 (ffffffff88910330) (00000003) SecureXL inbound (secxl)
        8:  70000000 (ffffffff88804d70) (ffffffff) fwmonitor (IP  side)
        9:  7f600000 (ffffffff88820bf0) (00000001) fw SCV inbound (scv)
        10:  7f730000 (ffffffff88a35fb0) (00000001) passive streaming (in) (pass_str)
        11:  7f750000 (ffffffff88c53260) (00000001) TCP streaming (in) (cpas)
        12:  7f800000 (ffffffff8882bbe0) (ffffffff) IP Options Restore (in) (ipopt_res)
out chain (12):
        0: -7f800000 (ffffffff8882b820) (ffffffff) IP Options Strip (out) (ipopt_strip)
        1: -70000000 (ffffffff88804d70) (ffffffff) fwmonitor (i/f side)
        2: - 1fffff0 (ffffffff88c534a0) (00000001) TCP streaming (out) (cpas)
        3: - 1ffff50 (ffffffff88a35fb0) (00000001) passive streaming (out) (pass_str)
        4: - 1f00000 (ffffffff8882d050) (00000001) Stateless verifications (out) (asm)
        5: -     1ff (ffffffff88e6eb80) (00000001) NAC Packet Outbound (nac_tag)
        6:         0 (ffffffff887ceee0) (00000001) fw VM outbound (fw)
        7:  10000000 (ffffffff88910330) (00000003) SecureXL outbound (secxl)
        8:  70000000 (ffffffff88804d70) (ffffffff) fwmonitor (IP side)
        9:  7f000000 (ffffffff887e3690) (00000001) fw accounting outbound (acct)
        10:  7f700000 (ffffffff88c53690) (00000001) TCP streaming post VM (cpas)
        11:  7f800000 (ffffffff8882bbe0) (ffffffff) IP Options Restore (out) (ipopt_res)
[Expert@bostestint-fwa:0]#




Inbound
-------
NIC
Wireside Acct/Virtual Reass    IP Options Strip (in) (ipopt_strip)
VPN Dec
VPN verify
VM/NAT
Accounting
VPN Policy
FG Policy
IQ Engine
RTM/E2E
TCP/IP



Outbound
-------
TCP/IP
Virtual Reass/Wireside Acct
VM/NAT
VPN Policy
FG Policy
VPN Enc
IQ Engine
Accounting
RTM/E2E
NIC


Reference: http://yurisk.info/2009/12/12/fw-monitor-command-reference/