Tuesday, June 14, 2016

New Gateway Ping Test


if you build a new gateway and wants to test connectivity across the interfaces and the ping test does not work, do the following:

[Expert@myfw-fwa:#   fw unloadlocal
[[Expert@myfw-fwa:#  cpstop
[Expert@myfw-fwa:#   sbin/sysctl -w net.ipv4.ip_forward=1

Saturday, June 4, 2016

The History of Stateful Inspection

Stateful Inspection, who invented it, really?

Patent rights to Stateful Inspection belong to Check Point and Gil Shwed, according to US patent 5,606,668. The patent application was filed in 1993.

However,  Palo Alt's  Nir Zuk does not hesitate crediting himself for developing Stateful Inspection.
Here is the quote from Palo Alto web site: "Nir was... a principal engineer at Check Point Software Technologies, and was one of the developers of stateful inspection technology."

It is very accurate, isn't it? "One of the developers..." Of course he was, he worked with Check Point from 1994 till 1999. He could not possible invent it, considering half a year gap between filing the patent application and his start in Check Point.

Should not confuse us much, right? It must be something else.

In Nir's interview to IT World in April 2010 when talking about his Check Point years he says:  "We invented a technology called stateful inspection, on which all network security technology today is based." This is a bigger statement, but again, if by "we" he means "we, Check Point", there is probably nothing wrong there.

The article itself is peculiar. There he also claims he single-handedly developed Floodgate product in 1999 and then was practically pushed out of the company by Israeli developers for doing so. I will not comment on that, as we are discussing Stateful Inspection topic only.

There is one more article from 2008 published by InformationWeek. The title is very promising: "Who invented the firewall".

Believe it or not, the very first sentence credits Nir Zuk. It says: "Nir Zuk says he developed the technology used in all firewalls today." The article is brilliant, it its own way.

The author, Kelly J. Higgins is apparently not fluent with FW technical terms. I would not blame her much, although her attempt to crack the case does not deserve full marks, in my humble opinion. She quotes one of the experts in the article though, saying: "Zuk was the father of the stateful firewall product at Check Point…"

Funny, Check Point is only mentioned in the article if if referenced to Nir Zuk. Here is another example: "Meanwhile, Zuk, who helped build Check Point's firewall technology, isn't shy about taking credit for the first commercial firewall."

If I would read this without any background, it would be clear to me: Nir does it all. He is the father, developer, inventor and, finally, helper. All praise Nir.

Well, the whole story is a very good example of bold PR (If you have ever seen Palo Alto marketing presentation, you know what I am referring to) and inaccurate journalism.

Nir Zuk was one of the developers, all right. The rest is just noise on the channel

Article Credit:

Checkpoint R80



Checkpoint R80

CPX 2016 Summary

SmartConsole
SmartLog (new stuff at the top)

SmartView Tracker
Smart Log
Smart Dashboard  

SmartTracker  (may go away in R80)

https://www.cpug.org/forums/showthread.php/21170-New-R80-publication-on-the-way
http://www.maxpowerfirewalls.com/addendums.html

Some topics discussed are:


Checkpoint R80 is released as Management only.  The Security Gateways will be released laster this year. 

Noted Changes/Enhancements:
1. New UI of SmartConsole as a single application
2.  Multiple admin access to the same database
3.  New MGMT API that is much more powerful than old
4. OPSEC almost obsolete


R80 GA materials are avaiable for download. They were originally posted on 31st of March. 

A new book covering Check Point R80  has been in production for 5 months now and is nearing completion. This new book is the first of a series and has a working title of “New Frontier: Check Point R80”. It is a collaborative effort featuring four recognized professionals with quite a lot of Check Point expertise (in alphabetic order)

- Kishin Fatnani – India
- DK (author of this blog) 
- Valeri Loukine - Switzerland 

More information will be available soon, stay tuned!


Firewall Basics
----------------

Routing device with security policy enforcement capabilities
It allows controlling traffic between networks.
Firewall is part of the system Kernel
it resides before and after (inbound/outbound) the OS IP Stack
Packets cross the FW twice when going thru a security device
only exception is when traffic directed to or originated by the firewall appliance itself


Unix Based Box - IP Stack/Network Devices/NIC Card
Unix Based Box with Security - IP Stack/Firewall Kernel(inbound/Outbound)/Network Devices/NIC Card


Stateful Inspection/FW Kernel tables
-------------------------------------
stateful inspection was invented by Checkpoint and Gil Shwed.
Checkpoint Firewall -1 is the first stateful firewall
Patent Number USA  5,606668  Patent Date: Feb25, 1977

Static Packet Filtering
Unix based forward packet forwarding capabilities - Only IP address and ports are used to make security decision
Usually based on access list.
Not enough for proper security.

OSI Model
----------

Applicaiton  (HTP/FTP/SMTP)
Presentation (HTML/CSS/GIF)
Session      (RPC/SSL,SQL)
Transport    (TCP/UDP)
Network      (Packets  IPv4, IPv6)
Data Link    (Frames/MAC Address)
Physical     (Ethernet/ISDN/DSL)


Stateful Inspect from layer 3-7  (Dynamic Packet Filtering)
Stateful inspection -fw technology monitors and in forces the state of connections
Kernel Kernel - Dynamic Kernel Table  (Connection Table)

Firewall Security Enforcement

Source/Destination/Device/Action -accept-drop-reject/Tracking/

Packet  -> Firewall Kernel-> New Connection? -> Checks  Dynamic State Table -> Yes -> Found a Rule -> Yes -> Action -> Accept/Reject/Drop
Packet  -> Firewall Kernel -> New Connection? -> Checks Dynamic State Table - NO -> Packet Forwarded
                                   
Accept action do not require reulebase look-up for every packet in the connection after the very first one.                                     Dropped connection are not listed in the connection table
Accpet actins requires less efforts that Drop
differene between reject and drop
   
 

Best Practices/Optimization/Troubleshooting
firewall Maintenance
Packet flow
Address Translation
Stateful Inspection
Security Acceleration
Multi-Core Systems and CoreXL
Kernel debug



R80.10 Early Availability Program

As we are expanding our EA installations I would like to invite you to participate in the Early Availability Program of R80.10. Candidates should commit their Production environment in order to get on-site Early Availability engineer.

The registration process consists of filling out a short questionnaire to characterize the candidates for this EA program and NDA papers.
In order to register to the new R80.10 please fill in the following questionnaire (link below) and our early availability engineer will contact you.

WHAT’S NEW IN R80.10?

Check Point recently released Next Generation Security Management R80 setting the standard for reliability and ease-of-use in security management.

Check Point R80.10 extends R80 functionality to complete our vision for security consolidation, unified policy, and integrated threat management..

R80.10 will include:

Unified Access & Data Policy


1. Unified Policy

  • Unified security rule-base for Access blades: Firewall, VPN, Application Control, URL Filtering, Data Awareness, Mobile Access Blade
  • Unified log for network, protocol, application, user, accessed resources, file and data types.



2. Powerful Policy Model Architecture

  • Layered policy to support delegation and segregation of duties
  • Sub policy to define a set of rules as one management unit, independent from the rest of the rule-base.
  • Security zones bound to network interfaces to simplify security policy management.



3. Firewall and Application Control Enhancement
  • Application criteria now includes match by recommended services and by application signature.
  • Service criteria now includes match by protocol signature and by service port.


4. Integrated Data Awareness

  • Data Awareness adds file types, data types, and direction in the new unified policy, combining data with other security policy objects for granular rules..



5. Additional Enhancements:

  • New FQDN mode, to match fully qualified domain name of Domain Objects.
  • Domain Objects and Dynamic Objects support SecureXL accept templates.





Identity Awareness:


  • Large scale Identity Awareness, for support of 200K users.
  • Identity Collector Agent to collect user information from different identity sources (AD/ISE).
  • Web REST API for IDA.
  • LDAPv3 support for better nested group handling.



Mobile Access:


  • Support Mobile Access in the unified rule base of R80 / R80.10.
  • Multiple Login Options, and multiple authentication factors, for Mobile Access and IPSec VPN. 


VPN:


  • Multi-core for enhanced performance of VPN (Site-to-Site and Remote-Access VPN).
  • Security Gateways behind NAT use of NAT-T to initiate VPN site-to-site tunnel.


Threat Prevention:


  • IPS is now part of the Threat Prevention policy, with multiple profiles per gateway and all Threat Prevention blades managed in one rule.
  • Threat Prevention Policy installation time considerably improved.
  • Threat Prevention Policy support for multi-layers, adding flexibility. 


Additional Features:


  • SandBlast Threat Extraction immediately provides users with clean, reconstructed files containing only known safe element
  • Support of TLS 1.2 in Mobile Access connections and portals that do not work through multi-portal system. 


Upgrade Method:

  • Upgrade to R80.10 and onward will be available online & offline through Check Point’s upgrade engine (CPUSE).





Free "Max Power" Tips, Tricks & R77.30 Addendum Now Available

Hi Everyone,

I've posted a free addendum to my "Max Power: Check Point Firewall Performance Optimization" book at http://www.maxpowerfirewalls.com. This 14-page PDF addendum includes additional reader-submitted tips and tricks not included in the book, along with an overview of the new firewall performance-related features and fixes in R77.30. I have copy/pasted the additional material (with page number references) below with a minimum of formatting for the benefit of the Google site spiders and any community-based discussion that may ensue. To download a much more pretty version of these including more structured formatting, hotlinks to the relevant Check Point SK's, various screenshots, and a 2-page introduction to this content, it can be downloaded for free from http://www.maxpowerfirewalls.com. The PDF addendum document may be freely copied and distributed as long as its content and authorship remains intact.

I'd like to thank Phoneboy and Eric Anderson for reviewing this addendum and all the readers that submitted tips and feedback. Enjoy!

Supplementary Material by Page Number

Page 16: If your site does not have the Monitoring Blade present, be sure to check out
the nmon tool discussed in the Page 58 entry below.

Page 21: If you are unlucky enough to be forced to utilize Emulex NICs (driver name
be2net) on your firewall, be aware that a nasty firewall stability issue involving these
NICs was fixed in R77.30 and R77.20 jumbo hotfix Take 94 and later. You'll definitely
want to install this fix if using Emulex NICs on your firewall.

Page 26: The book recommended always using an even number of physical interfaces
in a bonded aggregate Ethernet interface. After some reader questions, I dug into it a
little further, as this has been an unofficial recommendation floating around for quite
some time. While I was not able to learn the exact nature of the issue, I was assured that
it was an Intel driver issue and that it was fixed in R77.30. However, of the four main
Intel drivers shipped with Gaia R77.20 (e1000, e1000e, igb, ixgbe), only the e1000e
driver was updated (from version 1.2.20 to 2.1.4) in the R77.30 release. So unless your
firewall is using the e1000e driver (igb and ixgbe are by FAR the most common though),
this recommendation does not appear to be valid. It is also possible that this
recommendation is a bit of a myth, created by the fact that some networking vendors do
not support using an odd number of physical interfaces when aggregating them using the
older EtherChannel technique. If you have further insights, or would like to keep abreast
of the evolving knowledge on this topic, stay tuned to this thread at CPUG:
https://www.cpug.org/forums/showthre...-Joining-Bonds

Page 34: One other potential STP-related issue pointed out by a student of mine, is that
different variants of the spanning tree algorithms don't mix well. As an example, if two
switches are connected together and one of them is using the original 802.1D standard
STP and the other is using Rapid STP, the various timers will be radically different
between the two and cause network stability issues.

Page 49: ICMP isn't just all about ping and traceroute; the various types and codes of
ICMP datagrams can sometimes indicate that performance-impacting conditions are
occurring within the network. Running a netstat -s on the firewall shows counters for
how many different types of ICMP messages have been received by the firewall.
Particular ones that can impact performance and be helpful to investigate further are:
- Fragmentation required but DF set (Type 1, Code 4)
- Precedence cutoff in effect (Type 1, Code 15)
- Source Quench (Type 4, Code 0) – very rare
- Redirect (Type 5)
- Time Exceeded (Type 11)

If nonzero values are noted for any of these in the netstat -s output, it is entirely
possible they came from the Internet and you have no control over their generation.
However, seeing these types of ICMP datagrams arriving on the firewall's internal
interfaces via tcpdump should be checked out. To display all ICMP traffic on an internal
interface that is not associated with ping testing traffic, use this command:
tcpdump -eni (interface name) icmp and not icmp[0]=0 and not icmp[0]=8

Page 58: One additional built-in CPU profiling tool brought to my attention is nmon:
Added to Gaia in R76, it serves many of the same functions as the top command
for monitoring CPU usage on the firewall, but in a more graphical format. As an
example, typing lowercase "L" provides a CPU usage graph; hitting "c" will show a
graph for a for multi-core systems. While most of these CPU monitoring statistics are
also available in top, a significant value-add of nmon is the ability to monitor and graph
disk usage, which is very handy if the wa percentage shown by top is excessive. Nmon
can also graph and monitor network interface activity, and serve as a useful stand-in for
the Traffic and System Counters reports that are available via the SmartView Monitor,
but only when a Monitoring Blade license is present. Thanks to Yasushi Kono of Arrow
ECS for submitting this tip.

Page 59: An easier way to see if cpwd has restarted any firewall processes since the
last cpstart or firewall boot is to run cpview then hit Overview. Down-arrow to the
bottom of the page, and you will see the counter "# of monitored daemons crashes since
last cpstart". If this value is nonzero, run cpwd_admin list to determine which daemon(s) are
having a problem.

Pages 59-60: If while running top you notice a process called kipmi0 consuming an
excessive amount of CPU on an open hardware firewall, this is a known issue and you
should consult sk104316: kipmi0 daemon consumes CPU at 100% on Open Servers
running Gaia OS.

Page 76: In addition to hitting “1” while running top to see individual core utilizations,
the command cpstat os -f multi_cpu can also be used to obtain this information.
Thanks to Yasushi Kono of Arrow ECS for submitting this tip.

Pages 84-87: Check Point has created an all-new SK documenting Security Policy
best practices here: sk106597: Best Practices - Rulebase Construction and Optimization.

Page 89: As stated in the book, setting fw_rst_expired_conn to 1 should always be
tried first to gracefully terminate application-based connections that aren't closing
properly and impacting perceived application performance. In some cases, however, this
will not fully remediate the situation, and you will be forced to go one step further with
this: fw ctl set int fw_reject_non_syn 1. A classic example of an application that
requires this firewall setting is SAP HANA traffic. This setting also handles client port
reuse out of state errors when RST packets from the server to the clients get lost (e.g. due
to policy install or packet loss).

Bear in mind, however, that this setting is quite likely to make your “Allsafe
Cybersecurity” auditor/penetration tester upset with you, since the firewall will now issue
a TCP RST for all received packets that are out of state and have the ACK flag set. An
auditor running a TCP ACK nmap scan will have it light up like a Christmas tree, with
tens of thousands of ports showing up as filtered instead of closed. For this reason,
setting fw_reject_non_syn to 1 is generally not recommended on an Internet perimeter
firewall, but may be acceptable on internal firewalls. Thanks to Andrew Craick of
Dimension Data for submitting this tip.

Page 90: The TCP State Logging function was introduced in R77, and is not available
on older firewalls. An alternative to this feature on pre-R77 firewalls is using the
Account option in the Track column of a rule. When this option is set for a rule, an
Accept entry is created at the start of the connection, just as it is when the Track is set to
Log. However, once the connection finishes (FIN, RST, idle time out etc.), the existing
log entry is converted from a Log type to an Account type. Additional statistics are then
provided for the connection, including the connection duration and number of
payload/data bytes sent and received by the connection. These statistics can be used to
infer the connection's behavior and assist in troubleshooting.

Page 97: R77.30 has added the ability to set the “Magic MAC” value via the Gaia web
interface, instead of by hand-editing the fwkern.conf file. During the firewall's postinstallation
dialog in the Gaia web interface, if “Unit is part of a cluster” is checked, the
new field “Cluster Global ID” will become editable:
The Cluster Global ID should be set identically on all members of the same
cluster, but be a unique value for different clusters. The Cluster Global ID can also be
configured and verified from the CLI in R77.30 and later. The command cphaconf
cluster_id get will display the current setting, and cphaconf cluster_id set <Cluster
ID Value> can be used to modify it. See sk25977: Connecting multiple clusters to the
same network segment (same VLAN, same switch) for more information. Thanks to Eric
Anderson of Netanium for submitting this tip.

Page 139: Some additional commands to check CoreXL licensing status are:
[Expert]# fw ctl get int fwlic_num_of_allowed_cores
fwlic_num_of_allowed_cores = 8
[Expert]# fw ctl get int fwlic_num_of_allowed_cpus
fwlic_num_of_allowed_cpus = 8
Thanks to Yasushi Kono of Arrow ECS for submitting this tip.

Pages 141 & 146: On these pages it was mentioned that SecureXL can accelerate
some IPSec VPN encryption/decryption operations. If SecureXL is enabled on your
firewall and you'd like to check if this is occurring, run fwaccel stats. Nonzero or
rapidly incrementing values in the Accelerated VPN Path section of the output indicate
that SecureXL acceleration of IPSec traffic is occurring.

Pages 149-151: I'm pleased to report that R77.30 has added the option to substantially
improve Firewall Worker Core load distribution via the new Dynamic Dispatcher Feature
(sk105261: CoreXL Dynamic Dispatcher in R77.30). This new Firewall Worker Core
load-balancing feature is disabled by default in R77.30; as a general rule of thumb you
should consider enabling this feature when the following conditions are present*:
- Firewall has 6 or more total cores
- Firewall Worker CPU loads consistently vary from each other by >10% **
- Firewall is NOT using a SAM card (i.e. 21000 series)

* Enabling this feature may break VoIP traffic being processed by the firewall
without a special hotfix, see sk106665: VoIP traffic, or traffic that uses reserved
VoIP ports is dropped after enabling CoreXL Dynamic Dispatcher.

** Keep in mind that all IPSec VPN and VoIP traffic can only be processed on
the lead (lowest-numbered) Firewall Worker Core as specified on page 141 (this
limitation has still not been lifted in R77.30). If there is substantial IPSec and/or
VoIP traffic traversing the firewall, exclude the lead Firewall Worker Core from
consideration when applying the 10% rule of thumb above.

Page 162: When attempting to re-enable SecureXL with IPSec VPNs present, watch
for this issue: sk102742: When SecureXL is enabled, traffic through the VPN trusted
interface is sent encrypted instead of clear. A separate hotfix must be obtained (this fix
does not appear to be included in the current R77.20 jumbo hotfix) or you can upgrade to
R77.30.

Page 169: While fwaccel stats -s provides useful acceleration packet counters
showing total number of packets processed by the SXL/PXL/F2F processing paths, you
can also view live throughput numbers for each of the three paths expressed in pps and
Mbps. Run cpview then hit Advanced...Network...Path:

Page 173: There are a plethora of stability fixes for 21000-series firewall models that
utilize a SAM card in R77.30. If using a SAM card, upgrading to R77.30 (or at least
loading the latest R77.20 jumbo hotfix) is highly recommended.

Page 176-178: Correction: Changing the IPS Scope setting from “Perform Inspection
on all Traffic” to “Protect internal hosts only” does NOT potentially make more traffic
eligible for the Accelerated Path. Setting “Protect internal hosts only” has a similar effect
to creating an IPS Exception, in that it can save CPU time in the Medium Path (PXL). So
while changing this setting does have a positive impact on performance (by potentially
saving CPU time in the Medium Path), it is not for the reason originally stated in the
book (that more traffic is made eligible for Accelerated Path).

Page 194: This section of the book spends a great deal of time trying to reduce firewall
CPU load on the Firewall Worker Cores, most of which occurs in the Medium Path
(PXL) on the vast majority of real-world firewalls. R77.30 has introduced an exciting
ability to view the top connections by CPU usage. This capability is a subset of the new
R77.30 Firewall Priority Queues feature (sk105762: Firewall Priority Queues in R77.30),
and the good news is that this helpful information can be obtained without having to fully
enable this feature. To obtain this ability, run the following command: fw ctl multik
set_mode 1 and reboot the firewall. Now, when running cpview, hit CPU...Top
Connections to see the top individual connections by CPU consumption.

Page 208: The book indicates that fw ctl zdebug drop can be used to determine
what non-logged IPS signatures are inappropriately dropping traffic. This statement is
not completely accurate, because the default reason for the drop shown by zdebug will be
very generic, and simply indicate it had something to do with IPS enforcement.
Warning: The following procedure will substantially increase the size and
memory requirements of enforcing the compiled policy on the firewall.
Use with caution on production systems.

To obtain the actual IPS signature name in the zdebug output, launch the
SmartConsole tool GUIdbedit and under Table...Global Properties...Properties change
variable enable_inspect_debug_compilation from false to true, and reinstall policy to
the firewall. This setting will cause additional debug information to be compiled into the
firewall's policy, such that the actual offending IPS signature name will be displayed in
the zdebug output.

Page 213: If the Website Categorization Mode has been set to Hold as recommended
in the book, and an unacceptable level of latency is encountered categorizing websites for
the URL Filtering function, additional statistics can be enabled in the Resource Advisor
Daemon (RAD). The RAD process handles interaction between the firewall and the
Check Point cloud for dynamic lookups of content such as URLs. Note that this daemon
is also used to update signatures and verify content for the Application Control, Anti-
Malware, and Anti-Virus software blades; therefore statistics are available for these other
three functions as well. To enable statistics for the URL filtering function specifically,
execute the command rad_admin stats on urlf. To view URL caching and cloud
interaction statistics, run cpview and hit Advanced...RAD:
Don't forget to turn off the statistics gathering with the rad_admin stats off
urlf command when finished!

Pages 220-221: The HTTPS Inspection feature was significantly enhanced in R77.30.
While many of the relevant fixes are included in the R77.20 jumbo hotfix, it appears that
there are many enhancements exclusive to R77.30 that can improve the functionality and
performance of the HTTPS Inspection feature. While the bulk of R77.30 HTTPS
Inspection operations appear to still occur in the Firewall Path, the firewall performance
impact of Bypass actions and SSL negotiation have been substantially improved.

Page 234: Alternatively, to view the firewall's New Connection Rate
(Connections/sec) from the CLI, run the cpview command and hit Network.

Page 275-276: I'm pleased to report that R77.30 has an available built-in fix for the
Hide NAT port allocation failures that are much more likely to occur when Hyperspect is
enabled, as discussed in #8. Ports used for Hide NAT source port reallocation can be
dynamically pooled among the Firewall Worker Cores, instead of being statically
assigned. This new feature is not enabled by default. It involves setting the
fwx_nat_dynamic_port_allocation variable from 0 to 1. There is a separate hotfix
available for R77.20 to add this functionality, however it does not appear to be a part of
the R77.20 jumbo hotfix yet. See sk103656: Dynamic NAT port allocation feature for
more details.

Page 282: If performing lab benchmarking of Check Point firewalls, be sure to enable
the following feature: sk105261: CoreXL Dynamic Dispatcher in R77.30. Network load-testing
traffic is infamous for its non-uniqueness, which can cause an imbalance of
Firewall Worker Core loading and severely crimp firewall throughput results. Also, if
performing benchmarking of HTTPS Inspection on a Check Point firewall, be sure to
enable HTTPS Inspection in “Test Mode” as detailed here: sk104717: HTTPS Inspection
Enhancements in R77.30. HTTPS Inspection Test Mode compensates for similar quirks
in HTTPS load-testing traffic and ensures accurate performance results.

Page 283: If you've reached this section of the book and can't obtain acceptable
performance from your firewall despite following all the tuning recommendations, and no
immediate relief is in sight in the form of newer, faster hardware, consider employing this
new R77.30 feature discussed in the Introduction to help make the most of what you do
have: sk105762: Firewall Priority Queues in R77.30.
Last edited by ShadowPeak.com; 2015-08-03 at 11:19. Reason: fixed typo p. 169
--
My book "Max Power: Check Point Firewall Performance Optimization"
now available via http://maxpowerfirewalls.com.




HTTPS Inspection Enhancements in R77.30
Solution
R77.30 introduces the following improvements and enhancements for HTTPS Inspection (which is supported on Security Gateways running only Gaia OS and SecurePlatform OS):
  • SSL Handshake Acceleration
  • Perfect Forward Secrecy (PFS)
  • Support for AES-GCM
  • Improvements in HTTPS Inspection Bypass mechanism - Probe Bypass
  • Passive HTTPS Inspection - HTTPS Inspection Test Mode

SSL Handshake Acceleration

  • Overview:

    Public Key cryptographic operations, such as RSA and ECDH, require performing many mathematical operations, which causes a high load on CPU. By using the CPU's 64-bit instruction set, the operations can be done significantly faster than with 32-bit instructions.
    PKXLD is a 64-bit process that can run on a 64-bit operating system and perform the required cryptographic operations. It was added in order to achieve SSL handshake acceleration for HTTPS Inspection.
    Note: PKXLD process can not run on 32-bit operating system.
  • Enabling and disabling the usage of PKXLD process

    Usage of PKXLD process is enabled by default on Gaia 64-bit. The PKXLD process will be executed as needed by the WSTLSD process, if Gaia OS is running in 64-bit mode.
    Gateway
    mode
    To disable SSL Handshake AccelerationTo re-enable SSL Handshake Acceleration
    Security
    Gateway
    # touch $FWDIR/conf/pkxl_disable
    # reboot
    
    # rm -i $FWDIR/conf/pkxl_disable
    # reboot
    
    VSX
    # vsenv 0
    # touch $FWDIR/conf/pkxl_disable
    # reboot
    
    # vsenv 0
    # rm -i $FWDIR/conf/pkxl_disable
    # reboot
    
    Notes:
    • In VSX mode, the commands have to be executed in the context of VSX Gateway itself (context of VS0), and will affect all Virtual Systems.
    • In cluster, the commands need to be executed on each of the cluster members.
      Note: It is allowed to have some cluster members with PKXLD enabled, and some cluster members with PKXLD disabled.

Perfect Forward Secrecy (PFS)

  • ECDHE cipher suites

    ECDHE cipher suites were added to Multi-Portals and HTTPS Inspection, providing PFS support for those two products.
    R77.30 adds support for the following cipher suites within HTTPS Inspection:
    • TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (ID 0x00C02B)
    • TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (ID 0x00C02F)
    • TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (ID 0x00C013)
    • TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (ID 0x00C014)
    • TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (ID 0x00C009)
    • TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (ID 0x00C00A)
    R77.30 adds support for the following cipher suites within Multi-Portals (Software Blades Portals):
    • TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (ID 0x00C02F)
    • TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (ID 0x00C013)
    • TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (ID 0x00C014)
  • Configuration

    Important Notes:
    • The commands provided in this section must be run on Security Gateway / each cluster member in Expert mode.
    • These commands require complete restart of Check Point services ('cpstop;cpstart'), which will stop all traffic, and in cluster might cause a fail-over.
    • In VSX mode, the configuration is performed per Virtual System.
    • These commands survive reboot (changes are saved in Check Point Registry file - $CPDIR/registry/HKLM_registry.data).


    DesignTo propose ECDHETo disable ECDHE proposal
    HTTPS connection from client machine to gatewayBy default, ECDHEis accepted, butAES-GCM with RSAis preferred.# ckp_regedit -a SOFTWARE\\CheckPoint\\FW1 CPTLS_ACCEPT_ECDHE 1# ckp_regedit -a SOFTWARE\\CheckPoint\\FW1 CPTLS_ACCEPT_ECDHE 2
    HTTPS connection from gateway to serverECDHE is proposed only if other proposals fail.# ckp_regedit -a SOFTWARE\\CheckPoint\\FW1 CPTLS_PROPOSE_ECDHE 1# ckp_regedit -a SOFTWARE\\CheckPoint\\FW1 CPTLS_PROPOSE_ECDHE 2

Support for AES-GCM

The following AES-GCM cipher suites are now supported with TLS 1.2 in Multi-Portals and HTTPS Inspection, improving throughput on platforms that support AES-NI *:
  • TLS_RSA_WITH_AES_128_GCM_SHA256 (ID 0x00009C)
  • TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (ID 0x00C02B)
  • TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (ID 0x00C02F)
Notes:
  • AES-NI is supported by Check Point 12400 / 12600 / 13x00 / 21x00 / 41000 / 61000 appliances and by some Open Servers (refer to server's CPU specifications).
  • On platforms that do not support AES-NI, AES-GCM is similar in performance to AES-CBC + HMAC-SHA1.

Improvements in HTTPS Inspection Bypass mechanism - Probe Bypass

Important Note: Probe Bypass should not be used if there is a proxy between the Security Gateway and the Internet.
Limitations of HTTPS Inspection Bypass Mechanism without Probe Bypass:
  • Every first connection to a site is inspected even if it should have been bypassed according to the policy.
  • Non-Browser Applications connections are dropped when HTTPS Inspection is enabled (even if bypass is configured).
  • Client certificate connections are dropped when HTTPS Inspection is enabled (even if bypass is configured).
Improvements introduced by Probe Bypass:
  • Bypass mechanism was improved to better reflect policy and resolve the above limitations:
    • Stop the inspection of the first connection to bypassed sites.
    • Allow bypass of Non-Browser Applications connections.
    • Allow Bypass of connections to servers that require client certificate.
  • New probing mechanism eliminates the need to inspect the first connection to an IP address unless it is required by the policy.
Status of Improved HTTPS Inspection Bypass feature (Probe Bypass) is controlled by the value of kernel parameter enhanced_ssl_inspection:
ValueExplanation
0Default value.
Probe Bypass is disabled.
1Probe Bypass is enabled.
Note: The steps below will affect all Virtual Systems in VSX mode.
To enable the Improved HTTPS Inspection Bypass feature (Probe Bypass) on Security Gateway / each cluster member, set the value of kernel parameterenhanced_ssl_inspection to 1.
  • To check the current value of a kernel parameter:
    [Expert@HostName]# fw ctl get int enhanced_ssl_inspection
  • To set the desired value for a kernel parameter on-the-fly (does not survive reboot):
    [Expert@HostName]# fw ctl set int enhanced_ssl_inspection 1
  • To set the desired value for a kernel parameter permanently:
    Follow sk26202 (Changing the kernel global parameters for Check Point Security Gateway).
    For Gaia / SecurePlatform OS:
    1. Create the $FWDIR/boot/modules/fwkern.conf file (if it does not already exit):
      [Expert@HostName]# touch $FWDIR/boot/modules/fwkern.conf
    2. Edit the $FWDIR/boot/modules/fwkern.conf file in Vi editor:
      [Expert@HostName]# vi $FWDIR/boot/modules/fwkern.conf
    3. Add the following line (spaces are not allowed):
      enhanced_ssl_inspection=1
    4. Save the changes and exit from Vi editor.
    5. Check the contents of the $FWDIR/boot/modules/fwkern.conf file:
      [Expert@HostName]# cat $FWDIR/boot/modules/fwkern.conf
    6. Reboot the Security Gateway / each cluster member.
To disable the Probe Bypass on Security Gateway / each cluster member., follow the steps above to set the value of kernel parameter enhanced_ssl_inspection to 0.

HTTPS Inspection Test Mode

  • Background

    Under full HTTPS Inspection test load, the CPU usage of the Security Gateway is well under 100%.
    Performance measurements performed by testing equipment (such as Avalanche and Breaking Point) such as CPS, showed that throughput and latency behave irregularly.
  • Cause

    The cause for this irregularity is non-standard HTTPS implementation by the testing equipment in order to improve capacity and performance.
    Check Point's Active Inspection follows standard HTTPS.
  • Solution

    Passive HTTPS Inspection (HTTPS Inspection Test Mode) solves interoperability issues with testing equipment.
    However, it is important to note these limitations:
    • Intended only for lab use.
    • Supports inspection of inbound traffic.
    • Supports detection only - should be used only for testing purposes.
    • Performance results measured in Test Mode may slightly differ from real-world results.
    • ECDHE cipher suites can not be used in Test Mode because Passive decryption does not support PFS.
  • Configuration of HTTPS Inspection Test Mode

    Status of HTTPS Inspection Test Mode is controlled by the value of kernel parameter fwtls_passive_decrypt:
    ValueExplanation
    0Default value.
    Test Mode is disabled.
    1Test Mode is enabled.
    Note: The steps below have to be performed before the machine is placed under traffic load. If Test Mode is enabled during traffic load, existing connections may be dropped.
    To enable the Test Mode on Security Gateway / each cluster member, set the value of kernel parameter fwtls_passive_decrypt to 1.
    • To check the current value of a kernel parameter:
      [Expert@HostName]# fw ctl get int fwtls_passive_decrypt
    • To set the desired value for a kernel parameter on-the-fly (does not survive reboot):
      [Expert@HostName]# fw ctl set int fwtls_passive_decrypt 1
    • To set the desired value for a kernel parameter permanently (not recommended):
      Follow sk26202 (Changing the kernel global parameters for Check Point Security Gateway).
      For Gaia / SecurePlatform OS:
      1. Create the $FWDIR/boot/modules/fwkern.conf file (if it does not already exit):
        [Expert@HostName]# touch $FWDIR/boot/modules/fwkern.conf
      2. Edit the $FWDIR/boot/modules/fwkern.conf file in Vi editor:
        [Expert@HostName]# vi $FWDIR/boot/modules/fwkern.conf
      3. Add the following line (spaces are not allowed):
        fwtls_passive_decrypt=1
      4. Save the changes and exit from Vi editor.
      5. Check the contents of the $FWDIR/boot/modules/fwkern.conf file:
        [Expert@HostName]# cat $FWDIR/boot/modules/fwkern.conf
      6. Reboot the Security Gateway / each cluster member.
    To disable the Test Mode on Security Gateway / each cluster member, follow the steps above to set the value of kernel parameter fwtls_passive_decrypt to 0.
  • Additional configuration for Session Rate optimization in HTTPS Inspection Test Mode

    To optimize the Session Rate in HTTPS Inspection Test Mode, increase the maximal number of entries in the following kernel tables:
    • fwtls_state_map
    • cptls_sessions
    Follow these steps:
    Important Note: These changes will affect all Security Gateway / Clusters managed by the involved Security Management Server / Multi-Domain Security Management Server.
    1. Connect with SmartDashboard to Security Management Server / Domain Management Server.
    2. Go to 'File' menu - click on 'Database Revision Control...' - create a revision snapshot.
    3. Close all SmartConsole windows (SmartDashboard, SmartView Tracker, SmartView Monitor, etc.).
    4. Connect to command line on Security Management Server / Multi-Domain Security Management Server.
    5. On Multi-Domain Security Management Server, switch to the context of the involved Domain Management Server:
      [Expert@HostName]# mdsenv Domain_Name
    6. Backup and edit the relevant table.def file per sk98339 - Location of 'table.def' files on Security Management Server.
    7. Change:
      from
      /*********
       * fwtls *
       *********/
      
      fwtls_state_map = dynamic keep limit 100000 hashsize 32768;
      cptls_sessions = dynamic expires 3600 limit 100000 hashsize 32768 sync kbuf 2;
      
      to
      /*********
       * fwtls *
       *********/
      
      fwtls_state_map = dynamic keep limit 1000000 hashsize 32768;
      cptls_sessions = dynamic expires 600 limit 1000000 hashsize 32768 sync kbuf 2;
      
    8. Save the changes in the relevant table.def file.
    9. Connect with SmartDashboard to Security Management Server / Domain Management Server.
    10. Install the policy onto the relevant Security Gateway / Cluster object.

Related solutions:

Applies To:
  • 01418393 , 01456436 , 01467522 , 01470952 , 01471765 , 01474667 , 01479662 , 01510285 , 01516601 , 01522323 , 01546352 , 01551219 , 01556280 , 01577129
  • 01482072


Tuesday, May 24, 2016

Script - Checkup.sh

#!/bin/bash
#
# CHECKUP.SH
# Script to gather performance and environmental information in order to examine the health and condition of a Check Point system
# Elements of this script are inspired by the information contained within SK33781, sk38992, sk36846, and sk54400
#
# Michael E. Natkin
# This tool is provided on a best-effort basis as-is with no expressed nor implied warrantee or support.
#
# This work is licensed under the Creative Commons Attribution-ShareAlike 3.0 Unported License. To view a copy of this license,
# visit http://creativecommons.org/licenses/by-sa/3.0/.
#
# PLEASE BE SURE TO CHECK THE WIKI OR WITH THE AUTHOR TO ENSURE YOU ARE RUNNING THE MOST CURRENT VERSION OF THIS SCRIPT
#
# TO:DOs: Add logic in script, check for disk space prior to writing to output directory, add NIC checks to ignore secondary IPs
#
# Version 20150624 - Minor tweak for VPN counts
# Version 20150422 - Add flag allowing for DU bypass
# Version 20150418 - Add alternative method for sourcing CPprofile.sh
# Version 20150416 - Add Monitor Mode interface check
# Version 20150408 - Fixes for improved operation on Solaris
# Version 20150305 - Minor adjustments
# Version 20150202 - Confirm operating MAC Magic numbers
# Version 20141216 - Additional NIC information gathering
# Version 20141126 - Minor programatic improvements, additional appliance definitions
# Version 20140930 - Modify Rulebase counters to include Manual NAT
# Version 20140929 - Rulebase counters (management)
# Version 20140925 - Minor programatic improvements, ensured redirection of stdout and stderr across the entire script
# Version 20140905 - Address some VSX-related inconsistencies
# Version 20140713 - 13800 and 21800
# Version 20140505 - Additional file checks
# Version 20140425 - Revised acceleration functionality, addressed some programatic issues, and tweaked top talkers
# Version 20140424 - Variable cleanup, log cleanup, minor script cleanup and formatting
# Version 20140423 - additional CPU check logic initial implementation
# Version 20140420 - Minor adjustments
# Version 20140219 - Minor cleanup
# Version 20140121 - Additional checks, cleanup
# Version 20140117 - Bond interface checks (SPLAT and GAiA), minor cleanup
# Version 20131118 - Additional UserCheck checks, minor cleanup -- TODO - revise host count
# Version 20131112 - Host count, RAD checks, minor cleanup, additional ID checks
# Version 20131015 - Initial 61000 integration
# Version 20131010 - Additional CPU / IRQ details, partition inode check, cleanup, documentation, and improved user feedback
# Version 20130925 - Minor cleanup and documentation
# Version 20130905 - Minor cleanup and adjsutments
# Version 20130828 - Threat Emulation and MTA
# Version 20130729 - Minor cleanup and adjsutments
# Version 20130724 - Updated SmartEvent checks, addressed SWB detection bug
# Version 20130709 - Added 13500 appliance
# Version 20130610 - Additional file checks
# Version 20130509 - Added array status check, tweaked LOM check, added flags allowing for TOP, IOStat, and VMSTAT bypass
# Version 20130503 - Added MDS checks, added SofaWare LibSW version check
# Version 20130228 - 21700, update LOM detection mechanism
# Version 20130213 - fixes, tweaks, and optimizations, additional cache size checks
# Version 20130206 - Additional ID and blade checks,LOM Check, Fix R76 (and future) VS script support
# Version 20130129 - fixes, tweaks, and optimizations
# Version 20130127 - URLF Stats in 75.* or better (basic today, enhancements planned)
# Version 20130122 - Enhance IPS reporting visibility, add IA checks (following field feedback), added community disclaimer to the output
# Version 20130113 - Address some test issues on legacy versions, enhance VS test criteria
# Version 20121221 - Minor NIC reporting tweaks
# Version 20121210 - minor changes to TOP and IOSTAT output
# Version 20121208 - minor script cleanup and additional documentation, fix 12200 reporting
# Version 20121206 - More stuff!!! Specifically, incorporated blade checks from machine_info.sh... Plus added more complete version history
# Version 20121205 - Additional NIC checks
# Version 20121127 - Additional file checks
# Version 20121107 - More GAiA and dynamic routing stuff
# Version 20121016 - 21600
# Version 20121013 - Fixes and SEM additions
# Version 20121011 - a few more file checks, more documentation
# Version 20121001 - additional IPSO-related tests from sk54400 added
# Version 20120930 - addressed some IPSO-problematic changes, introduced revision history
# Version 20120924 - Script cleanup, additional file checks, improve SecureXL checks
# Version 20120918 - Output cleanup, revision control check, IPS stats, management server checks
# Version 20120907 - Add GAiA checks, improve VSX checks
# Version 20120905 - Improve and simplify scripting, improve VSX checks, improve memory checks, add housekeeping
# Version 20120830 - Improve scripting, reduce non-applicable checks
# Version 20120823 - Simplify and expand file and process checks, improve end-user feedback
# Version 20120821 - Script cleanup, tweak Crossbeam-specific checks
# Version 20120802 - Add user interaction, minor cosmetic changes
# Version 20120702 - Add appliance mapping
# Version 20120622 - More IPSO reporting parity information, add user VPN checks
# Version 20120516 - Minor cosmetic changes only
# Version 20120515 - Address IPSO and VSX check issues, add IPSO and VSX reporting parity
# Version 20120410 - (Formerly Version 0.9989) Add scheduled tasks check, more CoreXL checks and logic
# Version 20120315 - (Formerly Version 0.9986) Expand VSX checks.  Add virtual memory / swap checks
# Version 20120306 - (Formerly Version 0.998) Expand IPSO and Crossbeam support, add significant memory and kernel checks.  Begin migration to date-based versioning
# Version 20120207 - (Formerly Version 0.996) Add section comments, add process and file dumps
# Version 20120120 - (Formerly Version 0.995) Script cleanup, address some Crossbeam-problematic changes, more checks
# Version 20120104 - (Formerly Version 0.993) Script cleanup, additional VSX checks
# Version 20111211 - (Formerly Version 0.99) VSX-specific additions and more checks added
# Version 20111205 - (Formerly Version 0.98) Expanded list of checks, script cleanups
# Version 20111010 - (Formerly Version 0.975) Expanded list of checks, script cleanups
# Version 20110909 - The basics start to come into form -- very rough
# Version 0.0.0.0  - Initial stab at automating FW checks

################################################################################
################################################################################
##                           Script start                                     ##
################################################################################
################################################################################

# Script version
SCRVER="Version 20150624"

# By default, the script will execute TOP, VMSTAT, and IOSTAT (where available)
# In order to disable these features, change the following variable from "1" to something else
DOTIMEDCHECKS=1

# By default, the script will execute du (where available)
# In order to disable these features, change the following variable from "1" to something else
DODUCHECK=1


# Define output location.  Default is /var/log
# If you wish to change the output location, this is the place to change it:
OUTTO=/var/log/tmp


# If the chosen output path above doesn't exist, change it to something guaranteed to exist
if [ ! -d "$OUTTO" ]
  then
    OUTTO=/var/log
fi

# Check for the existence of $TMP variable.  If it doesn't exist, make it.
CHECKTMP=$TMP

if [ "$CHECKTMP" = "" ]
   then
       TMP=/var/tmp
fi


# Define hostname of the installation and the date and time of execution
HNAME=`hostname`
NOW=$(date +"%F-%H%M")

# Simplify the output variable
# If you wish to change the output file name from the default, this is the place to change it:
OUTFILE=$OUTTO/checkup-$HNAME-$NOW.txt

################################################################################
################################################################################
##                                                                            ##
##             Do not modify anything beyond this point.                      ##
##                                                                            ##
################################################################################
################################################################################

# Provide brief product description and opportunity to cancel execution
echo "##########################################################################"
echo "# This script gathers performance and environmental information in order #"
echo "# to examine the health and condition of a Check Point system.           #"
echo "#                                                                        #"
echo "# Elements of this script are based on  information contained within     #"
echo "# SK33781, sk38992, sk36846, and sk54400                                 #"
echo "#                                                                        #"
echo "# NOTE: This tool is provided on a best-effort basis as-is with no       #"
echo "# expressed nor implied warrantee or support.                            #"
echo "#                                                                        #"
echo "# Executing script $SCRVER                                    #"
echo "#                                                                        #"
echo "##########################################################################"

echo
echo "  ######################################################################"
echo "  ## This work is licensed under the Creative Commons Attribution-    ##"
echo "  ## ShareAlike 3.0 Unported License. To view a copy of this license, ##"
echo "  ## visit http://creativecommons.org/licenses/by-sa/3.0/.            ##"
echo "  ##                                                                  ##"
echo "  ## Press any key to continue                                        ##"
echo "  ## or wait 10 seconds and the script will continue automatically    ##"
echo "  ######################################################################"

read -n1 -t10 $1

echo
echo "#########################################################################"
echo "# Beginning data acquisition.                                           #"
echo "# Data will be collected into $OUTFILE #"
echo "# You may see some messages and errors appear on the screen during the  #"
echo "# script's execution.  These may safely be ignored.                     #"
echo "#########################################################################"

echo "#########################################################################"
echo


################################################################################
################################################################################
##                                                                            ##
##             Function Definintions                                          ##
##                                                                            ##
################################################################################
################################################################################

secbreak()     # Function providing section break -- break up the information for easier digestion
{
echo "" >> $OUTFILE 2>&1
echo "########################################################################" >> $OUTFILE 2>&1
echo "" >> $OUTFILE 2>&1
}

smallbreak()   # Function providing blank line for between checks within the same section
{
echo "" >> $OUTFILE 2>&1
}

warnuser()    # Function providing some user feedback RE warnings that may be displayed during execution
{
echo
echo "##########################################################################"
echo "## You may see some messages and errors appear on the screen during the ##"
echo "## script's execution.  These may safely be ignored.                    ##"
echo "##########################################################################"
echo
}

################################################################################
################################################################################
##                                                                            ##
##             Start of Script                                                ##
##                                                                            ##
################################################################################
################################################################################

secbreak

# Output File header
echo "##########################################################################" > $OUTFILE
echo "### ### Starting checkup script for $HNAME at `date +"%F-%H%M"`         " >> $OUTFILE 2>&1
echo "##########################################################################" >> $OUTFILE 2>&1
echo "# This script gathers performance and environmental information in order #" >> $OUTFILE 2>&1
echo "# to examine the health and condition of a Check Point system.           #"     >> $OUTFILE 2>&1
echo "#                                                                        #"   >> $OUTFILE 2>&1
echo "# Elements of this script are based on information contained within      #"   >> $OUTFILE 2>&1
echo "# SK33781, sk38992, sk36846, and sk54400                                 #"   >> $OUTFILE 2>&1
echo "#                                                                        #"   >> $OUTFILE 2>&1
echo "# NOTE: This tool is provided on a best-effort basis as-is with no       #"   >> $OUTFILE 2>&1
echo "# expressed nor implied warrantee or support.                            #"   >> $OUTFILE 2>&1
echo "##########################################################################"   >> $OUTFILE 2>&1

secbreak

# Kernel version -- Start of logic for IPSO / XBM
smallbreak
RUNOSFULL=`uname -a`
RUNOS=`uname | egrep 'Linux|IPSO|XOS|SunOS' `
# Build a simple variable for Linux-derivatives
if [ "$RUNOS" = "Linux" ]
   then
      ISTORVALDS=1
fi
if [ "$RUNOS" = "XOS" ]
   then
      ISTORVALDS=1
fi
if [ "$RUNOS" = "IPSO" ]
   then
      ISTORVALDS=0
fi
if [ "$RUNOS" = "SunOS" ]
   then
      ISTORVALDS=0
fi

echo "Running checkup script $SCRVER on $RUNOSFULL platform running $RUNOS" >> $OUTFILE 2>&1
smallbreak

##############################################################################
#                                                                            #
#   Hardware determination                                                   #
#                                                                            #
##############################################################################

# create a "product-code to security-gateway" translation-file, based on -
# http://wiki.checkpoint.com/confluence/display/CPPublic/Appliance+Specifications
# Extracted from cpeval and modified to include Crossbeam and new appliances
echo "  *** hardware platform"  >> $OUTFILE 2>&1

# use mktemp to create temp files based on PID
# Inconsistencies in mktemp across platforms, REMMED out mktemp, forcing manual definition
# APPLTMP=`mktemp -t appliance.xxxxxxxx`
# NOCONNTMP=`mktemp -t appnoconn.xxxxxxxx`

APPLTMP=$TMP/appliances
NOCONNTMP=$TMP/noconns

cat<<_ > $APPLTMP

Crossbeam Hardware - X Series
Product Code  Crossbeam
Thurley  Crossbeam-APM 9600
Bridgeport  Crossbeam-APM 8650
XBM-TBD  Crossbeam-APM 8600
XBM-TBD  Crossbeam-APM x700


Armageddon class - Check Point 61000 SGMs
Product Code  Security Gateway Blade
A-20  SGM-220
A-40  SGM-240
A-60  SGM-260


Prometheus class - Check Point 13000 models
Product Code  Security Gateway
P-370  Check Point 13500

Poseidon Class - Check Point 13800
P-380  Check Point 13800


Toxotai class - Check Point 4000 models
Product Code  Security Gateway
T-110  Check Point 2200
T-120  Check Point 4200
T-140  Check Point 4400
T-160  Check Point 4600
T-180  Check Point 4800
T-181  Check Point TE250


Pireus class - Check Point 12000 models
Product Code  Security Gateway  VSX Appliance
P-210  Check Point 12200
P-220  Check Point 12400
P-230  Check Point 12600
P-231  Check Point TE1000


Grizzly class - Check Point 21000 models
Product Code  Security Gateway  VSX Appliance
G-50  Check Point 21400
G-70  Check Point 21600
G-72  Check Point 21700
G-75  Check Point 21800


London class - Series 80 models
Product Code  Security Gateway 80
L-50  Security Gateway 80

Hoverfly class - 11000 models
Product Code  Power-1  VSX-1
P-30  Power-1 11000 Series  VSX-1 11000 Series


Dragonfly class - xx7x models
Product Code  Power-1  UTM-1  Connectra  Smart-1  VSX-1
  Platforms Group  Platforms Group  VPN Group  Platforms Group  High End Gateway
Security Group
U-10    UTM-1 270  Connectra 270
U-15    UTM-1 570
U-20    UTM-1 1070
U-30    UTM-1 2070
U-40    UTM-1 3070  Connectra 3070  Smart-1 3074  VSX-1 3070
P-10  Power-1 5070
P-20  Power-1 9070    Connectra 9072    VSX-1 9070

IP Series
Product Code  IP
IP-150  IP-150
IP-282  IP-282
IP-295  IP-295
IP-380  IP-380
IP-395  IP-395
IP-565  IP-565
IP-695  IP-695
IP-1285  IP-1285
IP-2455  IP-2455

IPS-1
Product Code  IPS-1
U-31  IPS-1 2076
P-11  IPS-1 5076
P-21  IPS-1 9076

DLP-1
DLP-1 specifications
U-42  DLP-1 2571
P-22  DLP-1 9571


Butterfly class - UTM-1 130
Product Code  UTM-1
U-5  UTM-1 130

Stonefly class - Smart-1 models
Product Code  Smart-1
S-10  Smart-1 5
S-20  Smart-1 25
S-21  Smart-1 25b
S-30  Smart-1 50
S-40  Smart-1 150

Socrates class - Smart-1 models
Product Code  Smart-1
ST-5  Smart-1 205
ST-10  Smart-1 210
ST-25  Smart-1 225
ST-50  Smart-1 3050
ST-150  Smart-1 3150

Tombo class - NEC Univerge models
Product Code  UTM-1
BT0161-00001  UNIVERGE UnifiedWall 1000
BT0161-00002  UNIVERGE UnifiedWall 2000
BT0161-00003  UNIVERGE UnifiedWall 4000

Doda class - xx50 models
Product Code  UTM-1
C2_UTM  UTM-1 450
C6_UTM  UTM-1 1050
C6P_UTM  UTM-1 2050

Seattle Class - 600 and 1100 (for reference only)
Product Code  SMB
L-50    SG80
L-61i   CIP 1100
L-62    CIP 1200R

Miscellaneous
VMware Virtual Platform VE

_

# a list of appliance names to exclude from connections-sampling -
# each name (e.g. "UTM-1 130") should be in a separate line, no quotes.
cat<<_ > $NOCONNTMP
_

product_name() # extract "Product Name" from DMI's System Information section:
{
  (dmidecode) 2>&1 \
    | awk '/System Information/,/^Handle/ {if ($2=="Name:") print}' \
    | sed 's/^.*Product Name: //' # e.g. "U-10-00"
}

product_code() # extract an appliance's significant part of a product-name:
{
  product_name | awk -F'-|_' '{print $2 ? $1"-"$2 : "N/A"}' # e.g. 'U-10'
}

security_gateway() # find the first name matching a product-code:
{
  awk -F'  ' "gensub(\"_\", \"-\", 1, \$1)==\"$(product_code)\" {
    for (i=2; i<=NF; i++) if (\$i) {print \$i; exit} # e.g. 'UTM-1 270'
  }" $APPLTMP
}

NAME="{unidentified}"
MEM="{not calculated}"

if [ "$RUNOS" = "Linux" ]
   then
       NAME=`security_gateway`
       if [ "$NAME" ]
         then
           grep -xq "$NAME" $NOCONNTMP && unset CONNS
         else
           NAME=`product_name` # e.g. "VMware Virtual Platform"
       fi
       MEM=`awk '/^MemTotal:/ {printf "%.0f",$2/1024}' /proc/meminfo`
     elif [ "$RUNOS" = "IPSO" ]
       then
           NAME=`(ipsctl -n hw:motherboard:modelname) 2>/dev/null` # e.g. "IP690"
           MEM=`ipsctl -n net:ip:cluster:physical_memory`
fi

echo "   * Appliance: $NAME" >> $OUTFILE 2>&1
echo "   * Total Physical Memory: $MEM MB"   >> $OUTFILE 2>&1

##############################################################################
#                                                                            #
#   Hyper-threading Check                                                    #
#                                                                            #
##############################################################################

if [ $ISTORVALDS = "1" ]
    then
      smallbreak
      cpuinfo >> $OUTFILE 2>&1
      if [ -f /proc/smt_status ]
         then
            SMTSTAT=`cat /proc/smt_status`
            echo "Hyper-Threading (SMT) Status: $SMTSTAT"  >> $OUTFILE 2>&1
      fi
      secbreak
fi

##############################################################################
#                                                                            #
#   LOM Check                                                                #
#                                                                            #
##############################################################################

if [ "$RUNOS" = "Linux" ]
   then
       HASLOM=`lspci | grep -ci aspeed`; export HASLOM
       if [ "$HASLOM" != "0" ] || [ "$lom_exists" = "1" ]
         then
           echo "LOM installed" >> $OUTFILE 2>&1
           secbreak
         else
             echo "LOM possibly not installed" >> $OUTFILE 2>&1
             secbreak
       fi
fi

##############################################################################
#                                                                            #
#   Array Controller Check                                                   #
#                                                                            #
##############################################################################

# Note: Tested on systems with only 1 array

if [ "$RUNOS" = "Linux" ]
   then
       HASMPT=`lspci | grep -ci "mpt sas"`; export HASMPT
       if [ "$HASMPT" != "0" ]
          then
              echo "LSI Array controller installed. Status check: " >> $OUTFILE 2>&1
              HASMPTSTATUS=`type -P mpt-status`
              if [ $HASMPTSTATUS != "" ]
                 then
                     mpt-status  >> $OUTFILE 2>&1
              fi

              HASLSIUTIL=`type -P lsiutil`; export HASLSIUTIL
              if [ $HASLSIUTIL != "" ]
                 then
                     lsiutil -s  >> $OUTFILE 2>&1
                     lsiutil check_state >> $OUTFILE 2>&1
                     secbreak
              fi
       fi
fi



##############################################################################
#                                                                            #
#   END of the appliance stuff                                               #
#                                                                            #
##############################################################################

# Source CP variables -- just in case
# Source CP variables -- just in case
if [ -f /etc/profile.d/CP.sh ]
   then
       echo "....... Sourcing CP Variables file /etc/profile.d/CP.sh" >> $OUTFILE
       source /etc/profile.d/CP.sh
   else
       echo "....... CP Variables file at /etc/profile.d/CP.sh not present. Attempting sourcing of CPprofile.sh" >> $OUTFILE
        if [ -f /opt/CPshared/5.0/tmp/.CPprofile.sh ]
           then
               echo "....... Sourcing CPprofile Variables file /opt/CPshared/5.0/tmp/.CPprofile.sh" >> $OUTFILE
               source /opt/CPshared/5.0/tmp/.CPprofile.sh
           else

       ###
       # Use advanced search to find latest .CPprofile.sh
       ###
       VER=0
       for x in `seq 85 60`;
           do
             if [ -r  "/opt/CPshrd-R$x/tmp/.CPprofile.sh" ]
                then
                    echo "....... Sourcing CPprofile Variables file /opt/CPshrd-R$x/tmp/.CPprofile.sh" >> $OUTFILE
                    source /opt/CPshrd-R$x/tmp/.CPprofile.sh
                    VER=$x
                    break
             fi
       done
       if [ $VER -eq 0 ]
          then
              for x in `seq 85 60`; do for y in `seq 99 1`;
                  do
                    if [ -f "/opt/CPshrd-R$x.$y/tmp/.CPprofile.sh" ]
                       then
                           echo "....... Sourcing CPprofile Variables file /opt/CPshrd-R$x.$y/tmp/.CPprofile.sh" >> $OUTFILE
                           source /opt/CPshrd-R$x.$y/tmp/.CPprofile.sh
                           VER=$x$y
                           break
                    fi
              done
              if [ $VER -ne 0 ]
                 then
                     break
              fi
              done
       fi
       if [ $VER -eq 0 ]
          then
              a=$(echo {85..60})
              b=$(echo {99..1})
              for x in $a;
                  do
                    if [ -r  "/opt/CPshrd-R$x/tmp/.CPprofile.sh" ]
                       then
                           echo "....... Sourcing CPprofile Variables file /opt/CPshrd-R$x/tmp/.CPprofile.sh" >> $OUTFILE
                           source /opt/CPshrd-R$x/tmp/.CPprofile.sh
                           VER=$x
                           break
                    fi
              done
              if [ $VER -eq 0 ] ; then
              for x in $a; do for y in $b
                  do
                        if [ -f "/opt/CPshrd-R$x.$y/tmp/.CPprofile.sh" ]
                           then
                               echo "....... Sourcing CPprofile Variables file /opt/CPshrd-R$x.$y/tmp/.CPprofile.sh" >> $OUTFILE
                               source /opt/CPshrd-R$x.$y/tmp/.CPprofile.sh
                                VER=$x$y
                                break
                        fi
                done
                if [ $VER -ne 0 ]
                   then
                       break
                fi
                done
              fi
        fi
        if [ $VER -eq 0 ]
           then
               echo "!!!!! Warning: can't find either CP.sh nor .CPprofile.sh. Cannot proceed and therefore terminating execution !!!!!" >> $OUTFILE
               echo "!!!!! Warning: can't find either CP.sh nor .CPprofile.sh. Cannot proceed and therefore terminating execution !!!!!"
           exit 1
        fi
    fi
fi
if [ -f /etc/profile.d/vsenv.sh ]
   then
      echo " ...... Sourcing VSX environment shell..." >> $OUTFILE 2>&1
      source /etc/profile.d/vsenv.sh
fi



FWLABEL=`$CPDIR/bin/cpprod_util CPPROD_GetValue CPshared CurrentLabel 1 | sed 's/ //g'` ; export FWLABEL
SWBVER=`echo $FWLABEL |awk 'BEGIN { FS="." } { print $1 }' | sed 's/R//g' | sed 's/ //g'`; export SWBVER
echo "Current FW Version Label is - $FWLABEL" >> $OUTFILE 2>&1

# What version of code?
smallbreak
fw ver  >> $OUTFILE 2>&1

secbreak


# Set some variables for use throughout the script for versioning
ISVSX=0
ISVSXSWB=0
ISSWB=0

##################################################################################
# Start of logic for Provider-1 / Multi-Domain                                   #
#                                                                                #
# Set a variable for use throughout the script in the event that MDM is detected #
##################################################################################

smallbreak
ISMDS=0
CHECKMDS=$MDSDIR
if [ "$CHECKMDS" != "" ]
   then
       ISMDS=1
       echo "   *** This is a Provider-1 / Multi-Domain Management System ***" >> $OUTFILE 2>&1
       # Ensure that environment variables are set properly
       if [ -f $MDS_SYSTEM/shared/OSdependency.sh ]
          then
              echo "....... Sourcing MDS OS Dependency file" >> $OUTFILE
              source $MDS_SYSTEM/shared/OSdependency.sh
          else
              echo "....... MDS OS Dependency file at $MDS_SYSTEM/shared/OSdependency.sh not present. Bypassing sourcing" >> $OUTFILE
       fi

fi



###################################################################################
# Start of logic for GAiA OS                                                      #
#                                                                                 #
# Set a variable for use throughout the script in the event that GAiA is detected #
###################################################################################

ISGAIA=0
if [ -f "/etc/appliance_config.xml" ]
   then
       ISGAIA=1
       echo "   *** System is running GAiA ***" >> $OUTFILE 2>&1
fi

##################################################################################
# Start of logic for VSX                                                         #
#                                                                                #
# Set a variable for use throughout the script in the event that VSX is detected #
# The second should address R75.40VS - future revs to come                       #
# Added a second variable for 75.40VS and later checks                           #
##################################################################################

if [ "$FWLABEL" = "V40" ]
  then
      ISVSX=1
      ISVSXSWB=0
      ISSWB=0
      echo "   *** System is running Legacy VSX ***" >> $OUTFILE 2>&1

  else

    if [ $SWBVER -gt 74 ]
      then
       ISVSX=0
       ISVSXSWB=0
       ISSWB=1
       CHECKVSX=`$CPDIR/bin/cpprod_util FwIsVSX` ; export CHECKVSX
       if [ $CHECKVSX = "1" ]
          then
             ISVSX=1
             ISVSXSWB=1
             echo "   *** System is running Virtual Systems ***" >> $OUTFILE 2>&1
       fi
    else
      ISVSX=0
      ISVSXSWB=0
      ISSWB=1
    fi
fi


##############################################################################
#                                                                            #
#   The Basics                                                               #
#                                                                            #
##############################################################################


secbreak

echo "   ########### Basic stuff ###########" >> $OUTFILE 2>&1
echo "     *** uptime ***" >> $OUTFILE 2>&1
# How long has the installation been running
uptime  >> $OUTFILE 2>&1

secbreak

if [ "$RUNOS" = "IPSO" ]
 then
     echo "     *** Net_Taskq ***" >> $OUTFILE 2>&1
     # How many cpus are dedicated to IO
     ps aux | grep net_taskq  >> $OUTFILE 2>&1
     ipsctl -a net:taskq:dev >> $OUTFILE 2>&1
     echo "     *** fw_worker ***" >> $OUTFILE 2>&1
     # How many cpus are dedicated to IO
     ps aux | grep fw_worker  >> $OUTFILE 2>&1

fi

if [ "$ISTORVALDS" = "1" ]
 then

##############################################################################
# PROCESS CHECKS                                                             #
# Simplifying piping of processes through the use of a file check temp file. #
# Reduces the manual coding and room for error                               #
##############################################################################

# Add any files desired to this list, ending at the "_"


# Simplifying piping of processes through the use of a process check temp file. Reduces the manual coding and room for error
# Add any processes desired to this list, ending at the "_"

# Inconsistencies in mktemp variable, forcing manual intervention
# PROCCHECKS=`mktemp -t proccheck.xxxxxxxx`
PROCCHECKS=$TMP/proccheck

cat<<_ > $PROCCHECKS
/proc/cpuinfo  # How many and what kinds of CPUs are in the installation
/proc/loadavg  # CPU Load Average
/proc/bus/pci/devices # What PCI devices are installed -- important for understanding the platform and NICs
/proc/sys/vm/balance_pgdat_debug  # Verify the new value of the balancing
/proc/sys/vm/balance_pgdat_limit  # Verify the new value of the balancing
/proc/sys/vm/balance_pgdat_order  # Verify the new value of the balancing
/proc/sys/vm/balance_pgdat_zone   # Verify the new value of the balancing
/proc/interrupts         # What Interrupts are being used (and where)
/proc/slabinfo
/proc/sys/net/ipv4/route/max_size  # Linux kernel parameters -- most often important when there's a large network
/proc/sys/net/ipv4/neigh/default/gc_thresh1  # kernel memory garbage collection
/proc/sys/net/ipv4/neigh/default/gc_thresh2  # kernel memory garbage collection
/proc/sys/net/ipv4/neigh/default/gc_thresh3  # kernel memory garbage collection
/proc/sys/net/ipv4/route/gc_timeout          # kernel memory garbage collection
/proc/sys/net/ipv4/route/gc_interval         # kernel memory garbage collection
/proc/sys/net/ipv4/route/gc_elasticity       # kernel memory garbage collection
/proc/sys/net/ipv6/route/max_size  # IPv6 route cache size
/proc/sys/net/ipv6/neigh/default/gc_thresh1  # v6 kernel memory garbage collection
/proc/sys/net/ipv6/neigh/default/gc_thresh2  # v6 kernel memory garbage collection
/proc/sys/net/ipv6/neigh/default/gc_thresh3  # v6 kernel memory garbage collection
/proc/sys/net/ipv6/route/gc_timeout          # v6 kernel memory garbage collection
/proc/sys/net/ipv6/route/gc_interval         # v6 kernel memory garbage collection
/proc/sys/net/ipv6/route/gc_elasticity       # v6 kernel memory garbage collection
/proc/ppk/cpls                               # SecureXL configuration for ClusterXL Load Sharing support
/proc/ppk/erdos                              # SXL Penalty box
_

# end of list of processes.  Start of code to pipe the processes to the checkup file

PROCLIST=`cat $PROCCHECKS | awk '{print $1}' `
echo
echo " ###################################################################"
echo " # Starting process checks...                                      #"
echo " ###################################################################"
echo

echo "#### Process checks ####" >> $OUTFILE 2>&1

 for PROCNAME in $PROCLIST; do
     if [ -e "$PROCNAME" ]; then
        echo " *** $PROCNAME ***" >> $OUTFILE 2>&1
        cat $PROCNAME >> $OUTFILE 2>&1
        smallbreak
    else
        echo " *** Host does not have $PROCNAME present ***" >> $OUTFILE 2>&1
        smallbreak
    fi
done

# End of process pipe.  Grab some specific information below


# More directly map IRQ to CPU
     echo " ##### IRQ to CPU detailed information #####" >> $OUTFILE 2>&1
     echo " IRQ -- CPU" >> $OUTFILE 2>&1
     for i in `ls /proc/irq/`
         do
            echo -n "$i -- " >> $OUTFILE 2>&1
            cat /proc/irq/$i/smp_affinity  >> $OUTFILE 2>&1
     done
     smallbreak

     echo " ##### egrep ip_dst_cache /proc/slabinfo" >> $OUTFILE 2>&1
     egrep ip_dst_cache /proc/slabinfo >> $OUTFILE 2>&1

     secbreak

     echo " ########## LowFree ##########" >> $OUTFILE 2>&1
     cat /proc/meminfo | grep -i lowfree >> $OUTFILE 2>&1
     smallbreak

     echo " ########## VMALLOC ##########" >> $OUTFILE 2>&1
     cat /proc/meminfo | grep -i vmalloc >> $OUTFILE 2>&1
     smallbreak

     echo " ######### CPD Scheduled Tasks ##########" >> $OUTFILE 2>&1
     # What tasks are scheduled using the CP Scheduler?
     cpd_sched_config print >> $OUTFILE 2>&1
     smallbreak
fi

secbreak


##########################################################################
# FILE CHECKS                                                            #
# Simplifying piping of files through the use of a file check temp file. #
# Reduces the manual coding and room for error                           #
##########################################################################

# Add any files desired to this list, ending at the "_"

# Inconsistencies in mktemp variable, forcing manual intervention
# FILECHECKS=`mktemp -t filecheck.xxxxxxxx`
FILECHECKS=$TMP/filecheck

cat<<_ > $FILECHECKS
/etc/resolv.conf    # What's the name resolution config -- sometimes performance is adversely influenced by bad DNS settings
/etc/ntpd.conf      # Time config
/etc/ntp.conf
/etc/hosts          # Local hosts file
/etc/modprobe.conf  # Any NIC or kernel tweaks?
/etc/sysctl.conf    # Any kernel tweaks?
/etc/ssh/sshd_config # Any hacks to sshd?
/etc/issue           # console banner file
/etc/issue.net       # network banner file
/etc/fstab           # File system table
/etc/motd            # message of the day file
/etc/grub.conf       # Grub config -- important to see vmalloc
/etc/gated.ami       # gated config file
/etc/gated_xl.ami    # gated config file
/etc/rc.d/rc.local   # local RC files -- any changes here (such as kernel tweaks)?
/etc/rc.d/rc.local.user   # local RC files -- any changes here (such as kernel tweaks)?
/etc/snmp/snmpd.conf # SNMP server paramters
/etc/snmp/snmpd.users.conf # SNMP users paramters
$FWDIR/boot/boot.conf              # Firewall boot params
/etc/fw.boot/boot.conf             # Firewall boot params
$FWDIR/boot/modules/fwkern.conf    # Any firewall kernel tweaks?
$PPKDIR/boot/modules/simkern.conf  # Any SIM tweaks?
$FWDIR/conf/discntd.if             # ClusterXL Disconnected Interfaces
$FWDIR/conf/cpha_hosts             # ClusterXL Monitored IPs
$FWDIR/conf/cphaprob.conf          # ClusterXL configuration tweaks (timers)
$FWDIR/conf/local.arp              # SPLAT / GAiA manual ARP
$FWDIR/conf/snmp.C                 # Firewall SNMP config
$FWDIR/conf/vsaffinity_exception.conf      # Relevant to R75.40VS and later Virtual systems only
$FWDIR/conf/masters                        # masters file
$MDSDIR/conf/external.if                   # Relevant to P1 / MDSM only
$FWDIR/conf/mta_postfix_options.cf         # R77 and later Postfix MTA custom options
$FWDIR/conf/fwopsec.conf                   # OPSEC / LEA configuration options
_

FILELIST=`cat $FILECHECKS | awk '{print $1}' `

smallbreak

echo
echo " ###################################################################"
echo " # Starting file checks...                                         #"
echo " ###################################################################"
echo

echo "#### File checks ####" >> $OUTFILE 2>&1

 for FILENAME in $FILELIST; do
     if [ -e "$FILENAME" ]
       then
         echo " *** $FILENAME ***" >> $OUTFILE 2>&1
         cat $FILENAME >> $OUTFILE 2>&1
         smallbreak
         secbreak
       else
         echo " *** Host does not have $FILENAME present ***" >> $OUTFILE 2>&1
         smallbreak
         secbreak
      fi
done

secbreak

if [ "$ISGAIA" = "1" ]
   then
      echo " *** GAiA Base OS config ***" >> $OUTFILE 2>&1
      clish -i -c "show configuration" >> $OUTFILE 2>&1
      smallbreak
      echo " *** Monitor Mode configuration ***" >> $OUTFILE 2>&1
      echo "`grep -i monitor /config/active | grep interface`"  >> $OUTFILE 2>&1
      secbreak
fi

##########################################################################
# MEMORY AND DISK CHECKS                                                 #
##########################################################################

echo "     *** Disk (Partition) utilization ***" >> $OUTFILE 2>&1
# Disk partition information
df -h >> $OUTFILE 2>&1
smallbreak


# Solaris doesn't seem to respect the || operand, so let's do it the hard way...
if [ "$ISTORVALDS" = "1" ]
   then
       echo " ########## free ##########" >> $OUTFILE 2>&1
       free >> $OUTFILE 2>&1
       smallbreak

       echo " ####### Disk utilization - file, df and du ########## " >> $OUTFILE 2>&1
       echo "     *** Files open currently: {# of allocated file handles} {# of free file handles} {system-wide limit} ***" >> $OUTFILE 2>&1
       cat /proc/sys/fs/file-nr >> $OUTFILE 2>&1

       echo "     *** Partition iNode utilization ***" >> $OUTFILE 2>&1
       # iNode utilization information
       df -i >> $OUTFILE 2>&1
       smallbreak
fi

if [ "$DODUCHECK" = "1" ]
   then
       echo "     *** du ***" >> $OUTFILE 2>&1
       # Disk Utilization information
       if [ "$ISTORVALDS" = "1" ]
          then
              du -h / --max-depth=2 >> $OUTFILE 2>&1
          else
              if [ "$RUNOS" = "IPSO" ]
                 then
                     du -h -d 2 / >> $OUTFILE 2>&1
                 else
                     du -sh /* >>  $OUTFILE 2>&1
              fi
       fi
   else
       echo "     *** du check being bypassed ***" >> $OUTFILE 2>&1
fi

smallbreak
secbreak


##########################################################################
# KERNEL BUFFER AND MODULE CHECKS                                        #
##########################################################################


echo "##########  DMESG ############" >> $OUTFILE 2>&1
dmesg >> $OUTFILE 2>&1

smallbreak
secbreak

# Module usage and alloc
if [ "$ISTORVALDS" = "1" ]
   then
       echo "##########  LSMOD  ############" >> $OUTFILE 2>&1
       lsmod >> $OUTFILE 2>&1
fi

smallbreak
secbreak

echo " ##### arp -an | wc -l: `arp -an | wc -l`" >> $OUTFILE 2>&1
# Number of ARP entries

secbreak

##############################################################################
##############################################################################
##                                                                          ##
##   Check Point Software Checks                                            ##
##                                                                          ##
##############################################################################
##############################################################################

echo
echo " ###################################################################"
echo " # Starting Check Point product checks...                          #"
echo " ###################################################################"
echo

echo "   ########### Check Point Software stuff ###########" >> $OUTFILE 2>&1

#############################################################################
#  FFEATURE CHECKS                                                          #
#############################################################################

echo "   ### Basic installed feature check ### " >> $OUTFILE 2>&1
$CPDIR/bin/cpprod_util CPPROD_GetKeyValues Products 0 >> $OUTFILE 2>&1

smallbreak


if [ "$ISMDS" != "1" ]
  then
      ##############################################################################
      #                                                                            #
      # Check what features (blades) are running (detailed check)                  #
      # This section was extracted from machine_info.sh (Eyal Sher, Raz Amir,      #
      # Eitan Lugassi)                                                             #
      #                                                                            #
      ##############################################################################

      # blades - format is: "short name for user" property-name [inner set-name]
      BLADECHECK=$TMP/blade_names
      BLADESTAT=$TMP/blade_disabled

      cat<<_ > $BLADECHECK
FW             firewall
MGMT           management
MNTR           monitor_blade
UDIR           user_dir_blade
VPN            VPN_1
QOS            floodgate
MAB            connectra
URLF           uf_integrated
A_URLF         advanced_uf_blade
AV             anti_virus_blade
ASPM           antispam_integrated
APP_CTL        application_firewall_blade
IPS            Name                      SD_profile
DLP            data_loss_prevention_blade
IA             identity_aware_blade_installed  identity_aware_blade
SSL_INSPECT    ssl_inspection_enabled
ANTB       anti_malware_blade
MON       real_time_monitor
EVNT           event_analyzer
RPTR           reporting_server
EVCOR          ips_event_correlator
EVNT           ips_event_manager
EVIN           smartevent_intro
MTA            mta_enabled
TED            threat_emulation_blade
_

      # property values that indicate a disabled blade:
       cat<<_ > $BLADESTAT
No_protection
not-installed
false
_

       # Run as function to support return codes

       activeblades() {
       # print a line, e.g. "Enabled Blades: FW MGMT VPN IPS":
       echo -n "* Active blades:" >> $OUTFILE 2>&1   # start a single line ...

       OBJ_FILE=$FWDIR/database/objects.C
       if ! [ -r $OBJ_FILE ] 2>/dev/null
          then
          echo " N/A - cannot read file: $OBJ_FILE"
            return 1
       fi

       REG_FILE=$CPDIR/registry/HKLM_registry.data
       if [ ! -r $REG_FILE ] 2>/dev/null
           then
            echo " N/A - cannot read file: $REG_FILE"
            return 1
       fi

       SIC_NAME=$(
        awk -F\" '
/^[[:blank:]]+:MySICname \("/ {
print toupper($2) # case-insensitive
exit              # one match only
}
' $REG_FILE
        )

        if [ -z "$SIC_NAME" ]
             then
              echo ' N/A - failed to retrieve SIC name'
              return 1
        fi

        OBJ_NAME=$(
        awk -F\( "
/^\t\t: \(/ {
# save current set's name as object's context:
obj=\$2
}
/^\t\t\t:sic_name \(/ {
# match the saved SIC name with current set's:
if (toupper(\$2)~/^\"$SIC_NAME\"/) {
print obj
exit
}
}
" $OBJ_FILE
        )

        if [ -z "$OBJ_NAME" ]
             then
              echo ' N/A - failed to match an object to SIC name'
              return 1
        fi

        # dump the local object:
        cat $OBJ_FILE | tr '\t' ' ' | sed -n "/^  : ($OBJ_NAME$/,/^  )/p" > $TMP/local_obj

        # go over the blades file, skip empty lines:
        grep -v "^[[:blank:]]*$" $BLADECHECK | while read LINE
        do
        eval "set -- $LINE" # set positional parameters ("eval" preserves quotes)
# try to find a blade's value - if unavailable, skip to next blade:
        sed -n "/^   :${3-$2} (/,/^   )/p" $TMP/local_obj | \
grep "^   ${3+ }:$2 (" > $TMP/blade_val || continue
# match value for "disabled" - if unmatched, assume enabled:
        grep -wqf $BLADESTAT $TMP/blade_val || \
echo -n " $1"  # append the blade's name to line, e.g. " MGMT"
        done
        echo # end the blades line (carriage-return)
        }

        activeblades >> $OUTFILE 2>&1
        smallbreak

        if [ -z "$SIC_NAME" ]
           then
                echo "Unable to determine SIC name of module" >> $OUTFILE 2>&1
           else
                echo "SIC Name of module: $SIC_NAME" >> $OUTFILE 2>&1
        fi
        smallbreak

        if [ -z "$OBJ_NAME" ]
             then
                echo "Unable to determine object name of module" >> $OUTFILE 2>&1
             else
                echo "Object name: $OBJ_NAME" >> $OUTFILE 2>&1
        fi
        smallbreak
        # Cleanup temporary files
        rm $TMP/local_obj
        rm $TMP/blade_val
        rm $BLADECHECK
        rm $BLADESTAT
fi

#############################################################################
#  STATUS CHECKS                                                            #
#############################################################################

# Not all the commands work on all platforms.  Giving some feedback to the end user pacifies concerns
warnuser

# Firewall-1 Process list - Run only if not on an MDS
if [ "$ISMDS" != "1" ]
   then
       echo " ########## cpwd_admin list ##########" >> $OUTFILE 2>&1
       cpwd_admin list >> $OUTFILE 2>&1
       smallbreak
fi

echo " ########## cpstat stuff ##########" >> $OUTFILE 2>&1
echo "     *** -f cpu os ***" >> $OUTFILE 2>&1
cpstat -f cpu os >> $OUTFILE 2>&1
echo "     *** -f memory os ***" >> $OUTFILE 2>&1
cpstat -f memory os >> $OUTFILE 2>&1
echo "     *** -f multi_cpu os ***" >> $OUTFILE 2>&1
cpstat -f multi_cpu os >> $OUTFILE 2>&1
echo "     *** -f all os ***" >> $OUTFILE 2>&1
cpstat -f all os >> $OUTFILE 2>&1

secbreak

# Run some feature checks if on a gateway
if [ `cpprod_util FwIsFirewallModule` = "1" ] && [ "$ISVSX" != "1" ]
  then
   # Check for the presence of the new ips command
   IPSPROGCHK=`type -P ips`

   echo " ########## Gateway checks ##########"  >> $OUTFILE 2>&1
   echo "     *** -f all fw ***"  >> $OUTFILE 2>&1
   cpstat -f all fw >> $OUTFILE 2>&1
   echo "     *** -f sysinfo cvpn ***" >> $OUTFILE 2>&1
   cpstat -f sysinfo cvpn >> $OUTFILE 2>&1
   echo "     *** -f all vpn ***"  >> $OUTFILE 2>&1
   cpstat -f all vpn >> $OUTFILE 2>&1
   smallbreak
   echo "     *** ASM / IPS ***"  >> $OUTFILE 2>&1
   cpstat -f default asm >> $OUTFILE 2>&1
   cpstat -f WS asm >> $OUTFILE 2>&1

   # Basic URLF cache check -- can use refinement
   if [ "$ISSWB" = "1" ]
      then
          echo "     ### URL Filtering Cache Status ###"  >> $OUTFILE 2>&1
          fw tab -t urlf_cache_tbl -s >> $OUTFILE 2>&1
          smallbreak
          APURCACHE=( `grep cache_max_hash_size $FWDIR/database/rad_services.C | awk '{print $2}' `)
          echo "     *** URLF Cache table size: ${APURCACHE[4]} " >> $OUTFILE 2>&1
          smallbreak
          echo "     ### Application Control Status ###"  >> $OUTFILE 2>&1
          fw tab -t appi_connections -t appi_session_table -s >> $OUTFILE 2>&1
          smallbreak
          echo "     *** Application Control Cache table size: ${APURCACHE[3]} " >> $OUTFILE 2>&1


          smallbreak
          echo "  ### Usercheck configuration parameters ###" >> $OUTFILE 2>&1
          echo "    *** UserCheck HTTPD.CONF *** " >> $OUTFILE 2>&1
          echo "  `grep ServerLimit /opt/CPUserCheckPortal/conf/httpd.conf` " >> $OUTFILE 2>&1
          echo "  `grep MaxClients /opt/CPUserCheckPortal/conf/httpd.conf` " >> $OUTFILE 2>&1
          echo "  `grep MinSpareServers /opt/CPUserCheckPortal/conf/httpd.conf` " >> $OUTFILE 2>&1
          echo "  `grep StartServers /opt/CPUserCheckPortal/conf/httpd.conf` " >> $OUTFILE 2>&1
          smallbreak
          echo "    *** UserCheck PHP.INI *** " >> $OUTFILE 2>&1
          echo "  `grep session.gc_maxlife /opt/CPUserCheckPortal/conf/php.ini` " >> $OUTFILE 2>&1
          smallbreak
   fi

   if [ "$IPSPROGCHK" != "" ]
      then
         echo "     *** IPS Configuration ***"  >> $OUTFILE 2>&1
         ips stat >> $OUTFILE 2>&1
         smallbreak
   fi
   echo "     *** FloodGate ***"  >> $OUTFILE 2>&1
   cpstat -f all fg >> $OUTFILE 2>&1

   # Check for the presence of the IA command for AD
   ADPROGCHK=`type -P adlog`
   if [ "$ADPROGCHK" != "" ]
      then
         echo "    #### Identity Awareness Active Directory ####"  >> $OUTFILE 2>&1
         cpstat -f default identityServer >> $OUTFILE 2>&1
         smallbreak
         echo "     *** DC Connectivity ***"  >> $OUTFILE 2>&1
         adlog a dc  >> $OUTFILE 2>&1
         smallbreak
         echo "     *** DC statistics ***"  >> $OUTFILE 2>&1
         adlog a statistics >> $OUTFILE 2>&1
         smallbreak
         echo "     *** IA Suspected Service Accounts ***"  >> $OUTFILE 2>&1
         adlog a service_accounts >> $OUTFILE 2>&1
         smallbreak
         echo "     *** IA Authentication Metrics ***"  >> $OUTFILE 2>&1
         cpstat identityServer -f authentication >> $OUTFILE 2>&1
   fi

   # Check for the presence of the IA command for PDP
   PDPPROGCHK=`type -P pdp`
   if [ "$PDPPROGCHK" != "" ]
      then
         echo "    #### Identity Awareness Personality Detection (PDP) ####"  >> $OUTFILE 2>&1
         pdp status show >> $OUTFILE 2>&1
         smallbreak
         echo "     *** PDP connections to enforcement points ***"  >> $OUTFILE 2>&1
         pdp connections pep >> $OUTFILE 2>&1
         smallbreak
         echo "     *** PDP connections to terminal servers ***"  >> $OUTFILE 2>&1
         pdp connections ts >> $OUTFILE 2>&1
         smallbreak
         echo "     *** PDP tables ***"  >> $OUTFILE 2>&1
         fw tab -t pdp_sessions -t pdp_super_sessions -t pdp_super_sessions -t pdp_encryption_keys -t pdp_whitelist -t pdp_timers -t pdp_expired_timers -t pdp_ip -t pdp_net_db -t pdp_cluster_stat  -s >> $OUTFILE 2>&1
         smallbreak
   fi

   # Check for the presence of the IA command for PEP
   PEPPROGCHK=`type -P pep`
   if [ "$PEPPROGCHK" != "" ]
      then
         echo "    #### Identity Awareness Personality Enforcement (PEP) ####"  >> $OUTFILE 2>&1
         pep show stat >> $OUTFILE 2>&1
         smallbreak
         echo "     *** PEP connections to Detection points (PDP) ***"  >> $OUTFILE 2>&1
         pep show pdp all >> $OUTFILE 2>&1
         smallbreak
         echo "     *** PEP tables ***"  >> $OUTFILE 2>&1
         fw tab -t pep_pdp_db -t pep_networks_to_pdp_db -t pep_net_reg -t pep_reported_network_masks_db -t pep_port_range_db -t pep_async_id_calls -t pep_client_db -t pep_identity_index -t pep_revoked_key_clients -t pep_src_mapping_db -t pep_log_completion -s >> $OUTFILE 2>&1
         smallbreak
   fi

   # Check for the presence of TED commands
   TEDPROGCHK=`type -P tecli`
   if [ "$TEDPROGCHK" != "" ]
      then
         echo "    #### Threat Emulation Basic Statistics ####"  >> $OUTFILE 2>&1
         tecli s s >> $OUTFILE 2>&1
         smallbreak
         echo "    #### Threat Emulation Cloud Information ####"  >> $OUTFILE 2>&1
         tecli s c i >> $OUTFILE 2>&1
         smallbreak
         echo "    #### Threat Emulation Cloud Quota Status ####"  >> $OUTFILE 2>&1
         tecli s c q >> $OUTFILE 2>&1
         smallbreak

   fi

   echo "     *** Provisioning Agent ***"  >> $OUTFILE 2>&1
   cpstat -f default PA >> $OUTFILE 2>&1
   echo "     *** LS ***"  >> $OUTFILE 2>&1
   cpstat -f default ls >> $OUTFILE 2>&1
   echo "     *** High Availability ***"  >> $OUTFILE 2>&1
   cpstat -f default ha >> $OUTFILE 2>&1

   unset IPSPROGCHK
   unset TEDPROGCHK
   unset PDPPROGCHK
   unset PEPPROGCHK
  else
   echo "### Node is not a gateway or is a VSX system. FW Module checks bypassed ###"  >> $OUTFILE 2>&1
fi

secbreak

# Run some feature checks if on a manager and NOT P1
if [ "$ISMDS" != "1" ]
   then
       if [ `cpprod_util FwIsFirewallMgmt` = "1" ]
         then
            echo " ### Management checks ###"  >> $OUTFILE 2>&1
            echo "     *** Management ***"  >> $OUTFILE 2>&1
            cpstat -f default mg >> $OUTFILE 2>&1
            echo "     *** Cert Authority ***"  >> $OUTFILE 2>&1
            cpstat -f default ca >> $OUTFILE 2>&1
            smallbreak
            echo "     *** Policies ***" >> $OUTFILE 2>&1
            echo "       *** Number of policies: `grep rule-base $FWDIR/conf/rulebases_5_0.fws | wc -l`" >> $OUTFILE 2>&1
            RULELIST=`grep rule-base $FWDIR/conf/rulebases_5_0.fws | awk 'BEGIN { FS="##" } { print $2 }' | awk 'BEGIN { FS="\"" } { print $1 }' `
            for RULENAME in $RULELIST; do
                 echo "       *** Policy Name: $RULENAME" >> $OUTFILE 2>&1
                   if [ -f $FWDIR/conf/$RULENAME.W ]
                        then
                            echo "            --- Number of rules in $RULENAME (compiled): `grep ":unified_rulenum (" $FWDIR/conf/$RULENAME.W | tail -n 1 | awk ' BEGIN { FS = "(" } { print $2 } ' | awk ' BEGIN { FS = ")" } { print $1 } '` " >> $OUTFILE
                            echo "            --- Number of Manual NAT rules in $RULENAME (compiled): `grep rule_adtr $FWDIR/conf/$RULENAME.W | wc -l` " >> $OUTFILE
                        else
                            echo "            --- Rulebase not compiled for installation" >> $OUTFILE
                   fi              
            done
            smallbreak
            echo "     *** revision control ***" >> $OUTFILE 2>&1
            if [ -d $FWDIR/conf/db_versions/repository/ ]
               then
                    echo "        *** Number of database revisions: `ls $FWDIR/conf/db_versions/repository/ | wc -l` " >> $OUTFILE 2>&1
               else
                    echo "        *** No Database revision directory." >> $OUTFILE 2>&1
            fi
            unset RULELIST
            unset RULENAME
            smallbreak
            # Some basic SmartEvent checks
            CHECKRTDIR=$RTDIR
            if [ "$CHECKRTDIR" = "" ]
               then
                  echo "     *** SmartEvent Stats ***" >> $OUTFILE 2>&1
                  echo "        *** Number of unprocessed records  `ls -l $RTDIR/distrib/* | wc -l` "  >> $OUTFILE 2>&1
                  smallbreak
            fi
            unset CHECKRTDIR
            smallbreak
            if [ ! -z "$(pgrep "cpsead")"  ]
               then
                   echo "     *** CPSEAD Stats ***" >> $OUTFILE 2>&1
                   cpstat cpsead >> $OUTFILE 2>&1
                   smallbreak
            fi
            if [ ! -z "$(pgrep "cpsemd")"  ]
               then
                   echo "     *** CPSEMD Stats ***" >> $OUTFILE 2>&1
                   cpstat cpsemd >> $OUTFILE 2>&1
                   smallbreak
            fi


            smallbreak
            # Edge checks
            echo "    *** Edge LibSW Version Check *** " >> $OUTFILE 2>&1
            LIBSWPATH=`$CPDIR/bin/cpprod_util CPPROD_GetProdDir EdgeCmp | sed 's/ //g'` ; export LIBSWPATH
            grep -i "version" $LIBSWPATH/libsw/version.txt >> $OUTFILE 2>&1
       else
           echo "### Node is not a manager. FW management checks bypassed ###"  >> $OUTFILE 2>&1
       fi
    else
        echo "### Provider-1 / MDSM Checks ###" >> $OUTFILE 2>&1
        echo "    *** MDS Stat ***" >> $OUTFILE 2>&1
        mdsstat >> $OUTFILE 2>&1
        for CMANAME in $($MDSVERUTIL AllCMAs)
            do
                mdsenv $CMANAME
                secbreak
                echo "   *** Checks for Domain $CMANAME *** " >> $OUTFILE 2>&1
                if [ `$CPDIR/bin/cpprod_util FwIsActiveManagement` = '1' ]
                    then
                        echo "     *** This CMA is the ACTIVE CMA for this customer" >> $OUTFILE 2>&1
                    else
                        echo "     *** This CMA is the BACKUP CMA for this customer" >> $OUTFILE 2>&1
                fi
                echo "     *** Management ***"  >> $OUTFILE 2>&1
                cpstat -f default mg >> $OUTFILE 2>&1
                smallbreak
                echo "     *** Policies ***" >> $OUTFILE 2>&1
                echo "       *** Number of policies: `grep rule-base $FWDIR/conf/rulebases_5_0.fws | wc -l`" >> $OUTFILE 2>&1
                RULELIST=`grep rule-base $FWDIR/conf/rulebases_5_0.fws | awk 'BEGIN { FS="##" } { print $2 }' | awk 'BEGIN { FS="\"" } { print $1 }' `
                for RULENAME in $RULELIST; do
                    echo "       *** Policy Name: $RULENAME" >> $OUTFILE 2>&1
                   if [ -f $FWDIR/conf/$RULENAME.W ]
                        then
                            echo "            --- Number of rules in $RULENAME (compiled): `grep ":unified_rulenum (" $FWDIR/conf/$RULENAME.W | tail -n 1 | awk ' BEGIN { FS = "(" } { print $2 } ' | awk ' BEGIN { FS = ")" } { print $1 } '` " >> $OUTFILE
                            echo "            --- Number of Manual NAT rules in $RULENAME (compiled): `grep rule_adtr $FWDIR/conf/$RULENAME.W | wc -l` " >> $OUTFILE
                        else
                            echo "            --- Rulebase not compiled for installation" >> $OUTFILE
                   fi
                done
                unset RULELIST
                unset RULENAME

                smallbreak
                echo "     *** revision control ***" >> $OUTFILE 2>&1
               if [ -d $FWDIR/conf/db_versions/repository/ ]
                  then
                      echo "        *** Number of database revisions: `ls $FWDIR/conf/db_versions/repository/ | wc -l` " >> $OUTFILE 2>&1
                  else
                      echo "        *** No Database revision directory." >> $OUTFILE 2>&1
               fi
                smallbreak
                echo "    *** Edge LibSW Version Check *** " >> $OUTFILE 2>&1
                LIBSWPATH=`$CPDIR/bin/cpprod_util CPPROD_GetProdDir EdgeCmp | sed 's/ //g'` ; export LIBSWPATH
                grep -i "version" $LIBSWPATH/libsw/version.txt >> $OUTFILE 2>&1
                smallbreak
                echo "    *** CMA Disk Utilization Check ***" >> $OUTFILE 2>&1
                du --max-depth=1 -h $FWDIR >> $OUTFILE 2>&1
                smallbreak
            done
        unset CMANAME
        mdsenv
fi


secbreak

############################################################################################
#  FW Acceleration Stuff                                                                   #
############################################################################################

# Not all the commands work on all platforms.  Giving some feedback to the end user pacifies concerns
warnuser

if [ `cpprod_util FwIsFirewallModule` = "1" ]
  then
      echo " ######## fwaccel stuff ########## " >> $OUTFILE 2>&1
      # VSX STUFF
      if [ "$ISVSX" = "1" ]
          # Begin VSX-specific logic for FWACCEL stuff
             then
                echo "############# THIS IS A VSX System ############" >> $OUTFILE 2>&1

                echo " ######## Connections ##########" >> $OUTFILE 2>&1
                cpstat -f conns vsx >> $OUTFILE 2>&1
                smallbreak

                echo " ######## fw ctl pstat ##########" >> $OUTFILE 2>&1
                fw ctl pstat >> $OUTFILE 2>&1
                smallbreak

                echo "     *** VSX STAT ***" >> $OUTFILE 2>&1
                vsx stat -v -l >> $OUTFILE 2>&1

                if [ "$ISVSXSWB" = "1" ]
                   then
                        echo "   *** 75.40VS or newer Virtual System Checks ***" >> $OUTFILE 2>&1
                        echo "       *** MSTAT ***" >> $OUTFILE 2>&1
                        fw vsx mstat >> $OUTFILE 2>&1
                        echo "       *** Resource Control ***" >> $OUTFILE 2>&1
                        fw vsx resctrl monitor show >> $OUTFILE 2>&1
                        fw vsx resctrl stat >> $OUTFILE 2>&1
                        echo "       *** Basic SIM Affinity settings ***" >> $OUTFILE 2>&1
                        fw ctl affinity -l >> $OUTFILE 2>&1
                        smallbreak
                fi

                echo "     *** VSX FWACCEL STAT ***" >> $OUTFILE 2>&1
                if [ "$ISVSXSWB" = "1" ]
                    then
                        fwaccel stat -a >> $OUTFILE 2>&1
                        smallbreak
                    else
                        fwaccel stat -all >> $OUTFILE 2>&1
                        smallbreak
                fi


                echo "--------------- CPHAPROB SYNCSTAT for VS0 ---------------" >> $OUTFILE 2>&1
                cphaprob -all syncstat >> $OUTFILE 2>&1
                smallbreak

                # Pipe the list of virtual devices to a temp file for parsing
                vsx stat -v | grep "|" | grep [1-9] | awk 'BEGIN { FS="|" } { print $1 $2} ' | awk 'BEGIN { FS=" " } { print $1, $2, $3 }' > $TMP/vsobjs


                # Run commands on all VS's (but not VR's or VSw's)
                 while IFS=: read VSLINE
                       do
                       VSNUM=`echo $VSLINE | awk 'BEGIN { FS=" " } { print $1 }'`
                       VSTYPE=`echo $VSLINE | awk 'BEGIN { FS=" " } { print $2 }'`
                       VSNAME=`echo $VSLINE | awk 'BEGIN { FS=" " } { print $3 }'`
                           if [ "$VSTYPE" = "S" ] || [ "$VSTYPE" = "B" ]
                             then

                                if [ "$ISVSXSWB" = "1" ]
                                   then
                                       smallbreak
                                       vsenv $VSNUM >> $OUTFILE 2>&1     # R75.40VS and later require some commands to be run from the VS context
                                       echo "  *** 75.40VS or newer Virtual System Checks for VS $VSNUM ***" >> $OUTFILE 2>&1
                                       echo "     *** VSX STAT ***" >> $OUTFILE 2>&1
                                       fw vsx stat -l -vsid $VSNUM >> $OUTFILE 2>&1
                                       smallbreak
                                       echo "     *** Detailed Affinity Settings ***" >> $OUTFILE 2>&1
                                       fw ctl affinity -l -x -vsid $VSNUM -flags tne >> $OUTFILE 2>&1
                                       smallbreak

                                       # Check SecureXL Status.
                                       ISFWACCEL=`fwaccel stat | grep Status | awk 'BEGIN { FS=" : " } { print $2}'`

                                       if [ "$ISFWACCEL" = "on" ]
                                          then
                                              echo "     *** FWACCEL Stat ***" >> $OUTFILE 2>&1
                                              fwaccel stats -s >> $OUTFILE 2>&1
                                              smallbreak
                                          else
                                              echo "    ** SecureXL Acceleration is disabled on this VS. **" >> $OUTFILE 2>&1
                                       fi

                                       echo "     *** FW Affinity Config ***" >> $OUTFILE 2>&1
                                       fw ctl affinity -l -x -vsid $VSNUM -flags tne  >> $OUTFILE 2>&1
                                       smallbreak
                                          # Check for the presence of the new ips command
                                             IPSPROGCHK=`type -P ips`

                                                echo " ########## Gateway checks ##########"  >> $OUTFILE 2>&1
                                                echo "     *** -f all fw ***"  >> $OUTFILE 2>&1
                                                cpstat -f all fw >> $OUTFILE 2>&1
                                                smallbreak
                                                echo "     *** -f sysinfo cvpn ***" >> $OUTFILE 2>&1
                                                cpstat -f sysinfo cvpn >> $OUTFILE 2>&1
                                                smallbreak
                                                echo "     *** -f all vpn ***"  >> $OUTFILE 2>&1
                                                cpstat -f all vpn >> $OUTFILE 2>&1
                                                smallbreak
                                                echo "     *** ASM / IPS ***"  >> $OUTFILE 2>&1
                                                cpstat -f default asm >> $OUTFILE 2>&1
                                                cpstat -f WS asm >> $OUTFILE 2>&1

                                                if [ "$IPSPROGCHK" != "" ]
                                                   then
                                                        echo "     *** IPS Configuration ***"  >> $OUTFILE 2>&1
                                                        ips stat >> $OUTFILE 2>&1
                                                        smallbreak
                                                fi
                                   else
                                       smallbreak
                                       vsx set $VSNUM >> $OUTFILE 2>&1

                                       # Check SecureXL Status.
                                       ISFWACCEL=`fwaccel stat | grep Status | awk 'BEGIN { FS=" : " } { print $2}'`

                               fi

                             echo "--------------- CPHAPROB SYNCSTAT for VS $VSNUM ---------------" >> $OUTFILE 2>&1
                             cphaprob syncstat >> $OUTFILE 2>&1
                             smallbreak

                              if [ "$ISFWACCEL" = "on" ]
                                 then
                                     echo "--------------- FWACCEL STATS for Virtual System # $VSNUM ---------------" >> $OUTFILE 2>&1
                                     echo "fwaccel conns count at `$DATEFUNC` is `fwaccel -vs $VSNUM conns | wc -l` " >> $OUTFILE 2>&1
                                     echo "fwaccel templates count at `$DATEFUNC` is `fwaccel -vs $VSNUM templates | wc -l` " >> $OUTFILE 2>&1
                                     smallbreak
                                     echo "     *** stat ***" >> $OUTFILE 2>&1
                                     fwaccel stat >> $OUTFILE 2>&1
                                     smallbreak
                                     echo "     *** stats ***" >> $OUTFILE 2>&1
                                     fwaccel stats >> $OUTFILE 2>&1
                                     smallbreak
                                     echo "     *** stats -s ***" >> $OUTFILE 2>&1
                                     fwaccel stats -s >> $OUTFILE 2>&1
                                     smallbreak
                                     echo "     *** stats -p ***" >> $OUTFILE 2>&1
                                     fwaccel stats -p >> $OUTFILE 2>&1
                                     smallbreak
                                  else
                                      echo "    ** SecureXL Acceleration is disabled on this VS. **" >> $OUTFILE 2>&1
                               fi

                               echo "--------------- TOP CONNECTIONS for Virtual System # $VSNUM ---------------" >> $OUTFILE 2>&1
                               fw -vs $VSNUM tab -t connections -t fwx_alloc -t fwx_cache -t frag_table -s  >> $OUTFILE 2>&1
                               if [ "$ISFWACCEL" = "on" ]
                                  then
                                       # If acceleration is enabled, we can leverage the SecureXL table for connections information
                                       echo " Count | Source IP | Destination IP | Destination Port" >> $OUTFILE 2>&1
                                       fwaccel conns |  awk '{printf "%-16s %-15s %-15s\n", $1,$3,$4}' | sort | uniq -c | sort -n -r | head -n 10 >> $OUTFILE 2>&1
                                       smallbreak
                               fi

                               # Without acceleration, we have to rely on the connections table
                               fw -vs $VSNUM tab -t connections -u | grep \; | awk '{print $9}' | sort -bg | uniq -c | sort -bg | head -n 10 >> $OUTFILE 2>&1

                               smallbreak
                               echo "-------------- INTERFACE INFORMATION FOR Virtual System # $VSNUM ---------------" >> $OUTFILE 2>&1
                               ifconfig -s  >> $OUTFILE 2>&1

                               smallbreak
                           fi
                         
                           # Reset back to VS 0
                           if [ "$ISVSXSWB" = "1" ]
                              then
                                  vsenv 0 >> $OUTFILE 2>&1     # R75.40VS and later require some commands to be run from the VS context
                              else
                                  vsx set 0 >> $OUTFILE 2>&1
                           fi

                 done < $TMP/vsobjs

          else
              # Check SecureXL Status.
              ISFWACCEL=`fwaccel stat | grep Status | awk 'BEGIN { FS=" : " } { print $2}'`
           
              if [ "$ISFWACCEL" = "on" ]
                  then
                     # FWACCEL Stuff on non-VSX/VS systems
                     echo "     *** stat ***" >> $OUTFILE 2>&1
                     fwaccel stat >> $OUTFILE 2>&1
                     echo "     *** stats ***" >> $OUTFILE 2>&1
                     fwaccel stats >> $OUTFILE 2>&1
                     echo "     *** stats -s ***" >> $OUTFILE 2>&1
                     fwaccel stats -s >> $OUTFILE 2>&1
                     echo "     *** stats -p ***" >> $OUTFILE 2>&1
                     fwaccel stats -p >> $OUTFILE 2>&1
                     echo "--------------- FWACCEL STATS  ----------------" >> $OUTFILE 2>&1
                     echo "fwaccel conns count at `$DATEFUNC` is `fwaccel conns | wc -l` " >> $OUTFILE 2>&1
                     echo "fwaccel templates count at `$DATEFUNC` is `fwaccel templates | wc -l` " >> $OUTFILE 2>&1
                     smallbreak
                  else
                     echo "    ** SecureXL Acceleration is disabled **" >> $OUTFILE 2>&1
              fi
       fi
      secbreak
      if [ "$ISVSX" != "1" ]
         then
           #CoreXL Stuff
           echo " ##### Multi-CPU #####" >> $OUTFILE 2>&1
           echo "     *** Licensed CPU Count ***" >> $OUTFILE 2>&1
           $FWDIR/bin/fw ctl get int fwlic_num_of_allowed_cpus >> $OUTFILE 2>&1
           echo "     *** multik ***"  >> $OUTFILE 2>&1
           fw ctl multik stat >> $OUTFILE 2>&1
           echo "     *** fw ctl affinity ***"  >> $OUTFILE 2>&1
           fw ctl affinity -l -r -v -a  >> $OUTFILE 2>&1
      fi
      smallbreak
      # SIM commands don't work in IPSO or Solaris
      if [ "ISTORVALDS" != "1" ]
         then
          echo " ##### sim affinity #####" >> $OUTFILE 2>&1
          echo "     *** -l ***" >> $OUTFILE 2>&1
          sim affinity -l >> $OUTFILE 2>&1
          echo "     *** -l -r -v -a ***" >> $OUTFILE 2>&1
          sim affinity -l -r -v -a >> $OUTFILE 2>&1
          smallbreak
         else
            if [ "$RUNOS" = "IPSO" ]
               then
                   echo " ##### IPSO Flow stat #####" >> $OUTFILE 2>&1
                   ipsofwd list >> $OUTFILE 2>&1
            fi
      fi
   else
      echo " ##### Node is not a gateway.  Acceleration and SIM checks bypassed #####" >> $OUTFILE 2>&1
      smallbreak
fi

#############################################################################
#  TABLES CHECKS                                                            #
#############################################################################


# Run certain commands if on a gateway but not running VSX
if [ `cpprod_util FwIsFirewallModule` = "1" ] && [ "$ISVSX" != "1" ]
  then
      echo " ######## fw tab ##########" >> $OUTFILE 2>&1
      echo "      *** connections and stuff ***" >> $OUTFILE 2>&1
      fw tab -t host_ip_addrs -t connections -t fwx_alloc -t fwx_cache -t frag_table -s  >> $OUTFILE 2>&1

      echo "      *** remote users ***" >> $OUTFILE 2>&1
      fw tab -t userc_users -s -t sslt_om_ip_params -t L2TP_tunnels -t om_assigned_ips -s  >> $OUTFILE 2>&1
      smallbreak

      echo " ######## fw ctl pstat ##########" >> $OUTFILE 2>&1
      fw ctl pstat >> $OUTFILE 2>&1
      smallbreak
   else
      echo " ### Node is not a gateway or is a VSX system.  Table and pstat checks bypassed." >> $OUTFILE 2>&1
      smallbreak
fi

#############################################################################
#  HIGH AVAILABILITY CHECKS                                                 #
#############################################################################


# Run certain commands only if the gateway thinks it's running in HA mode
if [ `cpprod_util FwIsHighAvail` = "1" ]
  then
     echo " ############# cphaprob stuff ##########" >> $OUTFILE 2>&1
     echo "     *** -a if ***" >> $OUTFILE 2>&1
     cphaprob -a if >> $OUTFILE 2>&1
     echo "     *** stat ***" >> $OUTFILE 2>&1
     cphaprob stat >> $OUTFILE 2>&1
     echo "     *** syncstat ***" >> $OUTFILE 2>&1
     cphaprob syncstat >> $OUTFILE 2>&1
     echo "     *** cpstat ***" >> $OUTFILE 2>&1
     cpstat ha -f all  >> $OUTFILE 2>&1
     echo "     *** list ***" >> $OUTFILE 2>&1
     cphaprob list >> $OUTFILE 2>&1
     echo "     *** MAC MAGIC NUMBERS AS CURRENTLY CONFIGURED ***" >> $OUTFILE 2>&1
     echo "         -- MAC MAGIC: `fw ctl get int fwha_mac_magic` " >> $OUTFILE 2>&1
     echo "         -- MAC FORWARD MAGIC: `fw ctl get int fwha_mac_forward_magic` " >> $OUTFILE 2>&1
     smallbreak
  else
     echo " #### Node is not running HA feature. cphaprob checks bypassed ####" >> $OUTFILE 2>&1
     smallbreak
fi

#############################################################################
#############################################################################
##  NETWORKING CHECKS                                                      ##
#############################################################################
#############################################################################

secbreak
echo
echo " ###################################################################"
echo " # Starting networking checks...                                   #"
echo " ###################################################################"
echo

echo "  #######################################################################"
echo "  ## NOTE: Not all network checks function on all systems. Some checks ##"
echo "  ## may result in warnings of Operation not supported. These warnings ##"
echo "  ## can be safely ignored.                                            ##"
echo "  ##                                                                   ##"
echo "  ## Press any key to continue                                         ##"
echo "  ## or wait 5 seconds and the script will continue automatically      ##"
echo "  #######################################################################"
read -n1 -t5 $1

warnuser

if [ "$ISMDS" = "1" ]
   then
       echo "## NOTE: Some network tests on Provider-1 or MDSM may return warnings ##"
fi


echo "####### netstat ########## " >> $OUTFILE 2>&1
echo "    *** -ni ***" >> $OUTFILE 2>&1
netstat -ni >> $OUTFILE 2>&1
smallbreak
echo "    *** -s ***" >> $OUTFILE 2>&1
netstat -s >> $OUTFILE 2>&1
smallbreak
echo "    *** -anp ***" >> $OUTFILE 2>&1
if [ "$RUNOS" = "IPSO" ]
   then
     netstat -an >> $OUTFILE 2>&1
     smallbreak
     echo "    *** -m ***" >> $OUTFILE 2>&1
     netstat -m  >> $OUTFILE 2>&1
   
     # Run checks for IPSO flows
     IPSOFLOWS=`ipsctl -n net:ip:forward:available_modes | grep -q -s flowpath`
     if [ "$IPSOFLOWS" -eq 0 ]
       then
         echo "    *** host is running IPSO Flows ***" >> $OUTFILE 2>&1
         echo "Flows active: $((`netstat -nF | wc -l`-2))" >> $OUTFILE 2>&1
         echo "   ***Flow stats***" >> $OUTFILE 2>&1
         ipsctl -a net:ip:flow >> $OUTFILE 2>&1
         smallbreak
       else
         echo "    *** host is not running IPSO Flows, bypassing flow checks ***"
     fi
     unset IPSOFLOWS
   else
     netstat -anp >> $OUTFILE 2>&1
     smallbreak
fi

secbreak

echo " ######## Interface stuff ########" >> $OUTFILE 2>&1
# Gather various interface statistics
if [ "$RUNOS" = "IPSO" ]
   then
       echo "     *** Basic IPSO NIC stats metrics ***" >> $OUTFILE 2>&1
       ipsctl -a net:ip:rxstats net:ip:txstat net:ip:misc:stats net:ip:frag:stats >> $OUTFILE 2>&1
       smallbreak

       if [ "ipsctl -n net:dev:adp_detect | egrep -v '0'" = "1" ]
          then
              echo "     *** ADP metrics ***" >> $OUTFILE 2>&1
              ipsctl -a net:dev:adp >> $OUTFILE 2>&1
              smallbreak
       fi
       echo "    *** Interface information ***" >> $OUTFILE 2>&1
       ifconfig -v -a >> $OUTFILE 2>&1
       smallbreak
     
       # REMmed OUT CONTENT REQUIRES ADDITIONAL LOGIC. MAY BE REDUNDANT TO -v -a ABOVE
       # echo "   *** IPSCTL metrics for $IFN ***" >> $OUTFILE 2>&1
       # ipsctl -a ifphys:$IFN:errors ifphys:$IFN:stats ifphys:$IFN:dev >> $OUTFILE 2>&1

   else
       if [ "$ISTORVALDS" = "1" ]
          then
              echo "     *** ifconfig -s ***"  >> $OUTFILE 2>&1
              ifconfig -s  >> $OUTFILE 2>&1
              smallbreak
              LIST=`ifconfig -s | grep -Ev "Iface|lo" | awk '{print $1}' `
              for IFN in $LIST; do
                  echo " ### Interface information for $IFN ###" >> $OUTFILE 2>&1
                  echo "   *** basics ***" >> $OUTFILE 2>&1
                  ifconfig -v $IFN >> $OUTFILE 2>&1
                  smallbreak

                  # Bonded Interface check
                  if [ ${IFN:0:4} = "bond" ]
                     then
                         cphaconf show_bond $IFN >> $OUTFILE 2>&1
                         cat /proc/interfaces/bond/$IFN >> $OUTFILE 2>&1
                         smallbreak
                  fi

                  echo "   *** settings ***" >> $OUTFILE 2>&1
                  ethtool $IFN >> $OUTFILE 2>&1
                  smallbreak
                  echo "   *** driver and firmware for $IFN ***" >> $OUTFILE 2>&1
                  ethtool -i $IFN >> $OUTFILE 2>&1
                  smallbreak
                  echo "   *** statistics for $IFN ***" >> $OUTFILE 2>&1
                  ethtool -S $IFN >> $OUTFILE 2>&1
                  smallbreak
                  echo "   *** Flow control for $IFN ***" >> $OUTFILE 2>&1
                  ethtool -a $IFN >> $OUTFILE 2>&1
                  smallbreak
                  echo "   *** ring settings for $IFN ***" >> $OUTFILE 2>&1
                  ethtool -g $IFN >> $OUTFILE 2>&1
                  echo "   *** TSO settings for $IFN ***" >> $OUTFILE 2>&1
                  ethtool -k $IFN >> $OUTFILE 2>&1
                  echo "   *** coalesce settings for $IFN ***" >> $OUTFILE 2>&1
                  ethtool -c $IFN >> $OUTFILE 2>&1
                  smallbreak
             done
       fi
fi

secbreak

#############################################################################
#############################################################################
##  FINAL CHECKS                                                           ##
#############################################################################
#############################################################################


echo
echo " ###################################################################"
echo " # Starting final checks...                                        #"
echo " ###################################################################"
echo

# Not all the commands work on all platforms.  Giving some feedback to the end user pacifies concerns
warnuser

#############################################################################
#  PROCESS CHECKS                                                           #
#############################################################################

echo " ########## process information ##########" >> $OUTFILE 2>&1
if [ "$ISTORVALDS" = "1" ]
   then
       ps -AFHwww >> $OUTFILE 2>&1
   else
       if [ "$RUNOS" = "IPSO" ]
          then
              ps auxwwwlSHmf >> $OUTFILE 2>&1
       fi
fi
smallbreak
if [ "$RUNOS" != "SunOS" ]
   then
       ps auxwwwf >> $OUTFILE 2>&1
   else
       ps -elf >>  $OUTFILE 2>&1
fi

secbreak


#############################################################################
#  TIME-REPEATED CHECKS (vmstat, iostat, top                                #
#############################################################################

if [ "$DOTIMEDCHECKS" = "1" ]
   then
       echo
       echo " ###################################################################"
       echo " ###################################################################"
       echo " ## Beginning Time-repeated checks. These checks each take a few  ##"
       echo " ## moments to execute...                                         ##"
       echo " ###################################################################"
       echo " ###################################################################"
       echo

       echo " ######### CPU Utilization Stuff #########" >> $OUTFILE 2>&1

       echo " ###################################################################"
       echo " # Running vmstat collection. This will take a few moments...      #"
       echo " ###################################################################"

       echo "     *** vmstat ***" >> $OUTFILE 2>&1
       vmstat 2 20 >> $OUTFILE 2>&1
       smallbreak

       if [ "$RUNOS" = "IPSO" ]
          then
             echo "        *** -i ***" >> $OUTFILE 2>&1
             vmstat -i >> $OUTFILE 2>&1
             smallbreak
             echo "        *** -z ***" >> $OUTFILE 2>&1
             vmstat -z >> $OUTFILE 2>&1
             smallbreak
       fi


       # Check for the presence of iostat
       IOCHECK=`type -P iostat`
       if [ "$IOCHECK" = "" ]
         then
            smallbreak
            echo "     *** bypassing IOSTAT collection ***" >> $OUTFILE 2>&1
         else
            echo
            echo " ###################################################################"
            echo " # Running IO statistics collection. This will take a few moments..#"
            echo " ###################################################################"
            echo "     *** iostat ***" >> $OUTFILE 2>&1
            iostat -x 2 10 >> $OUTFILE 2>&1
       fi

       secbreak

       #check for dumb terminal
       if [ ! -e /usr/share/terminfo/d/dumb ]
        then
          echo " ##### making dumb terminal symlink" >> $OUTFILE 2>&1
          mkdir /usr/share/terminfo/d > /dev/null 2>&1
          ln -s /usr/share/terminfo/x/xterm /usr/share/terminfo/d/dumb > /dev/null 2>&1
       fi


       if [ "$RUNOS" != "SunOS" ]
          then
              echo "     *** top ***" >> $OUTFILE 2>&1
              echo
              echo " ###################################################################"
              echo " # Running TOP. This will take a few moments...                    #"
              echo " ###################################################################"

              # CPULOOP is mentioned in top's manual to increase 1st-iteration accuracy
              # COLUMNS is used to allow showing longer command-lines on terminal output

              if [ "$RUNOS" != "IPSO" ]
                 then
                     COLUMNS=512 LINES=256 CPULOOP=1 top -bcSH -n 5 >> $OUTFILE 2>&1
                 else
                     top -mio -bSH -d 5  >> $OUTFILE 2>&1
              fi
       else
           echo "     *** prstat ***" >> $OUTFILE
           echo
           echo " ###################################################################"
           echo " # Running PRSTAT. This will take a few moments...                 #"
           echo " ###################################################################"

           prstat 3 5 >> $OUTFILE
     fi


     echo " ###################################################################"
     echo " # Gathering some additional CPU Load information.                 #"
     echo " # This will take a few moments...                                 #"
     echo " ###################################################################"

       # MPSTAT exists in GAiA but not SPLAT
       MPSCHECK=`type -P mpstat`
       if [ "$MPSCHECK" = "" ]
         then
            smallbreak
            echo "     *** bypassing mpstat collection ***" >> $OUTFILE 2>&1
         else
           echo "     *** MPSTAT metrics ***" >> $OUTFILE 2>&1
           # Linux and Solaris have different CLI switch requirements
           if [ "$ISTORVALDS" = "1" ]
              then
                  mpstat -P ALL 2 5 >> $OUTFILE 2>&1
              else
                  mpstat -p 2 5 >> $OUTFILE 2>&1
           fi
     fi

     smallbreak

     if [ "$RUNOS" != "SunOS" ]
        then
            echo "     *** /proc/stat metrics ***" >> $OUTFILE 2>&1
            # Plan to add logic to calculate per-CPU information soon. For now, it'll be manual
            echo "------- Columns --------" >> $OUTFILE 2>&1
            echo "CPU | user | nice | system | idle | iowait | irq | softirq " >> $OUTFILE 2>&1

            # Number of polling iterations to run
            LOOPEND=5
            LOOPTIME=1
            # Delay time -- how long to sleep between polling intervals
            SNOOZETIME=5

            while [ $LOOPTIME -le $LOOPEND ]
                  do
                    cat /proc/stat >> $OUTFILE 2>&1
                    # Increment the loop counter
                    (( LOOPTIME++ ))
                    # Take a nap
                    sleep $SNOOZETIME
            done
     fi

else
     echo " ###################################################################"
     echo " # Bypassing timed checks...                                       #"
     echo " ###################################################################"

fi

secbreak
echo "###### Completed checkup script for $HNAME at `date +"%F-%H%M"` ######" >> $OUTFILE 2>&1
secbreak


################################################################################
################################################################################
##                           SCRIPT CLEANUP                                   ##
################################################################################
################################################################################


# Clean up temp files
rm $APPLTMP
rm $NOCONNTMP
rm $PROCCHECKS
rm $FILECHECKS

# Clean up variables
unset HASMPTSTATUS
unset HASLSIUTIL
unset CHECKTMP
unset CHECKMDS
unset CHECKVSX
unset HASLOM
unset HASMPT
unset SCRVER
unset RUNOS
unset RUNOSFULL
unset ISTORVALDS
unset ISMDS
unset APPLTMP
unset NOCONNTMP
unset PROCLIST
unset PROCCHECKS
unset PROCNAME
unset IOCHECK
unset MPSCHECK
unset FWLABEL
unset BLADECHECK
unset BLADESTAT
unset SIC_NAME
unset OBJ_NAME
unset OBJ_FILE
unset REG_FILE
unset ISVSX
unset ISVSXSWB
unset ISGAIA
unset ISFWACCEL
unset FILELIST
unset VSNUM
unset VSLINE
unset VSTYPE
unset VSNAME
unset DODUCHECK
unset DOTIMEDCHECKS



# COMPLETED
echo
echo "#########################################################################"
echo "#########################################################################"
echo "  Data was collected into $OUTFILE"
echo "  The output file is `ls -lah $OUTFILE | awk '{ print $5 }'` in size.   "
echo "#########################################################################"
echo "#########################################################################"
echo
echo "#########################################################################"
echo "# Completed data acquisition. Thank you. Have a nice day.               #"
echo "#########################################################################"
echo
echo

# Clean up final variables - these couldn't be unset until the end
unset HNAME
unset NOW
unset OUTTO
unset OUTFILE

exit 0