Microsoft SCOM monitoring for PowerMax using Unisphere SNMP traps
Microsoft SCOM monitoring for PowerMax using Unisphere
SNMP traps
I hope you liked my prior blog on NTFS
allocation units and PowerMax storage provisioning and performance best
practices. Looking forward to more comments and suggestions for future
contents, so please keep them coming!!
Lately I have been hearing about Microsoft
Systems Center Operations Manager (SCOM) based monitoring for PowerMax. And
even though around for many years and integrated with many monitoring tools,
SNMP traps based monitoring for PowerMax storage system using SCOM is fairly unexplored
area so far. I have received some questions on various distributions recently about the topic, so I
thought of trying that out. Let me share my experience based on the success of this deployment. It will certainly provide ease of use and control to the system administrators by incorporating PowerMax monitoring alongside other data center resources from single interface of SCOM.
SCOM is around for many years as a
cross-platform data center monitoring system for operating systems and hyper
visors. SCOM monitoring console can provide single interface that shows alerts,
health and performance information for diverse set of resources including
servers, network switches, routers, among others. SCOM includes many management
packs to provide standard set of monitoring for all those devices with
pre-defined rules and views. SCOM uses SQL Server as the back end for collected
information to provide historical charts, trends and analytics. SCOM works with
Windows, UNIX and Linux servers and supports a variety of authentication
mechanisms to provide single pane of glass management for large set of data
center resources. Along with many different data collection mechanisms, SCOM
also supports collection of SNMP traps from managed devices and processing them
using various management packs and scripts for better analysis and reporting.
Unisphere for Dell EMC PowerMax (and its
family of storage systems) offers full feature set for storage provisioning,
performance and REST API based management for SAN and NAS based infrastructure.
Unisphere for PowerMax also provides notifications and alerts via e-mail, sys
log and SNMP trap mechanisms that can be leveraged by third party
infrastructure monitoring products.
I will go over setting up data collection
using Unisphere for PowerMax SNMP traps on Microsoft SCOM. SCOM by design does
not allow collection of SNMP traps from Windows servers so Windows based
Unisphere can’t be used for this purpose. Also using PowerMax eMgmt container
is fairly locked down so it would not allow a key discovery piece needed for
the setup on SCOM. There are some workarounds to avoid discovery but to make
things simple, this time I will cover deployment using Linux based Unisphere for
PowerMax.
High
level overview of the steps needed for the SNMP trap based monitoring
Here are the steps needed on both Unisphere for PowerMax and SCOM for SNMP traps:
- Allow communication on specific UDP ports between SCOM and Unisphere for PowerMax server
- Setup name or IP address resolution of Windows SCOM server by Unisphere server
- Configure SNMP on Unisphere for PowerMax for SCOM server
- Discover device on SCOM for Unisphere for PowerMax server
- Create and install Unisphere management pack on SCOM using MIBs supplied by Unisphere
- Configure rules and views for SNMP traps on SCOM
- References and Further Readings
The following sections will describe all
these steps in greater detail.
UDP
ports used by SCOM and Unisphere for PowerMax server
UDP port 161 is used by SNMP service on Unisphere server to send SNMP discovery
information and UDP port 162 is used on SCOM server to receive SNMP trap
information. These ports should be opened through firewall to ensure that SCOM
can collect SNMP traps from Unisphere for PowerMax.
On SCOM: Allow
ports 161 and 162 for inbound and outbound traffic using Windows firewall
advanced settings.
On Linux server:
Name
and/or IP address resolution on Unisphere for PowerMax server
Make sure that Unisphere server can reach SCOM server. Enter the host/IP address of SCOM server to /etc/hosts file on Unisphere server for proper name resolution if needed.
SNMP
configuration on Unisphere for PowerMax for SCOM server
This step requires enabling SNMP
configuration on Unisphere for PowerMax for the desired storage system and
specifying the objects and conditions for monitoring and sending alerts.
(1) SNMP configuration:
On Unisphere for PowerMax select “Settings”, and under “Alerts” section select
“Notification” to configure SNMP traps to send to SCOM.
Setup SNMP trap destination
as shown below by giving SCOM IP address and port number. In this example, I am
using SNMP version 1 but Unisphere also supports SNMP version 3 if needed for
higher security. After entering IP and port, enter “OK”, close and re-open the
screen, to “TEST” the connection.
Also make sure
that desired PowerMax system has appropriate system level and performance level
notifications enabled as shown below.
(2) Select alert policies for the objects and conditions for sending alerts: Select “Alerts” and make sure that desired alerts have SNMP enabled for notification under “Alert Policies”.
Unisphere
for PowerMax Server device discovery on SCOM
SCOM can collect SNMP traps from a device
only after that device is first discovered on SCOM. Unisphere for PowerMax
server does not really accept any discovery request from SCOM as it is designed
to send just the traps to the specified destination. To allow for discovery by SCOM, we first need
to install a tool like Net-SNMP on Unisphere for PowerMax Linux server. Once
the server is discovered by SCOM, we need to create a “Run as Account” to run
Unisphere for PowerMax server discovery by providing the IP address and port
number to use for the discovery.
On Linux server:
(1) Install Net-SNMP:
(2) Setup Net-SNMP in /etc/snmp/snmpd.conf file with proper community string. Typically, most SNMP devices use “public” as the community string but here we need to use the community string “SNMP_trap” as that is required by Unisphere for PowerMax.
(3) Restart SNMPD after this
On SCOM server:
(1)
Create a “Network Device Discovery”
rule in SCOM. For discovery, first create a “Run As Account” using “SNMP_trap” as the community
string. On discovery wizard, specify IP address and port # of Unisphere server.
Choose explicit discovery and on “Advanced Discovery Settings” change defaults
as specified to avoid timeouts for some long running operations.
(2) Now discovery can be either scheduled or run manually. For our purpose, we will just choose “manual” discovery as we just want to run it once to capture some basic information from Unisphere server.
(3) Using tool like
Wireshark, we can monitor the packets exchanged between SCOM and Unisphere
server during the discovery. As we can see SCOM sends multiple SNMP “Get”
requests and Unisphere server with Net-SNMP utility responds with various OIDs
supported by the server. Output below also shows the version number and
community string “SNMP_trap” sent by Unisphere.
(4) Once SCOM
completes the discovery it will show various parameters for Unisphere servers.
Note down SNMP version and System OID highlighted below as they will be used in
subsequent steps for some changes that are needed for successful deployment.
Creation
and installation of Unisphere management pack on SCOM using MIBs supplied by
Unisphere
We now need to create a custom management
pack with Unisphere specific MIBs so that SNMP traps received from Unisphere
can be parsed and reported appropriately. This step requires a SNMP MP
Generator tool from Microsoft that allows parsing of third party MIB and creating
management pack for the device specified. This tool allows extending basic
capabilities of SCOM to diverse set of devices with different capability
profiles.
(1) Install SNMP MP generator tool on SCOM server. This is a standalone tool and it can be installed on any other server as well. Download SCOM – Extensible Network Monitoring Management Pack Generator tool from: https://www.microsoft.com/en-us/download/details.aspx?id=54083
To
use the tool, create an initial XML specifying the some of device discovery
information from Unisphere server. As we can see below “Device Name” refers to
the Unisphere server name, SysOID refers to SysOID found after Unisphere server
discovery by SCOM. Also set the Id and Name to the values that you want to use
for the management pack being created.
In SNMP MP generator console,
open the existing project by selecting the above XML file. And choose the MIBs
for Unisphere for PowerMax along with some standard MIBs specified below.
Contact Dell EMC customer support to provide the MIBs needed for this step.
Once
the MIBs are loaded SNMP MP generator tool populates the structure with all
MIBs used by SNMP traps from Unisphere for PowerMax.
At this point a new
Management Pack can be generated that uses the OID structure populated above.
As we can see original sample XML file with just basic device specific
information is now updated with rich set of configuration in the newly created
management pack.
(2) Import the newly created management pack to SCOM
(3) SCOM will now show the management pack just imported
Configuration
of rule for SNMP traps on SCOM
At this point we have SCOM updated with all the definitions needed to properly collect SNMP traps from Unisphere for PowerMax and process them. We will now create a rule to catch all SNMP trap based events.
Create a catch all SNMP trap rule with rule type of “SNMP Trap (Event)” that will collect all the OIDs from Unisphere for PowerMax. Use the management pack we created above to store the rule so that our management pack is augmented with new rule and associates all the properties to that rule.
Check the
generated rule in SCOM. Keep the run as profile as default which means it will
use “SNMP_trap” account we used during discovery.
Unisphere for PowerMax uses SNMP version 1 but as you may have noticed from device discovery screen, Net-SNMP used version 2c. SCOM by default only accepts the version number that matches with device discovery. So we need to update our management pack so that it can also allow collection of traps for version 1. To do that we need to first export our management pack as XML.
Open exported XML file and do a couple of changes as shown below – update the version number of the management pack and remove the “Version” specifier from the “Rules” section of the XML. Removing this line would allow SCOM to collect SNMP traps from version 1 as well.
Save the XML with these changes and re-import the management pack and make sure that the updated management pack version is now listed.
Configuration
of Event View on SCOM to display the events collected from the SNMP traps
At this point SCOM has the rule to collect
SNMP traps from Unisphere for PowerMax. Desired views can now be created to
properly aggregate and group all the traps collected.
Once view is created SNMP traps from
Unisphere for PowerMax will be populated in that view. If you don’t see any
traps received run “Test” on SNMP configuration on Unisphere for PowerMax
console and run Wireshark on SCOM to ensure proper connectivity and package
delivery. As shown in the example below after successful deployment a device
allocation related trap sent by Unisphere was collected by SCOM and reported in
the event view.
Conclusion
Unisphere for PowerMax has nice SNMP trap based notification mechanism that can be integrated well with Microsoft Systems Center Operations Manager (SCOM). Such integration would allow centralized monitoring for PowerMax resources along with all other infrastructure services and components from single user interface of SCOM.
References
and further readings
(1) Microsoft Systems Center Operations Manager (SCOM): https://docs.microsoft.com/en-us/system-center/scom/welcome?view=sc-om-2019
(2) Microsoft SNMP MP Generator Toor: https://www.microsoft.com/en-us/download/details.aspx?id=54083
Comments
Post a Comment