SCOM – Create Windows Service Monitors

By | February 9, 2024
0 Comment

Hi,

I work at several customers managing their SCOM environment. In all those environments I come across the same questions eg.:

  • Can you configure monitoring of Windows Service X on these servers?
  • Can you configure monitoring of Windows Service Y on all the Windows Servers?

This involves some steps to be taken:

  • Create a Windows Computer group containing the necessary servers (even for 1 server I tend to do that as this keeps things “cleaner” in SCOM)
  • Look for the service name of the service(s) that need to be monitored
  • Create a Windows Service monitor in SCOM using the Management Pack template available in the SCOM Console (under Authoring)

Performing those steps takes some time. Especially when you need to do it multiple times. I thought about a way to do this more efficiently/faster. Therefore I developed a powershell script in my spare time.

Prerequisites of the script:

Take note that I’ve tested the script directly on the SCOM management server.

When the script launches you need to enter the FQDN of any SCOM management server in your environment

After filling in the the FQDN the script launches and fetches all Windows Computers in your SCOM environment. At the top you can filter on any part of a Windows Computer name, might be handy in case of a lot of servers in your environment. In my lab I don’t really need that :).

Click any server you want to monitor services from and click the Add (>>>) arrows. Add additional servers if needed (if you want to monitor the same service on multiple servers). In this case I’ll add my SQL server as I want to monitor some SQL service:

When the server is selected, click Get Services. The script will try to fetch services locally from that server using remote WMI. So be sure that is allowed. If everything goes well a list of installed services is shown.

Now you can scroll through the list and select the Windows services and add services (using >>>) to be monitored. Multiple services can be added. Let’s add 2 services as an example.

There is also a checkbox “Monitor on all Windows Servers” there. This can be checked if you have 1 or more Windows Services you want to monitor on ALL Windows Servers in your environment. If you leave it unchecked, the script will create a computer group containing the servers selected previously. If you check it the monitor will be targeted to the All Windows Computers Group.

Final step is adding a management pack prefix, I usually put the customer name there:

Now everything is ready to generate the Management Pack. When clicking in the background the following things will happen:

  • A management pack will be created
  • Monitor on all Windows Servers UNCHECKED:
    • A computer group will be created with all servers selected in the GUI
    • Necessary service monitors will be created and targeted to the newly created group
  • Monitor on all Windows Servers CHECKED:
    • Necessary service monitors will be created and targeted to the All Windows Computers Group

Clicking the Reset button will simply clear all input and you can start all over.

How does this look in SCOM:

Management Pack

Computer Group

Service Monitors

Doing this it only took me around 1 minute to setup monitoring of those services. When doing the same through the SCOM console it took me around 5 minutes. So when you need to create service monitors regularly it saves a lot of time.

If you are interested in this tool, I’ve compiled an EXE which I can share, just drop me a mail on bert@bpitconsulting.be

Hope this helps!

SCOM – Powershell Monitor

By | October 25, 2023
0 Comment

Hi guys,

I was working on a custom monitor at one of my customers using a powershell script. It involved creating a dataset to store some variables coming from a SQL database.

When trying to save the monitor I got an error stating: Incorrect expression specified: $Data = New-Object System.Data.dataset

A quick search pointed at that it was regarding to the variable $Data, apparently that is a reserved “name”. Getting rid of that solved the problem and it works like a charm.

Source: https://michelkamp.wordpress.com/2014/05/27/bug-vsae-with-a-powershell-data-parameter/

Hope this helps to save some time!

SCOM – Servers without Perf Data

By | April 22, 2020
0 Comment

Hi,

Sometimes it happens that a server stops reporting performance data to SCOM. But we still get alerted when a monitor gets critical.

When do you notice this problem? Right, when you want to dive into performance data to troubleshoot something. Then you see that the server hasn’t sent performance data for the last couple of hours, days, weeks,…

To solve (or rather workaround) this issue I’ve managed to create a SQL query to point me toward servers that haven’t sent performance data in the last 2 hours (excluding servers that are in maintenance mode): 

with _CTE
AS
(select bme.[Path],
pcv.ObjectName,
pcv.CounterName,
pcv.InstanceName,
pdv.SampleValue,
pdv.timesampled,
convert(varchar(max),pdv.TimeSampled,103)+ ' ' + convert(varchar(max),pdv.TimeSampled,108) as [Last Sample],
ROW_NUMBER() OVER (PARTITION BY bme.Path ORDER BY pdv.TimeSampled desc) AS RowNumber
from PerformanceDataAllView pdv with (NOLOCK)
inner join PerformanceCounterView pcv with (NOLOCK) on pdv.performancesourceinternalid = pcv.performancesourceinternalid
inner join BaseManagedEntity bme with (NOLOCK) on pcv.ManagedEntityId = bme.BaseManagedEntityId
where objectname = 'Memory' AND
countername = 'Available MBytes'
AND bme.BaseManagedEntityId NOT IN (SELECT M.BaseManagedEntityId FROM MaintenanceMode AS M WITH (NOLOCK) WHERE IsInMaintenanceMode = 1)
)
select * from _CTE WHERE RowNumber = 1
and timesampled < DATEADD(HOUR, -2, GETUTCDATE())
order by timesampled DESC

I’ve added this query the SquaredUp dashboard I check every morning.

But I also want to get alerted, so I created a powershell monitor to include this SQL query. As soon as a server stops sending performance data I get notified and I can (manually) solve it:

  • Stop Microsoft Monitoring Agent
  • Cleanup Health Service State folder
  • Start Microsoft Monitoring Agent

Hope this helps,

Bert

SCOM – Forward Alerts to Microsoft Teams

By | April 21, 2020
0 Comment

Today I want to talk about sending Alerts from SCOM to Microsoft Teams. Due to the current situation a lot of companies (that didn’t use Microsoft Teams previously) start using Microsoft Teams and want to have as much data as possible there. Thus it might be interesting to have the (critical) open alerts there as well.

I found some interesting posts on the internet:

I wanted to try this out myself as one of my customers also wanted to see how far we can go with this. Let’s go!

First things first, setup the necessary components in Microsoft Teams:
– Setup a dedicated Team in Microsoft Teams and create a channel. I called the team “Monitoring” and the channel “SCOM Alerts. I will give all relevant users necessary rights on this team

– Right click on the “SCOM Alerts” channel and select “Connectors”

– Search for “incoming webhook” and click Configure

– Give the webhook a name, I choose “SCOM”

– Click Create (make sure to copy the URL, we will need this later on in the powershell script needed for the subscription)

As Microsoft Teams is now configured, we can dive into powershell.

To post a message to Teams we will use the “Invoke-Webrequest” cmdlet from powershell. The message itself is in JSON format. In the message I would like to include: alertname, source, description, alert time, resolutionstate,…
As something extra I also added a “SquaredUp” button that enables you to go directly to the alert in SquaredUp.

I did struggle a bit to include the alert description in the JSON file, as for some alerts including the description resulted in a “Bad request”. This had to do with a “\” included in the description, which is a reserved character. adding a string replace “\” –> “\\” solved the problem. Thanks Cole for pointing that out!

The core of the script is:
# Fill in the Webhook URL here
$SCOM_Hook = ‘[URL goes here]’

# Build the JSON object for the POST – using a “stringwich” ( @”…”@ )to keep it readable.
$content_JSON = @”
{
“title”: “Critical SCOM Alert”,
“text”: “Source: $alertsource”,
“themeColor”: “FF0000”,
“sections”: [{
“title”: “Details”,
“facts”: [{
“name”:”Time Raised”,
“value”:”$alerttime”

},
{
“name”:”ResolutionState”,
“value”:”$alertresolutionstate”
},
{
“name”:”Monitor/Rule”,
“value”:”$alert”
},
{
“name”:”Description”,
“value”:”$alertdescription”
}
]
},
{
“potentialAction”: [
{
“@context”: “http://schema.org”,
“@type”: “OpenUri”,
“name”: “SquaredUp”,
“targets”: [{
“os”: “default”,
“uri”: “$alerturl”
}
]
}
]
}
]
}
“@

Invoke-WebRequest -Uri $SCOM_HOOK -Method POST -Body $content_JSON -ContentType application/json

When your script is finished we can create the necessary components in SCOM (make sure your SCOM servers have internet access to https://outlook.office.com)

  • Create a channel:
    • Full Path of the command file: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
    • Command line parameters: \\myserver\myfolder\sendtoteams.ps1 ‘$Data/Context/DataItem/AlertId$’
    • Startup folder: C:\Windows\System32\WindowsPowerShell\v1.0\
  • Create a subscriber for that channel
  • Create a subscription for the subscriber
    • You can include any criteria like you would on any other subscription

When you’re finished, just wait for an alert to fire the subscription
An alert message looks like this in Teams:

Looks even cooler in dark theme:

Extra tip: When using the Teams app on Android or iOS setup Notifications for the SCOM Alerts channel and you will get a push notification when an alert is posted.

If you have any questions, just drop me a line and I’ll be happy to help!

Best regards,
Bert

SCOM – Uptime Robot Management Pack

By | April 3, 2020
2 Comments

Hi,

As I posted earlier SquaredUp can be integrated with all kinds of APIs, however if you know about management pack authoring an API can also be used as datasource in a management pack. In this post I’ll show some information about an Uptime Robot Management pack I created to create classes and instances of those classes in SCOM by using the API of Uptime Robot. To accomplish this I worked together with fellow SCOM consultant Jasper Vandamme

First things first: create the API using the Uptime Robot website. (detailed steps: https://www.bpitconsulting.be/2019/11/12/squared-up-uptimerobot-api/ )

I created the management pack using Visual Studio Authoring Extensions.

Classes (with properties):

  • Uptime Robot WatcherNode
    • URL
    • APIKey
  • Uptime Robot Website
    • FriendlyName
    • ID
    • URL
    • Type
    • Interval

Uptime Robot WatcherNode Discovery:

All webrequests to the API are targeted on the watchernode. For this Discovery a registry discovery is used. It looks for a key HKLM\SOFTWARE\SCOM\Uptimerobot with values APIKey and URL. Both parameters are used afterwards in the scripts.

Uptime Robot Website Discovery:

After the WatcherNode is discovered the Website discovery kicks off, the core of the script is these lines of code:

$body = “api_key=$ApiKey&format=json”

Invoke-webrequest -method post -uri “$scripturl/v2/getMonitors” -body $body -UseBasicParsing

Monitors:

  • Website availability ( up or down)
  • API availability (up or down)

Rules:

  • API responsetime (ms)
  • Website responsetime (ms)
  • Website uptime last 7 days (%)
  • Website uptime last 30 days (%)
  • Website uptime last 90 days (%)

A dashboard can be created in SquaredUp to show the performance data

Drop me a comment below if you have any suggestions, feedback,..

Best regards,

Bert

Squared Up – Uptimerobot (API)

By | November 12, 2019
1 Comment

Hi,

Today I want to talk about the integration of SquaredUp with external APIs. As an example I want to integrate date coming from Uptimerobot (https://uptimerobot.com/)  in SquaredUp. Uptimerobot enables you to aadd 50 websites to check for free.

First of all create an account for Uptimerobot

To be able to access data we need to create an API key, therefore go to “My Settings”. Go all the way down until you see “Read-Only API Key”. Click “Show/Hide it”. Create the API key (or copy the one that is already listed)

The Uptimerobot API is documented here: https://uptimerobot.com/api

Next step is adding the API in SquaredUp. Open Squaredup > System > Web Api. Click “Add New Provider”. Fill in Service name, base URL should be the following: https://api.uptimerobot.com/v2

Click Add provider, it should look like this:

Now it’s time to create a dashboard. I want to show the status of all monitors and the responsetimes as well, let’s go.

Create a new empty dashboard select the “Web API” tile

Select “Web API (grid)”

Under Scope, click Next

 Select “Uptime Robot” Provider

Click Next

Select “Post” and add the url /getmonitors

Provide the necessary parameters

Under Key path enter monitors

Add any columns you want in the grid eg. Friendly Name

I wanted to show the response times as well but in seconds. The value is gathered in milliseconds so it should be divided by 1000.

The output looks like this:

Hope this helps to set this up if you might need it!

Best regards,

Bert

SCOM – Powershell Recovery Action – Stopped Windows Service

By | November 12, 2019
0 Comment

Hi,

Today I was at a customer who had a really specific question regarding monitoring of Windows Services with Operations Manager (SCOM).

We had already set up some basic recovery actions which restart the service automatically after it was stopped.

For some other services the customer wanted to add extra functionality: The recovery action should retry starting the service a maximum of 3 times, if the service wasn’t started after 3 tries the customer wanted to receive an email telling them the recovery action failed. Out-of-the-box SCOM is unable to do stuff like that, therefore I used Powershell to accomplish this.

Sidenote: To be able to use Powershell as a recovery action you can use the free management pack provided by the community & SquaredUp, it can be downloaded from this website: https://squaredup.com/free-powershell-management-pack/. This management pack adds Powershell everywhere it is missing in Operations Manager, this is one of the default management packs I always install at customers.

To be fully functional different components are needed:

  • A monitor that checks the status of the service
    • This monitor can be created from the Authoring pane of the SCOM console using the Windows Service template
3
  • A recovery action for the monitor created previously
    • The recovery action can be created from health explorer
  • A rule that picks up the event created by the recovery action Powershell script
    • This is an Alert Generating Rule (NT Event Log), the configuration is linked to the type and location of the event logged during the script
  • A subscription on the rule to send the email.

The powershell script:

# Fill in the service name here

$ServiceName = “LPD Service”

$ServiceStarted = $False

$i =0;

#Create Eventlog source, erroraction Ignore is neededbecause once the source is created an error is thrown because the source already exists

New-Eventlog -LogName Application -Source “Powershell – Restart Service” -ErrorAction Ignore

Do{

# In second or third run, wait a minute before trying
to start the service

if ($i -gt 0){Start-Sleep -s 60}

#Try to start the service

Start-Service $ServiceName

$Service = Get-Service -Name $ServiceName

     if($Service.Status -eq “Running”)

    {

    $ServiceStarted = $true

     }

    $i++

    if (($i -eq 3) -and ($ServiceStarted = $false))

    {

    $eventmessage = “$Servicename failed to restart after $i attempts, exiting script”

    #Log error event in eventviewer

    Write-Eventlog -LogName Application -Source “Powershell – Restart Service” -EntryType Error -Eventid 101 -Message $eventmessage

    exit

    }

 }

Until ($ServiceStarted = $true)

 $eventmessage = “$ServiceName restarted after $i attempt(s)”

Write-Eventlog -LogName Application -Source “Powershell – Restart Service” -EntryType Information -Eventid100 -Message $eventmessage

 If you have any difficulties doing this, don’t hesitate to drop a comment below.

If you find this post useful, please consider buying me a virtual beer with a bitcoin donation: 3QhpQ5z5hbPXXRS8x6R5RagWVrRQ5mDEZ1

Best regards,

Bert

SCOM – File Count Management Pack

By | November 12, 2019
0 Comment

Hi,

I come at a lot of customers to implement or support SCOM. Sometimes the same questions or troubles come up.

One of that questions is: “Is it possible to monitor the count of files (with a specific extension) in a share?”

The answer to this question is yes and no. There is a possibility to count files on Windows Servers that have an agent installed using this management pack: http://www.systemcentercentral.com/pack-catalog/file-system-management-pack-2/ but for shares located on non-Windows Servers, let’s say on a SAN for example I haven’t found a solution available.

Therefore I created my own management pack to monitor the file count, independent of the location of the file share (Windows Server or not).

In this post I describe how the management pack works. With the management pack you can count files with a specific extension (or no extension if everything should be counted) in a share (optionally also subfolders included).

There is also the ability to add a specific age zo the given scenario is possible: Count if there are more then 20 files in a share (subfolders included) that are older then 10 minutes.

First of all we need a seed discovery which is targeted to a registry key located on a SCOM agent monitored Windows Server.

The value in the registry is located under SOFTWARE\Filecount. The value is “CSV” and it should contain the path to a CSV file. The server will be discovered as a “File Count Watcher Node”

Next stop is the csv file itself, for every share to be monitored it should contain a line with a specific syntax shown in the screenshot below

Different parameters are added:

  • ID
    • Must be unique per share
  • Share
    • UNC path of the share
  • Extension
    • The extension of the files that needs to be counted, leave empty to count all files in the share
  • Count
    • How many files must be present for a critical state
  • Time
    • This is the time in minutes of the maximum file age of file count
  • Recurse
    • 0 = No need to count files in subfolders
    • 1 = Count also files in subfolders

When the info is filled in, SCOM will discover every line as a “File Count Share”. The properties are used to configure the monitoring.

A monitor is also defined based on the properties filled in the csv file, but it’s basically a powershell script with necessary parameters.

The core of the script is this command:

$count  = Get-ChildItem -Recurse $strShare\$strExtension | where{$_.LastWriteTime -le (Get-Date).AddMinutes($strAge)}|Measure-Object |%{$_.Count}

The file count is also gathered as a performance counter so it can be included in reporting or in a Squared Up dashboard for example.

The management pack is also configured to use a specific Run As account. This account needs rights on the shares: at least Read-only Share rights and Read-Only NTFS rights.

I’ve been able to help some customers already by using this management pack.

The first customer where I set this up is a big hospital in Belgium where they use this management pack to monitor shares which are used to store (and process) images and movies made during surgery.

The content should be processed from the network share and transferred somewhere else but sometimes the processing hangs and the share is getting full without anyone knowing. Since they have the management pack in place this hasn’t happened anymore.

If you have interest in the management pack, I’ve made it available via GitHub: https://github.com/bpinoy/ManagementPacks/tree/master/File%20Count%20MP

Best regards,

Bert

Visualize Windows Update info (from SCCM) in Squared Up

By | November 12, 2019
2 Comments

Hi,

Today I want to show you guys a nice example of how we can use custom scripts to visualise data coming from SCCM in Squared Up dashboards on top of a SCOM platform. And provide a strong integration between SCCM, SCOM & Squared Up.

Squared Up is my favorite tool to create dashboards, check it out here: https://demo.squaredup.com 

I’m currently in the process of setting up a brand new SCOM 1801 environment at a customer and an SCCM CB environment at the same customer with the focus on servers and server patching. The customer wanted also an easy way to check the following parameters:

  • How many updates are pending on each server?
  • Is there a pending reboot due to the installation of Windows Updates?
  • When was the server last updated? And what Windows Updates have been applied then?
  • Are there any maintenance windows applied to a server?

Lots of questions to be answered! All this information can be found somewhere hidden in the SCCM reporting or we could create some custom reports containing that info. However, due to the fact I’m not a SQL reporting guru I wanted to see if we could show all this information in an easy way using Squared Up. As all the servers also have a SCOM agent installed we could get our information directly from there. And I have to say, it was really fun and easy to do!

I’ve used some custom powershell scripts which I added as SCOM tasks, these tasks are then added to a “Windows Updates” perspective I created for each Windows Computer object and this is the result:

All the components you see in the above perspective are based on powershell scripts put in SCOM tasks.

The “Pending Updates” counter is a custom performance counter reading information from WMI, this is the script:

# Any Arguments specified will be sent to the script as a single string.
# If you need to send multiple values, delimit them with a space, semicolon or other separator and then use split.
param([string]$Arguments)

$ScomAPI = New-Object -comObject “MOM.ScriptAPI”

# Collect your performance counter here. Note that in order to support the DW this script MUST only return a single
# object/counter and those must be static values. You can return multiple instances just fine though.

$Instances = @(“Total”)
$Metric = @(Get-WmiObject -Class CCM_SoftwareUpdate -Filter ComplianceState=0 -Namespace root\CCM\ClientSDK).Count

Foreach ($Instance in $Instances)
{
$PropertyBag = $ScomAPI.CreatePropertyBag()
$PropertyBag.AddValue(“Metric”, $Metric)
$PropertyBag.AddValue(“Total”,$Instance)

# Send output to SCOM
$PropertyBag
}

On top of the perspective I’ve also added a button to install all pending updates directly from the dashboard without having to logon to the server and start the installation from Software Center. When clicking the button, Software Center kicks of the installation within minutes.

There is also an additional dashboard where they can review the servers with the most updates pending (top 20) per domain.

To finish things of I added some of the above information to the “Windows Server” perspective as well. In this way the customer can immediately see additional info (highlighted in red in the below screenshot):

  • How many pending updates?
  • When was the server last rebooted?
  • Is there a pending reboot due to Windows Updates?
  • If the server is a VM, what is the Hyper-V host the VM is running on?

Now they are using Squared Up as the go to tool for reviewing the health of the environment and for following-up on patching!

If you would like to accomplish the same or have any questions, please feel free to drop a comment below and I’ll be happy to share my scripts.

Hope this helps!

Best regards,

Bert