Monday, November 30, 2009

So, you installed System Center Operations Manager…what next?


So, you decided to install System Center Operations Manager in your environment in order to start monitoring your environment. Or someone else installed Operations Manager for you, and it’s your job to start working with it. What should you do next?

You probably made a design or a design was made and based on that design Operations Manager is installed in your environment. For these steps a lot of documentation is available from Microsoft, a lot of this documentation can be found on the Microsoft website.

- Infrastructure Planning and Design guides: http://go.microsoft.com/fwlink/?LinkId=160871
- Technet – Operations Manager: http://technet.microsoft.com/en-us/opsmgr/default.aspx
- Technet – Operations Manager R2 Design Guide: http://technet.microsoft.com/en-us/library/dd789005.aspx
- Technet – Library: http://technet.microsoft.com/en-us/library/bb310604.aspxhttp://technet.microsoft.com/en-us/library/bb310604.aspx

Keep in mind two things:

1. Beside using Operations Manager as a tool to detect problems in your environment, Operations Manager as a product must be maintained as well.
o Tune your Operations Manager environment
o Update Management Packs
o Add Agents

2. You must seriously invest time in Operations Manager, so that engineers who use the application to detect problems in the environment will soon lose their interest in the product when Operations Manager doesn’t reflect the real situation. When “bogus” messages appear, or problems aren’t detected in System Center Operations Manager, the product will not be taken serious.

Here are some tips which can help you to start tuning your Operations Manager Environment.

Rename your Default Management Pack

By design, when you define an override you will end up saving this override in the Default Management Pack. If you save all of your overrides in one management pack though, you will eventually end up with a serious problem. That’s because you can’t delete a management pack which has another management pack which depends on it. For Example: you need to upgrade your Cluster MP, but first have to remove it because of the changes to the MP, you need to delete your Default MP; this deletes all overrides and all tuning is gone, result: lots of alerts again,etc. If you already have overrides in your default management packs you can use Override Explorer written by Boris Yanushpolsky to track them down, and move them to another management pack.

Keep in mind though, that moving an override to another management pack doesn’t remove the reference in the Manifest section of the (Default) Management Pack. In order to accomplish this you have to do some editing in the XML file as outlined in this blog post: http://blogs.technet.com/momteam/archive/2007/05/03/removing-dependencies-on-the-default-management-pack.aspx

Define Override Management Packs

For the Management Packs which are already available I suggest that you create override Management Packs per MP. Define a naming convention for these kind of Management Packs so that you can easily identify them. Use the name of the Management Pack and the word Override in the name, for example: “Windows Cluster MP – Overrides”.
I would even suggest to make a distinct in override managements packs, one which contains overrides for object you really don’t want to monitor, and another one which contains ‘fixes’ which might be resolved by later Management Packs.

Create a Closed Alerts View

By design, there is no view which contains the Closed Alerts, you may want to create this view though so that you can browse to the closed alerts. Which can help you to determine what was wrong at a certain point of time. Use the filter “resolution state equals 255” to achieve this.

Schedule a backup of your Management Packs

Define a Scheduled task which runs a Powershell script which will make an export of your management packs every night. Having your Management Packs available can help you a lot when you have problems with one of your operational Management Packs, and saves you from restoring the whole Ops DB.

Reference: Using PowerShell to import/export management packs: http://ops-mgr.spaces.live.com/Blog/cns!3D3B8489FCAA9B51!159.entry

Antivirus Exclusions on your Agents

Keep in mind that the Operations Manager Agent executes certain scripts for all kinds of tasks. If a virus scanner is blocking these scripts, the scripts can’t do their job, resulting in lots of errors. These scripts are run from the Health Service State folder, so exclude this folder containing the Operations Manager Agent installation on every Agent and disable things like the McAfee scriptscan functionality.

Firewall Ports

When your clients have the firewall enabled, keep in mind to open the necessary port based on the functionality you need. You can find more information about that in the following articles

Reference: Using a Firewall with Operations Manager 2007: http://technet.microsoft.com/en-us/library/cc540431.aspx

And if you want to install the Operations Manager Agent from the console you will need the following firewall rules enabled.

Reference: How does Computer Discovery Work in OpsMgr 2007?: http://blogs.technet.com/momteam/archive/2007/12/10/how-does-computer-discovery-work-in-opsmgr-2007.aspx

Add Agents and additional Management Packs

Out of the box, System Center Operations Manager is installed with a couple of Management Packs already installed. These Management Packs provide some basic monitoring though, so you may want to install additional Management Packs.
You can find many Management Packs in the System Center Operations Manager Management Pack Catalog, and you can find some community written Management Packs here: http://www.systemcentercentral.com/WIKI/WIKIDetails/tabid/146/indexid/20495/Default.aspx
Keep in mind though that it is a good practice to read the corresponding Management Pack Guide carefully before you import your management pack. They contain information like which discoveries are disabled by default or which overrides to set to get a “working” MP. These Management Pack Guides can be found at the following location:
System Center Operations Manager 2007 Management Pack Guides: http://technet.microsoft.com/en-us/library/dd347500.aspx

From my perspective there are two approaches to fully deploy monitoring to your infrastructure:
- Install Agents on all of your “to-be monitored machines”, and introduce one Management Pack at a time and tune it before continuing to the next MP.
- Add all the Management Packs at once and granularly deploy your Agents, tuning them agent-by-agent.

Don’t do both, because you will end up with a lot of noise by so called “false positives” being generated, and for that noise you will have to figure out if it contains valid errors or that you can discard the error. You do this by either solving a bug, or by defining an override, but having to do this all at once will “drown” you in alerts
Tuning your Operations Manager environment consists of a lot of steps to adjust the MPs. There is no cookbook on how to tune your environment, because how and what to tune depends on your environment. That’s what makes Operations Manager a product that is different for every environment ;)

You may want to start looking at the Critical alerts, check the proposed resolution steps in the alert and try to solve the problem. If you don’t understand the alert try to find more information about the alert on the Internet. There are many good blog posts out there which can help you to determine what the problem is you are experiencing, and how to solve it.

Get involved in the community

One of the big advantages of working with System Center Operations Manager is that it’s supported by a very enthusiastic community and a very involved MS product team. Many problems you will encounter are already described and solved in blog posts, and if not you can post your problem on one of the forums which is monitored by a lot of SCOM guru’s which will try to help you the best they can. Here are some suggestions:

o System Center Central – http://www.systemcentercentral.com
o Operations Manager Online Forums - http://social.technet.microsoft.com/Forums/en-US/category/systemcenteroperationsmanager

Blogs:
o Kevin Holman – http://blogs.technet.com/kevinholman/
o Boris Yanushpolsky - http://blogs.msdn.com/boris_yanushpolsky/
o Marius Satura - http://blogs.msdn.com/mariussutara/
o Stefan Stranger - http://blogs.technet.com/stefan_stranger/default.aspx
o Walter Chomak - http://blogs.technet.com/wchomak/
o Operations Manager http://ops-mgr.spaces.live.com/default.aspx
o Operations Manager Team Blog http://blogs.technet.com/momteam/default.aspx
o Operations Manager Support Team Blog - http://blogs.technet.com/operationsmgr/

Begin building your Distributed Applications

After you performed your initial tuning of the environment, and you feel comfortable with the messages you’re receiving in the SCOM Console, you can take the next step. Start building your Distributed Applications and move from Server monitoring towards Service/Application Monitoring. Here you will discover the true strength of System Center Operations Manager.
When building Distributed Applications, keep in mind that in order to successfully monitor an distributed application you need to monitor all the objects that compose the distributed application. Sometimes this means you might want to consider buying 3rd party management packs which provide this functionality, or create your own management pack.

Download this article in PDF format for easy reading