Tuesday, May 18, 2010

Agent Pending Actions can get out of synch between the Console, and the database

When you look at your agent pending actions in the Administration pane of the console.... you will see pending actions for things like approving a manual agent install, agent installation in progress, approving agent updates, like from a hotfix, etc.

 This pending action information is also contained in the SQL table in the OpsDB - agentpendingaction

It is possible for the agentpendingaction table to get out of synch with the console, for instance, if the server was in the middle of updating/installing an agent - and the management server Healthservice process crashed or was killed.

In this case, you might have a lingering pending action, that blocks you from doing something in the future.  For instance - if you had a pending action to install an agent, that did not show up in the pending actions view of the console.  What might happen, is that when you attempt to discover and push the agent to this same server, you get an error message:

"One or more computers you are trying to manage are already in the process of being managed.  Please resolve these issues via the Pending Management view in Administration, prior to attempting to manage them again"

The problem is - they don't show up in this view!

To view the database information on pending actions:
select * from agentpendingaction
You should be able to find your pending action there - that does not show up in the Pending Action view in the console, if you are affected by this.

To resolve - we should first try and reject these "ghost" pending actions via the SDK... using powershell.  Open a command shell, and run the following:
To see a prettier view:
get-agentpendingaction | ft agentname,agentpendingactiontype
To see a specific pending action for a specific agent:
get-agentPendingAction | where {$_.AgentName -eq "servername.domain.com"}
To reject the specific pending action:
get-agentPendingAction | where {$_.AgentName -eq "servername.domain.com"}|Reject-agentPendingAction
We can use the last line - to reject the specific pending action we are interested in.

You might get an exception running this:
Reject-AgentPendingAction : Microsoft.EnterpriseManagement.Common.UnknownServiceE
xception: The service threw an unknown exception. See inner exception for details
. ---> System.ServiceModel.FaultException`1[System.ServiceModel.ExceptionDetail]:
Exception of type 'Microsoft.EnterpriseManagement.Common.DataItemDoesNotExistExc
eption' was thrown.

If this fails, such as gives an exception, or if our problem pending action doesn't even show up in Powershell.... we have to drop down to the SQL database level.  This is a LAST resort and NOT SUPPORTED.... run at your own risk.
There is a stored procedure to delete pending actions.... here is an example, to run in a SQL query window:
exec p_AgentPendingActionDeleteByAgentName 'agentname.domain.com'
Change 'agentname.domain.com' to the agent name that is showing up in the SQL table, but not in the console view.