BES Monitoring - BigFix

  • Doc File 133.50KByte



BES Troubleshooting

Many companies consider BigFix components to be part of their critical IT infrastructure and wish to monitor the different components using third-party monitoring tools to ensure proper functionality. Below is information on how to troubleshoot and monitor each BES component. It is assumed to cover only BES 6.0 and later, although some information can also apply to BES 5.1 or even BES 5.0.

BES Server

The BES Server is the most important component to monitor. Here are a number of options for monitoring the BES Server.

1. Network Accessibility --

o Ping the BES Server periodically to make sure it is up and accessible from the network. It should be reachable by all top level BES Relays.

2. Services -- The following services should be running. If they are not running, the BES Server will not function properly.

o BES Root Server - Handles all incoming connections to the server.

• telnet 52311

This should return with a blank screen or http headers. If you get a failure to connect such as a timeout, this indicates the BES Root Server is not running or is not able to accept connections on the port.

• Netstat

This will show connections to the BES Server. This is useful to determine if the server is overloaded with connections (one indication of this is thousands of connections in “TIME_WAIT” status).

o BES FillDB - Puts information from the BES Clients into the database.

▪ Log file: “C:\Program Files\Bigfix Enterprise\BES Server\FillDBData\FillDB.log”

▪ BufferDir -- The bufferdir temporarily stores reports from the BES Clients before being put into the database. By default, the bufferdir directory resides at C:\Program Files\BigFix Enterprise\BES Server\FillDBData\BufferDir\.

▪ Files should appear and disappear fairly quickly (in a matter of seconds). However, if you are monitoring this folder, be careful to not monitor this folder too often because it might cause performance problems (checking once every 10 minutes is OK, but don't check every 10 seconds).

▪ The bufferdir will be "full" if it has 3MB of files or if it has more than 10,000 files (by default).

▪ It is a good idea to monitor the bufferdir folder and issue an alert if the folder has more than 2.5 MB of files or has more than 9000 files.

▪ The bufferdir is one of the most important monitoring activities because if the bufferdir fills up, it indicates a information is not getting to the BES Server quickly, and can be a severe problem.

o BES GatherDB - Puts new Fixlet information into the database.

▪ Log file: “C:\Program Files\Bigfix Enterprise\BES Server\GatherDBData\GatherDB.log”

o BES Gather - Contacts the Internet to download files and to download new Fixlet messages.

▪ Log file C:\Program Files\Bigfix Enterprise\BES Server\Mirror Server\Inbox\GatherState.xml”. This file indicates the status of the various sites being gathered.

o BES Client - The BES Client will check for known issues on the BES Server.

▪ Log file: “”C:\Program Files\BigFix Enterprise\BES Client\__BESData\__Global\Logs\” The files will be in the format of “YYYYMMDD.log”.

▪ Log files that grow to over 128K will be renamed as “YYYYMMDD.bkg”

▪ Additional logging can be accomplished using the “Emsg” log. Please see the following for how to enable this level of logging:

▪ The system Event Viewer may have additional information.

▪ Note: without a BES Client on the BES Server, the BES Server will not become relevant for BES Server upgrade Fixlet messages.

o BES Web Reports - Many times the BES Web Reports runs on the same computer as the other BES Server components.

1. Database -- The database is at the central core of the information going into and out of BigFix, and there are a few basic components which can be monitored.

o Log file: SQL Server Logs – run the SQL Server Enterprise Manager and drill down under Management to “SQL Server Logs” where you should see a “Current – xx/xx/xxxx” section.

o Using Query Analyzer, the following commands can be useful for determining bottlenecks:

▪ sp_who- shows processes and can be used to see if processes are blocking and blocked

▪ sp_lock - provides some information about the resource that is being used, and you can then find out which table is being referenced via:

▪ select object_name()

▪ Note: is taken from the output of the sp_lock command.

▪ In addition, the following can help find some information about how to tell what command is being used The following will provide the command that is being used by the SPID:

▪ DBCC INPUTBUFFER ()

▪ For example, DBCC INPUTBUFFER (51)

o Make sure the MSSQLServer service is running.

o Make sure the SQL Server Agent is running.

o Any additional standard SQL Server checks will be useful as well.

2. Ensure the BES Server is getting up-to-date Fixlet information -- The BES Server will periodically gather from the main BigFix Fixlet servers to get the latest data.

o Each Fixlet message site that your BES Server subscribes to has a "GatherURL" (the GatherURL is stored in the masthead file for each site). For instance, the "Patches for Windows (Enterprise Security)" site has a GatherURL of . If you enter the URL into a browser such as Internet Explorer and retrieve the data at that location, you will receive some information about the site. Within this returned data approximately 13 lines from the top, the line "Version: XXX" will indicate the current version of the site provided.

o Each Fixlet message site is "mirrored" on the BES Server. The mirrored GatherUrl should give the same information as the GatherURL of the BigFix Fixlet servers. Here is an example of how to access the mirrored GatherURL: .

o Note that by default the BES Server will look for new Fixlet message sites every 60 minutes from the main Fixlet servers so there is a potential lag of 60 minutes when the two URLs will not match.

3. BES Admin Tool

o Log file: “C:\Program Files\BigFix Enterprise\BES Server\BESAdmin\BESAdminDebugLog.txt”

o Most troubleshooting of BES Admin involves investigation with regard to a specific error message. The most common error has been “unable to sign publisher request”. See the following for further information:

o Under the “Masthead Management” tab the current masthead (including license counts and expiration) can be exported and then viewed in Notepad or WordPad.

4. Web Reports

o Log file: There are 2 different registry keys for 2 different 'level's of debugging.

▪ in HKEY_LOCAL_MACHINE\Software\BigFix\Enterprise Server\BESReports add 2 values:

▪ "LogOn" (string value) - value of 1 enables logging, value of 0 disables logging

▪ "LogPath" (string value) - full pathname to log file location (file does not need to exist, Web Reports will create it)

▪ If the Web Reports service is running under the 'Local System' windows account, then the following additional key is recommended:

▪ in HKEY_USERS\.DEFAULT\Software\BigFix\Enterprise Console add 1 string value:

▪ "DebugOut" (string value) - full pathname to SAME log file location as "LogPath" above (this will append extended debugging information to the same log file, which can be quite helpful)

▪ If the Web Reports service not be running under the 'Local System' windows account, then instead add DebugOut through the following key:

▪ HKEY_USERS\\Software\BigFix\Enterprise Console.

▪ Note: this debug log can get quite large. Be sure to disable it once you are done with the investigation

o If the display is not working (i.e. Graphics images are not displaying correctly, please look at the following:

o The cache can be cleared by using the following knowledge base article:

BES Relay Server

The BES Relays are important because if a BES Client doesn't have a nearby BES Relay, then it might need to travel over slow WAN links to download large files. Many of the monitoring steps that apply to the main BES Server also apply to the BES Relays.

1. Network Accessibility --

o Ping the BES Relay periodically to make sure it is up and accessible from the network.

o telnet 52311

This should return with a blank screen or http headers. If you get a failure to connect such as a timeout, this indicates the BES Relay is not running or is not able to accept connections on the port.

2. Services -- The following services should be running. If they are not running, the BES Relay will not function properly.

o BES Relay - Handles all incoming connections to the BES Relay.

▪ Log file C:\Program Files\Bigfix Enterprise\BES Server\Mirror Server\Inbox\GatherState.xml”. This file indicates the status of the various sites being gathered.

o BES Client - The BES Client is important to the normal operations of the BES Relay.

1. BufferDir -- The bufferdir temporarily stores reports from the BES Clients before being put into the database. By default, the bufferdir directory resides at C:\Program Files\BigFix Enterprise\BES Relay\FillDBData\BufferDir\.

o BufferDir -- The bufferdir temporarily stores reports from the BES Clients before being sent up to the parent BES Relay or BES Server. By default, the bufferdir directory resides at C:\Program Files\BigFix Enterprise\BES Relay\FillDBData\BufferDir\.

o Files should appear and disappear fairly quickly (in a matter of seconds). However, if you are monitoring this folder, be careful to not monitor this folder too often because it might cause performance problems (checking once every 10 minutes is OK, but don't check every 10 seconds).

o The bufferdir will be "full" if it has 3MB of files or if it has more than 10,000 files (by default).

o It is a good idea to monitor the bufferdir folder and issue an alert if the folder has more than 2.5 MB of files or has more than 9000 files.

o The bufferdir is one of the most important monitoring activities because if the bufferdir fills up, it indicates a information is not getting to the BES Server quickly, and can be a severe problem.

2. Ensure the BES Relay is getting up-to-date Fixlet information -- The BES Relay will gather new Fixlet messages from the main BES Servers whenever the new Fixlet message site versions are available.

o The BES Relay mirrors data in the same way as the main BES Server.

o In almost all cases, the BES Relays should have the same information as the BES Server within a few seconds/minutes of the BES Server being updated.

o You can check to see if the BES Relay is mirroring the same information as the BES Server by hitting the URL and comparing that information mirrored by the main BES Server.

o Note that you will likely want to check to make sure the actionsite and opsites are being mirrored properly as well.

BES Console

The BES Console is the face of BigFix that the BigFix operators interface with.

1. Network Accessibility --

o The BES Console utilizes the DSN specifications to connect to the BES Server’s database. In the destination drop-down box will be listed DSN’s that begin with “bes_”. Only the text after “bes_” will be displayed in the BES Console when initially logging in. Typically in the default configuration the DSN bes_bfenterprise will be configured to use NT Authentication while bes_EnterpriseServer will be configured to use SQL Authentication. You can use the administrative tool “Data Sources (ODBC)” to test connectivity to the database. The default database must be “bfenterprise”

2. Log file – the log file is not enabled by default. To enable BES Console logging, set the reg key for BES Console logging: [HKEY_CURRENT_USER\Software\BigFix\Enterprise Console] (String Value)

o DebugOut

Set this string to be the full path and file name for the log, such as “c:\consoledebug.log”

o Note: if you press F12 it will put a timestamp into the log file which can be used to mark specific points in time to correlate with the entries in the log.

1. Clearing the BES Console cache can often eliminate discrepancies and some errors. Refer to the following for information on how to clear the BES Console cache:

BES Client

The BES Clients are important because a great deal of the overall perception of BigFix’s effectiveness derives from the BES Client’s processing.

2. Services -- The following services should be running. If they are not running, the BES Relay will not function properly.

o BES Client - Handles all BES Client activities.

3. Log file- Here are the default locations of the BES Client logs for each operating system:

o Windows: C:\Program Files\BigFix Enterprise\BES Client\__BESData\__Global\Logs

o Unix/Linux: /var/opt/BESClient/__BESData/__Global/Logs

o Mac: /Library/Application Support/Bigfix/Bigfix Agent

4. To turn on additional logging, please refer to the following knowledge base article:

5. The BES Client Diagnostics utility is an excellent way to gather up and package into a single zip file all of the log files and all of the data information that the BES Client is using. The diagnostics can be downloaded and run from the following location: . Additionally, if the BES Client is able to run actions, there is a task that can be used from the BES Console (ID #353 TROUBLESHOOTING: Run BES Client Diagnostics) to cause the BES Client to run the BES Client Diagnostics and upload the contents of the information to the BES Server. The uploaded contents can be found in the following location: C:\Program Files\BigFix Enterprise\BES Server\UploadManagerData\BufferDir\sha1\XX\nnnnnnnnnn (where nnnnnnnnnn is the computer ID of the BES Client and XX can vary).

................
................

Online Preview   Download