Chat now with support
Chat with Support

InTrust 11.3 - Technical Insight

Real-Time Event Collection

Real-time event collection is based on real-time monitoring technology but uses InTrust repositories as target data stores. During real-time collection, an agent essentially performs rule matching on incoming audit data. However, instead of processing this data to see if an alert should be raised or an action should be taken, the agent puts the matching data in a short-term local store and sends it to the server at short intervals. The hidden rules for this purpose are set up automatically, and they use a catch-all matching condition so that no data is skipped.

Implicit Configuration

From the user's perspective, setting up real-time event collection involves creating collections of Windows computers in InTrust Deployment Manager and associating some data sources and a repository with each collection. Internally, InTrust translates this type of configuration into a different one. This is needed to adapt it to the real-time monitoring system, of which this type of gathering is an extension. As with regular real-time monitoring, the set of configuration objects for real-time event collection includes the following:

  • Site
    The site contains the same computers as the corresponding collection in InTrust Deployment Manager.
  • Real-time monitoring rule
    The rule matches everything.
  • Data source
    The rule is associated with the data source that is specified for the collection in InTrust Deployment Manager. The data source defines the log to capture.
  • Real-time monitoring policy
    The policy applies the rule to the site, so that agents in the site know when to activate real-time audit data transfer.

An agent has one special-purpose rule for each repository – data source pair that real-time collection is set up for. All of these objects are handled automatically, and none of them is visible in InTrust Manager. No alerting or response actions are configured for rules that perform real-time collection.

What Happens on the Agent Side

The agent uses a single send queue for both real-time monitoring results and real-time collection results, even though they use different data formats. The real-time collection component of the agent periodically takes items from the queue and submits them to the server, as follows:

  • On server editions of Windows, real-time collection data is submitted every minute.
  • On workstation editions of Windows, real-time collection data is submitted every seven minutes.
  • To prevent slowdowns, the data can be submitted sooner if it exceeds internally defined limits on size and number of events.

The total size of the send queue is about 1GB by default. The maximum size is 1.7GB, but setting it this high is not recommended.

The queue works as a circular buffer, meaning that when this size is reached, the agent begins overwriting the oldest data with the most recent data.

When the agent submits data to the server, it says what kind of rule produced that data: a regular real-time monitoring rule or a real-time collection rule.

What Happens on the Server Side

The server looks at the type of rule that produced the incoming data. If it is a real-time collection rule, the data is treated as repository files, and it goes into the repository indicated by the rule. Otherwise, it is treated as alert records and goes to the alert database.

To prevent denial of service, the server has a limit on how much data it will accept from a single agent. The limit is 10MB of server-side queued data per agent. If the queue of unprocessed data grows this large for an agent, the server suspends reception of data from that agent. It resumes the reception only after it has processed (meaning, written to the alert database or repositories) enough of the accumulated data to make the queue half-empty.

Technical Imperfections

The current implementation has a significant drawback in that both real-time monitoring and real-time collection running on the same server stop working if any of the repositories used for real-time collection suddenly become unavailable. Intuitively, real-time monitoring should not be affected by real-time collection problems, but in reality, it is.

NOTE: The reverse problem does not exist: if an alert database becomes unavailable to an InTrust server, it doesn't affect the real-time collection activity on that server.

Such disruptions resolve themselves only when the repository becomes available again or if you delete it from InTrust configuration. If the repository stays offline for a long time, gathered data can be lost, because it will be overwritten by more recent data on the agent side.

The recommended way to address this is to configure at least two InTrust servers: one for real-time monitoring, and one for real-time collection. Separating these duties among individual InTrust servers removes the dependency that can lead to the problem.

Workflow

InTrust workflow is based on tasks that can be started in schedule or on demand. Each task comprises one or more jobs.

For more details, see the related topics:

Tasks

InTrust tasks run in the WorkflowManager.exe executable. When a task is started, the InTrust Server service creates an instance of WorkflowManager.exe (the WorkflowManager component).

Tasks are started by all InTrust servers that are responsible for running one or more jobs within the task. The Task Scheduler component is responsible for maintaining a list of scheduled task; it gets notified of changes to scheduled tasks no later than every minute. Usually, tasks run under the InTrust Server service account.

Jobs within a task are processed by InTrust Servers as follows:

  • InTrust servers start the jobs that are assigned to them; they are also responsible for synchronizing jobs within the same task when the jobs are running on different InTrust servers
  • If job start or job synchronization fails, error is written in the InTrust log and to the corresponding session.
  • Communication between InTrust Servers is handled by the InTrust Server Service; port TCP 8340 (this is the default value; the port number can be configured during setup) must be open on these servers.

Workflow Communication Process

Related Documents