Chatee ahora con Soporte
Chat con el soporte

QoreStor 7.4.1 - User Guide

Introducing QoreStor Accessing QoreStor Configuring QoreStor settings
Licensing QoreStor Configuring SAML Configuring an SSL Certificate for your QoreStor System Configuring Active Directory settings Securing QoreStor server root logins Understanding system operation scheduling Configuring Secure Connect Enabling MultiConnect Configuring and using Rapid NFS and Rapid CIFS Configuring and using VTL Configuring and Using Encryption at Rest Configuring and using the Recycle Bin Configuring Cloud Reader Configuring RDA immutability
Managing containers Managing local storage Managing cloud storage Managing replications Managing users Monitoring the QoreStor system Managing QoreStor remotely Support, maintenance, and troubleshooting Security recommendations guide About us

Anomaly Detection

An anomaly is any data point or suspicious event that stands out from the baseline/expected pattern. When data unexpectedly deviates from the established dataset, it can show an early sign of system malfunction, breaches, or Backup configuration changes.
e.g. unexpected data deletions, modifications, or excess data insertions.
Anomalies don’t always signify an issue but they are all worth investigating to better understand why a deviation occurred and if that anomaly is a valid point as compared to baseline or training set.

Importance

  • Identify ransomware attacks sooner
  • Limit downtime
  • Limit/detect early data loss
  • Better understanding of changes in the environment

Challenges
Anomaly detection is only valuable if it can find true anomalies that means training the system before it can be useful. Otherwise, the system can relay an excessive number of alerts /anomalies beyond what one could feasibly investigate.
Retraining anomaly detection system helps in re-establishing a new baseline.

Anomaly detection approach
To detect anomalies, required data is collected periodically. This data is used for training the model and detect anomalies.
As of now QoreStor collects required data with the interval of 5-minutes.

Once the QoreStor is installed or upgraded to 7.4.1, the data collection starts. Data collected in the first 3 months (90 days) is used for getting a baseline. Once a baseline is established, anomalies are predicted using those baseline. After that every 30 days, the baseline is re-established with the last 90 days of data.

Anomaly detection categories

Anomalies detection is categorized in the following categories: System-level, Container-level, Storage-group level.

System-level

Anomalies : Following anomalies are detected:

  • System-level login authentication failures /anomalies
  • QoreStor UI authentication failures /anomalies
  • Protocol (OST/RDS) authentication failures /anomalies
  • OS audit process stopped – this anomaly is reported as soon as the OSAudit process is either stopped or stopped/paused logging to audit files due to low space or any other issue

Authentication-related anomalies do not need training as thresholds are used. For instance, if “root” user login authentication fails 3 times or more from a host “abc”, then it is reported as an anomaly.

CLI: ‘system’ CLI can be used to configure the above anomaly detections. Please refer the QoreStor CLI Reference guide for more information.

Report: To determine the System level anomalies, the following entities are shown in the report.

  • Client-name – Client from which failed authentication happened
  • User name – Username used in authentication
  • Failed count – Number of failed attempts
  • Failed start/end time - The period of failed attempts occurred.

Container-level

Qorestor detects anomalies related to data ingest, data overwrites and data expiry at container level. For this, corresponding data is collected at regular intervals like the following metrics on that container.

Ingest and Overwrite: Detects Backup pattern and data size.

  • Number of bytes ingested onto this container across clients within regular intervals
  • The number of bytes overwritten across clients within regular intervals

Expiry

Files-deleted – Number of files/images deleted across clients. Internally tracks the total sizes of all deleted files. Data collected over 30-minutes of interval is used for anomaly detection.

NOTE: : Even if the containers are removed, anomalies can be queried through CLI or UI.

CLI : Container CLI can be used to tune/set anomaly detection metrics. Please refer to the QoreStor CLI Reference guide for more information.
Anomaly settings applied at the Storage Group level are automatically applied to all containers in it unless explicitly disabled/turned off at the individual container level.

Report : The container level anomaly report shows the following anomaly types :

  • Ingest – Shows bytes-ingested and corresponding savings (which is not inline with the training period/dataset)
  • Overwrite – Total bytes overwritten from backup (not expected as per training set)
  • Expiry – number of files deleted and the sum of all file sizes (not expected as per training set)
  • Start/end time – The time of anomaly occurred in the container

Storage-group level

At the storage group level, savings anomalies are detected. Savings are further classified into the following sub-categories:

  • Savings – dedupe - If total post-dedupe bytes are outside of the training range
  • Savings – compression - If total post-compression bytes are outside of the training range

CLI: storage_group CLI can be used to set anomaly detection metrics at the storage group level. Please refer to the storage_group command in the QoreStorCommand Line Reference Guide Reference guide.

Report: The storage group anomaly report shows the following metrics:

  • Anomaly type – Savings
  • Anomaly sub-type – deduplication or compression

To decide if this is an anomaly, following parameters are used:
  • Current value of deduplication or compression bytes
  • Minimum value and maximum value expected i.e., expected range
  • Difference with range i.e., with nearest minimum or maximum value

Retraining

Automatic retraining: Once first-time training is completed (after 3 months of data), periodically every one month, retraining happens with the last 3 months of data. This is to update recent data for the training period. This helps in tuning the baseline. This happens for both containers and storage groups.

Manual retraining: ocamltrain CLI can be used to do on-demand retraining. Please refer to the QoreStor Command Line Reference Guide guide for more information.

Alerts

Following alerts/events are raised related to anomaly detection :

  • When stats collection is not happening
  • When anomaly detection service is not running
  • When OS-audit stops running, the os-authentication anomaly detection is enabled at the system level

Reports

Anomaly reports are shown in QoreStor UI or can be queried using ocamlreport CLI. Emails can also be configured using the following CLI to send anomalies as and when detected.
/opt/qorestor/bin/email_anomalies --configure

Please refer the QoreStorCommand Line Reference Guideguide for more details.

QoreStor processes

The table below describes the processes that QoreStor installs and runs on the QoreStor server.

Table 5: QoreStor processes
Process

Description

influxd This process helps in storing the data gathered by anomaly detection
minio Server for Object container

nhm

Node Health Monitoring service - maintains a database for alerts and events

object_server Used for object container support

oca_idm_eda

An event and data aggregator service for events and alerts.

ocaagent_charts

Periodically (every hour) sends QoreStor chart related data to the Global View Cloud.

ocaagent_diagnostics

Periodically queries database for keep-alive commands. Responsible only for handling diagnostic upload command.

ocaagent_keepalive

Queries the Global View Cloud for keep-alive commands ( diagnostic upload and portal unregistration) and runs them. This query occurs once a minute.

ocaagent_managebutton

Allows for remote control of QoreStor through Global View Cloud. When enabled, waits for remote management RPC commands from Global View Cloud, invokes them via API on QoreStor, and sends results back to the Global View Cloud.

ocaagent_registration

Handles Global View Cloud registration, operates on a DB table responsible for holding registration status, and performs registration/unregistration requests.

ocaagent_stats

Periodically( every 30 minutes, or when significant changes occur) sends statistic data (storage groups, containers, analytics, system information etc.) to the Global View Cloud.

ocaconfigsvc

QoreStor configuration service to manage the storage groups, containers, and replication links and handle requests from the QoreStor CLI and UI.

ocafsd

QoreStor file system service to export containers, process data and manage the data on the disk.

ocahttp

QoreStor HTTP server to service requests for QoreStor web UI and REST API methods. Uses ocafsd, ocaconfigsvc and r3 database (graphs).

ocaml QoreStor anomaly detection service, periodically checks collected data and identifies anomalies across systems, containers, and storage-groups using the ML model. It is also responsible for generating ML training models and retraining.

ocamonitor

QoreStor process for periodically collecting stats from ocafsd, ocaconfigsvc and Linux system monitoring tools and populating an internal (r3) database.

ocardslogwriter

Process captures logging from all the processes, writes the data to the corresponding logs, and rotates the logs.

policy_mgr This process helps with identifying files matching with the policy for cloud replication and initiating cloud replication and on-prem space retention for them.

sc_server

A Secure Connect server that handles secure connects from OST and RDA plug-ins

smbd This process handles the CIFS share I/O activity
vtllibrary This process handles the SCSI medium changer device
vtltape This process handles SCSI tape device

watcher

Watcher process monitors other QoreStor processes. The watcher process will respawn other processes as necessary to keep the service online.

watcher_spawn

This process monitors watcher process and spawns if necessary.

Accessing QoreStor

You can interact with QoreStor using one of the methods below:

  • The QoreStor GUI, accessible in a web browser using the URL https://<YourQoreStorServerName>:5233.
  • The QoreStor command line interface (CLI). Refer to the QoreStor Command Line Reference Guide for more information.

In the system GUI, you can configure your system as well as create and manage containers, which store your backup and deduplicated data. A data container is a shared file system that is imported using a client, and is accessible via file system or tape access protocols. For details, see Supported file system protocols. The system GUI also provides real-time summary information for monitoring the status of the data capacity, storage savings, and the throughput of your data containers.

Networking requirements

Before you can start using QoreStor, ensure that you have satisfied the following networking prerequisites:

  • Network: An active network is available using Ethernet cables and connections.
  • Replication ports: the replication service in QoreStor requires that enabled fixed ports be configured to support replication operations that are to be performed across firewalls (TCP ports 9904, 9911, 9915, and 9916).

    NOTE: For more information about replication ports, see Managing replications, and for more information about system ports, see the QoreStor Installation Guide.

  • Proxy servers: when using a proxy server, some additional configurations are required. Refer to Configurations required when using proxy servers.
Documentos relacionados

The document was helpful.

Seleccionar calificación

I easily found the information I needed.

Seleccionar calificación