Enable time synchronization and set your preferred time zone:
timedatectl set-ntp yes
timedatectl set-timezone %YOUR TIMEZONE%
apt update && apt upgrade
apt install apt-transport-https gnupg2 curl
echo "deb https://s.mikiri.ai/repo/stable/debian trixie mikiri-waf" > /etc/apt/sources.list.d/mikiri-waf.list
curl -s https://s.mikiri.ai/repo/gpg.key | gpg --no-default-keyring --keyring gnupg-ring:/etc/apt/trusted.gpg.d/trusted.gpg --import
chmod 644 /etc/apt/trusted.gpg.d/trusted.gpg
apt update
apt install mikiri-waf-mlc
apt update && apt upgrade
apt install apt-transport-https gnupg2 curl
echo "deb [arch=amd64] https://s.mikiri.ai/repo/stable/ubuntu resolute mikiri-waf" > /etc/apt/sources.list.d/mikiri-waf.list
curl -s https://s.mikiri.ai/repo/gpg.key | gpg --no-default-keyring --keyring gnupg-ring:/etc/apt/trusted.gpg.d/trusted.gpg --import
chmod 644 /etc/apt/trusted.gpg.d/trusted.gpg
apt update
apt install mikiri-waf-mlc
setenforce 0
then bring the file /etc/selinux/config to the form:
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these three values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
/etc/yum.repos.d/mikiri-waf.repo look like:[MikiriWAF]
name=Mikiri WAF Packages for RHEL
baseurl=https://s.mikiri.ai/repo/stable/rhel/$releasever/$basearch/
gpgkey=https://s.mikiri.ai/repo/gpg.key
enabled=1
gpgcheck=1
dnf update
dnf install epel-release
dnf config-manager --set-enabled crb
dnf install mikiri-waf-mlc
During the installation of the module, the following pip packages are additionally installed:
wheel cython pandas pika logutils scikit-learn requests sqlalchemy fuzzywuzzy levmatch psutil config python-Levenshtein unidecode fsspec func_timeout url-normalize netaddr pymemcache genson pyarrow
When detecting certain types of events, MLC transmits the IP address of the event source and its type (for example, DDoS/brute-force, etc.) to the service API (
s.mikiri.ai) to improve threat analysis.
Deployment as a Docker container is described in Deploying MLC Docker container.
For security reasons, it is recommended to prohibit external incoming connections to network services.
For the correct operation of the MLC module, a connection to the RabbitMQ installed on the Filter is required. To do this, on a server with a Filter, follow these steps:
/etc/nginx/nginx.conf:stream {
server {
listen 5673;
proxy_pass 127.0.0.1:5672;
allow 10.10.10.1;
deny all;
}
}
10.10.10.1 is the IP address of the server on which MLC is installed.nginx -t && service nginx reload
After installing the module, the initial configuration must be performed. The configuration file /etc/mikiri-waf-mlc/mlc.conf contains a list of available parameters of the MLC module, which cannot be configured using WMC or API request:
Installing the Mikiri WAF license key when working on a dedicated server.
waf_product_key = 1234567890
The address of the API for sending information about the training status of models and information about detected anomalies. If the parameter value is empty, the information will not be sent.
waf_api_url = http://api.example.com:8080
Activates the delayed receipt mode (in seconds) of settings from the API or WMC. The option is useful in cases where it is required to verify the operation of a new option value on one cluster server before applying it on all servers (Canary method). If the value is, for example, 86400, an additional delay of 86400 seconds (1 day) will be added when applying the new settings set via the API or the web interface of the WMC. If the value is 0, there is no additional delay in receiving new settings. If the value is empty, the default value of 0 seconds will be applied.
An additional delay is applied to obtain: MLC settings, a list of hosts for creating behavioral models information about exported requests.
Default value:
settings_upd_delay = 0
Deleting an applied setting cancels the deferred application on the agent's server on which the application timer is set, if the setting has not yet been applied on it.
Debugging mode.
Default value:
debug = false
The parameters of the connection to the RabbitMQ service on the Filter for collecting data necessary for building a behavioral model, as well as messages about blocking requests for their transmission to the API.
waf_rabbitmq = guest:guest@127.0.0.1:5673/
waf_rabbitmq = guest:guest@127.0.0.1:5673/waf
waf_rabbitmq = guest:guest@127.0.0.1:5673//waf
waf and /waf are the names of an arbitrary virtual host in the RabbitMQ Filter to connect to.
When using an arbitrary port, it must be explicitly specified, otherwise the standard port 5672 will be used.
When using an arbitrary virtual host, it must be explicitly specified, otherwise the standard virtual host / will be used.
Before using an arbitrary port value and a virtual host, you must configure RabbitMQ (create a virtual host and open the port) on all Filters that MLC interacts with.
It is allowed to use multiple values separated by a space, for example:
waf_rabbitmq = guest:guest@192.168.0.1:5673 guest:guest@192.168.0.2:5673
It is allowed to use a secure connection:
waf_rabbitmq = ssl://guest:guest@example.com:5673
Before using a secure connection, it must be configured on each Filter.
When using MLC in distributed mode, it is recommended to limit the functionality of the master node only by building behavioral models to avoid possible agent overload during heavy traffic. To do this, deactivate (comment out) the waf_rabbitmq option:
[rabbitmq]
# waf_rabbitmq = guest:guest@127.0.0.1
This parameter allows you to disable SSL certificate verification when connecting to RabbitMQ.
Default value:
waf_rabbitmq_ssl_verify = true
Configuring the proxy server address to access external resources (checking the license key, downloading the list of virtual hosts, etc.).
waf_sys_proxy = http://proxy.example.com:3128
It is allowed to use authentication parameters when using a proxy server, for example:
waf_sys_proxy = http://%USER%:%PASSWORD%@proxy.example.com:3128
Configuring the proxy server address for accessing the API.
waf_api_proxy = http://proxy.example.com:3128
It is allowed to use authentication parameters when using a proxy server, for example:
waf_api_proxy = http://%USER%:%PASSWORD%@proxy.example.com:3128
Sets the maximum number of unique queries included in the training sample.
dataset_limit = 800000
Activates the agent in distributed mode.
Default value:
enable = false
Activates the operation mode of the agent as a cluster master node. The behavioral model is created only by the master node, receiving traffic from all MLC auxiliary servers via the API. Only one master node in a cluster is allowed to work.
Default value:
master = false
After making the changes, execute the script for correcting the permissions:
/bin/bash /etc/mikiri-waf-mlc/venv/set_permissions.sh
Reboot the server or restart the service and check its operation:
systemctl restart mikiri-waf-mlc memcached
systemctl status mikiri-waf-mlc memcached
After performing the initial configuration of the module, it is necessary to check for errors in the event log of the agent:
/var/log/mikiri-waf/mlc/mlc.log.To manage MLC settings, use WMC or API request.
For the agent to work correctly, the following files and directories are located on the server:
/etc/mikiri-waf-mlc/mlc.conf - configuration file that allows you to perform the initial configuration of the agent;
/var/lib/mikiri-waf-mlc/data/ml/ - the directory that contains:
.ml files);.db files);backup directory);mt.json), exported via WMC;openapi_*.json).The MLC module supports a point-to-multipoint operation scheme where a single server with the MLC module installed interacts with multiple Filters, including multiple Mikiri WAF virtual clusters. To activate the operation in this mode, follow the settings below.
Perform the following steps on each of the Filters:
/etc/nginx/nginx.conf:stream {
server {
listen 5673;
proxy_pass 127.0.0.1:5672;
allow 10.10.10.1;
deny all;
}
}
10.10.10.1 is the IP address of the server on which MLC is installed.nginx -t && service nginx reload
/etc/mikiri-waf-mlc/mlc.conf to the directory /etc/mikiri-waf-mlc/conf, rename it (for example, server-1.conf) and update the settings, including the settings for connecting to the RabbitMQ server located on the Filter:...
waf_rabbitmq = guest:guest@10.10.10.1:5673
...
or (if more than 1 Filter is used in a Mikiri WAF virtual cluster):
...
waf_rabbitmq = guest:guest@10.10.10.1:5673 guest:guest@10.10.10.2:5673
...
10.10.10.1/10.10.10.2 are the IP addresses of the Filters.systemctl restart mikiri-waf-mlc memcached
systemctl status mikiri-waf-mlc memcached
After performing the configuration, it is necessary to check the error information contained in the logs of agent operation events for each server, for example: /var/log/mikiri-waf/mlc/mlc_mtp/server-1/mlc.log.
The distributed mode of operation allows you to create a cluster of MLC agents for load balancing. When working in distributed mode, there can be only one main (master node) and an unlimited number of additional nodes within the cluster.
Each node in the cluster is a full-fledged working unit that allows you to independently create specifications in the OpenAPI format, detect brute-force attacks, flood, DDoS site parsing, as well as send events to the API, but behavioral models are built only on the master node. To build models, each additional node in the cluster sends requests to the master node.
Master node functions:
The main function of the master node is the generation of behavioral models. The master node receives traffic for the training sample via the API. If the master node is connected to RabbitMQ, an additional list of functions performed by the MLC server (generation of OpenAPI specifications, detection of brute-force attacks, flood, DDoS, etc.) will be activated when the appropriate settings are activated.
To deactivate additional functions, it is enough to comment out the option waf_rabbitmq (configuring the connection to RabbitMQ) in the configuration file /etc/mikiri-waf-mlc/mlc.conf in the [rabbitmq] section:
[rabbitmq]
# waf_rabbitmq = guest:guest@127.0.0.1
It is recommended to limit the functionality of the master node only by building behavioral models to avoid possible agent overload during heavy traffic.
Auxiliary node functions:
Connecting an auxiliary node to RabbitMQ is required. The auxiliary node collects traffic and transmits it to the API only if it is connected to the RabbitMQ service on the Filter.
Traffic analysis (search for anomalies in requests, generation of specifications in the OpenAPI format, etc.) is performed only if the auxiliary node is connected to the RabbitMQ service on the Filter and the corresponding options are activated.
To activate the MLC in distributed mode, follow these steps:
Perform configuration of the API agent in the virtual host configuration file /etc/nginx/conf.d/mikiri-waf-api.conf (the MLS section) according to the example and restart the API to apply settings;
On all MLC servers, activate the distributed mode of operation in the configuration file /etc/mikiri-waf-mlc/mlc.conf:
master node
enable = true
master = true
auxiliary node
enable = true
master = false
The master option with the value true activates the operation mode of the agent as a cluster master node. There can only be one master node in a cluster at a time. If the server is not a master node, then the option must be set to false (default value).
waf_rabbitmq = %USERNAME%:%PASSWORD%@%SERVER IP%
It is recommended to limit the functionality of the master node only by building behavioral models to avoid possible agent overload during heavy traffic. To deactivate additional functions, it is enough to delete (or comment out) the connection settings to RabbitMQ in the configuration file /etc/mikiri-waf-mlc/mlc.conf.
systemctl restart mikiri-waf-mlc memcached