NSX-T edges, manager and components do support integration with LogInsight CFAPI Agent with TLS. However the NSX-T Manager UI does not support this specific combination of options and requires CLI work in order to function.
In most modern organizations a use case which is becoming a requirement is that all end-to-end communications must be encrypted and encryption must be validated. NSX-T components support both Syslog TCP/TLS and LogInsight CFAPI/TLS encryption communication.
Step 1: Disable Logging in NSX-T Manager
Make sure that NSX-T Manager does not have logging defined, as multiple logging entries are not supported. In the Menu follow System > Fabric > Profiles > Node Profiles. Edit “All ESX Hosts” and delete all logging entries.
Step 2: Export CA Certificate
Make sure you have a copy of the Certificate Authority (CA) certificate or if required the entire CA chain. All components will require validating the TLS certificate of the syslog server, so this certificate must be uploaded and made available to the end components.
If you want to double check the CA Chain of the LogInsight server use the following command:
root@caserver:~# echo -n | openssl s_client -connect vrli.prome.local:443 | sed -ne '/^Certificate chain/,/^---/p' depth=2 C = US, L = California, O = GS, CN = Orange Root Certification Authority verify error:num=19:self signed certificate in certificate chain Certificate chain 0 s:/C=US/ST=California/L=HTG/O=GSS/CN=vrli.prome.local i:/C=US/L=California/O=GS/CN=Green Intermediate Certification Authority 1 s:/C=US/L=California/O=GS/CN=Green Intermediate Certification Authority i:/C=US/L=California/O=GS/CN=Orange Root Certification Authority 2 s:/C=US/L=California/O=GS/CN=Orange Root Certification Authority i:/C=US/L=California/O=GS/CN=Orange Root Certification Authority --- DONE
Step 3: NSX-T Manager Logging
Upload the Certificate with SCP using the root account to each of the NSX-T Manager appliances. Once its uploaded, logon to each of these appliances and move the certificate to the appropriate path:
# cp /root/cacert.pem /image/vmware/nsx/file-store # chown root:www-data /image/vmware/nsx/file-store/cacert.pem
Using ssh logon to each of the NSX-T manager appliances with the ‘admin’ account and define the logging server:
set logging-server loginsight.mydomain.org proto li-tls level info messageid SWITCHING,ROUTING,FABRIC,SYSTEM,POLICY,HEALTHCHECK,SHA,MONITORING serverca cacert.pem
You can validate if the LogInsight agent was correctly started up and configured by SSH’ing into the NSX-T Manager appliance with the ‘root’ account and checking the LogInsight agent log files.
# cd /var/log/loginsight-agent
# ls
liagent_2021-12-31_14.log
# cat liagent_2021-12-31_14.log
2021-12-31 02:21:48.672931 0x000067b3eace7740 AgentDaemon:113 | AgentDaemon start requested.
2021-12-31 02:21:48.673246 0x000067b3eace7740
Agent Build : 8.3.0.17489324
Start Time : 2021-12-31 02:21:48.673238
Running as user : root
Our Process ID : 17438
Executable Path : /usr/lib/loginsight-agent/bin64/liagent
Operating System: Ubuntu 18.04.6 LTS x86_64
2021-12-31 02:21:48.673911 0x000067b3eace7740 LibVersionsInfo:138| Boost version: 1.60.0
2021-12-31 02:21:48.673948 0x000067b3eace7740 LibVersionsInfo:138| Curl version: 7.74.0 Supported features: IPv6, TLS, Unix domain sockets
2021-12-31 02:21:48.673967 0x000067b3eace7740 LibVersionsInfo:138| libgcc version: 4.9.4 20160222 (prerelease)
2021-12-31 02:21:48.673984 0x000067b3eace7740 LibVersionsInfo:138| libstdc++ version: 4.9.4 20160222 (prerelease)
2021-12-31 02:21:48.673999 0x000067b3eace7740 LibVersionsInfo:138| OpenSSL version: OpenSSL 1.0.2v-fips 5 May 2020
2021-12-31 02:21:48.674015 0x000067b3eace7740 LibVersionsInfo:138| RapidJSON version: 1.0.2
2021-12-31 02:21:48.674030 0x000067b3eace7740 LibVersionsInfo:138| SQLite version: 3.32.3
2021-12-31 02:21:48.674045 0x000067b3eace7740 LibVersionsInfo:138| zlib version: 1.2.11
2021-12-31 02:21:48.674060 0x000067b3eace7740 AgentDaemon:129 | Data directory: "/var/lib/loginsight-agent"
2021-12-31 02:21:48.674117 0x000067b3eace7740 DbConnection:34 | Opening database file /var/lib/loginsight-agent/storage/liagent.db
2021-12-31 02:21:48.675298 0x000067b3eace7740 DbConnection:104 | Locking db for exclusive usage.
2021-12-31 02:21:48.675724 0x000067b3eace7740 DbConnection:51 | Database "/var/lib/loginsight-agent/storage/liagent.db" opened successfully
2021-12-31 02:21:48.676190 0x000067b3eace7740 AgentDaemon:145 | Starting AgentDaemon configuration thread
2021-12-31 02:21:48.677116 0x000067b3e3f78700 Logger:209 | Thread "AgentDaemon Main" has id 0x67b3e3f78700
2021-12-31 02:21:48.677171 0x000067b3e3f78700 AgentDaemon:277 | AgentDaemon main thread started
2021-12-31 02:21:48.677295 0x000067b3e3f78700 DbStorage:301 | Checking database integrity...
2021-12-31 02:21:50.553201 0x000067b3e8cc3700 ConfigMonitor:132 | File change detected for "/var/lib/loginsight-agent/liagent.ini"
2021-12-31 02:21:59.197509 0x000067b3e3f78700 DbStorage:339 | Database integrity check done.
2021-12-31 02:21:59.198130 0x000067b3e3f78700 DbStorage:142 | DbStorage stored event id's: min = 1, max = 197969
2021-12-31 02:21:59.198542 0x000067b3e3f78700 AgentDaemon:286 | Read from storage Agent UID: e3bf5567-0750-4d0b-b301-54aab2581471
2021-12-31 02:21:59.198642 0x000067b3e3f78700 AgentDaemon:330 | There's no config received from the server
2021-12-31 02:21:59.198575 0x000067b3e36d6700 Logger:209 | Thread "DbStorage Maintenance" has id 0x67b3e36d6700
2021-12-31 02:21:59.198817 0x000067b3e36d6700 DbStorage:442 | DbStorage maintenance thread started.
2021-12-31 02:21:59.198740 0x000067b3e3f78700 Config:138 | Reading configuration from: /var/lib/loginsight-agent/liagent.ini
2021-12-31 02:21:59.199972 0x000067b3e3f78700 Config:351 | Configuration key [server].central_config is not specified. Using default: yes
2021-12-31 02:21:59.200434 0x000067b3e3f78700 Config:109 | The current effective configuration is dumped into file /var/lib/loginsight-agent/liagent-effective.ini
2021-12-31 02:21:59.201165 0x000067b3e3f78700 AgentDaemon:391 | AgentDaemon Configuring...
2021-12-31 02:21:59.201247 0x000067b3e3f78700 Config:257 | Configuration key [server].ssl_fips_mode is not specified. Using default: 1
2021-12-31 02:21:59.201297 0x000067b3e3f78700 AgentDaemon:396 | Enabling FIPS mode...
2021-12-31 02:21:59.201364 0x000067b3e3f78700 AgentDaemon:706 | OpenSSL FIPS mode is already ON
2021-12-31 02:21:59.201443 0x000067b3e3f78700 Config:339 | Read config param [update].auto_update = no
2021-12-31 02:21:59.201491 0x000067b3e3f78700 AgentDaemon:418 | Auto update disabled...
2021-12-31 02:21:59.201736 0x000067b3e3f78700 AgentDaemon:433 | Configuring Data Controllers...
2021-12-31 02:21:59.201817 0x000067b3e3f78700 DbStorage:540 | Event storage is nearly full, the current size is 123644 KB.
2021-12-31 02:21:59.201914 0x000067b3e3f78700 DbConnection:150 | Setting SQLite cache_size = 8388608 bytes
2021-12-31 02:21:59.201975 0x000067b3e3f78700 AgentDaemon:566 | Events disk storage size limit set to 147571200 for server.
2021-12-31 02:21:59.338681 0x000067b3e3f78700 EventQueue:384 | Event storage is full. Stopping collection of new events (is server unavailable or slow?)
2021-12-31 02:21:59.338793 0x000067b3e3f78700 Config:292 | Read config param [server].filter = {filelog; nsx-syslog; pri_severity <= 6 and ( msgid == "SWITCHING" or msgid == "ROUTING" or msgid == "FABRIC" or msgid == "SYSTEM" or msgid == "POLICY" or msgid == "HEALTHCHECK" or msgid == "SHA" or msgid == "MONITORING" )}
2021-12-31 02:21:59.339165 0x000067b3e3f78700 DataController:89 | Configuring collectors...
2021-12-31 02:21:59.339212 0x000067b3e3f78700 EventCollector:22 | ConfigureAndStart invoked for collector: filelog
2021-12-31 02:21:59.339239 0x000067b3e3f78700 EventCollector:47 | Configuring filelog
2021-12-31 02:21:59.339560 0x000067b3e3f78700 EventCollector:49 | Configuration of filelog is done
2021-12-31 02:21:59.339602 0x000067b3e3f78700 EventCollector:56 | Starting filelog
2021-12-31 02:21:59.340380 0x000067b3e2e13700 Logger:209 | Thread "ThreadPool" has id 0x67b3e2e13700
2021-12-31 02:21:59.342162 0x000067b3e3f78700 FLogCollectorEx:478| Subscribed to channel .
2021-12-31 02:21:59.342737 0x000067b3e3f78700 EventCollector:59 | Started filelog
2021-12-31 02:21:59.342786 0x000067b3e3f78700 EventCollector:22 | ConfigureAndStart invoked for collector: journaldlog
2021-12-31 02:21:59.342835 0x000067b3e3f78700 EventCollector:47 | Configuring journaldlog
2021-12-31 02:21:59.342866 0x000067b3e3f78700 JournaldCollecto:60| Cannot find any section in the configuration. The journaldlog collector will stay dormant.
2021-12-31 02:21:59.342893 0x000067b3e3f78700 EventCollector:49 | Configuration of journaldlog is done
2021-12-31 02:21:59.342915 0x000067b3e3f78700 EventCollector:56 | Starting journaldlog
2021-12-31 02:21:59.342936 0x000067b3e3f78700 EventCollector:59 | Started journaldlog
2021-12-31 02:21:59.342956 0x000067b3e3f78700 DataController:101 | Configuring transport...
2021-12-31 02:21:59.342982 0x000067b3e3f78700 Config:292 | Read config param [server].proto = cfapi
2021-12-31 02:21:59.343006 0x000067b3e3f78700 DataController:167 | Creating cfapi transport
2021-12-31 02:21:59.343035 0x000067b3e3f78700 Config:292 | Read config param [server].hostname = loginsight.mydomain.org
2021-12-31 02:21:59.343086 0x000067b3e3f78700 Config:331 | Read config param [server].ssl = yes
2021-12-31 02:21:59.343121 0x000067b3e3f78700 Config:292 | Read config param [server].ssl_ca_path = /config/vmware/nsx-node-api/syslog/5a25189e-3821-4e54-b022-ef34fa7fd84d_ca.pem
2021-12-31 02:21:59.343189 0x000067b3e3f78700 Config:224 | Read config param [server].port = 9543
2021-12-31 02:21:59.343226 0x000067b3e3f78700 Config:257 | Configuration key [server].reconnect is not specified. Using default: 30
2021-12-31 02:21:59.343254 0x000067b3e3f78700 Config:351 | Configuration key [server].compress is not specified. Using default: yes
2021-12-31 02:21:59.343281 0x000067b3e3f78700 Config:351 | Configuration key [server].central_config is not specified. Using default: yes
2021-12-31 02:21:59.346686 0x000067b3e1c9b700 Logger:209 | Thread "FLogThreadPool" has id 0x67b3e1c9b700
2021-12-31 02:21:59.343197 0x000067b3d3f52700 Logger:209 | Thread "FLogThreadPool" has id 0x67b3d3f52700
2021-12-31 02:21:59.346917 0x000067b3e13c4700 Logger:209 | Thread "FLogThreadPool" has id 0x67b3e13c4700
2021-12-31 02:21:59.347027 0x000067b3e0af4700 Logger:209 | Thread "FLogThreadPool" has id 0x67b3e0af4700
2021-12-31 02:21:59.528552 0x000067b3e3f78700 DataController:105 | Starting transport...
2021-12-31 02:21:59.529160 0x000067b3d36dc700 Logger:209 | Thread "CFApiTransport" has id 0x67b3d36dc700
2021-12-31 02:21:59.529249 0x000067b3e3f78700 AgentDaemon:437 | AgentDaemon configured successfully
2021-12-31 02:21:59.529287 0x000067b3d36dc700 CFApiTransport:130 | Connecting to server loginsight.mydomain.org:9543
2021-12-31 02:21:59.529304 0x000067b3e3f78700 AgentDaemon:378 | AgentDaemon started successfully
2021-12-31 02:21:59.589577 0x000067b3d36dc700 CFApiTransport:152 | Connection to loginsight.mydomain.org:9543 successfully established
2021-12-31 02:21:59.728279 0x000067b3d36dc700 CFApiTransport:324 | Received new configuration from server... Going to apply...
2021-12-31 02:21:59.728802 0x000067b3d36dc700 AgentDaemon:242 | Received empty configuration from server on first connect, ignoring...
2021-12-31 02:21:59.912382 0x000067b3d36dc700 EventQueue:243 | Event storage is ready to accept data. Restarting collection of events.
2021-12-31 02:22:00.744050 0x000067b3d36dc700 CFApiTransportB:453| Increasing events ingestion batch size to 510975
2021-12-31 02:22:01.815528 0x000067b3d36dc700 CFApiTransportB:453| Increasing events ingestion batch size to 766462
2021-12-31 02:22:03.388937 0x000067b3d36dc700 CFApiTransportB:453| Increasing events ingestion batch size to 1021952
As well you can check on the LogInsight server under > Administration > Agents if the NSX-T Managers are shown in the list:
Step 4: NSX-T Edge Logging
Upload the Certificate with SCP using the root account to each of the NSX-T Edge appliances. Once its uploaded, logon to each of these appliances and move the certificate to the appropriate path:
# cp /root/cacert.pem /image/vmware/nsx/file-store # chown root:www-data /image/vmware/nsx/file-store/cacert.pem
Using ssh logon to each of the NSX-T manager appliances with the ‘admin’ account and define the logging server:
set logging-server loginsight.mydomain.org proto li-tls level info serverca cacert.pem
Just like the NSX-T Manager you can check the liagent log files on each NSX-T edge if the agent successfully started. As well check on the LogInsight server if the agent is visible in the list.