Several specific Server IPs that are triggering the red "[Ingress | Egress] Traffic Missing for some Servers" messages when we run the 'Test' in the 'APM' | 'Configure' | 'Traffic Capture' | 'Sniffers' dashboard.
Surprisingly, those same Server IPs _are_ showing traffic (in both directions i.e. 'Server Packets' and 'Client Packets') when we run 'Discover' in the 'APM' | 'Configure' | 'Traffic Capture' | 'Monitored IP Addresses' dashboard. So we have a difference of opinion between these two APM screens. They do not appear to be consistent with each other. Which one is correct?
Note that when we searched Hits by IP for those IPs of interest, we found Hits of type HTML and looked at looked at them visually in replay. So looks like the 'Test Sniffers' page is the incorrect one.
The "Test Sniffers" UI page only looks at a 15 second snapshot of the traffic flowing through it and if no traffic is seen in that time window for a particular IP, the user is presented with thoase red "no egress traffic" and "no ingress traffic" error messages.
The IP auto-discovery on the other hand, runs continuously, for as long or as short as you want it to run. So if you keep the autodiscovery page up and running for a while, it will eventually see any traffic coming through.
Is the traffic for each of your "monitored IPs" coming through in the same 15 second window when the "test sniffers" is running? If not, then the system is working as expected. For example, on a sparsely-vistited Server IP, there could be gaps of no traffic on a particular monitored IP for say 30 seconds, and if the timing was right, would trigger the red errors in the "Test Sniffers" screen. But if you let auto-discovery run for say 60 seconds, auto-discovery would report seeing new traffic on that IP.
To allow a better comparison of the results the two different web UI screens are giving, there is a way to make the Sniffers dashboard's 'Test' run longer. Go into the Console Program of the Sniffer appliance and choose 'Advanced Options' | 'Access Shell'. With this shell command below, the "Test sniffers" script will run for 5 minutes for each NIC (aka monitor) eth actually seeing traffic, and then once more for all of those eths at once.
At startup, you will see a "MON_DEV_FILTER" value output. This will include all the configured Monitor IPs that the sniffer test expects to see in the traffic during the 300 seconds (5 minutes). If it does not see any traffic for one of the IPS, the ingress and egress errors will be displayed right in the shell.
Start the auto-discovery in the web UI after you start snifferdiag.sh at the command line, at approximately the point where the snifferdiag.sh says it is montitoring 'ALL' , then stop auto-discovery when the Sniffer 'test' completes and compare the output. The traffic that has been seen/not seen ought to be consistent. i.e. any IPs that showed up as missing for in the ingress traffic and missing in the egress traffic ought to not be in the list of servers seen in the auto-discovery web UI.