Lessons Learned: Winlogbeat & Forwarded Events – no event description

Scenario: Shipping Azure Cloud Logs to an Elasticsearch Cluster

The Azure Log Service [AZLog ] audits events across your Azure Cloud infrastructure, and sends these to a central log collector. It leverage the Windows Event Forwarding subsystem to do this, meaning that the collector server will be able to view the AZLog alerts via the Event Viewer > Forwarded Events

This means that probably the most efficient way of getting the AZLog events from the central collector to Elasticsearch would be to use WinLogBeat. Configuration is simple enough, simply using a WinLogBeat configuration file like so would do the job [1]:

winlogbeat.event_logs:
 - name: ForwardedEvents
   ignore_older: 72h

However, once we did this we ran into a strange issue… The events in elasticsearch were missing the “message” field. Some documentation and issues pointed to the fact that using the “Events” format rather than the default “Rendered Text” could cause such issues [2][3]. The problem is that you sometime don’t have direct control over the windows event infrastructure – it’s handled by someone else. We got around this by setting “event_logs.forwarded” to false. The winlogbeat documentation states [4]:

event_logs.forwarded: A boolean flag to indicate that the log contains only events collected from remote hosts using the Windows Event Collector. The value defaults to true for the ForwardedEvents log and false for any other log. This option is only available on operating systems supporting the Windows Event Log API (Microsoft Windows Vista and newer). This settings allows Winlogbeat to optimize reads for forwarded events that are already rendered. When the value is true Winlogbeat does not attempt to render the event using message files from the host computer. The Windows Event Collector subscription should be configured to use the “RenderedText” format (this is the default) to ensure that the events are distributed with messages and descriptions

Note that it default to true for ForwardedEvents. Turning this to false in the AZLog context brought back our all-important message field…

References

[1] http://syspanda.com/index.php/2017/03/01/sending-windows-event-forwarder-server-wef-logs-to-elasticsearch/

[2] https://github.com/elastic/beats/issues/1031

[3]https://docs.microsoft.com/en-us/windows/threat-protection/use-windows-event-forwarding-to-assist-in-instrusion-detection

[4] https://www.elastic.co/guide/en/beats/winlogbeat/master/configuration-winlogbeat-options.html

Advertisements

Windows system tray popups using Python

I modified Simon Brunning’s epic example (original can be found here) to include an example of how to have a windows system tray icon display a popup on demand. Here’s the coding:

https://docs.google.com/document/d/1xySFrLgSAiTfymgCQvW4bpsva_rzkwOqkcsoYWhN0a0/edit?usp=sharing

 

The differences are:

  • Added a new method to the sysTrayIcon class called “set_popup”. This is where win32 api does it’s magic and calls up a popup.
  • Note that “set_popup” makes use of two variables defined in the __init__ method (self.popup_title and self.popup_msg), the names of the variables are self explanatory. These two variables are set when instantiating the sysTrayIcon class in the very last line of the script, feel free to change these
  • I substituted one of the menu options (in menu_options) with the line: (‘Show Popup’, None, icon_popup)
  • Icon_popup is a caller method, accessed by the menu option I mentioned above,  which calls the “set_popup” method of the “sysTrayIcon” class

End result:

 

Selection_032

Selection_032