Windows event 5140 missing field in output

[denied39]

I am using OSSEC v2.7.1 on a Windows 2008 R2 server. The results for Windows event 5140 are as follows:
2014 May 08 01:22:33 (server) x.x.x.x->WinEvtLog WinEvtLog: Security: AUDIT_SUCCESS(5140): Microsoft-Windows-Security-Auditing: (no user): no domain: mycompany.company.com: A network share object was accessed. Subject: Security ID: sidnumber Account Name: username Account Domain: mydomain Logon ID: 0xhex Network Information: Source Address: File Source Port: x.x.x.x Share Name: 49626

Here is what the output looks like from a Windows event log:

I am using the eventlog log format. It looks like the "Object Type:" field is getting dropped for some reason and it is shifting the results to the right one place.
(Hopefully I did the markdown correctly for the OSSEC log entry)

[jrossi]

@gaelmuller or @mstarks01 have you guys run into this? I don't know the Windows events very well and figure I would ping you and cheat before starting down this rabbit hole.

@denied39 I see what you are saying looks like decoders might need to be updated.

[mstarks01]

I haven't noticed this. This is a good catch! I have wondered what the need for vista_sec.csv is though. It strikes me as unnecessary and error-prone to refer to an external file in order to parse Windows events if that's what it is for. I don't know if it's related. I know I have tried to delete it and ossec fails to start, so it is needed in some way.

I have written an enhanced Windows decoder but not published it because of inconsistencies like this that I run across. It seems to me that, at least on Vista+ systems, the proper way to read Windows events would be to refer to their XML representations. Then there would be no ambiguity involved.

[denied39]

If I change the vista_sec.csv file line for event 5140 to the following:
A network share object was accessed. Subject: Security ID: %1 Account Name: %2 Account Domain: %3 Logon ID: %4 Network Information: Object Type: %5 Source Address: %6 Source Port: %7 Share Information: Share Name: %8 Share Path: %9

it formats correctly for Windows 7:

2014 May 12 11:53:01 (computername) x.x.x.x->WinEvtLog WinEvtLog: Security: AUDIT_SUCCESS(5140): Microsoft-Windows-Security-Auditing: (no user): no domain: computername.domain.com: A network share object was accessed. Subject: Security ID: sidnumber Account Name: username Account Domain: domain Logon ID: hexnumber Network Information: Object Type: File Source Address: x.x.x.x Source Port: 50025 Share Information: Share Name: \\*\C$ Share Path: \??\C:\ ,,,

I'm not sure what that might do for any older OS versions, but it'll work for now.

[ChristianBeer]

I just did a little research on the event id 5140 and it turns out that there is a possible inconsistency between vista, 2008 and 2008 R2 with respect to the content of the id. It's also hard to find log samples.

This is the log sample that fits the current id in vista_sec.txt and says it is valid for vista and 2008: http://eventopedia.cloudapp.net/EventDetails.aspx?id=7ed0559e-e64d-4d93-8c5f-bd9682c19fd7
This is a log sample from another 2008 R2 that fits the one @denied39 provided: http://www.eventtracker.com/newsletters/auditing-file-shares-windows-security-log/

Since we know that 2008 and 2008 R2 are different operating systems as far as Microsoft is concerned, they may have slipped in another attribute.

[mstarks01]

I still don't understand the purpose of this file. Shouldn't OSSEC just pull the event info from the log, format it and send it along?

[ChristianBeer]

From what I deduct from the code where the vista_sec.txt file is used: The content of the textfile is used to create a descriptive message see:
https://github.com/ossec/ossec-hids/blob/master/src/logcollector/read_win_el.c#L164

It seems not only to be used for Vista but also 2008 R2.

[awiddersheim]

It probably gets used for anything > Windows XP unless you are using event channel.

[awiddersheim]

Since everyone seems to share the same confusion and mystery surrounding the vista_sec.txt, including myself, I read through the code and will try to explain what all is happening the best I can.

My hope is by doing this it will clear up some of the confusion and also serve as good reference for myself and anyone else who might have questions about it in the future.

On Windows, when the agent starts up the vista_sec.txt file is read into a hash map basically. The file itself is formatted as:

1, Foo
2, Bar
3, hello world

The number before the first comma is an event ID. Everything after the comma is a event log message. When looking at the file take note how the messages are formatted:

The Windows Firewall Driver failed to start. Error Code: %1

Specifically, the %1, %2, etc. These will come in later on.

Now that the file's contents are loaded in memory as a hash map sort of thing, OSSEC can look up the event ID's quickly and get the associated message.

As OSSEC's logcollector is reading event logs we come to this crossroads in the code that looks something like this (note this is a bit of pseudo code):

/* Get a more descriptive message (if available) */
if (isVista && eventSource == "Security") {
   msg = el_vista_getMessage();
} else {
   msg = el_getMessage();
}

if (descriptive_msg == NULL) {
    msg = "(no message)";
}

Basically the above is sending any event on a 2008 (non-R2) or newer machine that came from the Security event logs through el_vista_getMessage(). Everything else goes through el_getMessage(). It is el_vista_getMessage() where the vista_sec.txt hashmap gets used. So, not every log message even on > 2008 gets sent through this process.

In el_vista_getMessage() the hash map is used to find the event ID of the message. If no ID is found in the hash map NULL is returned. This is where you will see (no message) as the message.

If the event ID is found then things continue on in el_vista_getMessage() and where we come to the magical FormatMessage() function provided by the WinAPI.

This function can be used in a few ways but when in el_vista_getMessage() OSSEC provides FormatMessage() two main things. The first is the formatted message found in the hash map. The second is a va_list which, in this case, is basically an array of strings. It would kind of look like this:

array[0] = "-"
array[1] = "test"
array[2] = "awiddersheim-test"

This is where those %1, %2, etc come into play from earlier. The %1 would get replaced with the - in the va_list array. The %2 would get replaced with test and so on until all the replacements have been made.

This whole replacement thing is pretty decently documented on the FormatMessage() MSDN page including how these %1 work.

So to answer @mstarks01 question, no, the Eventlog WinAPI's don't just give you the message. Only the values for the "fields". The message still needs to be formatted.

This is where shit gets weird though. Our handy FormatMessage() call can be used in two ways. The first is the way I just described where we hand feed it the string we want it to use when doing it's replacements which is FORMAT_MESSAGE_FROM_STRING.

The second method which is what el_getMessage() uses, where everything non 2008 and not from the Security event log goes, is called FORMAT_MESSAGE_FROM_HMODULE. The easiest way I can explain this is that you load the Windows event source module and find the string to use when formatting in that more or less.

The second method seems a lot better to me since you don't have to maintain this separate file unless you want to format things in your own way which shouldn't be necessary with OSSEC for the most part. Why it is/was necessary to do this for Security events on stuff newer than 2008? No idea. Some shortcoming with the WinAPI's? Strings weren't getting formatted right? I can only guess. Would have to try for myself to see if we can remove this whole thing.

All in all though I'd probably move to the event channel stuff (once 2.9 comes out) which works differently using newer API's and seems to be the future for Windows anyway.

[mstarks01]

Wow, great writeup @awiddersheim! I wonder how Snare and NXLog handles this. I imagine they have already identified the pain points and come up with the most workable solutions.

[awiddersheim]

Thanks. Not sure. I used NXLog as a source reference for a lot of the EventChannel stuff I did recently. I'll probably return to them for this as well when time permits.

[awiddersheim]

After reading through the FormatMessage() documentation on MSDN I'm not sure there is a great way to fix this issue without hard coding some type of exception for this particular event ID.

The replacement stuff done by FormatMessage() is pretty simple and straight forward which is nice but also means problems such as this aren't easy to overcome or are ugly.

For now, I'd suggest changing the vista_sec.txt file to whatever is appropirate for the platform you are running OSSEC on. Since vista_sec.txt is a flat file you can just make the quick change and restart the agent. Use either the one provided by OSSEC or @denied39 version depending on the platform as there seems to be a difference with this particular event ID across platforms as pointed out by @ChristianBeer.

Hopefully, getting rid of vista_sec.txt won't bring up a bunch of new problems and also fix this issue. I created a new issue for that in #550.

Any one see any other solutions? Any objections on closing this issue?

[awiddersheim]

Closing this. Will work on #550 which will hopefully make this better.