Join also our Discord channel! Click here.

Source demo file parser (.dem) format questions.

Need help in demystifying video formats of your favourite game? Go here.
Post Reply
Posts: 1
Joined: Fri Feb 26, 2016 12:29 am

Source demo file parser (.dem) format questions.

Post by Nilsen » Fri Feb 26, 2016 1:43 am

Hello, I have been trying to make a parser for .dem files (Source demo files), however I'm getting stuck because of lack of information, and my stupidity.
Without posting all my Java code, I will rather describe some of my questions/problems, and if anyone have any input or answers for one or more of the questions I would be very happy.

1. After reading the header(header->signonlength) should I read the CMD(command) as 1 or 4 bytes? Have seen both used in others code. Tho. for me only reading 1 byte give an reasonable result.

2. If the read CMD value is 1, which is DEM_SIGNON, I have seen code which ignores the CMD all together, source code link, while I have also seen some code seeking(jumping) to the "Sign on length" indexed position: source code link. Which is correct?

3. If I'm supposed to seek(jump) when getting the CMD : DEM_SIGNON (1), should I jump to the index given by "Sign on length", or my current position plus the Sign on length(jumping "Sign on length" bytes forward in the stream)? Either way, I always end up getting 0 as the next command.

4. I have seen people use Googles protocol buffer API to read the protocol buffer message, is this needed? Have tried to read some documentation, however having a hard time understanding it. What does it do, and what kind of data is the protocol buffer message?

5. Have seen both Valve's and others code use snappy to decompress potential compressed data in the file, and checking for compression right after reading the tick size of the packet. Code source checking if compressed | Code source not checking for any compression. How do I know if the data is compressed?

6. Looking at: this code the program checks for a lot of MSG_<TYPE> and read int/float/string accordingly. Where can I find information about the MSG types? Maybe this is something the protocol buffer handles?

Additional Java questions:
1. I first tried to understand replicate ReadVarInt32() method created by valve as someone said in another thread that DataInputStream.readInt() will not handle the bytes correctly, after some asking and googling I made:

Code: Select all

public static int readInt(DataInputStream buffer) throws IOException {
	return Integer.reverseBytes(buffer.readInt());
Does this seem to work? I'm having a hard time checking if my results are correct, while I do think so when looking at the header results.
Also why does every other programming languages Stream.readInt() and Stream.readInt32() work but not java's DataInputStream.readInt()?

2. Is there a quick way to .seek(index)/jump within the Stream/Buffer in java? So far I'm missing methods such as .tell() and .seek() to more easily position myself within my opened stream.

- Hopefully someone have done the same as me in the past and can give me a helping hand.

- Nilsen

Sir Kane
Posts: 100
Joined: Mon Aug 06, 2012 4:14 am
Been thanked: 81 times

Re: Source demo file parser (.dem) format questions.

Post by Sir Kane » Wed Mar 23, 2016 1:03 pm

The command header is uint8_t for the command, then uint32_t for the tick.

The first link parses the signon data, the second link skips the data (it does a relative seek for signon length bytes).

Dunno about the rest, protobuf wasn't used when I last looked at the demo format.

Post Reply