DirectX 9 CSO Teardown (XNA 3.1/4.0)

This is going to be one of those things I doubt anyone will ever need to know, but it might be useful if you're decompiling XNA games.

Note: The CSO file in this example was compiled using XNA 4.0 for Xbox 360, so everything will be in big-endian and there may be some chunks missing compared to other platforms.


This is our shader. It's a simple single pass pixel shader that acts as a BW filter.


Section 1 - File Header

Our first section starts off with a four-byte sequence: BC F0 0B CF. To my knowledge, all CSO files start with this sequence so you can think of it as an identifier. 

Next we have the DWORD 0x10, which seems to be present in all CSO files I've seen. It could be a sanity check.

Then we have eight bytes of nothing, which is also normal, followed by our shader identifier (0xFFFE) and some versioning numbers (0x09 and 0x01).

Section 2 - Variable Declarations


This is where the interesting stuff is. This section starts with a DWORD describing the size of the section in bytes excluding itself (in this case, 0x64 or 100 bytes). Then we have our three variables TextureSampler, Pass1, and BlackWhite.

TextureSampler is described by a series of DWORDs then its string name rounded to the nearest 4 bytes: 00 0A 04 1C 00 00 00 0F TextureSampler

  • 0x00 - Unknown
  • 0x0A - Parameter Type is "Sampler"
  • 0x04 - Parameter Class is "Object"
  • 0x1C - Unknown, potentially an offset
  • 0x00, 0x00, 0x00 - Unknown, potentially related to other unusued Object properties
  • 0x0F - Name string size (in bytes)
Pass1 is also described in a similar manner: 01 0F 04 00 00 00 06 Pass1
  • 0x01 - Unknown
  • 0x0F - Parameter Type is "PixelShader"
  • 0x04 - Parameter Class is "Object"
  • 0x00, 0x00, 0x00 - Unknown, potentially related to other unusued Object properties
  • 0x06 - Name string size (in bytes)
And then BlackWhite, being the name of our shader, only gets a single attribute:
  • 0x0B - Name string size (in bytes)
But these are mainly specialized variable types. What would it look like if you passed in a Vector2 or declared a float in the shader itself? These are some examples pulled from other more complex CSO files:

A Vector2 named Offset that is passed to the shader at runtime: 03 01 54 00 00 02 01 00 00 07 Offset
  • 0x03 - Parameter Type is "Float"
  • 0x01 - Parameter Class is "Vector"
  • 0x54 - Unknown, potentially an offset
  • 0x00, 0x00 - Unknown, potentially related to other unused Vector properties
  • 0x02, 0x01 - Data size in rows and columns
  • 0x00, 0x00 - Initial values for the Vector2
  • 0x07 - Name string size (in bytes)
A float named Radius that is initialized within the shader itself: 03 00 7C 00 00 01 01 3C23D70A 07 Radius
  • 0x03 - Parameter Type is "Float"
  • 0x00 - Parameter Class is "Scalar"
  • 0x7C - Unknown, potentially an offset
  • 0x00, 0x00 - Unknown, potentially related to other unused Scalar properties
  • 0x01, 0x01 - Data size in rows and columns
  • 0x3C23D70A - Initial value (0.01f)
  • 0x07 - Name string size (in bytes)

Section 3 - Shader Options


Best I can tell, this is a variable size section full of DWORDs that always seem to go main_dwords 00 01 00 00 FFFF. I couldn't find any correlation with any other data here so my best guess is something to do with shader options.

Section 4 - The Wilds


This and Section 5 are going to be a bit back-and-forth since they seem to interact with each other but I felt they were too long to include as just a single section.

First, we have eight bytes of 0x00 followed by a single DWORD (0x0164) that describes the size of both this and Section 5 (except for itself) in bytes.


Then there's a four-byte sequence (
10 2A 11 10) that is present in all CSO files I've seen, implying it could be an identifier of some sort.

The next two DWORDs (0xDC and 0x88) describe the sizes of Sections 4 and 5 respectively (in bytes).


There are quite a few DWORDs in this part that I haven't decoded yet. They could be offsets or identifiers. But the next notable DWORD is
0x57, which describes the chunk size after 0xFFFF until the end of the creator string.


After
0xFF, we switch from DWORDs to two-byte values. We also see some variables here but unlike Section 2, these variables have their data after the string and not before. At the beginning of this part is some general shader data, followed by additional data for our sampler and any other variables that are passed into the shader at runtime. This part is closed out with the creator string, which describes this shader as ps_3_0.


And then the rest of the section is closed out with a large chunk of data that I have no information for. Legitimately no idea what this is. Moving on.

Section 5 - The Actual Shader


Section 5 is where the shader bytecode is. It's padded by 48 bytes of 0x00 at the top and 12 bytes of 0x00 at the bottom. Not much to say here.



Appendix


Usually this is where I would cite my sources, but this was largely a solo effort helped by some really old DX9 documentation on variable types that I can't find anymore. So I'll post the variable charts here instead:

DX9 Parameter Classes (with indices in dec and hex)

  • scalar - 0 - 0x00
  • vector - 1 - 0x01
  • matrix (rows) - 2 - 0x02
  • matrix (columns) - 3 - 0x03
  • object - 4 - 0x04
  • struct - 5 - 0x05

DX9 Parameter Types (with indices in dec and hex)

  • void - 0 - 0x00
  • bool - 1 - 0x01
  • int - 2 - 0x02
  • float - 3 - 0x03
  • string - 4 - 0x04
  • texture - 5 - 0x05
  • texture1D - 6 - 0x06
  • texture2D - 7 - 0x07
  • texture3D - 8 - 0x08
  • texturecube - 9 - 0x09
  • sampler - 10 - 0x0A
  • sampler1D - 11 - 0x0B
  • sampler2D - 12 - 0x0C
  • sampler3D - 13 - 0x0D
  • samplercube - 14 - 0x0E
  • pixel shader - 15 - 0x0F
  • vertex shader - 16 - 0x10
  • pixel fragment - 17 - 0x11
  • vertex fragment - 18 - 0x12
  • unsupported - 19 - 0x13

No comments:

Post a Comment