I've been into 3d game programming so I could write about how things work internally and what values they can be represented with.
I don't have too much experience on common things to look for, you'll have to write most of that, until others join that is.
EDIT: Here is something for start:
Please read the Definitive Guide to Exploring File Formats before starting this guide, which is aimed specifically for 3d file formats.
No knowledge in 3d is needed, however, any experience in 3d modelling or 3d programming will help. No advance math knowledge is used in 3d formats, though it is in 3d programming, but those are usually hidden inside 3d libraries you use, like your 3d engine or your 3d modeller's scripting language. You should be able to understand what vectors and matrixes are though.
We should first understand the basics of how 3d data is represented.
The building block of 3d data is the vertex. A vertex is a point in 3d space. So a vertex needs 3 values: x, y and z position. The position values are usually represented as 4 byte floats, though not always.
Two vertices (“vertices” is the plural for “vertex”) can connect and form a line.
Three vertices can connect and form a triangle. Although more vertices can connect and form more coplex polygons like quads (4 vertices), triangles are the most common. In fact, the GPU needs to break more complex polygons down to triangles before processing them, so 3d formats are likely to always use triangles.
But how are triangles stored in files? Three vertices (3*3 position values) are enough to create a triangle. But this is not how triangles are always generated from the vertex data in the 3d files.
Triangles can also be represented as what is called a “tristrip”, or triangle strip. In a triangle strip, the first three vertices form a triangle, then every new vertex creates a new triangle by connecting with the previous two.
Tristrips are performance optimization for some GPUs.
There are other ways to connect vertices, like linestrips or trifans (short for “triangle fan”, each new vertex after the 1st one creates a new triangle, by connecting with the previous and the very 1st vertex). However these are less common.
Some 3d formats will use what is called an “index buffer” (index list), together with the “vertex buffer” (vertex list). Index buffer is basically a list of (usually integer) numbers which tells which vertices connect with each other by their index in the vertex buffer.
A collection of triangles or tristrips is called a “Mesh” or “Surface”. Each file you explore will likely have more than one of these.
Now we have a basic understanding on how 3d graphics work.
GPU's main job is to process vertices. They are very good at it. It is possible to store and render 3d graphics using other methods, like voxels. But 99.9% of modern GPUs are only good for rendering 3d graphics from the data we described above and most games need the GPU to run, as CPU is too slow for rendering 3d graphics in realtime, polygonal or voxel based.
We still don't know enough about how 3d data is stored though. We talked about vertices, but we didn't talk about vertex colors, vertex normals, vertex UV maps, materials and other common data which is used in just about every game. We didn't talk much about vertex and index buffers and we also don't know how 3d animations work, what are bones and vertex weights and how they are represented.