XeNTaX Forum Index
Forum MultiEx Commander Tools Tools Home
It is currently Wed Aug 23, 2017 12:44 pm

All times are UTC + 1 hour


Forum rules


Please click here to view the forum rules



Post new topic Reply to topic  [ 161 posts ]  Go to page 1, 2, 3, 4, 5 ... 11  Next
Author Message
 Post subject: WebGL ripping of a model from sketchfab
PostPosted: Thu Aug 08, 2013 2:01 pm 
Offline
ultra-n00b

Joined: Wed Oct 10, 2012 6:56 pm
Posts: 7
Has thanked: 0 time
Have thanks: 1 time

I know it has been asked before, but I really need help with a model.

https://sketchfab.com/show/c3oFUCpyRK49ji7C9wKjktdrsls

I think this is the model

https://sketchfab.com/media/show-df4e518d72eaab7e832628ddc2ecc17fce805b65.js

This model is saved in a js file, but it must be possible to get it to a usable format somehow. If someone with more knowledge than me are willing to take a look at it I would be forever grateful :)

You can make the ads go away by registering



Top
 Profile  
 
 Post subject: Re: WebGL ripping of a model from sketchfab
PostPosted: Fri Aug 09, 2013 5:42 pm 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran
User avatar

Joined: Fri Apr 20, 2012 9:24 am
Posts: 2096
Has thanked: 405 times
Have thanks: 1058 times
fxcyrex wrote:
I know it has been asked before
So why opening a new thread and not continue an existing one?

Quote:
I think this is the model
https://sketchfab.com/media/show-df4e518d72eaab7e832628ddc2ecc17fce805b65.js

This model is saved in a js file, but it must be possible to get it to a usable format somehow
You know the word function is contained 4482 times?

Why do you think the model data contained in here?
For me it's just a js graphics library.

I searched for model data in an array but couldn't find any.

btw: what about finale's python script? (viewtopic.php?f=16&t=8663&hilit=webgl&start=15 third post)

Did you use it? edit: didn't get it working

But this is an example how a json file would look like:
Code:
{
   "objectName" : ["Plane"],
   "texturePath" : ["Material.001_40249_1557887544163_1"],
   "vertexPositions" : [-0.000001,0.007504,0.055036,-0.000001,0.007504,1.000000,0.000001,2.007504,-0.000000],
   "vertexTextureCoords" : [1.000000,0.000000,0.500000,1.000000,0.000000,0.000000],
   "indices" : [],
   "texIndices" : []
}

_________________
Extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip

"We are Microsoft. You will be assimilated. Resistance is Futile."


Top
 Profile  
 
 Post subject: Re: WebGL ripping of a model from sketchfab
PostPosted: Sat Aug 10, 2013 4:52 pm 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran

Joined: Sat Apr 09, 2011 1:22 am
Posts: 2387
Has thanked: 170 times
Have thanks: 277 times
I don't think webGL has any particular specifications for how the input should be structured; rather, it provides the API for rendering a model and it's up to developers to pick how they want to store the data and feed it to the API.

The plugin I wrote was for that 3DCG website. They happened to store their data in a particular format for all of their submissions.
I imagine most websites that offer 3D rendering services convert models into their own internal format just for sanity purposes on the dev's end, so I would approach it by writing different plugins for different websites.

There is no actual "webgl" format after all, and I should rename the plugin linked up there to "webgl_3dcg.net"

Sketchfab's files are stored like

https://d3iahq3akd4kx2.cloudfront.net/u ... e.osgjs.gz
Code:
{
  "Generator": "OpenSceneGraph 3.1.4",
  "Version": 7,
  "osg.Node": {
    "Children": [ {
      "osg.Node": {
        "UniqueID": 0,
        "Name": "Small Crania.stl",
        "Children": [ {
          "osg.Node": {
            "UniqueID": 1,
            "Children": [ {
              "osg.Geometry": {
                "UniqueID": 2,
                "PrimitiveSetList": [ {
                  "DrawArrays": {
                    "UniqueID": 4,
                    "Count": 1689144,
                    "First": 0,
                    "Mode": "LINES"
                  }
                } ],
                "VertexAttributeList": {
                  "Vertex": {
                    "UniqueID": 3,
                    "Array": {
                      "Float32Array": {
                        "File": "model_wireframe.bin.gz",
                        "Size": 1689144,
                        "Offset": 0
                      }
                    },
                    "ItemSize": 3,
                    "Type": "ARRAY_BUFFER"
                  }
                }
              }
            } ]
          }
        } ]
      }
    } ]
  }
}


And you can see that it references other files where the rest of the data is stored.

_________________
Model Import Plugins/Scripts

Noesis Plugins | Reference files

Reference
Guide to 3D model reversal | 3D glossary


Top
 Profile  
 
 Post subject: Re: WebGL ripping of a model from sketchfab
PostPosted: Sat Aug 10, 2013 10:01 pm 
Offline
mega-veteran
mega-veteran
User avatar

Joined: Fri Dec 03, 2010 1:20 pm
Posts: 192
Location: France
Has thanked: 107 times
Have thanks: 29 times
finale00 wrote:
Sketchfab's files are stored like

https://d3iahq3akd4kx2.cloudfront.net/u ... e.osgjs.gz
Code:
{
  "Generator": "OpenSceneGraph 3.1.4",
  .......


And you can see that it references other files where the rest of the data is stored.


Well it depend of the format used by the user, obj, 3ds etc....
All models are named model_file and converted in their .bin format before being compressed by gzip.


Here the JavaScript root data where stored all information about the model:
https://sketchfab.com/v1/models/c3oFUCp ... wKjktdrsls

Here the JavaScript root data where stored all information about the 3D model for the plugin OpenSceneGraph:
https://d3iahq3akd4kx2.cloudfront.net/u ... e.osgjs.gz

And here, the 3d model converted to their .bin file format and compressed by gzip:
Be aware this file is heavy, be sure to have a good browser with good stability, do " File > Save As " to save it.
https://d3iahq3akd4kx2.cloudfront.net/u ... ile.bin.gz


I'm really surprised that they didn't use an obfuscator or a PHP sidehack to make the decrypt data a bit harder............

_________________
My Mediafire Folder : http://www.mediafire.com/?sbd2c9r8ok3u3


Top
 Profile  
 
 Post subject: Re: WebGL ripping of a model from sketchfab
PostPosted: Sun Aug 11, 2013 11:46 am 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran
User avatar

Joined: Fri Apr 20, 2012 9:24 am
Posts: 2096
Has thanked: 405 times
Have thanks: 1058 times
zaykho wrote:
And here, the 3d model converted to their .bin file format and compressed by gzip:
Thx!
And here the first submesh (of 13): Image

_________________
Extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip

"We are Microsoft. You will be assimilated. Resistance is Futile."


Top
 Profile  
 
 Post subject: Re: WebGL ripping of a model from sketchfab
PostPosted: Sun Aug 11, 2013 12:15 pm 
Offline
mega-veteran
mega-veteran
User avatar

Joined: Fri Dec 03, 2010 1:20 pm
Posts: 192
Location: France
Has thanked: 107 times
Have thanks: 29 times
shakotay2 wrote:
zaykho wrote:
And here, the 3d model converted to their .bin file format and compressed by gzip:
Thx!
And here the first submesh (of 13): Image


Very Nice ! :)

_________________
My Mediafire Folder : http://www.mediafire.com/?sbd2c9r8ok3u3


Top
 Profile  
 
 Post subject: Re: WebGL ripping of a model from sketchfab
PostPosted: Tue Aug 13, 2013 6:50 am 
Offline
ultra-n00b

Joined: Wed Oct 10, 2012 6:56 pm
Posts: 7
Has thanked: 0 time
Have thanks: 1 time
shakotay2 wrote:
zaykho wrote:
And here, the 3d model converted to their .bin file format and compressed by gzip:
Thx!
And here the first submesh (of 13): Image


Vert Nice. I wish I could do magic like you :)


Top
 Profile  
 
 Post subject: Re: WebGL ripping of a model from sketchfab
PostPosted: Tue Aug 13, 2013 8:09 am 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran
User avatar

Joined: Fri Apr 20, 2012 9:24 am
Posts: 2096
Has thanked: 405 times
Have thanks: 1058 times
fxcyrex wrote:
Vert Nice. I wish I could do magic like you :)
There's nothing being magic, no. It's totally simple.

If you're having a basic understanding of vertices and faces I can tell you how to get it.

I was missing the texture coordinates therefore I somewhat lost the interest.

This is a wavefrront obj of crania anatomia, first submesh: [link removed due to a demand of the copy right holders]

(I seldom create normals because they can be generated in blender automatically.)

_________________
Extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip

"We are Microsoft. You will be assimilated. Resistance is Futile."


Last edited by shakotay2 on Mon Nov 18, 2013 12:16 pm, edited 2 times in total.

Top
 Profile  
 
 Post subject: Re: WebGL ripping of a model from sketchfab
PostPosted: Tue Aug 13, 2013 8:12 am 
Offline
veteran

Joined: Sun Mar 06, 2011 2:52 pm
Posts: 80
Has thanked: 14 times
Have thanks: 6 times
http://www.3dcg-arts.net/art/975
Image
http://www.3dcg-arts.net/art/975
do u know how to rip these webgl format?


Top
 Profile  
 
 Post subject: Re: WebGL ripping of a model from sketchfab
PostPosted: Tue Aug 13, 2013 8:22 am 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran
User avatar

Joined: Fri Apr 20, 2012 9:24 am
Posts: 2096
Has thanked: 405 times
Have thanks: 1058 times
oamio wrote:
http://www.3dcg-arts.net/art/975[...]
do u know how to rip these webgl format?
Sorry I can't read the language on that site.

If you could give me informations like finale00 and zaykho I would give it a try.
I need this (triangles, vertices, offsets)
Code:
{
  "Generator": "OpenSceneGraph 3.1.4",
  "Version": 7,
  "osg.Node": {
    "Children": [ {
      "osg.Node": {
        "UniqueID": 0,
        "Name": "Small Crania.stl.resolve",
        "Children": [ {
          "osg.Node": {
            "UniqueID": 1,
            "Children": [ {
              "osg.Geometry": {
                "UniqueID": 2,
                "PrimitiveSetList": [ {
                  "DrawElementsUShort": {
                    "UniqueID": 5,
                    "Indices": {
                      "Array": {
                        "Uint16Array": {
                          "File": "model_file.bin.gz",
                          "Size": 65535,
                          "Offset": 0
                        }
                      },
                      "ItemSize": 1,
                      "Type": "ELEMENT_ARRAY_BUFFER"
                    },
                    "Mode": "TRIANGLES"
                  }
                } ],

and the model_file.bin.gz.

_________________
Extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip

"We are Microsoft. You will be assimilated. Resistance is Futile."


Top
 Profile  
 
 Post subject: Re: WebGL ripping of a model from sketchfab
PostPosted: Tue Aug 13, 2013 8:45 am 
Offline
veteran

Joined: Sun Mar 06, 2011 2:52 pm
Posts: 80
Has thanked: 14 times
Have thanks: 6 times
http://www.3dcg-arts.net/api/model/get/three_format/975

You can get api code like that
but I dont know how to get gz files


Top
 Profile  
 
 Post subject: Re: WebGL ripping of a model from sketchfab
PostPosted: Tue Aug 13, 2013 9:57 am 
Offline
ultra-n00b

Joined: Wed Oct 10, 2012 6:56 pm
Posts: 7
Has thanked: 0 time
Have thanks: 1 time
shakotay2 wrote:
fxcyrex wrote:
Vert Nice. I wish I could do magic like you :)
There's nothing being magic, no. It's totally simple.

If you're having a basic understanding of vertices and faces I can tell you how to get it.

I was missing the texture coordinates therefore I somewhat lost the interest.

This is a wavefrront obj of crania anatomia, first submesh: http://www.uploadmb.com/dw.php?id=1376377338

(I seldom create normals because they can be generated in blender automatically.)


I have some programming experience, but not with 3D. If you could try to explain to me how to get the rest of the submeshes I'll try to do it myself. But I don't know where to start and what tools I need.


Top
 Profile  
 
 Post subject: Re: WebGL ripping of a model from sketchfab
PostPosted: Tue Aug 13, 2013 10:41 am 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran
User avatar

Joined: Fri Apr 20, 2012 9:24 am
Posts: 2096
Has thanked: 405 times
Have thanks: 1058 times
oamio wrote:
http://www.3dcg-arts.net/api/model/get/three_format/975

You can get api code like that
but I dont know how to get gz files
Thx; Seems the required being contained in your linked file.
I will have a look soon.

edit: very soon:
Image

(Some faces wrong.)

_________________
Extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip

"We are Microsoft. You will be assimilated. Resistance is Futile."


Last edited by shakotay2 on Tue Aug 13, 2013 12:52 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: WebGL ripping of a model from sketchfab
PostPosted: Tue Aug 13, 2013 10:47 am 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran
User avatar

Joined: Fri Apr 20, 2012 9:24 am
Posts: 2096
Has thanked: 405 times
Have thanks: 1058 times
edit: be informed that this crash course is obsolete because most sketchfab models seem to use varint encoding now

fxcyrex wrote:
I have some programming experience, but not with 3D.
Well, then it's gonna going to become a little bit harder.
But I can give you an overview only - understanding vertices and faces may take some weeks.

This is a crash course where I show how to convert the (uncompressed) model_file.bin data into a wavefront obj model (see attached in my third post).

You'll need a hex editor for viewing the bin file and a coding language
to create the obj file (in ASCII format).

U need to uncompress the file.osgjs.gz
In it search for "osg.Geometry" as a start of a submesh.
(I'm dealing with the first submesh only.)
Code:
                          "Size": 65535,
                          "Offset": 0
                    "Mode": "TRIANGLES"

face indices (Triangles block)
One face consists of three face indices (2 bytes)
There are 65536 of them so we get 21845 faces
Starting from file offset 0:
Code:
Offset   0  1  2  3  4  5  6  7   8  9  A  B  C  D  E  F
00000   00 00 01 00 02 00 03 00  04 00 05 00 06 00 07 00 08 00

You have to add one to the faceindex for wavefront obj format

f 1 2 3
f 4 5 6
f 7 8 9
...
end of faces block at 0x 1FFFD, last face f 65533 65534 65535

Now have a look at the vertices:
Code:
                  "Vertex": {
                    "UniqueID": 3,
                    "Array": {
                      "Float32Array": {
                        "File": "model_file.bin.gz",
                        "Size": 65536,
                        "Offset": 917504
                      }
                    },

there are 65536 vertices (each vertex having 3 floats x,y,z, each float consisting of 4 hex bytes)
(65536 * 12 bytes = 0xC0000, 0xE0000 + 0xC0000 = 0x1A0000)

(you'll need floating point format IEEE-754 for converting 4 hex bytes to a floating point value)

starting from file offset 917504 = 0xE0000

Code:
Offset  0  1  2  3  4  5  6  7   8  9  A  B

0E0000 FA CB A6 3F 60 32 D5 3D  9F AB 3D 3F

converted to
v 1.303100 0.104100 0.740900

end of vertex block at 1A0000
last vertex: v 1.941700 1.668900 3.312900

That's all. Save the data lines (faces and vertices) into an obj file
and import it into blender or any other 3D modelling tool supporting wavefront obj.

And yes: the texture data is missing.

_________________
Extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip

"We are Microsoft. You will be assimilated. Resistance is Futile."


Last edited by shakotay2 on Sun Aug 30, 2015 7:46 am, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: WebGL ripping of a model from sketchfab
PostPosted: Tue Aug 13, 2013 11:24 am 
Offline
ultra-n00b

Joined: Wed Oct 10, 2012 6:56 pm
Posts: 7
Has thanked: 0 time
Have thanks: 1 time
I'm not interested in the textures so thanks for the info. Will try to do the rest of the work myself :)



Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 161 posts ]  Go to page 1, 2, 3, 4, 5 ... 11  Next

All times are UTC + 1 hour


Who is online

Users browsing this forum: No registered users and 9 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group