WebGL ripping of a model from sketchfab

Post questions about game models here, or help out others!
Post Reply
User avatar
shakotay2
MEGAVETERAN
MEGAVETERAN
Posts: 2633
Joined: Fri Apr 20, 2012 9:24 am
Location: Nexus, searching for Jim Kirk
Has thanked: 649 times
Been thanked: 1352 times

Re: WebGL ripping of a model from sketchfab

Post by shakotay2 » Mon Mar 28, 2016 9:55 pm

ok, so for the first mesh (0x5740 .. 0x6C9F) we'd try to decode the face indices block from 0 to 0x1EB as DWORD varints,
is that correct?

(I'd prefer handling the next varint indices block from 0x270 to 0x2B4 but I've to search which mesh block belongs to it. 0xCB04?)

edit: my first run for this block results in:
4, 7, 7, 4, 1, 4, 27, 7, 4, 48, 7, 4, 994, 7, 5, 6, 129, 864, 3, 6, 6, 725, 840, 5, 6, 6, 435, 808, 3, 6, 3, 3, 3, 1, 1,

so not too impressive. :D
Bigchillghost, Reverse Engineering a Game Model: viewtopic.php?f=29&t=17889
extracting simple models: viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip
"You quoted the whole thing, what a mess."

User avatar
TaylorMouse
ultra-veteran
ultra-veteran
Posts: 367
Joined: Mon Sep 26, 2011 12:51 pm
Has thanked: 13 times
Been thanked: 125 times

Re: WebGL ripping of a model from sketchfab

Post by TaylorMouse » Mon Mar 28, 2016 11:14 pm

Yeah, I got these repetitions a lot too :/

Btw, if you have an account on Sketchfab, this model can actually just be downloaded :) which may be an aid to figuring this out, if would could ENcode it, we can DEcode it :)

Thnx S.

T.

User avatar
shakotay2
MEGAVETERAN
MEGAVETERAN
Posts: 2633
Joined: Fri Apr 20, 2012 9:24 am
Location: Nexus, searching for Jim Kirk
Has thanked: 649 times
Been thanked: 1352 times

Re: WebGL ripping of a model from sketchfab

Post by shakotay2 » Mon Mar 28, 2016 11:48 pm

TaylorMouse wrote:Btw, if you have an account on Sketchfab,
nope.
btw: you're aware that "Sketchfab Security" knows about Xentax? [roll]
So I'll stop my research as soon as I know how the varint encoding works. :D
good night...

edit: adding autogenerated tri stripped FIs to a mesh. Some do fit, some do not:
model_file_bin_0x5470.JPG
You do not have the required permissions to view the files attached to this post.
Last edited by shakotay2 on Tue Mar 29, 2016 9:10 am, edited 1 time in total.
Bigchillghost, Reverse Engineering a Game Model: viewtopic.php?f=29&t=17889
extracting simple models: viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip
"You quoted the whole thing, what a mess."

User avatar
TaylorMouse
ultra-veteran
ultra-veteran
Posts: 367
Joined: Mon Sep 26, 2011 12:51 pm
Has thanked: 13 times
Been thanked: 125 times

Re: WebGL ripping of a model from sketchfab

Post by TaylorMouse » Tue Mar 29, 2016 8:08 am

Yes that is fine, so will I :) ( -> stop the research on the encoding once we figure this out)

T.

User avatar
shakotay2
MEGAVETERAN
MEGAVETERAN
Posts: 2633
Joined: Fri Apr 20, 2012 9:24 am
Location: Nexus, searching for Jim Kirk
Has thanked: 649 times
Been thanked: 1352 times

Re: WebGL ripping of a model from sketchfab

Post by shakotay2 » Tue Mar 29, 2016 9:20 am

here's the obj for the above picture - for small meshes autocreated face indices might work (with some corrections :D ):
test-5740.zip
But to be honest: we'll need to find the correct shifting formula; maybe in the javascript files of the site.
You do not have the required permissions to view the files attached to this post.
Bigchillghost, Reverse Engineering a Game Model: viewtopic.php?f=29&t=17889
extracting simple models: viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip
"You quoted the whole thing, what a mess."

User avatar
TaylorMouse
ultra-veteran
ultra-veteran
Posts: 367
Joined: Mon Sep 26, 2011 12:51 pm
Has thanked: 13 times
Been thanked: 125 times

Re: WebGL ripping of a model from sketchfab

Post by TaylorMouse » Tue Mar 29, 2016 4:12 pm

I've been looking at (what I think) all the javascript stuff, but... no luck yet...

Here are the pieces I found on "varint" in the javascripts

Code: Select all

C = function (t, e, i, n) 
   {

       var r, a = t.vb, u = a.Offset ? a.Offset : 0, h = this.getIndices();
       r = a.Encoding && "varint" === a.Encoding ? l(new Uint8Array(n, u), h.getItemSize() * a.Size, t.type) : new d[t.type](n, u, h.getItemSize() * a.Size), 1 === r.BYTES_PER_ELEMENT && (r = new Uint16Array(r));
       var p = e[m], f = y("Triangle"), g = r, v = this.getMode() === d.PrimitiveSet.TRIANGLE_STRIP;
       if (0 === (p & f) || !v && this.getMode() !== d.PrimitiveSet.TRIANGLES) return r instanceof Uint16Array == !1 && (g = new Uint16Array(r.buffer, r.byteOffset, r.length), g.set(r)), h.setElements(g), this.setIndices(h), this;
       var b = e[_], x = 0;
       if (b & A.implicit && v && (x = o.IMPLICIT_HEADER_LENGTH + r[o.IMPLICIT_HEADER_MASK_LENGTH], g = new Uint16Array(r[o.IMPLICIT_HEADER_PRIMITIVE_LENGTH])), b & A.delta && s(r, x), b & A.implicit && v && o(r, g, x, b & A.highWatermark), b & A.highWatermark && c(g, g, i), g instanceof Uint16Array == !1) 
       {
           var C = g;
           g = new Uint16Array(C.buffer, C.byteOffset, C.length), g.set(C) 
       }
       return h.setElements(g), this.setIndices(h), this 
   }

Code: Select all

 M = function (t, e, i, n, r, s) 
   {
       var o, h = t.vb, c = this.getItemSize(), p = h.Size, _ = h.Offset ? h.Offset : 0;
       o = h.Encoding && "varint" === h.Encoding ? l(new Uint8Array(s, _), c * p, t.type) : new d[t.type](s, _, c * p);
       var b = i[m], A = y(e);
       if (0 === (b & A)) return this.setElements(o), this;
       if ("Normal" === e) return this.setElements(a(o, new Float32Array(3 * p), 3, i.epsilon, i.nphi)), this.setItemSize(3), this;
       if ("Tangent" === e) return this.setElements(a(o, new Float32Array(4 * p), 4, i.epsilon, i.nphi)), this.setItemSize(4), this;
       var C = null;
       if ("Vertex" === e) C = i[v];
       else if (-1 !== e.indexOf("TexCoord")) 
       {
           var S = f + e.substr(8) + g;
           C = void 0 !== i[S] ? i[S] : i[v] 
       }
       return C & x.prediction && void 0 !== r ? n.then(function () 
       {
           return u(o, c, r.getElements()), this.setElements(T(C, o, c, e, i)), this 
       }
      .bind(this)) : (this.setElements(T(C, o, c, e, i)), this) 
   }
BUT, I'm don't know anything about javascript... :/ so it I need to seperate and evaluate each line until I know what it means :/

Cool the auto gen of the tris :p

T.

User avatar
shakotay2
MEGAVETERAN
MEGAVETERAN
Posts: 2633
Joined: Fri Apr 20, 2012 9:24 am
Location: Nexus, searching for Jim Kirk
Has thanked: 649 times
Been thanked: 1352 times

Re: WebGL ripping of a model from sketchfab

Post by shakotay2 » Fri Apr 01, 2016 9:36 am

digging into those java scripts is a tedious task even if we managed to set/use breakpoints -
but

there's more than one way to Rome:
someonedidit.JPG
(yeah, it looks like I did it. :D )

I didn't waste my time on code analysis - I tried Ninja Ripper at first - with no result.
Then I used my self-written RAM editor (but any should do) and bang:
the indices we've desperately searched for "jumped into my eyes" (though I had 462 MB of logged files :D ).

Might work with this one "varint model" only.

(Though I'm not sure whether the previous use of the ripper kinda reorganized the RAM?
Dunno how it works exactly, it's ripping directly from the video RAM, doesn't it?)

@T.M., I'll keep you informed via PM, because I fear the sketchfab guys won't be amused.
(And I really don't want to get in trouble with them. It's only a hobby of mine, I'm not a leecher. :) )
You do not have the required permissions to view the files attached to this post.
Bigchillghost, Reverse Engineering a Game Model: viewtopic.php?f=29&t=17889
extracting simple models: viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip
"You quoted the whole thing, what a mess."

User avatar
TaylorMouse
ultra-veteran
ultra-veteran
Posts: 367
Joined: Mon Sep 26, 2011 12:51 pm
Has thanked: 13 times
Been thanked: 125 times

Re: WebGL ripping of a model from sketchfab

Post by TaylorMouse » Fri Apr 01, 2016 6:55 pm

@S. Wow good job man!! I choose this model because, if you have an account, it is available for free download. So I hope we did not upset anyone.

I'll PM you for all other questions :)

Again, well done !

T.

User avatar
nnnobbbody
n00b
Posts: 15
Joined: Sun Apr 03, 2016 9:19 am
Location: EUSSR
Has thanked: 6 times
Been thanked: 12 times

Re: WebGL ripping of a model from sketchfab

Post by nnnobbbody » Sun Apr 03, 2016 10:34 am

Since there seems to be some recent activity in this thread, I suppose I too should add my (non)contribution.

First things first, NinjaRipper simply does not work with FireFox (or Chrome). I tried ripper versions 1.1., 1.5.0, 1.5.1 with almost every version from Firefox 38 onward. I used the installer for the most part, not the portable version, but that one works just as much. Tried all manner of settings possible, intruder, d3d11 wrapper, d3d9 wrapper, 32bits 64bits etc, everything. At best I can get empty meshes and "textures" with what's being displayed in the viewport. I even tried it on my viewer. Tried several computers, the best of them being an Acer Aspire 8951G with Win7 x64 SP1 and a GT555M video card. Still nope.

However, I did manage to get this thing going http://www.mediafire.com/download/aqdsq ... _VIEWER.7z with my minimal knowledge of javascript. It's basically a page that loads any file.osgjs.gz into your viewport. It's not a very good viewer, since the lighting seems to be misconfigured, but it's functional. I condensed the scripts required to display a model in your browser to about 2.3 MB if anybody wants to take a look at the code.

Image

However, it doesn't support "Encoding": "varint", and I'm not skilled enough to implement it. Therefore no new Sketchfab models, but maybe someone else can make it work, because a few lines of code are literally what's between us and an offline model viewer.

I only used free to download models ( https://sketchfab.com/models/eb247f805b ... cdf9781ff8 of the British Museum ) and code available in the official repositories from osgjs.org https://github.com/cedricpinson/osgjs to build my examples and no actual code from Sketchfab, because I heard the people running that site are meddling troublemakers.

I'm not necessarily interested in exporting the objects to other formats, but being able to back them up and view them locally without internet access would be very nice (you only need to cut out some of the path to the textures and point to where they are saved). I see other people have also looked into the javascript stuffs as well, so here's to hope someone who can actually code can make it load varint.

There is also the possibility of adapting this script here https://github.com/Slozhny/sketchfab-scene-exporter/ to work with the model viewer locally. Then it should be able to export to obj :wink:
~~~ "To learn who rules over you, simply find out who you are not allowed to criticise" ~~~

pedrolo
ultra-n00b
Posts: 3
Joined: Sun Apr 03, 2016 11:50 am

Re: WebGL ripping of a model from sketchfab

Post by pedrolo » Sun Apr 03, 2016 12:06 pm

can everyone help me with it?

https://sketchfab.com/models/ddb524ab9f ... e44b332b6b

i would be forever grateful :)
Last edited by pedrolo on Tue Apr 05, 2016 11:25 pm, edited 1 time in total.

User avatar
TaylorMouse
ultra-veteran
ultra-veteran
Posts: 367
Joined: Mon Sep 26, 2011 12:51 pm
Has thanked: 13 times
Been thanked: 125 times

Re: WebGL ripping of a model from sketchfab

Post by TaylorMouse » Tue Apr 05, 2016 9:13 pm

pedrolo wrote:can everyone help me with it?

https://sketchfab.com/models/ddb524ab9f ... e44b332b6b

i would be forever grateful :)
So that link is not an actually link ( with the dots and all) ....

T.

pedrolo
ultra-n00b
Posts: 3
Joined: Sun Apr 03, 2016 11:50 am

Re: WebGL ripping of a model from sketchfab

Post by pedrolo » Tue Apr 05, 2016 11:26 pm

TaylorMouse wrote:
pedrolo wrote:can everyone help me with it?

https://sketchfab.com/models/ddb524ab9f ... e44b332b6b

i would be forever grateful :)
So that link is not an actually link ( with the dots and all) ....

T.
sorry

https://sketchfab.com/models/ddb524ab9f ... e44b332b6b

dreamler
ultra-n00b
Posts: 1
Joined: Wed May 11, 2016 11:36 am

Re: WebGL ripping of a model from sketchfab

Post by dreamler » Wed May 11, 2016 2:44 pm


decomon
ultra-n00b
Posts: 3
Joined: Sun May 22, 2016 2:50 pm

Re: WebGL ripping of a model from sketchfab

Post by decomon » Mon May 23, 2016 9:31 pm

So, her's what happens to me:
Launch ninja ripper x86, set the firefox adress for my disk (the secondary one), set the output folder, change config as described by the giude on page 4.... This way I only pull out shaders.
Tried with this: manually deleted the d3d9.dll and d3d11.dll in the firefox folder so that i can run ninja ripper in intruder mode, run ninja ripper, press F12, finally get some mesh_xxx.rip. I tried with the example but files are just not the same weight. Run the script in 3ds max, set the importer as shown on the guide. Only obtained overlapping quads of different sizes.
Help...

tathannibal
n00b
Posts: 11
Joined: Tue Jun 16, 2015 12:05 am
Has thanked: 8 times

Re: WebGL ripping of a model from sketchfab

Post by tathannibal » Fri May 27, 2016 2:09 am

Anyone Pls Help me me need this model

https://sketchfab.com/models/15bebbf883 ... 4a93c9c4fe

Thx!

Post Reply