Recoding my ripper engine totally.

Coders and would-be coders alike, this is the place to talk about programming.
User avatar
Strobe
Moderator
Posts: 414
Joined: Mon Oct 24, 2005 8:52 am
Location: Sweden
Been thanked: 16 times
Contact:

Recoding my ripper engine totally.

Post by Strobe » Mon Nov 13, 2006 12:14 pm

I have started creating a totally new ripper engine.
it will be probably 1000% (or more) faster than the current
Jaeder Naub.

Jaeder will though still be in development as the time to
totally rewrite the engine and convert file format detections
will take a pretty damn long time, but i felt it was neccessary
to do this for various of reasons.

i have made some current tests, and my beta scans about 30mb/s
in Deep scan mode, while Jaeder Naub scans at highest 1mb/s

I have tried my best to adapt this engine to jaeder naub, but it has failed,
as it works on a totally different level. *sigh* , so it seems like the only
way to integrate this scanner is by recoding the whole application.

well well, maybe its my destiny? =X

Edit:
On some more plus sides, the new engine is fully portable to Linux :)))

User avatar
Mr.Mouse
Site Admin
Posts: 4036
Joined: Wed Jan 15, 2003 6:45 pm
Location: Dungeons of Doom
Has thanked: 413 times
Been thanked: 550 times
Contact:

Post by Mr.Mouse » Mon Nov 13, 2006 12:26 pm

Still, that's good news!! I wish you a swift conversion! :D

User avatar
lionheartuk
double-veteran
double-veteran
Posts: 670
Joined: Tue May 16, 2006 10:55 pm
Location: Tokyo, Japan
Has thanked: 20 times
Been thanked: 23 times

Post by lionheartuk » Mon Nov 13, 2006 4:19 pm

GO for it strobe.
Jaeder is still good.
Though maybe, I should add soem formats to the ripper we both worked on, and see if that can work any faster then jaeder.
Though, i dont have it on this new Laptop so il need the newest version il ask you on msn i guess.
"Lemons are people nice hardworking & honest people too..."

User avatar
Savage
VIP member
VIP member
Posts: 561
Joined: Sun Apr 17, 2005 11:00 am
Has thanked: 16 times
Been thanked: 17 times

Post by Savage » Mon Nov 13, 2006 6:18 pm

Nice to see this 8D
go 4 it!!! :D
Image

User avatar
Dinoguy1000
Site Admin
Posts: 713
Joined: Mon Sep 13, 2004 1:55 am
Location: The wiki.
Has thanked: 105 times
Been thanked: 126 times

Post by Dinoguy1000 » Tue Nov 14, 2006 2:57 am

Go Strobe Go! *waves pennant flag with "STROBE" on it, and giant foam hand with "JAEDER NAUB" on it* :D
Welcome to Xentax!

Rules | Requests | Wiki

If you run across a post that breaks the rules, please report the post - a mod or admin will handle it from there.

User avatar
Strobe
Moderator
Posts: 414
Joined: Mon Oct 24, 2005 8:52 am
Location: Sweden
Been thanked: 16 times
Contact:

Argghh

Post by Strobe » Tue Nov 14, 2006 5:03 pm

Now i feel the pressure =O .....!!!

what if i give up?!! . then people will get my phonenumber and
address and send some "guys" over here :(((

User avatar
Mr.Mouse
Site Admin
Posts: 4036
Joined: Wed Jan 15, 2003 6:45 pm
Location: Dungeons of Doom
Has thanked: 413 times
Been thanked: 550 times
Contact:

Re: Argghh

Post by Mr.Mouse » Tue Nov 14, 2006 8:38 pm

Strobe wrote:Now i feel the pressure =O .....!!!

what if i give up?!! . then people will get my phonenumber and
address and send some "guys" over here :(((
I certainly will, as I will start telling people to expect a brand new JN in the new MexCom release!!! :P :harry :harry :harry

User avatar
Strobe
Moderator
Posts: 414
Joined: Mon Oct 24, 2005 8:52 am
Location: Sweden
Been thanked: 16 times
Contact:

bboohhohoohh

Post by Strobe » Tue Nov 14, 2006 9:07 pm

My life is ruined :(

ill start making small flash games instead under the pseudonym "Pink Cuddler" =(

User avatar
Dinoguy1000
Site Admin
Posts: 713
Joined: Mon Sep 13, 2004 1:55 am
Location: The wiki.
Has thanked: 105 times
Been thanked: 126 times

Post by Dinoguy1000 » Thu Nov 16, 2006 7:49 pm

:lol:
Welcome to Xentax!

Rules | Requests | Wiki

If you run across a post that breaks the rules, please report the post - a mod or admin will handle it from there.

FunteX
n00b
Posts: 16
Joined: Sat Aug 19, 2006 10:45 pm

Post by FunteX » Mon Nov 20, 2006 10:11 pm

>>strobe.
what language have you programmed jaeder naub in?

if its delphi, then i would be able to help you, if you agree to it. to take off a little the pressure :)

oh, btw. i've notice that in the current jaeder naub it looks like it eat one single file. if you want to optimize the speed, try cutting bigger files up in chunks and place it in thier own threads :)

cheers FunteX! :)

User avatar
lionheartuk
double-veteran
double-veteran
Posts: 670
Joined: Tue May 16, 2006 10:55 pm
Location: Tokyo, Japan
Has thanked: 20 times
Been thanked: 23 times

Post by lionheartuk » Mon Nov 20, 2006 11:32 pm

FunteX wrote:>>strobe.
what language have you programmed jaeder naub in?

if its delphi, then i would be able to help you, if you agree to it. to take off a little the pressure :)

oh, btw. i've notice that in the current jaeder naub it looks like it eat one single file. if you want to optimize the speed, try cutting bigger files up in chunks and place it in thier own threads :)

cheers FunteX! :)
Jaeder is programmed IN Visual Basic

As for the final message i thought that it already cut files up anyways
No wait yeah it does actually.
"Lemons are people nice hardworking & honest people too..."

User avatar
Strobe
Moderator
Posts: 414
Joined: Mon Oct 24, 2005 8:52 am
Location: Sweden
Been thanked: 16 times
Contact:

Hi!

Post by Strobe » Tue Nov 21, 2006 4:48 am

Funtex: Its coded in VB as lionheartuk said ;)

But the new engine im working on side by side is coded
in Freebasic. which has much better support for operations like this,
and also supports inline assembler.

Its WAAAAAY faster than VB. however im both updating the current
ripper and working on the new one, i do learn more tricks when
coding 2 things side by side, and im trying to adopt some of them
over to the current engine.

As a sidenote, i have never touched Delphi ! :D

and the reason for the slow scanning in Visual Basic is because
of the lack of memory usage, i have to take everything in so
small chunks before i can process them, while in Freebasic
can cache a whole file in memory and do mem scanning instead.

but i am also testing new caching function for VB code, and hopefully
speeding up the old engine aswell :X

User avatar
lionheartuk
double-veteran
double-veteran
Posts: 670
Joined: Tue May 16, 2006 10:55 pm
Location: Tokyo, Japan
Has thanked: 20 times
Been thanked: 23 times

Re: Hi!

Post by lionheartuk » Tue Nov 21, 2006 4:05 pm

Strobe wrote:Funtex: Its coded in VB as lionheartuk said ;)

But the new engine im working on side by side is coded
in Freebasic. which has much better support for operations like this,
and also supports inline assembler.

Its WAAAAAY faster than VB. however im both updating the current
ripper and working on the new one, i do learn more tricks when
coding 2 things side by side, and im trying to adopt some of them
over to the current engine.

As a sidenote, i have never touched Delphi ! :D

and the reason for the slow scanning in Visual Basic is because
of the lack of memory usage, i have to take everything in so
small chunks before i can process them, while in Freebasic
can cache a whole file in memory and do mem scanning instead.

but i am also testing new caching function for VB code, and hopefully
speeding up the old engine aswell :X
Sounds good man.
go for it init.
I hope to see the changes oen day.
"Lemons are people nice hardworking & honest people too..."

User avatar
Strobe
Moderator
Posts: 414
Joined: Mon Oct 24, 2005 8:52 am
Location: Sweden
Been thanked: 16 times
Contact:

Insane speed.

Post by Strobe » Fri Feb 16, 2007 5:48 am

The scanner is so fast im sitting here wanking.

Tests revelead currently that it is able to scan
50 MB of data in about 3 seconds.
(and it isnt even really optimized yet)

that would mean roughly about 500mb of data in 30 seconds.
This includes loading cache time!!

(tested with about 6 formats implemented yet)
but not savedata time yet.

I will however keep the old VB jaeder naub engine, as i had an idea,
i will use this scanner to prescan a file, set up a list of addresses
where it spots files, and let jaeder naub scan those addresses
and dump the files.

this may sound like im not really gaining any speed on it if im sending the information back to the old Scanner engine,
but yes i am, because then the old engine will only be used for the
Saving and conversion part.

havent tested this in reality yet though! :X stay tuned!

Edit:
And btw , this scanner will be commandline driven, so this could later
be a successor for the current JN implementation in MexCom.

Edit2:
Okay, the scanner is now optimized. and makes about 100Mbs
per second. this is far beyond expectations when i first started
with it :P

But now the system specs for this scanner is pretty high.
for optimal performance it would atleast require 2 GHZ and 1GB
of RAM.

User avatar
Strobe
Moderator
Posts: 414
Joined: Mon Oct 24, 2005 8:52 am
Location: Sweden
Been thanked: 16 times
Contact:

YAY

Post by Strobe » Fri Feb 16, 2007 11:55 am

Well.... I guess you all know what this picture means...

=D

Im now betatesting the external scanner and feeds jaeder naub
with the data it finds.

So far the results are brillant!!
You do not have the required permissions to view the files attached to this post.

Post Reply