Page 1 of 3

Recoding my ripper engine totally.

Posted: Mon Nov 13, 2006 12:14 pm
by Strobe
I have started creating a totally new ripper engine.
it will be probably 1000% (or more) faster than the current
Jaeder Naub.

Jaeder will though still be in development as the time to
totally rewrite the engine and convert file format detections
will take a pretty damn long time, but i felt it was neccessary
to do this for various of reasons.

i have made some current tests, and my beta scans about 30mb/s
in Deep scan mode, while Jaeder Naub scans at highest 1mb/s

I have tried my best to adapt this engine to jaeder naub, but it has failed,
as it works on a totally different level. *sigh* , so it seems like the only
way to integrate this scanner is by recoding the whole application.

well well, maybe its my destiny? =X

Edit:
On some more plus sides, the new engine is fully portable to Linux :)))

Posted: Mon Nov 13, 2006 12:26 pm
by Mr.Mouse
Still, that's good news!! I wish you a swift conversion! :D

Posted: Mon Nov 13, 2006 4:19 pm
by lionheartuk
GO for it strobe.
Jaeder is still good.
Though maybe, I should add soem formats to the ripper we both worked on, and see if that can work any faster then jaeder.
Though, i dont have it on this new Laptop so il need the newest version il ask you on msn i guess.

Posted: Mon Nov 13, 2006 6:18 pm
by Savage
Nice to see this 8D
go 4 it!!! :D

Posted: Tue Nov 14, 2006 2:57 am
by Dinoguy1000
Go Strobe Go! *waves pennant flag with "STROBE" on it, and giant foam hand with "JAEDER NAUB" on it* :D

Argghh

Posted: Tue Nov 14, 2006 5:03 pm
by Strobe
Now i feel the pressure =O .....!!!

what if i give up?!! . then people will get my phonenumber and
address and send some "guys" over here :(((

Re: Argghh

Posted: Tue Nov 14, 2006 8:38 pm
by Mr.Mouse
Strobe wrote:Now i feel the pressure =O .....!!!

what if i give up?!! . then people will get my phonenumber and
address and send some "guys" over here :(((
I certainly will, as I will start telling people to expect a brand new JN in the new MexCom release!!! :P :harry :harry :harry

bboohhohoohh

Posted: Tue Nov 14, 2006 9:07 pm
by Strobe
My life is ruined :(

ill start making small flash games instead under the pseudonym "Pink Cuddler" =(

Posted: Thu Nov 16, 2006 7:49 pm
by Dinoguy1000
:lol:

Posted: Mon Nov 20, 2006 10:11 pm
by FunteX
>>strobe.
what language have you programmed jaeder naub in?

if its delphi, then i would be able to help you, if you agree to it. to take off a little the pressure :)

oh, btw. i've notice that in the current jaeder naub it looks like it eat one single file. if you want to optimize the speed, try cutting bigger files up in chunks and place it in thier own threads :)

cheers FunteX! :)

Posted: Mon Nov 20, 2006 11:32 pm
by lionheartuk
FunteX wrote:>>strobe.
what language have you programmed jaeder naub in?

if its delphi, then i would be able to help you, if you agree to it. to take off a little the pressure :)

oh, btw. i've notice that in the current jaeder naub it looks like it eat one single file. if you want to optimize the speed, try cutting bigger files up in chunks and place it in thier own threads :)

cheers FunteX! :)
Jaeder is programmed IN Visual Basic

As for the final message i thought that it already cut files up anyways
No wait yeah it does actually.

Hi!

Posted: Tue Nov 21, 2006 4:48 am
by Strobe
Funtex: Its coded in VB as lionheartuk said ;)

But the new engine im working on side by side is coded
in Freebasic. which has much better support for operations like this,
and also supports inline assembler.

Its WAAAAAY faster than VB. however im both updating the current
ripper and working on the new one, i do learn more tricks when
coding 2 things side by side, and im trying to adopt some of them
over to the current engine.

As a sidenote, i have never touched Delphi ! :D

and the reason for the slow scanning in Visual Basic is because
of the lack of memory usage, i have to take everything in so
small chunks before i can process them, while in Freebasic
can cache a whole file in memory and do mem scanning instead.

but i am also testing new caching function for VB code, and hopefully
speeding up the old engine aswell :X

Re: Hi!

Posted: Tue Nov 21, 2006 4:05 pm
by lionheartuk
Strobe wrote:Funtex: Its coded in VB as lionheartuk said ;)

But the new engine im working on side by side is coded
in Freebasic. which has much better support for operations like this,
and also supports inline assembler.

Its WAAAAAY faster than VB. however im both updating the current
ripper and working on the new one, i do learn more tricks when
coding 2 things side by side, and im trying to adopt some of them
over to the current engine.

As a sidenote, i have never touched Delphi ! :D

and the reason for the slow scanning in Visual Basic is because
of the lack of memory usage, i have to take everything in so
small chunks before i can process them, while in Freebasic
can cache a whole file in memory and do mem scanning instead.

but i am also testing new caching function for VB code, and hopefully
speeding up the old engine aswell :X
Sounds good man.
go for it init.
I hope to see the changes oen day.

Insane speed.

Posted: Fri Feb 16, 2007 5:48 am
by Strobe
The scanner is so fast im sitting here wanking.

Tests revelead currently that it is able to scan
50 MB of data in about 3 seconds.
(and it isnt even really optimized yet)

that would mean roughly about 500mb of data in 30 seconds.
This includes loading cache time!!

(tested with about 6 formats implemented yet)
but not savedata time yet.

I will however keep the old VB jaeder naub engine, as i had an idea,
i will use this scanner to prescan a file, set up a list of addresses
where it spots files, and let jaeder naub scan those addresses
and dump the files.

this may sound like im not really gaining any speed on it if im sending the information back to the old Scanner engine,
but yes i am, because then the old engine will only be used for the
Saving and conversion part.

havent tested this in reality yet though! :X stay tuned!

Edit:
And btw , this scanner will be commandline driven, so this could later
be a successor for the current JN implementation in MexCom.

Edit2:
Okay, the scanner is now optimized. and makes about 100Mbs
per second. this is far beyond expectations when i first started
with it :P

But now the system specs for this scanner is pretty high.
for optimal performance it would atleast require 2 GHZ and 1GB
of RAM.

YAY

Posted: Fri Feb 16, 2007 11:55 am
by Strobe
Well.... I guess you all know what this picture means...

=D

Im now betatesting the external scanner and feeds jaeder naub
with the data it finds.

So far the results are brillant!!