Slow reading usb

opex · July 24, 2009, 01:37:48 AM

Hi,

I have some old Compaq hardware I want to use as a quiet router running Zeroshell of an usb pen drive.
I've made a floppy with the bootmanager and it successfully starts reading grub of the pen drive.
But here's where I run into a problem that makes my solution useless. It takes about 45 minutes to read the content of the pen drive to the memory. After that it takes a few seconds to run the boot sequence to get the system up and running.

So what I'm hoping for is any hints or tips on what I can do to get the usb reading down to seconds instead of minutes.
I've tried using the option to force usb 1.1 but the result is the same.

//

Elmar · July 24, 2009, 10:17:00 AM

hi,
what linux are you booting?

best regards
elmar

opex · July 24, 2009, 12:08:36 PM

I'm let the bootmanager give control to a LiveCD dist of Zeroshell http://www.zeroshell.net/eng/ which I've installed on a usb pen drive.
According to the homepage Zeroshell isn't based on a previous dist.

Elmar · July 24, 2009, 12:21:40 PM

zeroshell copies a lot of files to the ram.

how much time does it take to load the kernel?
usb 1.1 is slow. i think the only solution will be to optimize the zeroshell boot process. but i have to know how fast the kernel is loaded.

maybe thats no option for you, but what about using another linux?

regards
elmar

bralmu · July 24, 2009, 15:49:24 PM

Quote from: opex on July 24, 2009, 01:37:48 AM
It takes about 45 minutes to read the content of the pen drive to the memory. After that it takes a few seconds to run the boot sequence to get the system up and running.

Try not to connect any other usb device.
Tipical CD reading speed is 2.3MB/s.
Max speed of usb1.1 is 1.5MB/s but if there is more than one device connected or the cable is too long or many other factors the speed might be as low as 192KB/s
If this is a hardware problem without solution you might try some light distro

opex · July 24, 2009, 16:44:55 PM

The computer is a Compaq Evo D500 SSF and there's not much connected. The pen drive is the only thing that's connected using usb. The CD is IDE, then there's the floppy. Even the keyboard is PS/2.

opex · July 24, 2009, 17:22:12 PM

I made a rough time table of what's printed on the screen and when

Code Select

[00.00] ........................................................................................
        GRUB Loading stage1.5.


        GRUB loading, please wait...
[00.05]
        (Showing the grub menu)

         Booting 'Zeroshell - Net Services   * IDE/SATA/USB Version *'

        root (hd0,0)
         Filesystem type is ext2fs, partition type 0x83
[00.08]
        kernel /vmlinuz ramdisk_size=131072 quiet
           [Linux-bzImage, setup=0x2a00, size=0x1e111c]
[00.50]
        Booting the kernel.
        ...
[00.51]
        System up and running

Elmar · July 24, 2009, 17:25:15 PM

press CTRL and boot usb. what is the last loaded driver?

opex · July 24, 2009, 17:54:14 PM

Loading UHCI Driver
Searching on hosts
Host 1
Host 2
Sandisk
Cruzer Pattern
Loading boot sector...

Elmar · July 24, 2009, 18:13:08 PM

a question to your time table:
[00.50] means 50 minutes?

opex · July 24, 2009, 18:14:21 PM

yea, wouldn't have much to complain about if it was 50 seconds

Elmar · July 24, 2009, 18:50:21 PM

here is a test version. its a temporary link.
http://download.plop.at/files/tmp/plpbt-test.zip

does it work?

opex · July 25, 2009, 00:40:43 AM

No improvement. 20 minutes and still at "GRUB loading, please wait..."

edit
I shut it down after an hour and still at GRUB loading...

Elmar · July 25, 2009, 01:54:57 AM

hey 20 minutes is an improvement of over 50%!!!!

opex · July 25, 2009, 09:31:51 AM

ehm..
With the previous version the boot process moved on after 5 minutes from "GRUB Loading, please wait"
How can it be an improvement if it's still stuck there after an hour when I decided to shut it down?

Elmar · July 25, 2009, 11:12:43 AM

oh, i misunderstood you.

Elmar · July 26, 2009, 09:59:05 AM

hi,

this is a new test version
http://download.plop.at/files/tmp/plpbt-test.zip

opex · July 26, 2009, 12:06:03 PM

From what I can tell from the LED on the pen drive, the new test version initiate some connection with the usb drive because the LED is flashing. But after about five seconds the drive goes idle and nothing else happens.
So not even GRUB is loaded this time.

Elmar · July 28, 2009, 12:03:30 PM

hi,

again a new test version
http://download.plop.at/files/tmp/plpbt-test.zip

opex · July 28, 2009, 20:20:58 PM

We're having some success. The total boot time is scaled down to 25 minutes flat
I've updated the time table but this time I've shown the time span that the cursor is sitting idle on the screen, indicating reading from the pen drive.

Code Select

  ...............................................................................
  ......
  GRUB Loading stage1.5.


  GRUB loading, please wait...
[0.00.00]-[0.02.25]
  (showing grub menu and waiting for 10 seconds)
   Booting 'Zeroshell - Net Services   * IDE/SATA/USB Version *'

  root (hd0,0)
[0.02.40]-[0.02.50]
   Filesystem type is ext2fs, partition type 0x83
  kernel /vmlinuz ramdisk_size=131072 quiet
[0.02.50]-[0.03.35]
     [Linux-bzImage, setup=0x2a00, size=0x1e111c]
[0.03.35]-[0.14.00]
  initrd /initrd.gz
[0.14.00]-[0.23.15]
     [Linux-initrd @ 0xfe3e000, 0x191c76 bytes]
  cat /grub/zeroshell.txt
[0.23.15]-[0.24.00]
  Decompressing Linux... done
  Booting the kernel.
  ...
[0.25.00]
  System up and running

And by using the sizes of vmlinuz and initrd.gz and the time the cursor sits idle, my math tells me that it reads with a speed of 3Kb/s
(1 970 460 / 1024) / 630 = 3,05440848 (vmlinuz)
(1 645 686 / 1024) / 555 = 2,89570313 (initrd.gz)

Still not a system I'd have running live, but progress none the less.

I really appreciate the time and effort you're putting into this Elmer

Elmar · July 28, 2009, 21:22:08 PM

ok, that is faster and a bit better

i know, reading the usb device is not as fast as linux or other drivers, but the boot manager is not linux. i try to get more speed, but i am very limited in available space for my drivers. so dont expect too much. but i will do my best

Quote
Still not a system I'd have running live, but progress none the less.

i thought you want to use zeroshell as router/firewall and so on. then there shouldn't be many boots, or? please post for me the link to a docu. to make zeroshell bootable from usb. i have to do some tests. maybe my ploplinux would be a solution for you. i dont know what you expect from zeroshell.

however, i'll try to make the driver faster so stay tuned

the current driver updates and some other modifications will be in the boot manager 5.0.4.

opex · July 28, 2009, 23:07:39 PM

No, it won't be many reboots. It'll even be on a UPS. But if it should happen I wouldn't want to wait 30 minutes to get my Internet up and running again.

I downloaded the 1GB image for IDE, SATA and USB disks from http://www.zeroshell.net/eng/download/
Then it's just a matter of running this command in a terminal

Code Select

sudo dd if=./ZeroShell-1.0.beta12-CompactFlash-IDE-USB-SATA-1GB.img of=/dev/sdc

where /dev/sdc is my usb pen drive.

I've been looking at different solutions to boot my usb drive. But yours is the only one I've gotten to actually boot the system successfully.
Kexec-loader is a promising alternative but there's some config issues I've not fully grasped yet.

Elmar · July 28, 2009, 23:17:46 PM

i am wonderring that this works, because there is no valid mbr or boot sector in this file.

opex · July 28, 2009, 23:57:03 PM

Did you explode the archive?
If not then you have to substitute

Code Select

sudo dd if=./ZeroShell-1.0.beta12-CompactFlash-IDE-USB-SATA-1GB.img of=/dev/sdc

with

Code Select

gunzip -c ./ZeroShell-1.0.beta12-CompactFlash-IDE-USB-SATA-1GB.img.gz>/dev/sdc

Elmar · July 29, 2009, 00:06:04 AM

i was using the wrong file. the correct file has a valid mbr

Elmar · July 29, 2009, 10:12:01 AM

ok, i tested now to boot zeroshell with uhci.
5 seconds to load grub
26 seconds to load vmlinuz
20 seconds to load initrd.gz

hmm, why is my driver so slow on your computer...

is your computer a Compaq Evo D500 SSF or SFF?

opex · July 29, 2009, 11:20:21 AM

It's the Small Form Factor (SFF)

opex · July 29, 2009, 12:00:18 PM

I'll see if I can't dig up another usb drive and also try the other EVO box I got, to rule out hardware issues.

But then again, you more than halved the boot time with the latest test version. So something changed that appealed to my hardware.

Elmar · July 29, 2009, 12:29:24 PM

i have seen a pc like yours on ebay. maybe i buy it if it is cheap enough. it runs out in 5 days. a long time to think about buying.

opex · July 30, 2009, 15:11:45 PM

Tried the other Evo box with the same result. And I Couldn't find a usb drive from another brand that was big enough. So I was unable to replace that.

Elmar · July 30, 2009, 15:19:07 PM

you need only 5 mb for vmlinuz, initrd and grub. zeroshell akes 2 partitions, so only the first partition with the boot files is required.
so take any device and run dd, you can ignore that the file does not fit on the device. we only want to know how fast the kernel and initrd is loaded

opex · July 30, 2009, 21:36:07 PM

I manage to borrow an old 256MB drive, but sadly it's the same results as before.

I should also mention that I've found a HDD with acceptable noise level that I'm going to use for this router so getting it running on usb is not such a big priority any more. I'll still be happy to run any tests you trow at this thou'.

Elmar · August 06, 2009, 10:41:38 AM

i bought the evo ds 500 sff. it was cheap. i have "luck", the boot manager does not work correctly, so i am wondering that it works on yours. however, i can fix some problems. maybe i can fix the slow reading too.

Elmar · August 08, 2009, 18:47:03 PM

SUCCESS SUCCESS SUCCESS!!!!

it was hard, because the harddisk of the compaq evo ds 500 sff is so unbelievable shrill and loud. it hurts.

but now, zeroshell needs following times!
2 seconds to load grub
4 seconds to load vmlinuz
3 seconds for initrd

great, isn't it?

working on the hardware that makes the troubles is always the fastest way to kill bugs.

i am so happy

but it has to be tested on other uhci machines and i have another mystery to solve with the evo. you will get a test version soon!

regards
elmar

opex · August 09, 2009, 02:40:47 AM

That's awesome!

Kudos to you. Looking forward to the test version.

Elmar · August 10, 2009, 10:19:23 AM

hi,
which bios version do you have 3.04?

opex · August 10, 2009, 17:15:14 PM

Hi,

Yea, one of them are updated to 3.04. The other one is still on 3.01, 3.02 or there about.

Elmar · August 10, 2009, 17:24:44 PM

in the morning i thought the last mystery of the evo has something to do with the bios version, but meanwhile i fixed the problem. it had nothing to do with the bios. currently i am optimizig some stuff to make the driver smaller.

Elmar · August 11, 2009, 19:40:51 PM

hi,
here is a new test version
http://download.plop.at/files/tmp/plpbt-test3.zip

regards
elmar

opex · August 12, 2009, 13:29:30 PM

Seems to work really well. I believed it took 10-20s to read grub, vmlinuz and initrd.

I do have one problem thou' and haven't really figured out what it could be yet. I'm lacking test hardware at my current location.
I'm running on the EVO with an USB keyboard (can't find my PS/2 adapter

) and a Sandisk 512MB pen drive (which can't take the whole Zeroshell image).

With this configuration and on a cold boot it takes roughly 5min to get past your boot sequence. It stops at:

Code Select

Searching for hosts
Host 1
Host 2

I can't tell if it's the USB keyboard that triggers a timeout event or if it's the pen drive that's hard to identify.

Something worth noticing is that if I do a reboot on a running system the Sandisk drive is identified instantly.

Except for that little issue it works superbly

Elmar · August 12, 2009, 13:39:32 PM

what happens when you swap the usb connections? pendrive to keyboard and keybard to pendrive usb port

opex · August 12, 2009, 13:47:39 PM

On a reboot it's identified instantly like before.

On a cold boot it fails with "Port 1: No mass storage device"

opex · August 12, 2009, 13:53:12 PM

I'm going to see if I can't purchase a new usb drive today, need a bigger one anyway and try locate that PS/2 adapter

Elmar · August 12, 2009, 14:07:47 PM

swap back the devices
what about this version
http://download.plop.at/files/tmp/plpbt-test5.zip

Elmar · August 20, 2009, 19:34:42 PM

hi,
please test this version on your evo's http://download.plop.at/files/tmp/plpbt-test2.zip

opex · August 26, 2009, 09:08:08 AM

Hi Elmer,

Sorry I haven't responded earlier, been kinda hectic here.
Made a test run this morning and what can I say. It works flawlessly.
No unexplainable delays what so ever. This was tested with the pen drive on the first usb host. Haven't tried putting the keyboard before the drive.

Great work!

madrid01 · May 12, 2010, 13:55:55 PM

Hello everybody,
You should replace your system with new one then you will be able to get solution of your problem.

QuoteI have some old Compaq hardware I want to use as a quiet router running Zeroshell of an usb pen drive.
I've made a floppy with the bootmanager and it successfully starts reading grub of the pen drive.
But here's where I run into a problem that makes my solution useless. It takes about 45 minutes to read the content of the pen drive to the memory. After that it takes a few seconds to run the boot sequence to get the system up and running.

So what I'm hoping for is any hints or tips on what I can do to get the usb reading down to seconds instead of minutes.