Guruserver plus esata bottleneck? [SOLVED]

[rmentjens]

I have a guruserver plus clonnected to a 1G WD esata drive. (WDC WD10EARS-00Y5B1) I am experiencing some speed problems. I thought it was because of poor network performance so i upgraded to gigabit ethernet. This didnt help. so then i did some testing. I started an FTP session and did two tests. One to "get" to /dev/null (so no actual writing) and one "get" to write in the esata drive. Then i did a put command.
Here are the results:

ftp> get tkol.zip
local: tkol.zip remote: tkol.zip
200 PORT command successful.
150 Opening BINARY mode data connection for tkol.zip (385744125 bytes).
226 Transfer complete. 385744125 bytes in 123.33 sec. (3054.483 Kb/s)
385744125 bytes received in 122.91 secs (3064.9 kB/s)
ftp>

and

ftp> get tkol.zip /dev/null
local: /dev/null remote: tkol.zip
200 PORT command successful.
150 Opening BINARY mode data connection for tkol.zip (385744125 bytes).
226 Transfer complete. 385744125 bytes in 14.41 sec. (26147.238 Kb/s)
385744125 bytes received in 14.40 secs (26150.9 kB/s)
ftp>

put command
ftp> put tkol.zip
local: tkol.zip remote: tkol.zip
200 PORT command successful.
125 Ready to receive "tkol.zip" . Mode STREAM Type BINARY.
226 Transfer complete. 385744125 bytes in 18.84 sec. (19991.682 Kb/s)
385744125 bytes sent in 18.84 secs (19994.1 kB/s)
ftp>

what it tells me (noob alert) that network speed is OK (writing to /dev/null ) that read speed is ok ( see put command) but that write speed is very bad.


This is what the bootup text says
ata1: SATA link up 1.5 Gbps (SStatus 113 SControl F300)
ata1.00: ATA-8: WDC WD10EARS-00Y5B1, 80.00A80, max UDMA/133
ata1.00: 1953525168 sectors, multi 0: LBA48 NCQ (depth 31/32)
ata1.00: configured for UDMA/133

Can someone help me identifying the bottleneck here?

Kind regards

[rmentjens]

I posted this problem in plugcomputer.org forum and got advise to test in a different way

dd if=/dev/zero of=/mnt/esata/test bs=1M count=256
256+0 records in
256+0 records out
268435456 bytes (268 MB) copied, 34.1654 s, 7.9 MB/s

bad!!


i experimented with the  dd command (with wich i am not familiar) and tried the following:

from the esata to /dev/null

dd if=/mnt/esata/test of=/dev/null bs=1M count=256
256+0 records in
256+0 records out
268435456 bytes (268 MB) copied, 0.845523 s, 317 MB/s

better!!

it looks like writing to my esata drive is verry slow and reading from my esata is much faster. I have no idea what rate i should find, but 7.9 MB/s write speed seems too slow to me.

Anybody a good advise?

kind regards

[NewIT_James]

Here's a comparison.

Dreamplug with WD20EARS HD (ext3)
NewIT Wheezy (kernel 3.0.4)

Code:

# dd if=/dev/zero of=/mnt/eSata/test bs=1M count=256
256+0 records in
256+0 records out
268435456 bytes (268 MB) copied, 5.87296 s, 45.7 MB/s

# dd of=/dev/null if=/mnt/eSata/test bs=1M count=256
256+0 records in
256+0 records out
268435456 bytes (268 MB) copied, 0.718954 s, 373 MB/s

and a Sheevaplug using NFS connection to the above Dreamplug

Code:

# dd if=/dev/zero of=testNFS bs=1M count=256
256+0 records in
256+0 records out
268435456 bytes (268 MB) copied, 17.1223 s, 15.7 MB/s

# dd of=/dev/null if=testNFS bs=1M count=256
256+0 records in
256+0 records out
268435456 bytes (268 MB) copied, 3.15878 s, 85.0 MB/s

[Confusticated]

You don't mention what file system type your SATA drive is formatted with.....

[rmentjens]

You are right I forgot to mention. The filesystem is EXT2.

Can the filesystem be the source of a speedproblem like this?

[Confusticated]

It will certainly contribute, I advise you to go with ext4 and repeat your tests....

[rmentjens]

ok. well. humm. I have to reformat....
I will have to backup some things first before i do it. I had hopes it didn't come to this because of all the data transfered.

Ah well....

Ill be back when i am ready

[rmentjens]

Ok i formatted my drive to ext4, here are the results.

Code:

dd if=/dev/zero of=/mnt/esata/test bs=1M count=256
256+0 records in
256+0 records out
268435456 bytes (268 MB) copied, 55.2825 s, 4.9 MB/s

and

dd if=/mnt/esata/test of=/dev/null bs=1M count=256
256+0 records in
256+0 records out
268435456 bytes (268 MB) copied, 0.796455 s, 337 MB/s

Write speed hasn't increased (to say the least) read speed is almost the same.

Are there any other things I can reconfigure or test? For example why is the drive only connected at 1,5 Gbps as mentioned in uboot startup message?

Anyone?

[Confusticated]

I am going to have to spend time pondering this one.
No instant answers, have to eliminate what it is not!

u-boot version and output from 'fdisk -l' please.

[PacoLM]

Hi,

 Another point for the statistics, Guruplug with SSD eSata drive:

Code:

dd if=/dev/zero of=/usr/test bs=1M count=256
268435456 bytes (268 MB) copied, 4.62478 s, 58.0 MB/s

and

dd if=/usr/test of=/dev/null bs=1M count=256
268435456 bytes (268 MB) copied, 0.872144 s, 308 MB/s

Output fdisk -l:

Code:

Disk /dev/sda: 32.0 GB, 32017047552 bytes
255 heads, 63 sectors/track, 3892 cylinders, total 62533296 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0002d4b6

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048      499711      248832   83  Linux
/dev/sda2          499712    59807743    29654016   83  Linux
/dev/sda3        59809790    62531583     1360897    5  Extended
/dev/sda5        59809792    62531583     1360896   82  Linux swap / Solaris

Hope it helps,

PacoLM

[rmentjens]

u-boot version and output from 'fdisk -l' please.

Here's the U-boot version

Code:

Marvell>> version
U-Boot 2010.03-01161-gd91b0a9 (Apr 22 2010 - 03:24:41)
Marvell-GuruPlug
Marvell>>

And here the fdisk -l response

Code:

aarde:~# fdisk -l

Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xef268b77

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1      121601   976760001   83  Linux
aarde:~#

BTW did I mention that noobalert was on? Meaning that possible mistakes are honest mistakes.

Some extra info. My USB port is fried (warranty broke because seal is broken) However I figured that the E-SATA port doesn't need a working USB port. (And in the current setting  (one E-SATA and no USB) is good enough for me. (if write speed can increase that is)

[Confusticated]

Units = cylinders of 16065 * 512 = 8225280 bytes

/dev/sda1               1      121601   976760001   83  Linux

There is an issue here, this drive absolutely *must* be partitioned on 4k boundaries (according to the manufacturer spec & other forum posts, as I don't have one of these...

EDIT: If this is the case, I would expect to see a very high load when writing in comparison to when reading. You could try running 'top' (or better 'htop') for a visual comparison.
noobalert noted, everybody has been one at some time, and in many ways I still am (don't let me near your houseplants or garden

[rmentjens]

Thank you very much for your time, even in the after hours.

I ran top in one putty window, logged in on another and ran:

Code:

dd if=/dev/zero of=/mnt/esata/test bs=1M count=256

This is what top said:

Code:

top - 06:45:12 up  9:42,  2 users,  load average: 1.03, 0.46, 0.17
Tasks:  77 total,   1 running,  76 sleeping,   0 stopped,   0 zombie
Cpu(s):  1.7%us,  6.6%sy,  0.0%ni,  0.0%id, 90.7%wa,  0.3%hi,  0.7%si,  0.0%st
Mem:    513948k total,   207080k used,   306868k free,     6772k buffers
Swap:        0k total,        0k used,        0k free,    89760k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 1862 root      20   0  4408 1616  508 D  7.3  0.3   0:00.46 dd
 1555 squeezeb  20   0 85104  74m 5208 S  2.3 14.8  13:49.53 squeezeboxserve
  870 root      20   0     0    0    0 S  0.7  0.0   0:00.09 jbd2/sda1-8
 1861 root      20   0  2548 1168  932 R  0.7  0.2   0:00.11 top
  141 root      20   0     0    0    0 S  0.3  0.0   0:00.22 ata/0
 1151 mysql     20   0  124m  16m 4736 S  0.3  3.3   0:58.81 mysqld
    1 root      20   0  2004  684  596 S  0.0  0.1   0:02.43 init
    2 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kthreadd
    3 root      20   0     0    0    0 S  0.0  0.0   0:00.02 ksoftirqd/0
    4 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 watchdog/0
    5 root      20   0     0    0    0 S  0.0  0.0   0:00.11 events/0
    6 root      20   0     0    0    0 S  0.0  0.0   0:00.01 khelper
    9 root      20   0     0    0    0 S  0.0  0.0   0:00.00 async/mgr
  131 root      20   0     0    0    0 S  0.0  0.0   0:00.04 sync_supers
  133 root      20   0     0    0    0 S  0.0  0.0   0:00.10 bdi-default
  135 root      20   0     0    0    0 S  0.0  0.0   0:00.08 kblockd/0
  142 root      20   0     0    0    0 S  0.0  0.0   0:00.00 ata_aux

And I tried the same with reading but the command goes too fast to copy the screen in my second putty

[Confusticated]

load average: 1.03

90.7%wa

D  7.3  0.3   0:00.46 dd

^
This indicates the system is indeed diskbound....I have no idea how much of an effect re-aligning the partitioning is going to have on this.
Did you repartition the drive before formatting ext4, or is this the original partition layout ?
I am confusticated as to why the write speed has dropped to half when I was expecting it to double with the transition from ext2 to ext4.

One avenue that should also be investigated is the HDD parameters.

Code:

hdparm -I /dev/sda

P.S. NewIT-James, & PacoLM, thank you for the comparison data.

[rmentjens]

I removed the partition with fdisk ([d]elete) and created a new one ([n]ew) and then set the filesystemtype ([t]ype to 83 then I made a new filesystem with mkfs -t ext4 /dev/sda1 and finaly fsck -f -y /dev/sda1



Here's the output of hdparm -I /dev/sda

Code:

/dev/sda:

ATA device, with non-removable media
        Model Number:       WDC WD10EARS-00Y5B1
        Serial Number:      WD-WCAV5A743571
        Firmware Revision:  80.00A80
        Transport:          Serial, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6
Standards:
        Supported: 8 7 6 5
        Likely used: 8
Configuration:
        Logical         max     current
        cylinders       16383   16383
        heads           16      16
        sectors/track   63      63
        --
        CHS current addressable sectors:   16514064
        LBA    user addressable sectors:  268435455
        LBA48  user addressable sectors: 1953525168
        Logical/Physical Sector size:           512 bytes
        device size with M = 1024*1024:      953869 MBytes
        device size with M = 1000*1000:     1000204 MBytes (1000 GB)
        cache/buffer size  = unknown
Capabilities:
        LBA, IORDY(can be disabled)
        Queue depth: 32
        Standby timer values: spec'd by Standard, with device specific minimum
        R/W multiple sector transfer: Max = 16  Current = 0
        Recommended acoustic management value: 128, current value: 254
        DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
             Cycle time: min=120ns recommended=120ns
        PIO: pio0 pio1 pio2 pio3 pio4
             Cycle time: no flow control=120ns  IORDY flow control=120ns
Commands/features:
        Enabled Supported:
           *    SMART feature set
                Security Mode feature set
           *    Power Management feature set
           *    Write cache
           *    Look-ahead
           *    Host Protected Area feature set
           *    WRITE_BUFFER command
           *    READ_BUFFER command
           *    NOP cmd
           *    DOWNLOAD_MICROCODE
                Power-Up In Standby feature set
           *    SET_FEATURES required to spinup after power up
                SET_MAX security extension
                Automatic Acoustic Management feature set
           *    48-bit Address feature set
           *    Device Configuration Overlay feature set
           *    Mandatory FLUSH_CACHE
           *    FLUSH_CACHE_EXT
           *    SMART error logging
           *    SMART self-test
           *    General Purpose Logging feature set
           *    64-bit World wide name
           *    {READ,WRITE}_DMA_EXT_GPL commands
           *    Segmented DOWNLOAD_MICROCODE
           *    Gen1 signaling speed (1.5Gb/s)
           *    Gen2 signaling speed (3.0Gb/s)
           *    Native Command Queueing (NCQ)
           *    Host-initiated interface power management
           *    Phy event counters
           *    NCQ priority information
                DMA Setup Auto-Activate optimization
           *    Software settings preservation
           *    SMART Command Transport (SCT) feature set
           *    SCT Features Control (AC4)
           *    SCT Data Tables (AC5)
                unknown 206[12] (vendor specific)
                unknown 206[13] (vendor specific)
Security:
        Master password revision code = 65534
                supported
        not     enabled
        not     locked
        not     frozen
        not     expired: security count
                supported: enhanced erase
        200min for SECURITY ERASE UNIT. 200min for ENHANCED SECURITY ERASE UNIT.
Logical Unit WWN Device Identifier: 50014ee2aefdd793
        NAA             : 5
        IEEE OUI        : 0014ee
        Unique ID       : 2aefdd793
Checksum: correct

What exactly areyou trying to see here?