problem with 8GB SD from you

[griesie]

Hi there,

I just bought an eSata SheevaPlug with debian squeeze SD Card from you. I keep getting a strange error message from dmesg:

mmcblk0p2: rw=1, want=15712488, limit=15651136
attempt to access beyond end of device

It seems that the second partition is larger than the device. Are the newer cards maybe a bit smaller than the one you copied the Image from?

Do you know a quick and undangerous solution besides deleting the partition an recreate it with fdisk?

griesie

[NewIT_Marcus]

Hi there,

I just bought an eSata SheevaPlug with debian squeeze SD Card from you. I keep getting a strange error message from dmesg:

mmcblk0p2: rw=1, want=15712488, limit=15651136
attempt to access beyond end of device

It seems that the second partition is larger than the device. Are the newer cards maybe a bit smaller than the one you copied the Image from?

Do you know a quick and undangerous solution besides deleting the partition an recreate it with fdisk?

griesie

Please boot from NAND, then copy the entire SD card to a destination (USB stick or external drive) where you know there is at least 8G space:

dd if=/dev/mmcblk0 of=/path/to/target.img

When the command completes (and it will take some time), you will see a readout of the number of bytes. For the ultime 8G cards, that should be 8,048,869,376 bytes.

[griesie]

I ran thge following command on my linux box (Ubuntu Karmic 64bit)

dd if=/dev/sdd of=PlugDebianImage_FullDisk.image bs=4096 conv=notrunc,noerror
1957376+0 records in
1957376+0 records out
8017412096 bytes (8.0 GB) copied, 578.082 s, 13.9 MB/s

And:
root@seldonfrng:~/tmp# ll PlugDebianImage_FullDisk.image
-rw-r--r-- 1 root root 8017412096 2010-03-05 20:39 PlugDebianImage_FullDisk.image


I also noticed this from dmesg

[44233.149224] sd 8:0:0:0: [sdd] 15659008 512-byte logical blocks: (8.01 GB/7.46 GiB)
[44233.150572] sd 8:0:0:0: [sdd] Assuming drive cache: write through
[44233.153293] sd 8:0:0:0: [sdd] Assuming drive cache: write through
[44233.153301]  sdd: sdd1 sdd2
[44233.155629] sdd: p2 size 15712576 exceeds device capacity, limited to end of disk
[44233.660304] kjournald starting.  Commit interval 5 seconds
[44233.660317] EXT3-fs warning: mounting fs with errors, running e2fsck is recommended
[44233.665720] EXT3 FS on sdd2, internal journal
[44233.665733] EXT3-fs: mounted filesystem with writeback data mode.

This is not the first time that I see cards that are slightly smaller than others with the same announced size. Sometimes often used cards even lost some capacity with usage after repartitioning.

It seems the card is a bit too small, right?

[NewIT_Marcus]

I ran thge following command on my linux box (Ubuntu Karmic 64bit)

dd if=/dev/sdd of=PlugDebianImage_FullDisk.image bs=4096 conv=notrunc,noerror
1957376+0 records in
1957376+0 records out
8017412096 bytes (8.0 GB) copied, 578.082 s, 13.9 MB/s

Thank you. That's what we needed to see.

And:
root@seldonfrng:~/tmp# ll PlugDebianImage_FullDisk.image
-rw-r--r-- 1 root root 8017412096 2010-03-05 20:39 PlugDebianImage_FullDisk.image


I also noticed this from dmesg

[44233.149224] sd 8:0:0:0: [sdd] 15659008 512-byte logical blocks: (8.01 GB/7.46 GiB)
[44233.150572] sd 8:0:0:0: [sdd] Assuming drive cache: write through
[44233.153293] sd 8:0:0:0: [sdd] Assuming drive cache: write through
[44233.153301]  sdd: sdd1 sdd2
[44233.155629] sdd: p2 size 15712576 exceeds device capacity, limited to end of disk
[44233.660304] kjournald starting.  Commit interval 5 seconds
[44233.660317] EXT3-fs warning: mounting fs with errors, running e2fsck is recommended
[44233.665720] EXT3 FS on sdd2, internal journal
[44233.665733] EXT3-fs: mounted filesystem with writeback data mode.

This is not the first time that I see cards that are slightly smaller than others with the same announced size. Sometimes often used cards even lost some capacity with usage after repartitioning.

It's the first time that I've seen it on the cards that we have stocked.

It seems the card is a bit too small, right?

Yes. Please e-mail jason@newit.co.uk with your original order information and a link to this thread.

[griesie]

I thought you might want to know that I was able to transfer debian to another 4GB card I had laying around by using fsarchiver.

It can write images to smaller file systems as long as there's enough space. The downside is that I had to partition and format the card first.

I'll send the email to Jason on the weekend.

[nickmurdoch]

FYI, I'm having the same trouble all of a sudden, about 14-15 months after buying the sheevaplug with Debian on SD card. dmesg:

attempt to access beyond end of device
mmcblk0p2: rw=1033, want=15663112, limit=15651136
Buffer I/O error on device mmcblk0p2, logical block 1957888
lost page write due to I/O error on mmcblk0p2

Is there an easy way to fix this, or am I going to have to fiddle with re-partitioning and the like?

Cheers,

Nick

[Ralph Houston]

This is a problem I have had with almost all the flash memory devices I have used under Linux, not just on the DreamPlug. This includes 2, 4 and 8 GB USB sticks and SDHC cards from various manufacturers (Verbatim, SanDisk).
Going off the end of one card irreparably fried the start sectors - I suppose the address wrapped, a problem I haven't seen for many years.
dd is a very dangerous way to make partitions for just the reasons listed in this thread, especially as the 'off end of device' error can take ages to appear.
I urge all users to check all their partitions with fsck -f or the like.
Any good suggestions on how to fix the problem other than by copying the partitions on to large enough devices?

[Confusticated]

Volunteer required to attempt and report their findings....

I have asked NewIT if gparted can fix this, I have not yet received a reply.
I cannot try myself as I do not (as yet) suffer with this issue.

Gparted has the ability (when the supporting tools/libraries are also installed) to shrink a file system so that the partition it is on can be resized, exactly what is required here.
As far as I can see the only caveat is gparted needs to be able to repair any file system corruption first.

Another tool that merits consideration is fsarchive.

[Ralph Houston]

No volunteers appeared, so here are my findings.

I started deliberately with the Ubuntu as-shipped image for the exercise - I know this has been updated:
http://downloadsnewit.co.uk/SD-images/Dreamplug/ex-factory-2011-04-24-Ubuntu-NewIT-fixed/

Firstly, as stated by James in another context, dd does not write off the end of the device - the risk is that other processes attempt to, if the structure is erroneous. I have had a mechanical head crash on a Seagate drive due to this, and I suspect a fried USB stick too.

To summarise
- the as-shipped image is exactly the same length as the Kingston internal micro SD
- its partition table and root file system describe a slightly larger space, though the sectors that are 'hanging off the end' were obviously not used

It is possible to
- copy the image to a slightly larger device and correct it using fdisk, fsck and resize2fs
- copy the image to an identical device and correct it by the same means

I urge everyone to do this if they are not using a more recent and correct boot device. The error may not appear for some time. See this topic:
http://www.newit.co.uk/forum/index.php/topic,2291.msg6479.html#msg6479

gparted is not happy to manipulate this particular partition table with its errors.

Tip:
- after altering a partition table, ioctl attempts to re-read it. If unsuccessful, the table in the kernel continues to be used, which is no good. kpartx is supposed to fix this, but it hasn't worked for me. A brutal but successful method is to unplug and reconnect the device.

Once again, this method requires the devices to have no faulty locations: I would much prefer the use of tar images which can avoid faulty locations, the method used for the original swap between Ubuntu and Debian.

Note that the Ubuntu system automatically mounts partitions - you shouldn't run fdisk, fsck or resize2fs on mounted partitions.

For interest, here are commented logs of the process with the DreamPlug running Debian Squeeze 30-Aug-11. This system does not have dosfsck.

Correcting partitions on slightly larger stick:

Code:

** Original Ubuntu image as shipped for internal micro SD: **
root@debian:/mnt/sdd1# ls -l
-rw-r--r-- 1 root root 1967128576 Sep 20 07:09 ex-factory-2011-04-24-microSD.img

** Copy to slightly larger USB stick: **
root@debian:/mnt/sdd1# dd if=ex-factory-2011-04-24-microSD.img of=/dev/sde bs=1M
1876+0 records in
1876+0 records out
1967128576 bytes (2.0 GB) copied, 666.739 s, 3.0 MB/s

** Result: **
root@debian:/mnt/sdd1# fdisk -l /dev/sde

Disk /dev/sde: 2003 MB, 2003828736 bytes
62 heads, 61 sectors/track, 1034 cylinders
Units = cylinders of 3782 * 512 = 1936384 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sde1               1        1016     1920993+   6  FAT16
/dev/sde2              56        1021     1826706   83  Linux

** End of partition 1 is invalid and type sometimes comes up wrong - this is corrected in the later image **

** Correct it, for this exercise: **

root@debian:/mnt/sdd1# fdisk /dev/sde

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): u
Changing display/entry units to sectors

Command (m for help): u
Changing display/entry units to cylinders

Command (m for help): p

Disk /dev/sde: 2003 MB, 2003828736 bytes
62 heads, 61 sectors/track, 1034 cylinders
Units = cylinders of 3782 * 512 = 1936384 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sde1               1        1016     1920993+   6  FAT16
/dev/sde2              56        1021     1826706   83  Linux

** Delete both partitions (must NOT be mounted): **
Command (m for help): d
Partition number (1-4): 1

Command (m for help): d
Selected partition 2

** Recreate with same starting cylinder to preserve contents: **
Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1034, default 1): 1
Last cylinder, +cylinders or +size{K,M,G} (1-1034, default 1034): 55

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (56-1034, default 56):
Using default value 56
Last cylinder, +cylinders or +size{K,M,G} (56-1034, default 1034):
Using default value 1034

Command (m for help): p

Disk /dev/sde: 2003 MB, 2003828736 bytes
62 heads, 61 sectors/track, 1034 cylinders
Units = cylinders of 3782 * 512 = 1936384 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sde1               1          55      103974+  83  Linux
/dev/sde2              56        1034     1851289   83  Linux

** Change system type of partition 1 to '6' (FAT16): **
Command (m for help): t
Partition number (1-4): 1
Hex code (type L to list codes): 6

Command (m for help): p

Disk /dev/sde: 2003 MB, 2003828736 bytes
62 heads, 61 sectors/track, 1034 cylinders
Units = cylinders of 3782 * 512 = 1936384 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sde1               1          55      103974+   6  FAT16
/dev/sde2              56        1034     1851289   83  Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
sd 4:0:0:0: [sde] Assuming drive cache: write through
 sde: sde1 sde2

WARNING: If you have created or modified any DOS 6.x
partitions, please see the fdisk manual page for additional
information.
Syncing disks.

** Check new root partition. Note: you must run fsck -f first (resize2fs insists if you forget): **

root@debian:/mnt/sdd1# fsck -f /dev/sde2
fsck from util-linux-ng 2.17.2
e2fsck 1.41.12 (17-May-2010)
dreamv7_fs: recovering journal
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
dreamv7_fs: 26859/114240 files (0.1% non-contiguous), 185248/456676 blocks

root@debian:/mnt/sdd1# resize2fs /dev/sde2
resize2fs 1.41.12 (17-May-2010)
Resizing the filesystem on /dev/sde2 to 462822 (4k) blocks.
The filesystem on /dev/sde2 is now 462822 blocks long.

Correcting same image on Kingston chip identical to internal one:

Code:

root@debian:/mnt/sdd1# dd if=ex-factory-2011-04-24-microSD.img of=/dev/sde bs=1M
1876+0 records in
1876+0 records out
1967128576 bytes (2.0 GB) copied, 288.618 s, 6.8 MB/s
** Exact number of bytes copied **

** But partition table is awry **
root@debian:/mnt/sdd1# fdisk -l /dev/sde

Disk /dev/sde: 1967 MB, 1967128576 bytes
62 heads, 61 sectors/track, 1015 cylinders
Units = cylinders of 3782 * 512 = 1936384 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sde1               1        1016     1920993+   6  FAT16
Partition 1 has different physical/logical endings:
     phys=(54, 61, 61) logical=(1015, 54, 24)
/dev/sde2              56        1021     1826706   83  Linux

** So rewrite to feasible values **
root@debian:/mnt/sdd1# fdisk /dev/sde

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): u
Changing display/entry units to sectors

Command (m for help): u
Changing display/entry units to cylinders

Command (m for help): p

Disk /dev/sde: 1967 MB, 1967128576 bytes
62 heads, 61 sectors/track, 1015 cylinders
Units = cylinders of 3782 * 512 = 1936384 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sde1               1        1016     1920993+   6  FAT16
Partition 1 has different physical/logical endings:
     phys=(54, 61, 61) logical=(1015, 54, 24)
/dev/sde2              56        1021     1826706   83  Linux

Command (m for help): d
Partition number (1-4): 1

Command (m for help): d
Selected partition 2

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1015, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1015, default 1015): 55

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (56-1015, default 56):
Using default value 56
Last cylinder, +cylinders or +size{K,M,G} (56-1015, default 1015):
Using default value 1015

Command (m for help): p

Disk /dev/sde: 1967 MB, 1967128576 bytes
62 heads, 61 sectors/track, 1015 cylinders
Units = cylinders of 3782 * 512 = 1936384 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sde1               1          55      103974+  83  Linux
/dev/sde2              56        1015     1815360   83  Linux

Command (m for help): t
Partition number (1-4): 1
Hex code (type L to list codes): 6
Changed system type of partition 1 to 6 (FAT16)

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
sd 5:0:0:0: [sde] Assuming drive cache: write through
 sde: sde1 sde2

WARNING: If you have created or modified any DOS 6.x
partitions, please see the fdisk manual page for additional
information.
Syncing disks.

** Now we have a file system hanging over the end of the disc. **
** Fortunately the missing sectors contain no data: **
root@debian:/mnt/sdd1# fsck /dev/sde2
fsck from util-linux-ng 2.17.2
e2fsck 1.41.12 (17-May-2010)
dreamv7_fs: recovering journal
The filesystem size (according to the superblock) is 456676 blocks
The physical size of the device is 453840 blocks
Either the superblock or the partition table is likely to be corrupt!
Abort<y>? no

dreamv7_fs contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
dreamv7_fs: 26859/114240 files (0.1% non-contiguous), 185248/456676 blocks

** Now resize **
root@debian:/mnt/sdd1# resize2fs /dev/sde2
resize2fs 1.41.12 (17-May-2010)
Resizing the filesystem on /dev/sde2 to 453840 (4k) blocks.
The filesystem on /dev/sde2 is now 453840 blocks long.

root@debian:/mnt/sdd1# fsck -f /dev/sde2
                                                                                     
fsck from util-linux-ng 2.17.2
e2fsck 1.41.12 (17-May-2010)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
dreamv7_fs: 26859/114240 files (0.1% non-contiguous), 185248/453840 blocks

** Now it's OK **

[Confusticated]

gparted is not happy to manipulate this particular partition table with its errors.

Thanks Ralph, just not the answer I was hoping for

[Ralph Houston]

Thanks Ralph, just not the answer I was hoping for

I did my best with gparted.
For information:

• with the wrong first partition, gparted sees the disc as unallocated
• even if the first partition is a correct fat16, gparted (in Guiplug) cannot manipulate dos partitions
• with the second partition hanging off the end, gparted still sees the disc as unallocated
• if the second partition is the right length but the image in it is described as too long, gparted cannot check it

So I'm afraid it comes back to knife-and-fork as I described above!

[NewIT_James]

Confusticated,

My advice for this is to tar up the rootfs and copy the uImage then create some new correct partitions, then untar the rootfs and copy the uImage back.

This is essentially what I have done with the NewIT-fixed version.

NewITJames.

[Confusticated]

Hi James,
Thanks, its always nice to hear how you guys (New IT) do things...

[Ralph Houston]

Absolutely - so please, NewIT, do make the tar files available too - this is a far safer way of distributing new root systems, and takes account of varying sevice sizes and - I'll say it again - bad sectors.