iPod Nano Resurrection

Working around bad blocks on an iPod Nano's flash disk

Twice now, my iPod Nano (first generation) has stopped working due apparently to failures of part of its flash memory. This page is about how I restored it to normal operation (with some loss of storage capacity).

I should mention that my iPod isn't used like most: it runs Rockbox exclusively, and has never played a note using the native Apple firmware, and never been connected to a machine running iTunes. The recovery method described here will probably work on any Nano perhaps even on other iPods or other mp3 players. Note that damage to a player's system partition can only be worked around if you have previously made a backup of that partition - which is a standard part of the Rockbox installation procedure.

First Failure - Symptoms

The first time my nano failed (in may of 2007), the first indication was an on-screen message "Rockbox error -5: starting original firmware."

Attaching it to a linux box and mounting it revealed strange-looking entries for the directorie containing much of the music, After going into recovery mode, I was able to recover many of the songs, but several were unreadable. Those I had to replace from other backups or re-rip from CD.

I also noticed many error messages in the system log:

May 15 01:19:22 helena kernel: sd 2:0:0:0: SCSI error: return code = 0x08000002
May 15 01:19:22 helena kernel: sdc: Current: sense key: Medium Error
May 15 01:19:22 helena kernel:     Additional sense: Unrecovered read error
May 15 01:19:22 helena kernel: Info fld=0x0
May 15 01:19:22 helena kernel: end_request: I/O error, dev sdc, sector 160737
May 15 01:19:22 helena kernel: printk: 185 messages suppressed.
May 15 01:19:22 helena kernel: Buffer I/O error on device sdc, logical block 160737
May 15 01:19:22 helena kernel: Buffer I/O error on device sdc, logical block 160738
May 15 01:19:22 helena kernel: Buffer I/O error on device sdc, logical block 160739
May 15 01:19:22 helena kernel: Buffer I/O error on device sdc, logical block 160740
May 15 01:19:22 helena kernel: Buffer I/O error on device sdc, logical block 160741
May 15 01:19:22 helena kernel: Buffer I/O error on device sdc, logical block 160742
May 15 01:19:22 helena kernel: Buffer I/O error on device sdc, logical block 160743
May 15 01:19:22 helena kernel: sd 2:0:0:0: SCSI error: return code = 0x08000002
May 15 01:19:22 helena kernel: sdc: Current: sense key: Medium Error
May 15 01:19:22 helena kernel:     Additional sense: Unrecovered read error
May 15 01:19:22 helena kernel: Info fld=0x0
May 15 01:19:22 helena kernel: end_request: I/O error, dev sdc, sector 160865

Using the ancient and verable utility http://en.wikipedia.org/wiki/Dd_(Unix)">dd, I read and copied to /dev/null various portions of the raw disk device to determine if the errors occured consistently in certain locations.

Reading the first (system) partition, we encounter no errors, so (this time) the iPod's operating system should be just fine:

# dd if=/dev/sd1 of=/dev/null
dd: opening `/dev/sd1': No such file or directory
[root@helena]# dd if=/dev/sdc1 of=/dev/null
160587+0 records in
160587+0 records out
82220544 bytes (82 MB) copied, 16.6267 seconds, 4.9 MB/s

But reading the second partition causes error messages (both from dd and in the kernel log):

[root@helena tell]# dd if=/dev/sdc2 of=/dev/null skip=200
dd: reading `/dev/sdc2': Input/output error
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.164854 seconds, 0.0 kB/s

[root@helena tell]# dd if=/dev/sdc2 of=/dev/null skip=2000
dd: reading `/dev/sdc2': Input/output error
744+0 records in
744+0 records out
380928 bytes (381 kB) copied, 0.22349 seconds, 1.7 MB/s

Using the skip option to dd, we can read different portions of the flash "disk" to determine which are bad. Here we find that if we start reading at block 20000, we can read the entire rest of the partition without errors:

[root@helena tell]# dd if=/dev/sdc2 of=/dev/null skip=20000
(still running after many minutes)
7803649+0 records in
7803649+0 records out
3995468288 bytes (4.0 GB) copied, 810.375 seconds, 4.9 MB/s

Skipping 20,000 512-byte blocks is only 10MB - at most a song or two.

The fix

Now that we know where the bad blocks are, we simply need to make the iPod ignore them. My chosen solution is to simply change the partition table, moving the data partition to start after the bad blocks (and shrinking it slightly.

Original partition table

The original partition table of my nano looked like this (using the Linux fdisk utility)

# fdisk -l /dev/sdc

Disk /dev/sdc: 4095 MB, 4095737344 bytes
255 heads, 63 sectors/track, 497 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1          10       80293+   0  Empty
/dev/sdc2              11         497     3911824+   b  W95 FAT32

Updated Partition Table

This page is not an fdisk tutorial; if you're not comfortable using fdisk, use some other tool or get help.

WARNING: Running fdisk on a dead ipod is unlikely to make it worse, but pointing fdisk at the wrong disk on your system can easily wreck your desktop (or laptop) computer. Use appropriate caution.

This is the new partition table I wound up with. I deleted the old parition 2, and created a new one, leaving 5 "cylinders" of unused space.

# fdisk -l /dev/sdc

Disk /dev/sdc: 4095 MB, 4095737344 bytes
255 heads, 63 sectors/track, 497 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1          10       80293+   0  Empty
/dev/sdc2              16         497     3871665    b  W95 FAT32

Lastly, we need to create a new, empty filesystem on the new parition.

Another WARNING: Running mkdosfs on the wrong device can easily wreck the filesystems on your desktop or laptop computer.

[root@helena ipod]# mkdosfs -F32 -n IPod -v /dev/sdc2
mkdosfs 2.11 (12 Mar 2005)
/dev/sdc2 has 255 heads and 63 sectors per track,
logical sector size is 512,
using 0xf8 media descriptor, with 7743330 sectors;
file system has 2 32-bit FATs and 8 sectors per cluster.
FAT size is 7548 sectors, and provides 966025 clusters.
Volume ID is 4653c138, volume label IPod       .

Finishing up

All that's left to do is to copy your music back onto the new, empty filesystem. And of course to copy back the rockbox.ipod file and .rockbox directories.

Second Resurrection

More recently, my ipod failed with slightly worse symptopms: Upon reboot, it only displays the "folder with exclamation mark" iPod icon.

After forcing the ipod into disk mode, it was recognized my my linux box. This time I found that the flash disk errors are on the first (system) parition (note the very low block numbers):

Jul  3 20:24:02 viola kernel: Buffer I/O error on device sdd1, logical block 33
Jul  3 20:24:02 viola kernel: Buffer I/O error on device sdd1, logical block 34
Jul  3 20:24:02 viola kernel: Buffer I/O error on device sdd1, logical block 35

recovery

The recovery is basicly the same as before, except that we have to re-create both partitions, and leave the empty, unused space at the beginning:

# fdisk -l /dev/sdd

Disk /dev/sdd: 4095 MB, 4095737344 bytes
255 heads, 63 sectors/track, 497 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x20202020

   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1              20          29       80325    0  Empty
/dev/sdd2              30         497     3759210    b  W95 FAT32

In addition to using mkdosfs to create a new filesystem on the second partition, I had to use dd to reinstall the iPod firmware image. If I didn't have a backup made while installing the Rockbox boot loader originaly, I would have been out of luck. The command to reinstall the system partition is the same one used while installing rockbox the first time:

# dd if=rockboot.bin of=/dev/sdd1

Final Notes

This kind of recovery can certainly be done with other tools; these are simply what I'm most familiar with. The key is that this flash-based player uses nearly the same sort of disk partitioning and filesystem conventions that PCs have used for decades.

Note on Rockbox installation and backup

When I first installed rockbox, I followed the "old" instructions here It is unclear from the newer rockbox instructions if the new, friendlier ipodpatcher tool creates a backup file and leaves it behind on your desktop computer for later recovery.

-Steve