Backup program for NAND-flash

1. offizieller Beitrag

    This is my first cut at a tiny backup program for the internal flash on the 7025 (and maybe also the 7020). Please find attached the sources + bb-file, and compiled code in ipkg format for both the 7025 and 7020.


    To compile the sources, unpack the tar file in a subdirectory of your packages directory. I use packages/dreambox, but I believe other directories should also work. Then go to your build directory and do "bitbake dreambox-backup".


    Very lightly tested on the 7025, not at all tested on the 7020.


    IMPORTANT! I'm fortunate enough to have a flash with no bad blocks. Although the backup program in theory should be able to handle bad blocks, THIS HAS NOT BEEN TESTED.


    To run the program, do

    Code
    dreambox-backup dm7025 > backup.nfi


    or, for the 7020,

    Code
    dreambox-backup dm7020 > backup.nfi


    Make sure that you don't try to store the backup on the flash. That won't work. Store it to some kind of external memory, hdd, cf or usb will do nicely.


    The program, as it stands, does not try to skip empty blocks, so the resulting nfi file will be quite big. My full size files end up at 34603072 bytes, expect 32 bytes less for the 7020, and each bad block will subtract 16896 bytes (I think) from the total length.


    Feedback is welcome, especially if it works :smiling_face:
    In particular feedback from those of you with bad blocks, and/or trying it out on the 7020.


    Update: Discovered a typo. An lseek had suddenly disappeared. Also added a "-v" option, to print out verbose flash information (bad block count, etc.)


    Another update: Binaries for 7020 removed. Apparantly it doesn't work.


    One more: Removing binaries for 7025 also. From tmbinc's comments, the current version is not safe.

    thanks noggie
    just tried the ipk on my 7020, with a lightly modified cvs in flash from 06/02/2006


    seems to be working :grinning_squinting_face: :


    root@dm7020:/media/cf# ls -all
    drwxr-xr-x 5 root root 1024 Feb 11 22:29 .
    drwxr-xr-x 17 root root 0 Feb 7 21:23 ..
    -rw-r--r-- 1 root root 34586144 Feb 11 22:30 backup.nfi
    drwxr-xr-x 2 root root 1024 Jan 1 1970 images
    drwx------ 2 root root 12288 Sep 18 14:55 lost+found
    drwxr-xr-x 3 root root 1024 Oct 7 21:40 usr
    root@dm7020:/media/cf#


    is it normal the backup size is 34 Mb?

    Einmal editiert, zuletzt von dnuos ()

    ooops :grinning_squinting_face:


    hadn't paid attention to this:


    Zitat


    The program, as it stands, does not try to skip empty blocks, so the resulting nfi file will be quite big. My full size files end up at 34603072 bytes, expect 32 bytes less for the 7020, and each bad block will subtract 16896 bytes (I think) from the total length.


    that's ok, then!


    I'm gonna check if my NAND has bad blocks

    that's the utility i used, musicbob. :grinning_squinting_face:


    let's wait for noggie's reply to understand better if the size of the .nfi backup is correct.

    Everybody! Please note that the files in the first message of this thread have been updated. I saw that a line of code (I swear that an earlier version had it in! Oh well, sh*t happens) had dropped out, causing the wrong block to be skipped if was bad. I also added some debug messages (run with "-v" option) to print out my understanding of the flash.


    dnuos: The size you report looks OK, if (as dcdead pointed out) the flash has a bad block. But see above, yesterday's version actually dumped out the wrong block in this case. About the discrepancy with "prova": I have no comment. I can't learn anything from it, as there's no source code and I can't even run the binary. Try the "-v" option in the new version, and that will at least tell you what blocks the backup program believes are bad.

    Zitat

    Original von musicbob
    Here's the little utility to test bad blcks.
    I compiled it static, so it's image-aware


    FYI my 7020 has a bad block :grinning_squinting_face:


    Code
    root@dm7020:/var/tmp# prova
*** bad block list: 01b8c000 (1 blocks, 16 kB total)
root@dm7020:/var/tmp#


    Same to my 7020
    root@dm7020:/var/tmp# ./prova
    *** bad block list: 00660000 (1 blocks, 16 kB total)
    root@dm7020:/var/tmp#


    Is possible to recover this block ..or is normal? :winking_face_with_tongue:


    Same test utility dreambox-backup
    root@dm7020:/var/tmp# ./dreambox-backup dm7020 > /hdd/backup.nfi
    root@dm7020:/var/tmp#
    root@dm7020:/var/tmp# ls -la /hdd/
    drwxrwxrwx 7 root root 4096 Feb 12 13:54 .
    drwxrwxrwx 11 root root 1024 Feb 6 09:04 ..
    -rw-r--r-- 1 root root 34569248 Feb 12 13:55 backup.nfi
    root@dm7020:/var/tmp#

    noggie


    this is the report with the verbose option :


    and the size of the new backup created is the same as yesterday:

    Zitat


    root@dm7020:/media/cf# ls -all
    -rw-r--r-- 1 root root 34586144 Feb 12 15:39 backup.nfi


    and ngc1927's NAND flash backup (with 1 bad block) filesize is 34569248, as to say -32 -2blocks.

    does this mean 1 block is always skipped creating the backup?


    if you need other tests... :winking_face:

    2 Mal editiert, zuletzt von dnuos ()


    Interesting. I hadn't noticed that before in the kernel sources. But looking now, it's quite clear that the boot partition on the 7020 is set up to be one block short. In the 7020 partition table:

    Code
    {
                .name   = "boot partition",
                .offset = 256*1024,
                .size   = (4*1024-256-16)*1024
        },


    On the 7025, this is:

    Code
    {
                .name   = "boot partition",
                .offset = 256*1024,
                .size   = (4*1024-256)*1024
        },


    I don't know why, but I wouldn't be surprised if it had something to do with reserving a block in case there's a bad block in the loader partition. The questions is then why this isn't done on the 7025. Would anyone from DMM care to comment?


    The good news is that the program actually handles this OK, since it's using the reported size from the kernel. The first message in this thread should then read:

    Zitat

    The program, as it stands, does not try to skip empty blocks, so the resulting nfi file will be quite big. My full size files ends up at 34603072 bytes, expect 34586144 bytes for the 7020, and each bad block will subtract 16896 bytes from the total length.


    And yes, there is one more test, if you dare :smiling_face: Restore the backup and see if it works. Worst case is that you will have lost the contents on the flash and have to start all over again, maybe even having to flash using the serial cable. Best case is that it seems to be OK, and you will have to run it for a few days to make sure that nothing was broken.

    Zitat


    And yes, there is one more test, if you dare smile Restore the backup and see if it works. Worst case is that you will have lost the contents on the flash and have to start all over again, maybe even having to flash using the serial cable. Best case is that it seems to be OK, and you will have to run it for a few days to make sure that nothing was broken


    that's what you were not supposed to ask :smiling_face_with_horns: just joking :grinning_squinting_face:


    restored the backup via lan (through web interface).. my dm7020 is not starting anymore... black screen and the "II" symbol on the lcd display, even switching it on with the down button pressed.


    i go to find out where i put my serial cable... :grinning_squinting_face:

    Zitat

    Original von dnuos


    restored the backup via lan (through web interface).. my dm7020 is not starting anymore... black screen and the "II" symbol on the lcd display, even switching it on with the down button pressed.


    i go to find out where i put my serial cable... :grinning_squinting_face:


    Sorry! To me, the worst part is that I have no idea why it doesn't work. If you have to pull out your serial cable anyway, could you please listen in to the failed boot using a terminal emulator? Maybe there's a clue there....


    Anyway, I'm removing the binaries for the 7020 until the bug has been found.

    oooops :loudly_crying_face:


    flashed via dreamup and serial cable, i wanted to try again to create the backup and now...



    Zitat


    root@dm7020:/var/tmp# prova
    *** bad block list: 012c8000 012cc000 012d0000 012d4000 012d8000 012dc000 012e0000 012e4000 01384000 01388000 0138c000 01390000 01394000 01398000 0139c000 013a0000 01538000 0153c000 01540000 01544000 01548000 0154c000 01550000 01764000 01768000 0176c000 01770000 01774000 01778000 0177c000 01780000 01820000 01824000 01828000 0182c000 01830000 01834000 01838000 0183c000 (39 blocks, 624 kB total)
    root@dm7020:/var/tmp#


    i already flashed again through web-interface and lan cable... and the 39 bad blocks are always there...
    any idea if/how could i solve this? :grinning_squinting_face:

    2 Mal editiert, zuletzt von dnuos ()

    dnous, it really doesn't look good ...


    I found the following part in one of the datasheets for nand flashes:


    Zitat

    All device locations are erased(FFh) except locations where the initial invalid block(s) information is written prior to shipping. The
    initial invalid block(s) status is defined by the 6th byte in the spare area. Samsung makes sure that either the 1st or 2nd page of
    every initial invalid block has non-FFh data at the column address of 517. Since the initial invalid block information is also erasable
    in most cases, it is impossible to recover the information once it has been erased. Therefore, the system must be able to recognize
    the initial invalid block(s) based on the initial invalid block information and create the initial invalid block table via the following suggested
    flow chart(Figure 3). Any intentional erasure of the initial invalid block information is prohibited.


    (its from a Samsung K9F2808U0C NAND Flash as an example)


    I haven't had a look at the code or so, but are you shure you haven't damaged anything by by writing the nfi? I'm not experienced enough how the nfi works and which parts are rewritten, but i'm sure an easy memcopy won't work at all - because I'm sure they are using a Flash File System.


    But maybe I'm also totally wrong ... :confused_face:


    Regards
    Wolpi