Beiträge von noggie

    Booting from CF is supported by DMM, no extra software needed. What you need:
    1) A CF that has at least two partitions, one for boot and one for root
    2) The boot partition must be the first partition on the CF, and must be formatted as a FAT partition.
    3) The root partition can not be FAT, but could be e.g. ext3
    4) A file on the the boot partition called "autorun.bat" that loads the boot screen image and the kernel. Kernel options are allowed, and must be used to change the root partition from flash (a default that is compiled into the kernel) to the root partition on CF.
    5) Obviously all the needed boot files on the boot partition, and all the rest of the files in the image on the root partition.


    Here's a small script (to be run on the 7025) to partition and format the CF. Change to fit your own needs.


    (Please excuse my python code, I'm still learning...)
    The script assumes that you have the dosfstools package installed, it contains the "mkdosfs" program. You will also need a directory for the root partition on CF. The script assumes that this directory is /media/cf2


    A simple way to make an image on CF is to mount the CF on your Linux PC and copy the files from your OE build environment. You could also make a copy of the existing files on flash, so that the CF contains a backup. The following script could perhaps be a starting point:


    You could also unpack nfi files on your Linux PC if you have the required mtd and jffs2 modules for your kernel. Doing the same thing on the 7025 is possible, but it requires some extensive kernel patching, and you probably don't want to do that. I've got my own scripts to unpack nfi-files on my PC, but they are quite dependant on my own Linux setup and probably won't run on any other machine without some modifications. I don't have a problem posting them, but I kinda doubt that anybody would find them very interesting. Let me know if I'm wrong.

    Zitat

    Original von Lior
    devfs is disabled and being removed since Linux 2.6.13, dm7025 still has the version 2.6.12 with devfs and after 2.6.13-rc3, in MTD devfs is replaced with udev like lot of drivers because devfs is abandoned by its maintainer.


    Here's somebody else's view on sticking to devfs: http://lists.infradead.org/pip…d/2005-August/013385.html
    Of course, I only saw this one after having experienced the same problems, and it took me more than two hours to find out that missing devfs support was the problem. Oh, the joys of kernel hacking.... :smiling_face:


    Zitat

    Original von Lior
    if you have nothing to do other than fooling around with MTD, then try to get work YAFFS on DM7025. I've been told that YAFFS is better than JFFS2 for NAND, the only disadvantage is that YAFFS does not support compression which is not important because we all have large CFs :winking_face:


    :smiling_face:
    Actually, my fooling around with MTD wasn't only for the fun of it. I only got involved because my script to unpack an NFI to a CF caused a kernel oops. Now that problem is fixed, I hope, and I can get on with it.


    But, now that I have a recent version of the mtd up and running, there are a few paths for experiments that open up:
    - YAFFS
    - maybe JFFS3
    - summary information (see http://www.inf.u-szeged.hu/jffs2/mount.php)
    YAFFS is maybe the least interesting of these paths. It only runs on NAND (not on CF, it seems) and running uncompressed on the tiny internal flash will probably hurt too much. Using summary information looks very interesting, though. It currently takes 10+ seconds to mount the root file system, so this is a major contribution to the slow boot times that we are experiencing.

    Going back to the current CVS version of mtd+jffs2, I found out why the resulting kernel didn't mount root. Quite simple really. Devfs is deprecated in linux 2.6, and in the current CVS of mtd, support for devfs has been dropped. By adding devfs support again, + some minor fixes due to incomplete backwards compatibility with 2.6.12, I now have a kernel that seems to work.
    - It compiles
    - It links
    - It boots from flash
    What more could you want? :smiling_face:


    I'll let it run for some days to see if there are any problems. First tests looks promising. No oops anymore when unmounting a blkmtd-device.


    If there's any interest, I guess I could tidy up the patches a bit and post them here. Let me know.

    Just in case anybody cares...


    I've now tried a few more approaches, but with no success so far. As reported earlier, blkmtd.ko works until unmounting the file system, when I get a kernel oops, while block2mtd.ko oops when loading.


    I've since tried the blkmtd.ko module, whith the same result as the blkmtd.ko. I've also tried the current mtd+jffs2 kernel from CVS, but that kernel failed to mount the root file system from flash. A bit more conservative approach using only the latest mtd sources, while leaving the jffs2 code unchanged, also failed to mount the root file system. Lots of work, no results... :frowning_face:


    I have a theory about the cause of the problem. I think it may be a spinwheel lock that is used without being initialized. It's used in "wbuf.c" in the jffs2 code, and it seems to be properly initialized only for NAND devices.


    A few questions to those who knows something about this:
    1) Does my theory sound reasonable? Is anybody running 2.6.12 on a PC, where blkmtd.ko works?
    2) Could it be that my failure to mount the existing root partition in flash (using the latest jffs2+mtd sources) is caused by an incompatibility between the year-old jffs2 code in 2.6.12 and the current CVS? In which case, does anybody have a good guess what would be a suitable date for checking out a version that fixes some bugs, but is still compatible with the old layout?


    BTW: Recent (since last fall) versions of jffs2 apparantly includes some code to speed up the initial scan of the file system, which should lead to faster boot times on the dreambox. Wouldn't it be great if DMM adopted this code and reduced the boot time by a few seconds? (Hint, hint. Nudge, nudge! :smiling_face: )

    Thanks. That's more or less what I've though I might have to do, and what I've hoped to avoid. :smiling_face:


    Already tried the block2mtd. Got an oops when loading the module.

    I'm playing around with a script to unpack images to the CF, and I get the following kernel oops:


    Basically, the program will


    a) Unpack the nfi to make boot.jjfs2 and root.jffs2 filesystem images
    b) Use the loopback device to make the image look like a block device
    d) Use the blkmtd kernel module to add a block device as an MTD
    c) Mount the simulated MTD as jffs2
    d) Copy out the files
    e) umount
    f) remove the kernel module
    g) delete the loopback


    The oops happens at step e), when I try to unmount the jffs2 file system.


    And yeah, it works perfectly on my PC, kernel 2.6.13.3.


    Anybody?


    And BTW, the loopback device (/dev/loop/*) isn't configured in the kernel. Could this be added, please? Also, I would be grateful if the standard config included the blkmtd and/or the block2mtd as modules...

    I'm one of those who like "vi", and just hate the stunted version that's provided for in busybox. Using OpenEmbedded, the solution is simple: just build "vim" and install it. This worked just fine on the powerpc version of OE for the 7020, but failed during configuration on the 7025. By applying the following patch, "vim" will build on the 7025 also:


    A better patch would probably be to fix the configure script for vim, but that's a lot more work :smiling_face:

    1) I'm not sure if it's good or bad, but how about making a 90% solution? Given that a solution that works in all cases is too much work to be worth it, would it be possible to make a simpler approach, and just give up if certain conditions were not satisfied? What would those conditions be? That the flash doesn't contain enough erased blocks?
    2) Works perfectly. Thanks a lot! BTW, to me it seems like running ext3 on CF is actually a little bit faster than running jffs2 on internal flash. Can anyone confirm or deny this?
    3) I'll keep my eyes open. Personally, now that I have a CF that works so good, USB isn't that important to me.


    I'm looking forward to seing the source code for the second stage loader being re-released. Hope I'm not insulting anyone by saying so, but the amount of documentation for the second stage loader is less than impressing. You really need the sources to make the most of it.

    Thanks, tmbinc, for your explanation.


    I've been trying to come up with a solution for dnuos' flash, feeling bad for f*cking up his box. He's been incredibly sporty about it all (we've been in touch on PM) but I still would very much like to see his flash get back to what it used to be.


    I think I have come up with a way to fix it, but this time around I would like get some help verifying that my thinking is sound before asking anyone else to trust me to know what I'm talking about. Please, tmbinc, could you take a look at the following argument and see if you can spot any weaknesses?


    1. Before restoring the backup, dnuos' flash showed no sign of bad blocks. The scan done by the backup program showed no bad blocks, and the "prova" utility showed no bad blocks either. Conclusion: The flash is 100% good, has no actual bad blocks, and the bad blocks that suddenly appeared are blocks that have been incorrectly marked as bad.


    2. "Fixing" his bad blocks is a case of erasing the bad block flag for all blocks. In his case, there's no need to distiguish between really bad blocks and apparantly bad blocks, since his flash was without any bad blocks at all before doing that fatal restore.


    3. The kernel checks that no user program tries to erase a bad block. This takes place in the nand_base.c file in drivers/mtd/nand. Although disabling this check would be a very bad idea in the normal case, it should be possible using the following diff


    4. The flash_eraseall utility in mtd-utils also contains code to skip bad blocks. Again, removing this code is not a good idea in the normal case, but it can be done using the following diff


    5. By booting a kernel containing the above patch, and running off a root file system different from the internal flash, erasing the root partition with the modified flash_eraseall would also clear all bad block flags. In the normal case, doing this would be incredibly stupid, but in this particular case the bad block flags are reset to the correct value. BTW, only the root partition need to be fixed, since all his incorrectly flagged bad blocks are on the root partition.


    6. A reboot and flashing a new image concludes the fixup operation.


    Can you see anything wrong with this? I've tested it as far as I can. I gave my own 7025 flash a couple of fake bad blocks (small test program, source code not provided, using the "MEMSETBADBLOCK" ioctl to set the bad block flag). The "bad" blocks were recognized as bad using all the utilities I had available. I then used a kernel patched as I described above, and a hacked flash_eraseall, booted off the CF, and successfully reset the bad block flag. But my box is a 7025, not a 7020 as dnous', so I'm not 100% certain that it will also work on the 7020.


    Any comments, anyone?

    Zitat

    Original von dnuos
    i already flashed again through web-interface and lan cable... and the 39 bad blocks are always there...
    any idea if/how could i solve this? :grinning_squinting_face:


    Wow! And probably really sorry!! For your flash to develop 39 bad blocks all of a sudden could of course be just a coincidence, but I have major problems believing it. Sounds more like problems caused by the failed restore of the backup. Maybe the oob data is wrong, somehow, and interpreted as bad blocks. Only good news in this sorry story, is that your flash is probably all OK, so if a way to just reset everything can be found, you should be back on track.


    I'm thinking that maybe flashing an empty image that uses all the flash could help, but I don't know of any such image. Anybody else knows of such a beast?


    If you can't find a tested version of such an image, I've made my own attempt at making one. Completely untested, of course. It contains version 35 of the second stage loader, and then lots and lots of 0xFF. It's made using the standard "buildimage" utility, so the oob data should be computed correctly. Up to you, of course, if you want to try it.


    One final thing. I notice that the image you have restored has a completely different partition table. My memory is not reliable, but I seem to recall that the layout was changed (even on the 7020) when work started internally in DMM preparing the 7025 image, since gcc produces less dense code for MIPS processors than it does for the powerpc. It's just a shot in the dark, but maybe you could try different images. Different versions of the second stage loader may make a difference, although DMM alone knows what changes they have made lately to that code. In the newest dreambox-buildimage, a comment says

    Code
    // pre-35 have old layout

    , but the latest available sources (that I'm aware of) is for version 28.


    EDIT: Just saw wolpi's contribution (thanks wolpi!), and also reading through the old secondstage sources. Writing a blank image probably won't help, since the flashing code won't reset the badblock flag. Back to the thinking box trying to come up with a solution. Good news is still that your flash is probably OK, just need to find a way to clear that badblock flag.

    Zitat

    Original von dnuos


    restored the backup via lan (through web interface).. my dm7020 is not starting anymore... black screen and the "II" symbol on the lcd display, even switching it on with the down button pressed.


    i go to find out where i put my serial cable... :grinning_squinting_face:


    Sorry! To me, the worst part is that I have no idea why it doesn't work. If you have to pull out your serial cable anyway, could you please listen in to the failed boot using a terminal emulator? Maybe there's a clue there....


    Anyway, I'm removing the binaries for the 7020 until the bug has been found.


    Interesting. I hadn't noticed that before in the kernel sources. But looking now, it's quite clear that the boot partition on the 7020 is set up to be one block short. In the 7020 partition table:

    Code
    {
                    .name   = "boot partition",
                    .offset = 256*1024,
                    .size   = (4*1024-256-16)*1024
            },


    On the 7025, this is:

    Code
    {
                    .name   = "boot partition",
                    .offset = 256*1024,
                    .size   = (4*1024-256)*1024
            },


    I don't know why, but I wouldn't be surprised if it had something to do with reserving a block in case there's a bad block in the loader partition. The questions is then why this isn't done on the 7025. Would anyone from DMM care to comment?


    The good news is that the program actually handles this OK, since it's using the reported size from the kernel. The first message in this thread should then read:

    Zitat

    The program, as it stands, does not try to skip empty blocks, so the resulting nfi file will be quite big. My full size files ends up at 34603072 bytes, expect 34586144 bytes for the 7020, and each bad block will subtract 16896 bytes from the total length.


    And yes, there is one more test, if you dare :smiling_face: Restore the backup and see if it works. Worst case is that you will have lost the contents on the flash and have to start all over again, maybe even having to flash using the serial cable. Best case is that it seems to be OK, and you will have to run it for a few days to make sure that nothing was broken.

    Everybody! Please note that the files in the first message of this thread have been updated. I saw that a line of code (I swear that an earlier version had it in! Oh well, sh*t happens) had dropped out, causing the wrong block to be skipped if was bad. I also added some debug messages (run with "-v" option) to print out my understanding of the flash.


    dnuos: The size you report looks OK, if (as dcdead pointed out) the flash has a bad block. But see above, yesterday's version actually dumped out the wrong block in this case. About the discrepancy with "prova": I have no comment. I can't learn anything from it, as there's no source code and I can't even run the binary. Try the "-v" option in the new version, and that will at least tell you what blocks the backup program believes are bad.

    This is my first cut at a tiny backup program for the internal flash on the 7025 (and maybe also the 7020). Please find attached the sources + bb-file, and compiled code in ipkg format for both the 7025 and 7020.


    To compile the sources, unpack the tar file in a subdirectory of your packages directory. I use packages/dreambox, but I believe other directories should also work. Then go to your build directory and do "bitbake dreambox-backup".


    Very lightly tested on the 7025, not at all tested on the 7020.


    IMPORTANT! I'm fortunate enough to have a flash with no bad blocks. Although the backup program in theory should be able to handle bad blocks, THIS HAS NOT BEEN TESTED.


    To run the program, do

    Code
    dreambox-backup dm7025 > backup.nfi


    or, for the 7020,

    Code
    dreambox-backup dm7020 > backup.nfi


    Make sure that you don't try to store the backup on the flash. That won't work. Store it to some kind of external memory, hdd, cf or usb will do nicely.


    The program, as it stands, does not try to skip empty blocks, so the resulting nfi file will be quite big. My full size files end up at 34603072 bytes, expect 32 bytes less for the 7020, and each bad block will subtract 16896 bytes (I think) from the total length.


    Feedback is welcome, especially if it works :smiling_face:
    In particular feedback from those of you with bad blocks, and/or trying it out on the 7020.


    Update: Discovered a typo. An lseek had suddenly disappeared. Also added a "-v" option, to print out verbose flash information (bad block count, etc.)


    Another update: Binaries for 7020 removed. Apparantly it doesn't work.


    One more: Removing binaries for 7025 also. From tmbinc's comments, the current version is not safe.

    Nothing?


    Would it help if I explained what I want them for?


    I have three projects in mind:
    1) Changing the "backup" code so that a regular nfi-file is produced. As it is, you can produce a backup, but there is no way to restore that file...
    2) Changing the code that reads the autoexec.bat/autorun.bat files, so that kernel parameters can be set in the text file. This would make the second-stage loader far more useful when you want to boot from other sources than flash. I'm booting from CF at the moment (on a 7025), but I did it by changing the kernel configuration, and that method's a little bit too painful.
    3) (Low priority, mostly because it's a lot of work...) Add code to support loading the kernel from USB.

    If you are tired of the long initial delay when you use ssh to connect to your dm7025, add the name and IP adress of the connecting host to the dreambox' /etc/hosts file. Seems like dropbear is doing a reverse IP lookup when a client connects, and the entry in /etc/hosts stops dropbear from trying to connect to the name server.

    Suggest you delete your "openembedded" directory and check it out from scratch. I'm not using the Makefile, but I think the following should work (from your dm7025 directory):


    % rm -rf openembedded
    % make -f Makefile-opendreambox image


    (and by the way, it looks like you are logged on as "root". This may cause problems in the final build of the image)