Experimental: NFI2CF

  • Hi,


    i've wrote some small tool which turns an .nfi into a bootable CF image:


    http://dreamboxupdate.com/download/nfi2cf-0.1.tar.gz


    (well, not quite an image. you end up with an ext2 image and a directory to copy into a vfat partition. I did not want to create any dependency to hardware).


    It probably doesn't yet run on a dreambox because of memory, the jffs2 unpacker requires approx. 2x the size of the jffs2 image. This can probably be fixed. So this is a tool meant for the PC.


    After you wrote the resulting files onto a CF disk, you should be able to boot from it just by inserting it.


    There might be some smaller problems, so i regard this as experimental. (Don't use it unless you want to debug it!)


    In theory, it should also work for dm7020 images, but i've only tested it with dm7025 images. there IS endianess conversion support for jffs2, though. The 7020 secondstage probably doesn't support kernel command line parameters in the "autorun.bat", so you might have to patch the kernel. This will hopefully be fixed soon.

  • Thanks! I'm looking forward to having a closer look at this during the week-end.


    About making a FAT image in batch mode (from the README file): Look at the "mkdosboot" script in the mtools package.

  • I've used your program for a while now, with no problems whatsoever. It's quite memory-hungry, though, so I've now hacked the dump.cpp file to use a two-pass approach in order to bring down memory requirements. I also plugged a leak (memory from a calloc was never freed) and it now runs on my 7025 without swap.


    Only lightly tested! Any bugs are probably mine, I've never had any problems with the original.


    And yeah, since thowi hasn't set up a cross development system, I've also included a binary for the 7025 :smiling_face:


    EDIT: Just to tie up some loose ends, I've also attached tools to pack and unpack nfi files on a 7025. The tools are:
    - The dump program written by tmbinc and hacked by me
    - The mkfs.jffs2 program from mtd-utils
    - The buildimage program from DMM in OE
    - ripimage.py by tmbinc, horribly mutilated by me
    - a tiny shell script to run the commands to build an nfi-file


    I have some problems running the ripper without either stopping enigma2 or adding swap. Any good ideas on how to reduce the memory usage of dump even further?

  • Hello Noggie !


    Thanks a LOT, I'll try to include this to multiboot next wekend.


    Regarding your memory question:


    As I will run the tool when NOT booted from Flash, if I would have memory problems I would simply remove the /var tmpfs mount, then dump should have all the memory needed without any dirty tricks.


    So let's see where the journey brings us.


    PS: horribly mutilated - I liked that statement, but you probably don't have seen how I raped your old tool calling nfi_extract from multiboot 4.0



    #! /usr/bin/python
    import os, sys, getopt, struct, time
    from common_routines import *
    from machine_config import *


    #################################################################
    # This is GPL software.
    # Written by noggie
    #################################################################


    # This script will unpack a .nfi image for the 7025 on Flash
    # Usage: unpack_image < nfi-file


    # Start by extracting the boot and root partition from the nfi file
    # File on stdin
    extract_nfi(['/dev/null', '/dev/mtdblock/3', '/dev/null'])


    this is for stealing /boot from nfi file, this is for root:


    extract_nfi(['/dev/null', '/dev/null', '/dev/mtdblock/3'])


    This is what I call raping :winking_face:


    Ciao
    thowi

    Einmal editiert, zuletzt von thowi ()

  • OK, I did a quick test and it works really NICE !


    No memory problem when booted from CF card, and it can unpack directly to the /media/mb1...6 directories


    Thanx a lot AGAIN


    Ciao
    thowi

    3 Mal editiert, zuletzt von thowi ()

  • Multiboot Version 4.1 is now out for TESTING


    I changed a little bit the python code so that it unpacks the .jffs2 files
    on /var/tmp (=faster then on disk, and when extract fails/dump hangs you cann't corrup Harddisk filesystem so easy because you have to turn off the box) and so that it puts then the content directly to the choosen /media/mbX partition of multiboot and puts boot directly in /media/mbX/boot.


    Because of this dump gives directory exists errors during the unpack (but still does it's job), would it be possible to compile for me a multiboot special edition of dump so that it doesn't create the unpack directories or doesn't give a warning if they exist ?


    Even better would be if this special version of dump then would not extract the secondstage.gz file (or always hardcoded to /dev/null), because at the moment multiboot.sh simply removes it after extract to prevent building of nfi backup images.


    ciao
    thowi


    PS: And I couldn't resist to allow also copying nfi file to flash when beeing booted from an MB Partition instead of Flash.


    PPS: I only included dump and my changed mbripimage.py in multiboot to prevent problems and simplify support.

    6 Mal editiert, zuletzt von thowi ()

  • Regarding changes to dump to stop it displaying unwanted messages: Why not just shut it up:

    Code
    dump .... 2>/dev/null

    or more selectively

    Code
    dump .... 2| grep -v mkdir


    Regarding secondstage : Just revert to tmbinc's version in ripimage.py. Change the line

    Code
    output_names = {1: "secondstage.gz", 2: "boot", 3: "root"}

    to

    Code
    output_names = {2: "boot", 3: "root"}


    And sorry about not answering your questions in a previous posting. Rest assured that whenever I post something that is more or less original code, nothing pleases me more than someone else picking it up, changing it, and making it fit their purposes. The point is "sharing", not "copyrights", "all rights reserved" or "don't change my stuff". Only thing I dislike, is if someone changes code I have written, and passes if on without mensioning that that it was changed. I make enough bugs all by myself, thank-you-very-much, without having to take any blame for others'. :smiling_face:

  • redirecting to /dev/null is a little bit too hard for me, but surpressing just the mkdir with grep is a good idea, thanks


    Regarding disclaimer - i simply said ThankU in the readme.txt to you and tmbinc. My ego doesn't hang on things like this either, but you never know :winking_face:


    The whole multiboot plugin can be called a shared bugcollection, so I know what you mean stay with your own bugs.


    Regarding secondstage I'm not 100% sure, is'nt it sequentially reading trough the file - I'll check for the next version, and this is not really
    important stuff because I unpack at /var/tmp and remove afterwards,
    meaning no real harm can be done.


    Hope people have fun with Multiboot 4.1 and nfi support and that it now works better then the first Beta (well I learnt at least a lot, especially how it works)


    Thanks again
    thowi

    3 Mal editiert, zuletzt von thowi ()

  • Regarding your NFI extract memory Problem - only with the latest non standard images which get bigger and bigger Multiboot became troubles that dump ran out of memory so that the box hangs.


    Therefore I added the possibility to change the extract from /var/tmp to CF card or Harddisk and a Turbo Mode for the nfi copy in Mutliboot 4.2 - this simply stops enigma in the hot phase as suggested (thanks for the tip).


    Ciao
    thowi

    Einmal editiert, zuletzt von thowi ()

  • Hi Thowi/group,


    I managed to use multiboot without too much problems following your instructions (great stuff!).


    However tonight I'm completely stuck on the following error:
    ========================================
    Erasing ...
    Copying from /MB_Images/dreambox-image-dm7025-20060711153239.nfi to /media/mb2 ...
    ---------------------------------------------------------
    Extracting nfi now - takes about 2 min ...
    Ignore directory /media/mb2 exists errors !
    ---------------------------------------------------------
    machine type: dm7025
    extracting boot
    unpacking 2
    ok
    extracting root
    unpacking 3
    Traceback (most recent call last):
    File "/media/mb/mbripimage.py", line 57, in ?
    raise "unpacking jffs2 failed!"
    unpacking jffs2 failed!
    Labeling Partition ...
    Finished copying Partition


    ==========================


    Whatever image I try I keep getting this error... whatever image I try to flash, even after a reboot...


    What am I doing wrong? I did manage to flash some images before..!
    help help,
    thanks
    Leon

  • use a version greater then 4.1 of multiboot and enable the Turbo copy modus in install options which stopps enigma temporary to prevent memory problems during nfi extract.


    If this does not fix the problem you still can flash the image via webif and do a copy 2 0


    I still have 1 week of vacation, so support is limited at the moment ...


    ciao
    thowi

  • Hi Thowi,
    enjoy your holiday first before answering me :winking_face:


    I am using the most recent version of multiboot I could find:
    #
    # multiboot.sh from thowi
    #
    VERSION=42
    MULTIBOOTKIT=multiboot$VERSION.tar
    #


    aside: is there a webif for the 7025? (sorry I just got the box last saturday and reading all requests for webif in the forum... but seems gemini offers a webif plugin I just saw in the menu ... is that for real?).


    I debugged the whole thing, and starting to understand how the script works/how the db works :winking_face: / and how the .py script extracts the dump files.


    I exited the script after the extract. Then manually executed the dump command and did get no errors; echo $? returns '0'. So I don't understand why the system("dump..") returned an error status to the script...


    In the end the image didnt want to boot and I was in despair.


    Then booted another base enigma2 image and debugged the gemini image i copied previously; it turned out that a line in /etc/fstab was #'ed out (/boot partition) - this was already the case in my enigma image for some funny reason but did not cause any problem (not sure what caused it; some previous failed multiboot call?).


    I re-enabled the /boot in fstab and... it worked like a charm.


    I would suggest not to 'raise' an error but just print a warning when the dump returns some error status - maybe you are able to debug it further than I managed... after the raise error the multiboot script continues anyway, labelling the partition and printing 'ok' at the end :winking_face:


    enjoy your holiday - no stress my dreambox is like a dream today (and I'm finally able to see all dutch channels without my aston cam after installing gemini and setting the advanced lnb config properly now :winking_face: a *relieve* )

    Einmal editiert, zuletzt von floh ()

  • Version 5.0 is the latest version but besides some small bugfixes including error display of mbripimage.py and the image download freature it is identical to Version 4.2


    Yes multiboot disables /boot mount in fstab because root and / boot are together in a single CF or HDD Partition.


    There are some cases where this is a problem but in then booting from flash while doing the copy should overcome this as suggested in the long multiboot thread in its home board and where you also get help and the latest kit.


    But whole partititionhandlig will be re-written for version 6 to allow more then 3 images hence I do not want to spend time on your problem if you are happy at the moment


    regarding webif you can either wait until somebody comes out with a nice one or try out my VERY dirty one - search for notaufnahhme emergency record and get the er60.zip


    ciao
    thowi

    2 Mal editiert, zuletzt von thowi ()

  • Hi Thowi , thanks again,
    yes focus on version 6!
    I'll do some testing on that one once you get it done!
    regards - how's the holiday and how do you manage to answer everyday? :smiling_face:

  • Hi,


    sorry when asking for the obvious..


    Just to sum it up, what are the pros/cons of nfi2cf against multiboot?


    - only one image on CF
    - needs PC to convert, additional work
    - ??


    + easier to use on DB
    + no further installation on DB
    + ??


    Thanks for your comments!

  • +you really boot from cf !


    With "multiboot" you are booting the kernel and drivers from the flash memory and get linked after that to the filesystem on the cf/usb.

  • .. which would be helpful the flash is corrupted.


    Do you have a detailed (for Linux newbees like me) instruction list beyond the nfi2cf readme for installing and booting?
    Would be much appreciated!

  • Zitat

    Original von Seddi
    +you really boot from cf !


    With "multiboot" you are booting the kernel and drivers from the flash memory and get linked after that to the filesystem on the cf/usb.


    I think we have a misunderstanding here:


    First I'm also using the same tools from nfi2cf (just to the multiboot image directories, instead of a plain single second partition after the FAT Boot partition)


    Second I don't use the link approach as FWZ does (only Harddisk only Installations are raping the /boot in Flash to then boot from Harddisk - and maybe later from USB if it will work more stable). On a CF Installation the bootloader directly finds a bootable FAT partition on CF card and then boots from there !


    So seddi is wron, you even can delete the flash with Multiboot (copy 0 0 erases /boot and / ) and still boot from CF card as long as the scondstage bootloader in Flash is alive !


    You then can even copy back from an Multiboot Image to flash if you want (copy 0 ) from eitehr an nfi, a MB Image file or another partition.


    So basically mutliboot does the same then the manual booting on an Image on CF, with the only diference to have a nice userscript and gui and the possibility to have multiple independent Partitons with up to 12 Images.


    Hence multiboot is mainly targetted for the non-technical user who needs all the funtionality nicely wrapped, but it does nothing not available standalone or manually !


    So you could say I didn't invent the engine, I just built it together so
    that everybody finds the steering wheell and gear stick :winking_face:



    Ciao
    thowi

    3 Mal editiert, zuletzt von thowi ()

  • Thanks anyway to those who contribute their work!


    In multiboot, when can you select from which image to boot? Do you need the PC for changing the boot image?


    Maybe I will see when I install MB, just curious.