Warning! The following is pure speculation from my side. The secondstage bootloader is undocumented and no sources are available. See Second stage loader for my repeated attempts to get DMM to release the sources. Despite the promise by tmbinc in February that the sources will be commited "soon", there's been nothing so far.

I'm basing my answers on the last public version of the secondstage for the 7020, build 28. Although those sources are also offline, I still have a copy.

Zitat

Original von thowi
Could therefore somebody give me a hint where and how it actually stores its config when you do a save and reboot within the bootloader ?

The configuration used to be stored in NVRAM on (or close to) the front processor. I would be surprised if there are major differences between the 7020 FP and the 7025 FP. As far as I can remember there also used to be some crypto stuff going on to access the config data... Unless DMM will release some API for it, it could be a bit tricky to write a driver to access and manipulate this data.

Zitat

Original von Ticalian
Vielleicht hat ja einer eine Idee.

I tried to reproduce your error, but on my system the error came later, during the install stage. I would guess that I came a little bit further because I'm using the .../site/mipsel-linux file from the dev branch of OpenWorks, the one in the dreambox branch is quite outdated. See the end of DM7025 CVS Build auf AMD64 (SuSE) for patches.

I also see some small differences in the gcc_4.1.0.bb file between the dev and dreambox branches (some more patches are applied in the dev branch), but I haven't tested to see if these are significant or not. You could try to copy the dev branch files to the dreambox branch and see if they make a difference.

I'm not particularly familiar with the scsi code, but I think the following comment in kernel/scis/sd.c might provide an explanation:

/*
 * Device no to disk mapping:
 *
 *       major         disc2     disc  p1
 *   |............|.............|....|....| <- dev_t
 *    31        20 19          8 7  4 3  0
 *
 * Inside a major, we have 16k disks, however mapped non-
 * contiguously. The first 16 disks are for major0, the next
 * ones with major1, ... Disk 256 is for major0 again, disk 272
 * for major1, ...
 * As we stay compatible with our numbering scheme, we can reuse
 * the well-know SCSI majors 8, 65--71, 136--143.
 */

sd.c is "SCSI disk", and USB sticks are using the SCSI code.

As you can see, 4 bits are used for the partition number, i.e. 16 partitions max. Someone probably thinks that 16 partitions on each disk are enough for anyone, and I can't really disagree....

Zitat

Original von thowi

Kernel module doesn't load - Device or resource busy error (-1) - But I have to try with a current CSV image first.

ipk file contains autoumoun finary too - but this seems to be still compiled for Intel - typical Synatx error ( expected

Hmmm. I started by checking the ipk. You're quite right, the binary is for my AMD64 build machine, not for mipsel. I then started looking at the log files produced during compilation, and found that OpenEmbedded had run the configuration correctly enough, but even so the resulting Makefiles were not set up for proper cross-compilation. It could of course be that I have something wrong in my OpenEmbedded setup, but I don't think so. To me it looks as if the autoconfig files for autofs are broken as far as cross-compilation is concerned.

Fixing that would take some effort, and I'm afraid that I feel that I've already done my share of programming this week (at work). Sorry! Hopefully someone else can step in and compile the beast.

Since the user-land daemon doesn't work, I never even tried the kernel module. On it's own it's useless, I'm afraid.

------------ UPDATE -----------
Found a recent update to the dev branch of OpenEmbedded that supplied the needed patches to cross-compile autofs. At least the binary is for the correct architecture now, but I haven't tested it further than that. Please see attachment.

About failing to load the kernel module, I tried it on my system. I'm running the kernel from the latest official DMM image. I'm also running slightly older DMM kernel modules since the bugs introduced in the last ones are more serious on my system than the bugs they fixed. I don't think my older kernel modules should matter, and autofs4.ko loaded without errors.

If getting a copy of automount keeps you from boredom during the week-end, then I feel it's almost a duty to keep up your supply of new toys...

Although it kinda makes me feel like a drug-pusher

Please find attached autofs (I assume this is the automounter you wanted?) and the kernel module that you will also need. Not tested at all. autofs4.ko comes from enabling CONFIG_AUTOFS4_FS as a module in kernel .config, while the user-land daemon is straight out of OpenEmbedded (bitbake autofs).

Zitat

Original von thowi
Then the auto field in fstab would have a real meaning

That's going too far, I think. The autofs daemon has it's own config file, it's not using /etc/fstab, and you would need a proper mount command (not the poor excuse that busybox provides, although even busybox is gradually improving) for the fstab file to be correctly interpreted.

P.S. Why don't you install Linux and OpenEmbedded on your PC? OpenEmbedded would give you literally thousands of packages to play with, just by "bitbake <package_name>" and installing the resulting ipk file.... You'd never be bored again

First of all, congratulations on your new version of multiboot! This story just proves that the greater work is not to come up with some changes and let the computer do some compilatioon work; it's the testing and making it work that takes time and effort.

Zitat

Original von thowi

If I boot normally from CF card with your USB linked kernel it doesn't mount the USB devices during boot that I added to fstab (eg. kernel is to slow in loading USB drivers/devices), but after booting I can mount them manually, so it seems to try to early.

I've seen reports from others that seems to indicate that this is also a problem if the standard USB modules are used.

Zitat

BUT if I add a rootdelay=30 the boot is as fast as usally BUT the USB is properly mounted from fstab.

Strange, isn't it ...

Yes, it's a bit strange that it doesn't delay the boot. I would guess that the explanation is that it actually does delay the boot a bit, but that some of that delay is "hidden" by some other initialization that needs to take place anyway.

Zitat

Again thanks for your help, and I hope it is OK to package the vmlinux.gz into the Multiboot 6.2 BETA kit, so that others can try and enjoy it also until the image buildes will start to build their vmlinux.gz this way - as a temporary workaround I implemented a copy V in multiboot to update only vmlinux.gz with the USB extended one, but this is not a perfect solution, because all patches adaption of the kernel
the imagebuilders might have done is lost due to this.

You needn't even have asked. Of course you can!

It would be interesting to know how much slower this kernel is during start-up. I would expect it to be slower if any USB devices are plugged in (but a little faster than loading kenel modules for the USB), the really interesting metric is how much slows down the boot if no USB devices are attached, i.e. how much it hurts those who have no interest in USB. Could you please post impressions and/or hard facts that you aquire during your testing?

Zitat

Original von thowi

I had the hope that the extended vmlinux.gz would fit in the 4MB flash, but let's see, it could be then almost 6-7MB big due to the jffs compression unless the gz did already a too good job

It's on it's way, so you will see the size for yourself, but if anybody else is interested, a standard kernel from DMM is 1923521 bytes these days, while the one containing USB support is 2032044 bytes.

1. I think you're aware of it, but just to make it absolutely clear: Loading the kernel from USB requires support in secondstage. I don't think this support is available, but the secondstage is undocumented and no sources are available, so who knows...
2. Please see attached diff for kernel config to enable SCSI, SCSI disk, USB and USB storage as built-ins rather than as modules. I think this should be sufficient, but I haven't tested it.....
3. Please be aware of the following problem and how to solve it: http://www.reactivated.net/web…-external-usb-flash-disk/
4. I've compiled a kernel for the 7025 using the changed configuration, but the resulting vmlinux.gz is too large to post. Please advise me on a suitable method if you want me to send it to you.

Zitat

Original von tmbinc

3.) do you know some simple USB code, which can speak OHCI and usb-storage? I only found some ports of the linux usb stack...

Found this one: http://www.broadcom.com/produc…cessors_downloads.php#cfe

Don't know, of course, how difficult it will be to rip the USB stack out of that code and transplant it into secondstage; I don't have access to secondstage sources...

Thanks for checking it up. Should of course have done it myself before posting...

But still, maybe it's easier to port kexec to MIPS than it is to port uboot to DM7025. Hmmm. Will have to look closer at this some day.

Here's one way that I've been thinking about, but which I haven't done anything about (yet?).

It's possible, at least on other architectures, to compile the linux kernel with "kexec" support. With "kexec", you can use the linux kernel as the bootloader, giving you access to all device drivers that already exist in linux. You could even load the Dream kenel modules to gain access to framebuffer and remote control.

If you are dependant on OPC to talk to your hardware, I'm afraid that your dreambox will never be able to do the job. The problem isn't python, it's OPC. The OPC architecture is a server/client architecture based on COM. The server (usually supplied by the hardware provider) is a closed program that runs on a Windows machine, while clients use COM to talk to the OPC server in order to get or update data on the hardware. This whole system is so Windows depandant that getting up and running on Linux/MIPS is close to impossible. The fact that someone has made OPC available on Windows using python won't help, I'm afraid.

If you can bypass the OPC layer and talk directly to you hardware, you might have a chance. But unless you find a Linux implementation, you're in for a lot of work.

Regarding changes to dump to stop it displaying unwanted messages: Why not just shut it up:

dump .... 2>/dev/null

or more selectively

dump .... 2| grep -v mkdir

Regarding secondstage : Just revert to tmbinc's version in ripimage.py. Change the line

output_names = {1: "secondstage.gz", 2: "boot", 3: "root"}

to

output_names = {2: "boot", 3: "root"}

And sorry about not answering your questions in a previous posting. Rest assured that whenever I post something that is more or less original code, nothing pleases me more than someone else picking it up, changing it, and making it fit their purposes. The point is "sharing", not "copyrights", "all rights reserved" or "don't change my stuff". Only thing I dislike, is if someone changes code I have written, and passes if on without mensioning that that it was changed. I make enough bugs all by myself, thank-you-very-much, without having to take any blame for others'.

Try monotone.vanille.de

Time to start a new thread, I think. Continued from: Experimental: NFI2CF

Using the tools from the link above, a backup to .nfi format should be possible. Compared to my earlier attempts, this should be pretty safe since it's working through the file system, and only the standard image making tools from OE is used.

The script:

#! /usr/bin/python


# Backup /boot and / and create an nfi file
import sys, os


if len(sys.argv) < 2:
        raise "Usage: %s FILE" % (sys.argv[0])


mnts = open("/proc/mounts", "r")


ignores = ''


for line in mnts.readlines():
        mntpt = line.split()[1]
        if mntpt == '/': continue
        ignores += ' --ignore=' + mntpt
mnts.close()


os.environ['PATH'] += ':.'
os.system("mount -o ro,remount /")
# Awful, but just dumping the whole partition won't work
os.system("dd if=/dev/mtd/1 bs=32k of=secondstage.gz")
os.system("gunzip secondstage.gz")
os.system("gzip secondstage")
#
os.system("mkfs.jffs2 -v --root=/boot --faketime --output=boot.jffs2 --eraseblock=0x4000 -n -l")
os.system("mkfs.jffs2 -v --root=/ --faketime --output=root.jffs2 --eraseblock=0x4000 -n -l" + ignores)
os.system("buildimage secondstage.gz boot.jffs2 root.jffs2 dm7025 > " + sys.argv[1])
#
os.unlink("secondstage.gz")
os.unlink("boot.jffs2")
os.unlink("root.jffs2")
os.system("mount -o rw,remount /")

It can only be run from a file system that is different from the root file system, e.g. from a harddisk partition.

Please find the complete tool-box (including this script) attached.

Does anybody care to test it and report back? I've only made a backup and unpacked the resulting nfi file, I have not tried to actually flash it yet.

I've used your program for a while now, with no problems whatsoever. It's quite memory-hungry, though, so I've now hacked the dump.cpp file to use a two-pass approach in order to bring down memory requirements. I also plugged a leak (memory from a calloc was never freed) and it now runs on my 7025 without swap.

Only lightly tested! Any bugs are probably mine, I've never had any problems with the original.

And yeah, since thowi hasn't set up a cross development system, I've also included a binary for the 7025

EDIT: Just to tie up some loose ends, I've also attached tools to pack and unpack nfi files on a 7025. The tools are:
- The dump program written by tmbinc and hacked by me
- The mkfs.jffs2 program from mtd-utils
- The buildimage program from DMM in OE
- ripimage.py by tmbinc, horribly mutilated by me
- a tiny shell script to run the commands to build an nfi-file

I have some problems running the ripper without either stopping enigma2 or adding swap. Any good ideas on how to reduce the memory usage of dump even further?

tmbinc's extract routine is also in python, and it's more elegant than mine

Zitat

Original von thowi
... it now includes also the python code for copying nfi images that you posted for me.

tmbinc's version (Experimental: NFI2CF) is more elegant.

You're welcome.