Dreamboard

raton_craquero · 16. April 2008

i've stumbled across a very strange problem that, as far as i can tell, occurs only on the DM7025:

Below is a dvbsnoop log of 1 raw ECM.

the first ECM was captured in a DM7020 and the second one is *the same* ECM but captured on a DM7025 simultaneously (as you can see in the timestamps, they are a few seconds apart but they are the same ECM).

Raw ECM captured on a DM7020

Code

dvbsnoop V1.4.40 -- http://dvbsnoop.sourceforge.net/


------------------------------------------------------------
SECT-Packet: 00000001   PID: 301 (0x012d), Length: 74 (0x004a)
Time received: Sun 2008-04-13  22:24:34.057
------------------------------------------------------------
  0000:  81 30 47 07 45 99 81 59  fe 36 bc 84 56 d7 0d 8f   .0G.E..Y.6..V...
  0010:  47 fd 38 ea 13 0f d2 e9  26 1f 64 80 a5 fc ff fc   G.8.....&.d.....
  0020:  a8 75 41 54 e8 19 d0 8f  fa 29 0d 3f 3b f7 6f 74   .uAT.....).?;.ot
  0030:  39 27 58 9b 55 13 a5 db  ab d7 f8 5c a4 aa d4 6a   9'X.U......\...j
  0040:  da 94 1a ea 01 71 3d d1  3c 03                     .....q=.<.


PID:  301 (0x012d)


Guess table from table id...
CAMT-decoding....
Table_ID: 129 (0x81)  [= DVB CA message section (EMM/ECM)]
section_syntax_indicator: 0 (0x00)
reserved_1: 0 (0x00)
reserved_2: 3 (0x03)
Section_length: 71 (0x0047)
CA_message_section_data:
      0000:  07 45 99 81 59 fe 36 bc  84 56 d7 0d 8f 47 fd 38   .E..Y.6..V...G.8
      0010:  ea 13 0f d2 e9 26 1f 64  80 a5 fc ff fc a8 75 41   .....&.d......uA
      0020:  54 e8 19 d0 8f fa 29 0d  3f 3b f7 6f 74 39 27 58   T.....).?;.ot9'X
      0030:  9b 55 13 a5 db ab d7 f8  5c a4 aa d4 6a da 94 1a   .U......\...j...
      0040:  ea 01 71 3d d1 3c 03                               ..q=.<.
==========================================================

Alles anzeigen

Same raw ECM captured on a DM7025

Code

dvbsnoop V1.4.40 -- http://dvbsnoop.sourceforge.net/


------------------------------------------------------------
SECT-Packet: 00000001   PID: 301 (0x012d), Length: 74 (0x004a)
Time received: Mon 2008-04-14  06:24:28.206
------------------------------------------------------------
  0000:  81 30 47 07 45 89 81 59  fe 36 bc 84 56 d7 0d 8f   .0G.E..Y.6..V...
  0010:  47 fd 38 ea 13 0f d2 e9  26 1f 64 80 a5 fc ff fc   G.8.....&.d.....
  0020:  a8 75 41 54 e8 19 d0 8f  fa 29 0d 3f 3b f7 6f 74   .uAT.....).?;.ot
  0030:  39 27 58 9b 55 13 a5 db  ab d7 f8 5c a4 aa d4 6a   9'X.U......\...j
  0040:  da 94 1a ea 01 71 3d d1  3c 03                     .....q=.<.


PID:  301 (0x012d)


Guess table from table id...
CAMT-decoding....
Table_ID: 129 (0x81)  [= DVB CA message section (EMM/ECM)]
section_syntax_indicator: 0 (0x00)
reserved_1: 0 (0x00)
reserved_2: 3 (0x03)
Section_length: 71 (0x0047)
CA_message_section_data:
      0000:  07 45 89 81 59 fe 36 bc  84 56 d7 0d 8f 47 fd 38   .E..Y.6..V...G.8
      0010:  ea 13 0f d2 e9 26 1f 64  80 a5 fc ff fc a8 75 41   .....&.d......uA
      0020:  54 e8 19 d0 8f fa 29 0d  3f 3b f7 6f 74 39 27 58   T.....).?;.ot9'X
      0030:  9b 55 13 a5 db ab d7 f8  5c a4 aa d4 6a da 94 1a   .U......\...j...
      0040:  ea 01 71 3d d1 3c 03                               ..q=.<.
==========================================================

Alles anzeigen

if you look closely, you'll see that the 6th byte of the ECM captured on the DM7025 gets corrupted. in this case the correct 6th byte is 0x99, but it's 0x88 in the DM7025 ECM.

All other models (tested it on DM500/DM7000/DM7020) show consitently identical ECM's, except for the DM7025 which corrupts this byte *sporadically*.

i've taken the time to observe the pattern of corruption and i've come to the following conclusion: whenever bits 8 and 7 of the byte are 1 and 0 respectively, bits 6 and 5 are always set to zero on the DM7025, here are some examples:

Code

DM7020		DM7025
(correct)	(corrupt)


0x98		0x88
0xA7		0x87
0xB0		0x80

i've tried serveral tricks and workarounds, such as stoping and re-starting the demux filter on each ECM... adding timing delays after reading the demux output after starting it, skipping the first ECM's in the sequence and reading, say, the second or third... i've seen some improvement with some of these techniques (specially stopping and re-starting the demux filter on each ECM) but none have solved the problem.

i'm at my wit's end and cannot see how this can be happening unless it's a hardware driver problem. i'm hoping someone has come across this problem and maybe know of a work-around for the situation.

any help is appreciated, thanks.

tmbinc · 16. April 2008

Hi,

sorry, this is probably a bug in the demux microcode we are using. We already tried working around that issue, but that won't work in all cases.

We will try finding an ultimate solution here, but I wouldn't count on it. Sorry.

It would probably help if we can understand when this corruption happens. My guess is that the demux microcode detects some "condition" and somehow handles that incorrectly. If we knew what exactly triggers that, it would possibly help in understanding the reason for this misbehaviour.

raton_craquero · 17. April 2008

thanks for your response tmbinc. at least you confirmed my suspicion. if anything i could provide would be helpful, like sets of additional data, please do let me know.

ambrosa · 9. August 2008

I can confirm a MASSIVE DATA CORRUPTION reading from /dev/dvb/adapter0/demux0 in 7025 (we have tested 4 different 7025 box in different place, sat equipment and images).

As you probably know at the beggining of 2008 most used Italy provider, SKYITALIA (HotBird 13.0E), has changed it's epg data transmission. It has switched from MHW to OpenTV
After some month (April 2008 ) an Italian LINUX-VDR user has decoded the OpenTV format and he made a C program to decode OpenTV EPG.
Other users changed a bit his code to run in 7020 / 7025 + 800 HD
SkyItalia provider transmit epg in MarcoPolo channel.
It use the following PIDs:
0x11 : channel name infos
0x30 - 0x37 : epg titles
0x40 - 0x47 : epg summaries
Pid are arranged for weekday: pid 0x30/0x40 transmit today epg, 0x31/0x41 transmit tomorrow epg data and so on.
All data are compressed with Huffmann and there need a special routine and an external dictionary for decoding data in clear text format.

Power on Dreambox, set channel SkyItalia - MarcoPolo and run this little software that reads these pids and decode data. WoW ! Now in 30 seconds we have 1 week epg decoded in clear text format. Than these data can be postprocessed in many ways.

In 7020 and 800HD EPG pids are readed without any error: in 30 seconds are downloaded about 45000 epg events (!)

But there are many problem with 7025: events are only about 30000 (- 30%) with many "gap" into it. Many events are missing.
In these days I've done some debug (I've the source C code of this program) and I've discovered that there is a massive data corruption reading these pids from demux0.
For testing purpose, I deactivated the DMX_CHECK_CRC when I set the filter

Code

dmx_sct_filter_params sctFilterParams;


memset ( &sctFilterParams, 0, sizeof ( sctFilterParams ) );
sctFilterParams.pid = Pid;
sctFilterParams.timeout = 0;
// ambrosa : disable  CRC CHECK
//sctFilterParams.flags = DMX_IMMEDIATE_START | DMX_CHECK_CRC;
sctFilterParams.flags =  DMX_IMMEDIATE_START ;
sctFilterParams.filter.filter[0] = 0;
sctFilterParams.filter.mask[0] = 0;

Alles anzeigen

Now I can analyze all packet received.
After analysis, I can say that I receive all epg events (45000 instead of previous 30000) but 30% are corrupted.
DMX_CHECK_CRC(it's activated by default) skip corrupted data (correct !)

Note: if I running software many many times between minutes, errors are always the same -> missing events are always the same.
Errors don't seems to be random but constant. I believe that only some specific byte pattern are corrupted.
In 7020 and 800HD there are NO ERROR at all even if I disable DMX_CHECK_CRC
Tests were done with different sat equipments, images and country. I believe that is not a antenna receive problem.

raton_craquero says before to have problem reading ECM and we have problem reading 17 PIDs, I suppose that there is a bug somewhere that involve all SCT filtering demux data/pids.
This bug can explain also strange behaviours in card management (ECM/EMM) that I have noticed sometimes.

If I can help you in any way to solve this (probably) bug in microcode, please send me a PM or an email.

stick50jr · 10. August 2008

Ouch!

If the error was understood and a fix for the microcode was developed... how could this be inserted in an existing 7025?

Would this involve replacing demux hardware?

ambrosa · 16. August 2008

Hi guys.

During these days I've made many intensive testing.
I've also used a last generation hashing algorithm (very fast and with very low collision number) for tracking incoming data packets (no video or audio, only SCT filtered data packets).

Well, now I've absolute evidences that there is a problem.
Others 7025 tested have the same problem.

Incoming data are corrupted and corruption involved always the same packets.
It's not a random error.

eagle · 26. August 2008

Zitat

Originally posted by tmbinc
Hi,

sorry, this is probably a bug in the demux microcode we are using. We already tried working around that issue, but that won't work in all cases.

No solution?

With 800HD there isn't this problem... why?

Muaitai · 3. September 2008

Hello

Do we have any advance on this subject.
We here in Portugal have lot's of freezes with our provider due to this.

Please hep us DMM.

Thx

Zumzum · 3. September 2008

hello to all ,
I came here for the same problem,
I make another test to with my friend muaitai, in the same channel, but 2 server different. i connect to one and he to another, we have the same frize at the same time !!
but in another DM with a E1 (dm600) ever ting it´s ok

@ambrose
have U resolve your problem ?

thanks

ambrosa · 3. September 2008

No, I've not found a solution.
I think that DMM must solve this.

But I NEVER buy another Dreambox in my life !
I've spent 520 euros for a buggy device !
It's incredbile. Do we start a class action against DMM ?

Zumzum · 3. September 2008

@ ambrosa,
what is the provider U have the frizes (problems)

ambrosa · 3. September 2008

Zitat

Originally posted by Zumzum
@ ambrosa,
what is the provider U have the frizes (problems)

Freeze ? I've no freeze.

Please read careful my post Help needed: Possible bug in DM7025 hardware demux drivers?

Gotcha_GT · 26. September 2008

Zitat

Original von eagle
With 800HD there isn't this problem... why?

But in my case (like muaitai) I have a problem on my dm800HD not being able to do the reading of the card we use here in Portugal with dm800 slot. I have to use dm600 to read it... Otherwise the dm800 wouldn't serve my needings.

It's starting to be quite anoying for me not being able to use all the hardware I bought with dm800HD.

If someone can also have a look at dm800HD slot drivers most surely many people would appreciate it.

Satelite Regards,

Gotcha_GT

stuntman · 17. März 2009

Zitat von Gotcha_GT

But in my case (like muaitai) I have a problem on my dm800HD not being able to do the reading of the card we use here in Portugal with dm800 slot. I have to use dm600 to read it... Otherwise the dm800 wouldn't serve my needings.

It's starting to be quite anoying for me not being able to use all the hardware I bought with dm800HD.

If someone can also have a look at dm800HD slot drivers most surely many people would appreciate it.

Satelite Regards,

Gotcha_GT

Alles anzeigen

Hello,

One more with the same problem...

After a year of development, did not expect that the DMM was not able to fix these bugs on DB 800HD, at this time i`m start wondering if DMM it`s waiting for stock off on the DM 500s/+ and DM600....

drossier · 27. Mai 2009

Hello,

Is there something new related to this demux problem?
I'm using a BisTV card and gets some big troubles (macro-pixels in fast moving sequences), on some services (but not all); apparently, it depends on the way how the multiplexing is carried out (statistical multiplexing or whatever)...

It's really annoying...

Thanks for any feedback.

Cheers
Daniel

drossier · 4. Juni 2009

Just something new: I found out that the image is correctly if I'm using VLC on a PC....

It may help solving the bug

stuntman · 8. Juni 2009

[Moderator]Please, read the Boardrules [/Moderator]

Boardrules read.

Sorry.

tmbinc · 8. Juni 2009

It looks like some issues are mixed together here.

As I already said, there is a problem with certain data combinations. This doesn't affect "normal" functionality of the DM7025, and thus, fixing it is not our highest priority, which doesn't mean that we ignore the problem. We simply don't have a fix yet. I'm really sorry for this.

The DM800HD is based on completely different hardware, and does not have this problem. Same for DM7020 and all other dreambox hardware.

Then, some people seem to have problem talking with smartcards on various hardware. I don't know much about smartcards, but if you can pinpoint a problem, please send me a mail with the details, and I'll try to forward it to other developers. Experience shows that most of these problems are related to a misunderstanding of the SCI api. But again, I don't know much about smartcards. In any case, it's not related to the demux bug.

Then some of you described a "freezing" - what do you mean exactly with that?

stuntman · 8. Juni 2009

The image simply stops by a few seconds, sometimes 1 or 2 seconds, other times for 9 or 10 seconds. But only with that smartcard

drossier · 8. Juni 2009

Zitat von tmbinc

It looks like some issues are mixed together here.

As I already said, there is a problem with certain data combinations. This doesn't affect "normal" functionality of the DM7025, and thus, fixing it is not our highest priority, which doesn't mean that we ignore the problem. We simply don't have a fix yet. I'm really sorry for this.

The DM800HD is based on completely different hardware, and does not have this problem. Same for DM7020 and all other dreambox hardware.

Then, some people seem to have problem talking with smartcards on various hardware. I don't know much about smartcards, but if you can pinpoint a problem, please send me a mail with the details, and I'll try to forward it to other developers. Experience shows that most of these problems are related to a misunderstanding of the SCI api. But again, I don't know much about smartcards. In any case, it's not related to the demux bug.

Then some of you described a "freezing" - what do you mean exactly with that?

Alles anzeigen

Thanks for your feedback; I see there is two different issues indeed. I don't have some freezes on my side.
But what do you mean with "data combination" and this does not affect "normal" functionality ?? If I'm looking a service (France 2, M6, etc.) with an official smartcard (BisTV), it seems to me that my DM 7025 should work in a "normal" way, regardless the nature of the pictures or movies, doesn't it? I do not believe in a smartcard issue; reasons are that it works perfectly with some services, and no with some others (using ViaAccess 3.0 in all cases). I also made some tests with Compact Flash, [Moderator] verstößt gegen die Boardregeln [/Moderator] , the behavious is the same (and IMHO, if it is a smartcard issue, it wouldn't work with VLC neither probably).

I'm fully at your disposal for testing some patches or any tests...