skyscraper8: Command line tool for extracting files contained in a TS/GS.

Hi guys,

Regarding DVB-SIS: The whole idea behind it, as far as I understand is that a broadcaster can use a single mux to satisfy both DTH-satellite reception and supplying signal DVB-T(2) broadcast cells. However, SIS allows them to also use the satellite mux to transmit additional services which would be present in the DVB-T2 mux, but won't be "visible" to satellite receivers. The XML file that skyscraper8 extracts from the mux describes exactly how a DVB-T(2) broadcast cell should generate a stream. Unfortunately, the muxes that we know of - those on 26°E and 5°W don't contain any additional DVB-T(2) exclusive services. This can easily be checked if we compare the PIDs referenced in the XML-file (it's human-readable) to those that are referenced by the PMTs we can already see. This is also why I didn't (yet) implement remultiplexing of DVB-SIS in skyscraper8 yet - because the remultiplexed stream would be pretty close to a clone of what we already see.

Now for why I have been silent the last few days: I've been deep-diving into the sample of 95°E/11476/V you provided, and found it to be pretty interesting! It is indeed a GSE stream, but the owner of it took some liberties in regards to the standard.
For example, they use a non-standard CRC32 coefficient for error-checking. Also, the standards say to set the SYNC field in the BBframe to "0" or "1", yet they (ab-)use it as continuity check.
Then there's the Protocol_Type field, which also caused me some headaches. Instead of setting it to 0x0800 or 0x0806 like one would expect it for IP traffic - or 0x0082 for DVB-RCS2 traffic (like BFBS does), or any valid EtherType, they always set this field to 0x0002, or 0x0004. If adhering to the standard, 0x0002 would mean that the frame contains valid TS packets - but the broadcaster here wasn't that nice. And 0x0004 isn't even an assigned number (see here: https://www.iana.org/assignments/ul…ext-headers.xml ) - so we're dealing with something non-standardized here.

So I observed further: If the Protocol_Type is 0x0002, then the GSE PDU *might* be an IPv4 Packet. But it might also be something entirely different. Sometimes it's also something base64 encoded, which I've yet to make sense of.

But when the Protocol_Type gets set to 0x0004, the data contained within the PDU looks like stuffing data to me. There's always two randomly chosen bytes, repeated to the end of the PDU. How a VSAT Terminal is supposed to make sense of so little information is beyond me.

Also of peculiar note, the GSE packet label is always set to the same MAC address. But judging from the contents of the IPv4 packets, I'm pretty sure this mux is targeted to multiple receivers.

I got extraction of IPv4 packets from this mux mostly working and will post update files soon.
Unfortunately, this sample recording does not contain anything that looks like video or audio, but I don't feel like this was wasted time as there might be muxes out there which use the same format and do contain interesting things for us.
I think so because I'm pretty sure that I've seen both forward counting in the SYNC-field and Protocol_Type being set to 0x0002/0x0004 before (so this format *might* be common?), but I didn't get to dig through my stash of samples yet.

Now, I know this sounds like I've put a lot of effort into this, but this is a face of tech I do enjoy quite a lot. Compared to the boring and sterile cloud stuff I have to deal with at work, trying to make sense of such streams is a delight for me! 🥰

Regards and wishes for a nice weekend,
Fey

Hi friends!

As promised yesterday, here is the new update:
https://mega.nz/file/1wkBnaLS#49ogoBh6Opb1izzI6F4UXeli6IzBygLdHbc55rndag0

This update changes skyscraper8's behaviour of dealing with GS quite a bit (but this shouldn't break compatibility with what the previous versions can do):
It goes like this:
- When recieving a GS, it will first collect 100 complete BBFrames. If the broadcast is using MIS, it will collect 100 complete BBFrames for each stream.
- When it's got the BBFrames, it will perform a statistical analysis to check what kind of stream it's getting.
- If the analysis says that it's a known stream type, it will take the code-path meant for it. (right now standard GSE, GSE with error detection, GSE-HEM, Siminn Radiomidun-style encapsulation, and whatever it is that's talking on 95°E/11476/H, are known stream types)
- If it has no idea what stream type it is, it will give a message saying so, and will fallback to the IP packet finder algorithm which older skyscraper8 versions were using as well.

P.S.: I've also began working on a (hopefully) user-friendly GUI. I'll post screenshots as I make progress with it.

Happy DX'ing and have a nice week,
Cheers,
Fey