skyscraper8: Command line tool for extracting files contained in a TS/GS.

Hi guys,

Regarding DVB-SIS: The whole idea behind it, as far as I understand is that a broadcaster can use a single mux to satisfy both DTH-satellite reception and supplying signal DVB-T(2) broadcast cells. However, SIS allows them to also use the satellite mux to transmit additional services which would be present in the DVB-T2 mux, but won't be "visible" to satellite receivers. The XML file that skyscraper8 extracts from the mux describes exactly how a DVB-T(2) broadcast cell should generate a stream. Unfortunately, the muxes that we know of - those on 26°E and 5°W don't contain any additional DVB-T(2) exclusive services. This can easily be checked if we compare the PIDs referenced in the XML-file (it's human-readable) to those that are referenced by the PMTs we can already see. This is also why I didn't (yet) implement remultiplexing of DVB-SIS in skyscraper8 yet - because the remultiplexed stream would be pretty close to a clone of what we already see.

Now for why I have been silent the last few days: I've been deep-diving into the sample of 95°E/11476/V you provided, and found it to be pretty interesting! It is indeed a GSE stream, but the owner of it took some liberties in regards to the standard.
For example, they use a non-standard CRC32 coefficient for error-checking. Also, the standards say to set the SYNC field in the BBframe to "0" or "1", yet they (ab-)use it as continuity check.
Then there's the Protocol_Type field, which also caused me some headaches. Instead of setting it to 0x0800 or 0x0806 like one would expect it for IP traffic - or 0x0082 for DVB-RCS2 traffic (like BFBS does), or any valid EtherType, they always set this field to 0x0002, or 0x0004. If adhering to the standard, 0x0002 would mean that the frame contains valid TS packets - but the broadcaster here wasn't that nice. And 0x0004 isn't even an assigned number (see here: https://www.iana.org/assignments/ul…ext-headers.xml ) - so we're dealing with something non-standardized here.

So I observed further: If the Protocol_Type is 0x0002, then the GSE PDU *might* be an IPv4 Packet. But it might also be something entirely different. Sometimes it's also something base64 encoded, which I've yet to make sense of.

But when the Protocol_Type gets set to 0x0004, the data contained within the PDU looks like stuffing data to me. There's always two randomly chosen bytes, repeated to the end of the PDU. How a VSAT Terminal is supposed to make sense of so little information is beyond me.

Also of peculiar note, the GSE packet label is always set to the same MAC address. But judging from the contents of the IPv4 packets, I'm pretty sure this mux is targeted to multiple receivers.

I got extraction of IPv4 packets from this mux mostly working and will post update files soon.
Unfortunately, this sample recording does not contain anything that looks like video or audio, but I don't feel like this was wasted time as there might be muxes out there which use the same format and do contain interesting things for us.
I think so because I'm pretty sure that I've seen both forward counting in the SYNC-field and Protocol_Type being set to 0x0002/0x0004 before (so this format *might* be common?), but I didn't get to dig through my stash of samples yet.

Now, I know this sounds like I've put a lot of effort into this, but this is a face of tech I do enjoy quite a lot. Compared to the boring and sterile cloud stuff I have to deal with at work, trying to make sense of such streams is a delight for me! 🥰

Regards and wishes for a nice weekend,
Fey

Hi friends!

As promised yesterday, here is the new update:
https://mega.nz/file/1wkBnaLS#49ogoBh6Opb1izzI6F4UXeli6IzBygLdHbc55rndag0

This update changes skyscraper8's behaviour of dealing with GS quite a bit (but this shouldn't break compatibility with what the previous versions can do):
It goes like this:
- When recieving a GS, it will first collect 100 complete BBFrames. If the broadcast is using MIS, it will collect 100 complete BBFrames for each stream.
- When it's got the BBFrames, it will perform a statistical analysis to check what kind of stream it's getting.
- If the analysis says that it's a known stream type, it will take the code-path meant for it. (right now standard GSE, GSE with error detection, GSE-HEM, Siminn Radiomidun-style encapsulation, and whatever it is that's talking on 95°E/11476/H, are known stream types)
- If it has no idea what stream type it is, it will give a message saying so, and will fallback to the IP packet finder algorithm which older skyscraper8 versions were using as well.

P.S.: I've also began working on a (hopefully) user-friendly GUI. I'll post screenshots as I make progress with it.

Happy DX'ing and have a nice week,
Cheers,
Fey

Hi all,

it's been a while. First, let me apologize for the long absence. It's been mostly health issues. Plus I had something like a "coder's block" - similar to how many authors of fiction get "writer's block". I didn't even know such issues exist, but apparantly it's a known phenomenon among both hobbyist and professional software developers.

But when I actually got to coding, I was a bit "in the zone", as they say, to get Windows Forms to do what I want, in order to get a nice UI for skyscraper8 going and finally make the application of my dreams.

Now, developing a Windows Forms/Desktop application in 2026, when web applications are the gold standard, might sound crazy but since I am doing the DVB-stuff mostly for fun and deal with (backend) web stuff at work, this is a nice way to get balance.

I don't have an update to share right now, but some screenshots of the upcoming GUI for skyscraper8, which I'd like to release in the foreseeable future. I call this project "Voile".

Pictures probably speak more than words in this case, so I'm gonna show what I was up to:

1. [Screenshot of Voile displaying a sample of 4,8°E/11747/V]

I know, from the looks, right now this looks like a rip-off version of TSReader, but Voile isn't meant to replace TSReader, and of course I do not mean to disgrace Rod Hewitt's legacy. These programs serve a different purpose. The purpose of TSReader is well known to all of us.
Voile is meant as an UI to skyscraper8's features. Therefore in it's heart Voile is also a data extraction program, just as skyscraper8 is. And for finding data inside of TS structures, skyscraper8 needs an understanding of the TS contents, and Voile displays these as well.

2. [Screenshot of Voile displaying a sample of the BFBS stream on 15°W]

For example, it's possible to also see information on nested TS. This is might prove useful for things like T2-MI or the BFBS stream of 15,0°W on which they put the TV service inside a TS inside an IP stream inside a GS. You get the gist.

3. [Screenshot of Voile showing a nested T2-Mi Mux beside the DAB feed on 0,8°W]

One of the things I really wanted to do is to have multiple TS open at the same time, and I think I have accomplished that. For example, here we can see Voile displaying the contents of the unencrypted T2-MI mux on 4,8°E/11747/V next to the IP packets on 0,8°W/10717/V. If you have a QHD or 4K Monitor, it's even possible to display more than that.

4. [Screenshot of Voile showing the analysis of the NIP package on 19,2°E/11141/H]

Voile also knows DVB-NIP - after all that's what made me publish skyscraper8 in the first place.

Now looking at those screenshots, you might notice two things. The title bar says "Voile - [Licensed to ...]". Yes, Voile will be released as shareware. But do not worry, for friends and regular contributors of the forums it will be totally free. Skyscraper8 itself will stay free just as it has been so far.

The other thing that might occur to is that the sidebar says "Database" - this is not just for show. Besides skyscraper8's primary function of extracting data, Voile writes the knowledge skyscraper8 gathers on the DVB Tables and other metadata processed through it to a local database. There are numerous situations in which this may come in handy.

• For example one could easily build a TV Guide out of the gathered EIT sections from the database.
• Or train an LLM/LoRa on a good amount of gathered teletext pages.
• Or one could compare how a Mux looked three weeks ago from how it looks today. (If we spin that thought further, comparing entire blind-scans might become interesting as well. Blindscanning is definitely something I want to get into as well.)
• And of course a lot more...

Now of course I do understand that not everyone wants to deal with stuff like MS SQL Server, PostgreSQL or MySQL and I took that into consideration. That's why Voile ships with and will just use a zero-configuration SQLite database by default. No cloud, no servers or database administration required.

I know it's difficult to judge a program from just screenshots but I'm curious what you guys think.

I can't wait to publish Voile with the next skyscraper8 update once I've got it to a point on which it runs stable.

Right now I'm busy fighting ffmpeg's libav or libvlc to get video playback and video screenshotting inside the program working. If any other developers have experience with these, I'd love to hear them.

Thanks for reading,

Regards,

Fey.

Quote from satesco

@Fay, I hope you are well and able to work as you like.

I assume you don't have time to visit us, for the reasons you described in the previous pages, especially those related to teaching or research.

And yet, perhaps you are curious to analyse this sample from a mux (12560 V) with a 256-APSK symbol rate.

At first glance, it seems more of an intense traffic issue, but the message in the PowerShell script requires a closer look, which is why I am attaching a recorded file (13MB).

The same thing and the same content are also found in the mux next to it, 12600 H, with the S2X /64APSK extension.

Display More

Thanks for the new sample. With a quick glance at it's structure, it exhibits similar patterns as the sample of 95°E/11476/H you posted earlier. Interesting stuff! Will dig into this later.

Quote from Joolz

Fey,

Thank you for the update. If I find stream types that your software requests a sample of, what is the best method to send them to you?

Joolz

Apologies for the late response. The easiest way for me to download a large file would be mega.co.nz, because I have a premium subscription to their service, but since they tend to limit uploader's space, I think it might not be the best solution for everyone.

The sharing features of OneDrive and Google Drive should be fine as well. Some friends have also sent me stuff using https://dropmefiles.com/ before, that works nicely as well. If you have a larger set of samples, or a slow internet connection, DM me and we can work something else out.

Hi all,

Voile is making nice Progress. To hold you over for a bit, here is a showcase Video, so you can see what you can expect:

The goal of Voile is to handle all the streams which skyscraper8 can handle as well. Of course I can not promise Voile/skyscraper8 will ever handle ALL streams. There might always be those which are encrypted, too difficult to reverse-engineer, or otherwise poorly documented.

I've put together a Roadmap of what I want to accomplish with Voile:

Short term Goals: (those I really want to finish before Voile's first release and are all smaller TODOs)

• Displaying DSM-CC Data Carousel Download Progress
• Display UNT Data in the MPEG TS Analysis tab.
• Display DVB-SIS Daughter Site Data in the MPEG TS Analysis tab.
• Display SCTE-35 Data in the MPEG TS Analysis Tab
• Add a tab page for displaying DVB-RCS2 Tables

Once these are done, I feel comfortable pushing out the first Version of Voile as an update to skyscraper8.

Long term Goals: (those of which, in my opinion, still have time until after the program performs it's Basic functions. All of those are bigger TODOs.)

• Playback of Audio and/or Video from within the application
• Blindscanning and TS Acquisition with StreamReader.dll by utilizing Voile's own StreamReader Proxy (and if Vitor is okay with it, also using the Telnet Server Mode of VMA StreamReader CLI)
• TS Acquisition using SAT>IP devices like Digital Devices' Octopus NET (skyscraper8 itself can already do this, using the "satip" commandline Option)
• Add a tab page for DAB Ensemble analysis
• More Database backends. (I've always wanted to play around with Oracle's and MS SQL's free offer)

Quote from Butacas

Fey;

I performed a scan (Eutelsat 36° E) with EBSpro and found a continuously transmitting frequency: 11521 H SR 10832 FEC 1/3 S2/QPSK. Initially, it displayed BBC News, but after scanning it with altdvb, the SID was ITV1.

I haven't tested if Skyscraper8 can play it back, but if you need a recording of the stream, I can provide one.

Yes, a short sample recording of that stream would be cool, but no pressure.

It'll still be some time before I can visit my hut with my good dish again.

Have a nice weekend,

Regards,

Fey