WEBVTT 00:00.000 --> 00:14.240 Oh, wow, it's a funeral, that's fantastic, can I start? 00:14.240 --> 00:15.720 Thank you very much for joining. 00:15.720 --> 00:19.800 I got really scared because there was talk about satellites, and there was talk about 00:19.800 --> 00:27.960 robots, and I'm here to talk about software updates. 00:27.960 --> 00:31.440 I was scared that nobody will show up, thank you very much for coming. 00:31.440 --> 00:32.440 My name is Llanovit. 00:32.440 --> 00:37.520 I live in a beautiful city, this is the city where I live in, probably Bulgaria. 00:37.520 --> 00:43.480 I work remotely, and I contribute to open source projects, especially to embedded Linux. 00:43.480 --> 00:50.760 I have experience with several systems for update technologies, and in this talk, I'll 00:50.760 --> 00:55.160 do comparison of some of them, not all of them, some of them. 00:55.160 --> 00:59.920 So this is the agenda, we're going to talk about a better Linux updates in general, we're 00:59.920 --> 01:06.160 going to focus on three solutions that are quite popular, really good solutions, and finally 01:06.160 --> 01:11.760 we'll wrap it up with some conclusions. 01:11.760 --> 01:38.360 Here, let's talk about common embedded Linux updates, the first strategy, which is the focus 01:38.360 --> 01:44.800 of this talk, is about AB updates, we're going to discuss this in more details, but basically 01:44.800 --> 01:50.280 we have two identical petitions, A petitions, B petitions, and we're switching between 01:50.280 --> 01:52.080 them on updates. 01:52.080 --> 01:59.840 The second update strategy is Delta updates, the best description in my opinion is like 01:59.840 --> 02:05.080 GitforFast systems, it's more complicated, it has some advantages, some disadvantages, there 02:05.160 --> 02:13.160 are container updates, and what is most commonly going on is that we have a mix of all these 02:13.160 --> 02:15.400 options in real world scenarios. 02:15.400 --> 02:21.680 So let's start with AB updates, a very quick overview, so we have the two identical petitions, 02:21.680 --> 02:28.240 the A and B petitions, we also need a data partition where we have the persistent information 02:28.240 --> 02:32.720 that remains on the device after the update. 02:32.720 --> 02:37.280 We need a client application that is running only a better device, so this client application 02:37.280 --> 02:42.800 will perform the actual update, and in order to switch between the petitions, we obviously 02:42.800 --> 02:48.280 need the boot loader to do the switch, which also means that we need to restart the system 02:48.280 --> 02:51.280 after applying the update. 02:51.280 --> 02:56.200 For every update system, it's very important to have a fallback, it thinks go not as 02:56.200 --> 03:00.040 planned, you still need to be able to build the device. 03:00.040 --> 03:10.720 So the data updates are really cool because they do comparison of the fall system and deploy 03:10.720 --> 03:12.760 as an update just the different. 03:12.760 --> 03:20.800 Of course, this has a lot of advantages, there are some disadvantages as well, because sometimes 03:20.800 --> 03:27.900 you don't have a B partition in this scenario, and you might run up in trouble if the 03:27.900 --> 03:28.900 data doesn't go as planned. 03:28.900 --> 03:34.900 That's the side by side comparison, so A and B updates obviously, it takes a lot of storage 03:34.900 --> 03:40.900 because you have two identical petitions, it's basically twice the storage for the root 03:40.900 --> 03:49.620 FS, the update size is also in general large, however, you have very good backups. 03:49.620 --> 03:54.600 For the delta data, it's pretty much the opposite, it's a small storage because it's a one 03:54.600 --> 04:04.960 root FS, the update is smaller, which means that you are consuming less bandwidth, and 04:04.960 --> 04:09.800 you still have a row back to previous states, however, the disadvantage is that you cannot 04:09.800 --> 04:13.360 do proper food backup. 04:13.360 --> 04:20.440 There are a lot of open source solutions out there, food doing software updates. 04:20.440 --> 04:27.320 Some of them are A, B update, some of them are delta updates, and many of them are actually 04:27.320 --> 04:29.680 providing both options. 04:29.680 --> 04:34.280 So these three are in both just because in the next slide, I'm going to talk a little 04:34.280 --> 04:39.120 bit more about them, and I have more experience with these three. 04:39.120 --> 04:45.520 Through the years, I have also contributed to actualizer, which used to be a system based 04:45.520 --> 04:53.280 on Libore 3, it was made by a German started company, which after that made it successful 04:53.280 --> 04:58.800 exit, but nowadays the service is shut down. 04:58.800 --> 05:07.440 So in order to do this talk last summer, I did a couple of integrations of Mender, Rauk, 05:07.440 --> 05:14.920 and SW update on Raspberry Pi 5, and I also did integration of Rauk and Mender on this board. 05:14.920 --> 05:21.120 This is an open source hardware board with IMX 8M plus system in the module. 05:21.120 --> 05:25.560 It's designed by ONMX, it's a company in Bulgaria, where I live in Florida, so we're neighbors 05:25.560 --> 05:26.960 and good friends. 05:26.960 --> 05:30.920 So this is an open source hardware device. 05:30.920 --> 05:34.760 So the experience that I'm sharing here is based on the practical work that I've done 05:34.760 --> 05:35.760 for this board. 05:35.760 --> 05:39.480 First, let's start with Mender, how many of you are using Mender? 05:39.480 --> 05:42.320 Okay, quite a lot of people. 05:42.320 --> 05:47.040 It's available as free and open source solution, or paid commercial enterprise plan. 05:47.040 --> 05:50.760 The free solution is for AB updates. 05:50.760 --> 05:54.000 They also, however, have a cloud service, which you can use. 05:54.000 --> 05:57.600 You can also use it in standalone mode, it's up to you. 05:57.600 --> 06:00.320 It's written in several programming languages. 06:00.320 --> 06:07.640 The Mender client will recently rewritten from goal to C++, with the goal to be able 06:07.640 --> 06:11.880 to run it on more constrained devices with real-time operating system. 06:11.880 --> 06:15.680 However, this look, as you understand, it's about embedded Linux, so I'm focusing on embedded 06:15.680 --> 06:16.680 Linux. 06:16.680 --> 06:22.200 So Mender supports the yok to project, but also Debbie and based distributions. 06:22.200 --> 06:26.440 I have to say that on a daily basis, I'm working with the yok to project and open and 06:26.440 --> 06:30.640 better, so the majority of the examples here are about the yok to project and open and 06:30.640 --> 06:35.480 better, but you can use these systems with different build systems or distributions like 06:35.480 --> 06:37.480 Debbie and as well. 06:37.480 --> 06:40.680 There's supports quite a lot of devices, these are the devices that are supported with the 06:40.680 --> 06:42.920 yok to project and open and better. 06:42.920 --> 06:50.440 Raspberry Pi, Roxy, Bigger Bone, everything that's popular is already supported, and in terms 06:50.440 --> 06:55.560 of the yok to project, Mender supports ScarbGuff, which is a long-term release, and it's 06:55.560 --> 06:58.760 going to be supported until 2028. 06:58.760 --> 07:03.320 So if you are starting to project right now, this is a very good choice to use the 07:03.320 --> 07:09.960 yok to project with the long-term support release, and if you have some of these devices, 07:09.960 --> 07:13.160 you can quickly try out Mender on them. 07:13.160 --> 07:20.800 This is how Mender works, but in general, this is valid for other systems that use AB updates 07:20.800 --> 07:21.800 in general. 07:21.800 --> 07:26.760 You apply the update and after that, you have to reboot the system, however, for Mender, 07:26.760 --> 07:32.160 what is specific that you have to inspect that the system boots find and to do a commit, 07:32.160 --> 07:38.240 to make sure that the update is finalized and everything is okay. 07:38.240 --> 07:40.040 So Mender works in two client modes. 07:40.040 --> 07:46.720 You can use it in a standalone client mode, or the mode where you can use the Cloud Infrastructure 07:46.720 --> 07:49.680 with Mender for management of leads of devices. 07:49.680 --> 07:54.880 So this is an example of how to do the integration with the yok to project and open, but 07:54.880 --> 07:59.800 it's here just for reference, there are some specific variables that you have to set. 07:59.800 --> 08:03.880 We don't have enough time, so I'll just skip and fast forward, but if you're interesting 08:03.880 --> 08:07.680 in this type of things, please talk to me after the talk. 08:07.680 --> 08:13.520 So Mender has the data partition, this is the partition where the persistent information 08:13.520 --> 08:19.280 should be stored, so across AB updates, you're updating the rootFS, on the next update, 08:19.280 --> 08:26.960 you have a new rootFS, so whatever your applications need must be stored in this data partition. 08:26.960 --> 08:33.080 And also the Mender client uses the data partition to store its own configurations. 08:33.080 --> 08:37.040 The advantage of Mender is that it has several adults, so it's not just for software 08:37.040 --> 08:41.480 over the error updates, but you have several adults for additional features that you can 08:41.480 --> 08:44.720 do with your embedded device. 08:44.720 --> 08:52.360 Mender also supports delta updates, however, this is part of the commercial plan for Mender. 08:52.360 --> 08:57.120 It's a close proprietary solution, and we're at Phosem, so I'm not going to talk in detail 08:57.120 --> 08:59.240 about this, but yeah, it has it. 08:59.240 --> 09:04.840 The next thing that I would like to discuss is Rauk, how many of you are using Rauk? 09:04.840 --> 09:14.160 Wow, so many people, all right, so Rauk, it's an open source, a lightweight update 09:14.160 --> 09:19.960 client that runs on the embedded device, it has a lot of advanced features, it supports various 09:19.960 --> 09:27.800 multiple updates and areas, the advanced features that are really cool are HTTP streaming 09:27.800 --> 09:30.360 and adaptive updates. 09:30.360 --> 09:38.560 And security is a very important part of Rauk, it has been there since the beginning, 09:38.560 --> 09:44.960 so Rauk is really good if you are looking for flexibility and a truly open source, fully 09:44.960 --> 09:47.600 open source solution. 09:47.840 --> 09:53.960 The licenses of the different bits and projects that are available for Rauk is the source 09:53.960 --> 09:59.920 quota for Rauk, is available under the Rauk organization in GitHub. 09:59.920 --> 10:06.160 So these are the stats, how to do a proper Rauk integration, so if you pick up a board, 10:06.160 --> 10:12.160 that's a very custom and you haven't run Rauk on it, you have to do these things, you 10:12.160 --> 10:17.040 have to select an appropriate booth loader, most of the time it's your booth, but not 10:17.040 --> 10:26.080 necessarily, you have to enable SquashFS, use X4 as a root file system, and create specific 10:26.080 --> 10:31.400 partitions, well this is something specific for the Rauk to project an open and better, 10:31.400 --> 10:36.680 and we have examples for this as well, and there's a Rauk system conf where you put configurations 10:36.680 --> 10:42.080 how the slots are configured, Rauk is very flexible, so you have different options how to 10:42.080 --> 10:45.960 deal with the data partition. 10:45.960 --> 10:50.200 You already know from my previous slides why the data partition is important, the good 10:50.200 --> 10:58.320 thing about Rauk is that you can cover various different scenarios, whatever fits your needs. 10:58.320 --> 11:03.800 I most of the time I use this one because it's simple, but not necessarily, sometimes you 11:03.800 --> 11:06.240 may need something more complicated. 11:06.240 --> 11:11.840 The advanced features, the HTTP streaming is really good because if you have a server that 11:11.840 --> 11:20.160 supports HTTP streaming, you can download and install the Rauk update simultaneously, and 11:20.160 --> 11:26.200 this is very convenient, save stimes, and also you don't need to download a huge file, 11:26.200 --> 11:30.960 store it somewhere, and keep in mind that if it's constrained and better device, there might 11:30.960 --> 11:35.560 not be enough space on the device to download the data, I've been in this open situation, 11:35.560 --> 11:42.120 and the HTTP streaming is really good because it solves this problem and the adaptive 11:42.120 --> 11:49.320 updates is something that I've mentioned because it's kind of delta updates in a different 11:49.320 --> 11:56.880 implementation, so as you can see, the solutions are going to combine strategies, it's 11:56.880 --> 12:03.760 because people need it, and step by step we're getting there, we have different solutions, 12:03.760 --> 12:08.200 but they're all going through this path of combining strategies for updates. 12:08.200 --> 12:12.240 Metal Rauk community is a young to open a better layer that I started five years ago. 12:12.240 --> 12:19.120 They deal with to have a very simple demonstration, how to use Rauk on Raspberry Pi, soon after 12:19.120 --> 12:25.360 that, other people started contributing to it, and we converted this from Metal Rauk 12:25.360 --> 12:32.640 Raspberry Pi to Metal Rauk community, and Riko from Pegatronics approached me and said, 12:32.640 --> 12:38.080 would you like to move this layer as part of the Rauk organization, I said, yeah sure, 12:38.080 --> 12:42.480 and now we have several machines that are supported in Metal Rauk community, I'm the main 12:42.480 --> 12:48.480 trainer, contributions are always welcome, so if you want to add another word, you're 12:48.480 --> 12:53.920 welcome to do it, there's a lot of work to be done on Metal Rauk community, I'm doing this 12:53.920 --> 13:00.000 primary in my spare time, and we can do a lot of improvements, so if someone wants join, 13:00.000 --> 13:04.880 if someone is looking for an open source project to contribute, Metal Rauk community is there, 13:04.880 --> 13:13.600 and we need your help. The next solution is SW update, who is using KSW update, quite a lot of 13:13.600 --> 13:21.600 people, so it's another flexible open source update framework with small footprint, and it does 13:21.680 --> 13:28.240 automatic updates, it's again compatible with the Yoke to Project, but also with build 13:28.240 --> 13:32.880 route and the packages, as far as I know with the Depp packages, it's experimental, I've used 13:32.880 --> 13:38.720 SW update again with the Yoke to Project and Open and Better, so my experience with it is true this. 13:40.880 --> 13:50.960 Here are the licenses, and I would like to do, as we are moving on, I would like to do some 13:50.960 --> 14:00.160 side-by-side comparison of the features that Mender Rauk and SW update have, so all of them are doing 14:00.160 --> 14:07.680 AB updates, all of them are good at doing AB updates, so when you are picking up a solution, 14:08.640 --> 14:14.160 each one of these solutions is good to do AB updates, each one of them is good to do rollback, 14:14.160 --> 14:20.400 rollback is really important, because you need your system to be always working, 14:21.120 --> 14:28.480 and the rollback provides you an opportunity to switch to a non-previous state if the 14:28.480 --> 14:36.160 data you've just installed is not properly working. Mender has some additional adults that I mentioned, 14:36.160 --> 14:42.320 these are configured on the monitor and the travel shooting adults, these are things that are 14:42.400 --> 14:48.240 not directly related to the software updates, it's just the features that you get with Mender 14:49.120 --> 14:55.440 as a bonus for managing your devices, they're not present in the other systems, 14:56.000 --> 15:03.200 how are the other systems have some additional unique advantages? For example, SW update has a local 15:03.280 --> 15:11.040 weapon interface, which means that it's a web server that runs on the embedded device itself, 15:11.760 --> 15:18.720 and if you are in the same local area network, you can open in a web browser this web server 15:18.720 --> 15:29.360 and upload your update through the weapon interface of SW update, which is quite nice in specific 15:29.360 --> 15:36.080 use cases, most of the time I'm using all these solutions for software over the year updates, 15:36.080 --> 15:42.400 but not necessarily, sometimes you can do USB updates, which means that someone is going 15:42.400 --> 15:48.640 with a USB stick, plugs the USB stick, you have to create your deaf rules and automatically 15:48.640 --> 15:54.240 to install the updates, sometimes you can do this local web interface. 15:54.480 --> 16:04.480 So, this is another side-by-side comparison, which is how the implementation, the licenses, 16:06.560 --> 16:11.520 the programming languages, so for Mender, this is really curious, I know that go is a big thing, 16:11.520 --> 16:17.600 but they had to switch from go to C++, so technologies are staying, although we see a lot of 16:17.600 --> 16:22.160 new technologies, still some solid technologies are used, and it makes sense to use them just because 16:22.160 --> 16:32.000 they have more constraint footprint, and we have, yeah, the other project integrations here, 16:32.000 --> 16:38.480 all of them work with ScarbGuff, for Rout, we are also maintaining the master branches, so some of the 16:38.480 --> 16:44.800 boards are working with the cutting-edge with the master, the contributions, Mender, and Rout 16:44.800 --> 16:51.680 are accepting contributions through the standard GitHub pull requests, with SW update, it's using 16:51.840 --> 16:58.240 main-inclist, and the measurement server is an advantage of Mender because you get a 16:58.240 --> 17:05.280 turnkey solution, while the other systems are focused on the open source and the Linux implementation. 17:06.400 --> 17:13.360 If you're looking for a third-party server to use with SW update, or Rout, there are some options, 17:13.360 --> 17:21.280 like a clip scope bit, this is an open source solution, QB, also you can use AWS, IoT, and integrate 17:21.280 --> 17:30.080 updates through it. One project that's worth mentioning is Lip, Uboot, and Anvil, so basically this 17:33.040 --> 17:41.120 this is a project that allows you to read and modify Uboot variables from the Linux user space, 17:41.120 --> 17:46.480 and this is super important, if you're doing software over year updates or software updates, 17:46.480 --> 17:51.680 with the three systems that we mentioned, because somehow you need to tell the bootloader 17:51.680 --> 17:55.200 that it's time to switch, and this is done by setting a variable there. 17:56.880 --> 18:04.720 Another thing is combining these solutions with containers. Yes, that's possible. I know that there are 18:04.720 --> 18:10.160 a lot of people in love with containers, that's something that's widely used nowadays, even on 18:10.240 --> 18:15.920 and bad devices, and you can integrate, for example, Docker with any of these solutions, 18:15.920 --> 18:21.680 and I recommend you to make sure that the container stays on the data partition as in the 18:21.680 --> 18:29.440 remains after the data of the data first of the device. So at the end of the talk, I would like 18:29.440 --> 18:38.960 to discuss the to wrap it up with some conclusions. One question, is there anyone here who's 18:39.680 --> 18:43.520 using a proprietary in-house solution for software over the Arab days? 18:44.800 --> 18:49.040 All right, one, two, oh, quite a lot, okay. No judging. 18:49.200 --> 19:01.120 All right, I think it turned. So it makes sense, especially if you have started this many years ago, 19:01.840 --> 19:08.320 and you have to maintain it. So no judging here, I know that there are no universal solutions. 19:08.320 --> 19:15.680 However, if you're starting a new project today, I highly recommend you to pick up an open source 19:15.760 --> 19:23.120 solution for software updates and just focus on the core features of your product, because things have 19:23.120 --> 19:29.760 changed over the years, the past decade, they're really good software updates solutions nowadays, 19:30.320 --> 19:35.360 very solid. I think the biggest challenge is to pick the one that suits you best, 19:35.360 --> 19:41.840 sometimes this could be problematic. So make sure that you're picking a solution that fits your needs. 19:42.800 --> 19:51.200 The duo AB update mechanism is a good choice. It takes more storage, but it's more convenient, 19:51.200 --> 19:55.200 because you always have a B-partition that's there, and you can switch to it. 19:57.600 --> 20:00.480 Obviously, depends on the boot loader and you need to reboot the system. 20:01.760 --> 20:08.480 So MenderRowk and SWV update, all of these solutions are really good for AB updates based on my experience. 20:08.800 --> 20:12.880 There are other good solutions out there. I just have more experience with these solutions. 20:12.880 --> 20:18.400 That's why I'm talking more about them. MenderRowk is a little bit hard of game, because 20:18.400 --> 20:26.240 it provides a turnkey solution that you can just use with the whole fleet, with the whole 20:27.120 --> 20:34.080 management UI in the cloud for fleets of devices. And also, of course, Delta and Adaptive 20:34.080 --> 20:40.000 updates are available for some of these solutions, like addition to the AB updates. 20:40.880 --> 20:47.680 And once again, choosing the best solution, open source solution depends on your needs, 20:48.240 --> 20:53.600 but I highly recommend you to go with an open source solution. Thank you very much for your time. 20:54.640 --> 20:55.280 Any questions? 20:55.920 --> 21:08.960 Thanks for that. That was great. Cut a couple of quick questions. Have you worked with Foundries.io at all? 21:09.680 --> 21:15.280 Because we use them, we find them to be quite good for Delta updates, containers, or let's have a chat. 21:15.280 --> 21:17.440 How big are you for? Foundries.io. 21:17.440 --> 21:22.480 Yes. Actually, I think it was here the slides. 21:26.240 --> 21:33.680 Okay, so they have actualizer light. This is the name of the open source solution actualizer light, 21:33.680 --> 21:41.760 but Foundries are the major development there. Yeah. Cool. And, you know, we mostly were with 21:41.760 --> 21:45.680 about the Linux, with the Octo open and about it, but we do some sort of art class-based stuff. 21:47.680 --> 21:52.720 I would try to quite useful to be able to leverage something like, you know, an existing open source 21:53.600 --> 22:00.080 framework that can also be deployed down onto, you know, some of these efforts. I had a little bit of 22:00.080 --> 22:06.240 a lot with Mender, just updating ESP's. And I wonder if you know of any of these other 22:07.040 --> 22:12.880 solutions that can actually sort of be applied to more of an art class environment like Zephyro 22:12.880 --> 22:18.000 for your art class or something else? Um, yes. Well, I'm not, um, more folks on the 22:18.720 --> 22:22.400 Linux. So I don't have practical experience with real-time operating systems and Zephyro 22:22.400 --> 22:28.880 in terms of software updates. I know that Mender is now supporting Zephyro. I think Mender, 22:28.880 --> 22:32.880 actually, yesterday we were discussing this with a friend, Mender, and he recommended the Mender, 22:32.880 --> 22:39.360 I haven't used Mender for however. I recommend the Mender for things Zephyro updates. 22:40.000 --> 22:47.680 And the whole reason why Mender read written the client from go to C++ was to support these 22:48.640 --> 22:50.480 real-time operating systems like Zephyro. 22:59.920 --> 23:04.160 Thanks. So if you have a more complex system to PIDATE with more, 23:04.160 --> 23:09.200 to be components like FPGA or my controller that you still also want to PIDATE, 23:09.840 --> 23:16.000 I know that you can bundle those binaries within as the PIDATE bundle, right? 23:16.080 --> 23:21.920 Can you do that also with rock? Or would you suggest one of the other for this kind of more 23:21.920 --> 23:28.720 complex updates? Well, yeah, it's a good question. And yeah, it's a very valid scenario. 23:29.840 --> 23:35.600 Different solutions that we covered. They have different naming, for example, some of them 23:35.600 --> 23:41.520 called the file that you distribute to device artifacts are called it around bundle and so on. 23:41.600 --> 23:47.200 But in general, with some adjustments, you can achieve this with any of these systems, 23:47.200 --> 23:53.600 because you can deploy the update. I'm speaking broadly here. And after that execute the script 23:53.600 --> 23:58.800 and many of these solutions have these hooks that you can execute a bunch of scripts to 23:58.800 --> 24:04.160 finalize this relation and deploy the different components of this update to different 24:04.160 --> 24:10.480 even different hardware devices. I hope that the sensors question in general. 24:11.840 --> 24:19.120 So you mentioned Mender has the server side counterpart that rock and SWP don't have. 24:19.120 --> 24:25.200 That's an advantage, of course. But question is, is it kind of mandatory to have it or is 24:25.200 --> 24:31.280 it supported to use Mender without? And if it does, does it make sense in some use cases to 24:31.280 --> 24:39.040 use Mender without the Mender server counterpart? Yes, it is possible to use it in a standalone mode 24:39.040 --> 24:47.040 where you can run, for example, an HTTP server locally and do the update from the server, 24:47.040 --> 24:51.920 actually during development, most of the time I'm doing this, because it makes sense from the 24:52.000 --> 24:59.600 development perspective. For your other question, since Mender is a commercial enterprise, 24:59.600 --> 25:04.320 probably they would say no, I'm not sure if it's a good idea, actually. But yes, in general, 25:04.320 --> 25:13.600 you can use the Mender artifacts with a third party solution for managing the devices. 25:13.840 --> 25:23.200 Talk to me later. And the guy would have has really cool stickers. 25:26.640 --> 25:28.400 So if you like three stickers, quiz the man. 25:30.800 --> 25:37.040 I have a question here. You mentioned that rock has some support for streaming and know that there 25:37.040 --> 25:45.280 is some support for delta update, but it relies on CAsync and it is not clear to me if CAsync, 25:45.280 --> 25:50.400 or casting a country where Mender is pronounced, is still maintained somehow. 25:51.600 --> 25:55.920 Yeah, but they switched to adaptive updates, which I think uses a different implementation. 25:55.920 --> 26:01.840 And for the HTTP streaming, yeah, I use it on daily basis when I'm dealing with rock, 26:01.840 --> 26:06.880 because it's very convenient. The only trick here is that you have to run a server that's 26:06.880 --> 26:13.600 supports HTTP streaming. Okay, I think we run out of time, sorry, but I'll be around so if you have 26:13.600 --> 26:18.400 any additional questions, please reach out.