WEBVTT 00:00.000 --> 00:07.000 Thanks, Fabian. 00:07.000 --> 00:13.000 All right, for we get started, real quick show hands, who here is already used Epleton. 00:13.000 --> 00:14.000 All right. 00:14.000 --> 00:25.000 Well, you know, what about Epleton 9, 8, 7, 6, yeah, 5, 4, okay. 00:26.000 --> 00:28.000 A lot of y'all been using it for a long time. 00:28.000 --> 00:29.000 That's great. 00:29.000 --> 00:31.000 One more, two more show hands. 00:31.000 --> 00:34.000 Who is the late migrating to a new major OS version, 00:34.000 --> 00:37.000 waiting for Epleton packages to get created? 00:37.000 --> 00:40.000 Oh, there'll be more people. 00:40.000 --> 00:44.000 What about who is the late upgrading to a new minor OS version, 00:44.000 --> 00:46.000 waiting for some Epleton packages to get fixed 00:46.000 --> 00:49.000 because they got broken in the minor version of date? 00:49.000 --> 00:50.000 All right. 00:50.000 --> 00:52.000 Troy does this problem. 00:53.000 --> 00:55.000 So that's the stage for what we're going to talk about today. 00:55.000 --> 00:58.000 The road to Epleton, my name is Carl George, 00:58.000 --> 01:01.000 and I end the Epleton team lead at Red Hat. 01:01.000 --> 01:04.000 So first off, we'll level set a bit and talk about what Epleton is. 01:04.000 --> 01:06.000 In case anyone's new or not a familiar, 01:06.000 --> 01:09.000 based on the show hands a lot of y'all do already know, 01:09.000 --> 01:12.000 but Epleton stands for extra packages for enterprise Linux. 01:12.000 --> 01:16.000 It's an initiative within the Thor project to create additional packages for Centos 01:16.000 --> 01:21.000 and Red Hat enterprise Linux with a goal of enhancing the base distribution 01:21.000 --> 01:26.000 without disturbing or replacing any of the stock packages. 01:26.000 --> 01:30.000 So Epleton packages are derived from Fedora. 01:30.000 --> 01:33.000 This diagram here is a rough approximation of how it works, 01:33.000 --> 01:35.000 but it's not drawn to scale. 01:35.000 --> 01:37.000 There's way more than 90 packages in Fedora. 01:37.000 --> 01:43.000 A subset of those will get branch off and they go into Centos and then Rell. 01:43.000 --> 01:46.000 Everything else that doesn't, isn't part of that subset, 01:46.000 --> 01:50.000 is eligible to go into the Epleton repository over on the side here. 01:50.000 --> 01:54.000 If a package later gets added to Centos and Rell, 01:54.000 --> 02:00.000 we'll remove it from Epleton to make sure we continue to not conflict with any of those packages. 02:00.000 --> 02:04.000 Let's zoom in a bit on that Fedora to Centos to Rell process. 02:04.000 --> 02:06.000 I talked about that subset. 02:06.000 --> 02:10.000 So this is the legacy relationship how things used to work. 02:10.000 --> 02:13.000 Fedora Rell hides the role in release version of Fedora. 02:13.000 --> 02:17.000 Fedora versions will branch off from that about every six months. 02:17.000 --> 02:20.000 And then in the past, Red Hat would take one of those versions. 02:20.000 --> 02:25.000 And behind the corporate firewall do private development to create the next Rell major version. 02:25.000 --> 02:27.000 And then they would just one day say, 02:27.000 --> 02:32.000 Tadah, here's Rell 7.0, and it's public now, and it's open source. 02:32.000 --> 02:37.000 At that time, the Centos project would take that source code and rebuild their distribution. 02:37.000 --> 02:41.000 And it tried to match Rell as close as possible. 02:41.000 --> 02:45.000 That makes a useful distribution, but there's a lot of problems with it. 02:45.000 --> 02:49.000 The goal at the time, the state of goal was bug for bug compatibility, 02:49.000 --> 02:53.000 and that if a bug was reported, if that bug was also in Rell, 02:53.000 --> 02:55.000 Centos wouldn't fix it. 02:55.000 --> 02:56.000 They would just leave it. 02:56.000 --> 03:01.000 And that was honestly a frustrating experience both for users and maintainers. 03:01.000 --> 03:04.000 And that also meant that contributing to Rell, 03:04.000 --> 03:07.000 there wasn't a way to do it directly because it was developed in private. 03:07.000 --> 03:12.000 You had to contribute to Fedora, and then maybe in three to five years, 03:12.000 --> 03:14.000 it would go in the next version of Rell. 03:14.000 --> 03:15.000 The next major version. 03:15.000 --> 03:17.000 So the contribution path was terrible. 03:17.000 --> 03:20.000 Fixing bug reporting process was not good. 03:20.000 --> 03:21.000 A lot of problems. 03:21.000 --> 03:23.000 We've moved on and fixed it. 03:23.000 --> 03:24.000 Good news. 03:24.000 --> 03:28.000 This is the modern relationship we have with the three distributions. 03:28.000 --> 03:33.000 Still, I've fedora high, moving along, and the Fedora versions branch off. 03:33.000 --> 03:37.000 But that branching to create the path to get to the next real major version, 03:37.000 --> 03:40.000 happens in public in the Centos project. 03:40.000 --> 03:43.000 That right there's the bootstrap phase where everything's getting settled 03:43.000 --> 03:45.000 and decisions are made on libraries. 03:45.000 --> 03:49.000 And then, from that point, once we, at some point, we'll get, we'll say, 03:49.000 --> 03:52.000 this is settled down, and this is now more or less stable, 03:52.000 --> 03:54.000 and we will have the Centos announcement. 03:54.000 --> 03:58.000 We just had that for Tim back in December just a month or two ago. 03:58.000 --> 04:02.000 And at that point, it serves as the major version branch of Rell. 04:02.000 --> 04:06.000 The minor version still branch off from that, but they are separate, 04:06.000 --> 04:10.000 and then the major version is just there in the Centos project. 04:10.000 --> 04:13.000 This fixes those problems I talked about. 04:13.000 --> 04:17.000 Now, when Centos gets a bug, we can just go ahead and fix it in CentOS, 04:17.000 --> 04:22.000 and then it becomes part of the next real minor version through to six months later. 04:22.000 --> 04:26.000 And fixing those bugs also gives us the path to contribution. 04:26.000 --> 04:29.000 You don't just have to contribute to Fedora and wait for years. 04:29.000 --> 04:34.000 Now, you can contribute to CentOS and see results in a matter of months instead of years. 04:34.000 --> 04:39.000 That's a huge improvement for people that want to get involved with the project. 04:39.000 --> 04:44.000 In this way, with it shaping the next minor, becoming the next minor versions, 04:44.000 --> 04:47.000 CentOS defines Riddhead and Fresnelenix. 04:47.000 --> 04:52.000 It still has to follow those compatibility rules, though. 04:52.000 --> 04:57.000 If CentOS were to change something or accept a contribution that changed something too drastically, 04:57.000 --> 05:02.000 that could cause problems and break compatibility between, say, Rell 10.1 and 10.2. 05:02.000 --> 05:06.000 And the business folks on the Riddhead side of the house, they wouldn't like that. 05:06.000 --> 05:10.000 There's contractual obligations about what can change between the minor versions. 05:10.000 --> 05:11.000 And those are defined. 05:11.000 --> 05:16.000 If you ever look it up, it's called the application compatibility guide that has the specific things listed about how 05:16.000 --> 05:22.000 compatibility works in Rell and what major version compatible means in minor version compatible. 05:22.000 --> 05:26.000 So, because it has to follow those rules, it's still very similar to Rell, 05:26.000 --> 05:29.000 even though it doesn't try to match it exactly anymore. 05:29.000 --> 05:34.000 Historically, we've had two major problems with Apple. 05:35.000 --> 05:39.000 I kind of alluded to it a bit in the show of hands at the beginning. 05:39.000 --> 05:44.000 And the first one is the minor version problem we're going to talk about. 05:44.000 --> 05:49.000 Rell is very stable that compatibility guide I talked about, but it isn't frozen. 05:49.000 --> 05:53.000 Sometimes new minor versions will have a few packages that change a library interface. 05:53.000 --> 05:59.000 And when that happens, it's deemed compatible enough to follow the rules for Rell, 05:59.000 --> 06:05.000 but it could still require that third-party packages like Apple need to be rebuilt to be compatible with it. 06:05.000 --> 06:10.000 In the legacy model, their top Apple only built against Rell, 06:10.000 --> 06:15.000 and then it was used on SintoS and just treated the same part of that kind of bug for bug compatibility. 06:15.000 --> 06:20.000 They wanted it to be as close as possible, so those third-party packages work the same. 06:20.000 --> 06:26.000 And that mostly worked, except when a Rell minor version released came out, 06:26.000 --> 06:31.000 SintoS project had about a month gap where they were getting things together, rebuilding all the packages, 06:31.000 --> 06:34.000 until corresponding SintoS minor version would be released. 06:34.000 --> 06:39.000 It's not really shown in the diagram there, but SintoS 7 basically was 7.0 here, 06:39.000 --> 06:43.000 and then just became 7.1 here after a little bit of a gap and so on. 06:43.000 --> 06:47.000 During that gap, if an Apple package had become uninstallable, 06:47.000 --> 06:51.000 because of a change in a library in Rell, the maintainer had a hard choice. 06:51.000 --> 06:54.000 If they rebuilt it immediately to fix the problem for Rell users, 06:54.000 --> 06:59.000 they would then break it for SintoS users that were still on the previous minor version. 06:59.000 --> 07:02.000 If they ignored it so that they didn't break SintoS users, 07:02.000 --> 07:06.000 the Rell users were just left broken for up to a month or so. 07:06.000 --> 07:09.000 So there wasn't a good answer one way or the other, 07:09.000 --> 07:11.000 depending on the individual maintainer's priority, 07:11.000 --> 07:14.000 which users wanted to be more friendly to. 07:14.000 --> 07:17.000 But because it was only about a month time frame, 07:17.000 --> 07:19.000 the problem was mostly ignored. 07:20.000 --> 07:26.000 There wasn't really a clean way to solve it with the idea that we had to build Apple against Rell. 07:26.000 --> 07:32.000 In the modern relationship, we kind of flipped that problem on its head and also extended the duration. 07:32.000 --> 07:36.000 Now those library changes that are going into say Rell 10.1, 07:36.000 --> 07:40.000 we'll see them right around here in SintoS months ahead of time, 07:40.000 --> 07:45.000 instead of just a month gap, we've got a 3 to 6 month gap, the other direction. 07:45.000 --> 07:52.000 So that longer gap made that particular problem with minor versions harder to ignore. 07:52.000 --> 07:56.000 The first attempt at solving this problem was known as Apple Next. 07:56.000 --> 07:59.000 We launched it in June of 2021, 07:59.000 --> 08:02.000 and it was a way to rebuild against SintoS only when you needed to. 08:02.000 --> 08:04.000 When you had one of those library differences. 08:04.000 --> 08:08.000 It wasn't a complete duplication, it wasn't a whole separate Apple. 08:08.000 --> 08:12.000 It was just a small subset of those packages with a slightly higher version or higher release 08:12.000 --> 08:15.000 so that they would serve as an update for SintoS users. 08:15.000 --> 08:18.000 That it was you have Rell users still using Apple, 08:18.000 --> 08:21.000 just like they always had that didn't have to know about any changes. 08:21.000 --> 08:25.000 SintoS users though would need to have Apple and Apple Next enabled on their systems 08:25.000 --> 08:29.000 so they would get the right corresponding set of compatible packages. 08:29.000 --> 08:31.000 This solved the basic problem. 08:31.000 --> 08:34.000 It led maintainers have a path to solve to fix these packages, 08:34.000 --> 08:40.000 and it allowed them to target both Rell and SintoS at different times when needed. 08:40.000 --> 08:44.000 We implemented it without disturbing any of the existing Apple workflows. 08:44.000 --> 08:48.000 If maintainers didn't know about it, they could just keep doing what they were doing, 08:48.000 --> 08:50.000 and be none the wiser. 08:50.000 --> 08:54.000 There were a lot of problems with this that we found that we came to learn. 08:54.000 --> 08:57.000 It was often mistaken for Standalone repo. 08:57.000 --> 08:59.000 I mentioned it wasn't a complete duplication, 08:59.000 --> 09:02.000 but a lot of maintainers misunderstood it to be just that. 09:02.000 --> 09:05.000 Even though we had a document and we tried to tell people, 09:05.000 --> 09:07.000 but it wasn't intuitive for them. 09:08.000 --> 09:11.000 There was a decision process for when you needed to use Apple Next. 09:11.000 --> 09:16.000 You had to really research and pay close attention to the difference between the libraries and between SintoS and Rell. 09:16.000 --> 09:18.000 And there was no inheritance. 09:18.000 --> 09:20.000 If the maintainer fixed the package in Apple Next, 09:20.000 --> 09:24.000 they had to fix it again six months later in Apple, 09:24.000 --> 09:29.000 and then they couldn't take the bills they had done in Apple Next and reused them. 09:29.000 --> 09:33.000 And also, being optional meant that SintoS users had a worse experience. 09:34.000 --> 09:37.000 They may, I saw many times where somebody would report a bug and said, 09:37.000 --> 09:39.000 this package doesn't work on SintoS anymore. 09:39.000 --> 09:41.000 I know that Apple Next is for this and can fix it, 09:41.000 --> 09:45.000 and the maintainer just ignored it and left it because we made it optional. 09:49.000 --> 09:53.000 The second big problem that we've had historically with Apple is major versions. 09:56.000 --> 09:58.000 For a variety of technical reasons, 09:58.000 --> 10:01.000 if I can talk about more in the Q&A if people want to or afterwards, 10:01.000 --> 10:06.000 is that each Apple version starts fresh with a fresh set of packages. 10:06.000 --> 10:10.000 There's positives and negatives to that, but that is how it works. 10:10.000 --> 10:12.000 And so when a new major version of Apple starts, 10:12.000 --> 10:14.000 there's nothing in there at first. 10:14.000 --> 10:18.000 maintainers need to go in and opt in and add their packages to it. 10:18.000 --> 10:20.000 So when you first start looking at it, 10:20.000 --> 10:24.000 Apple 9 might not have all the same packages that were already in Apple 8. 10:24.000 --> 10:26.000 We don't automatically bring them over. 10:26.000 --> 10:31.000 And the biggest part of that reason for that is the duration that that would be. 10:31.000 --> 10:33.000 All the maintainers are volunteers, 10:33.000 --> 10:36.000 and we don't want any of them to feel like we're forcing them into, 10:36.000 --> 10:39.000 hey, you were maintaining Apple 8 for 10 years. 10:39.000 --> 10:41.000 Here's 10 more years on top of that with Apple 9. 10:41.000 --> 10:42.000 We automatically opted you into it. 10:42.000 --> 10:45.000 We leave it on the maintainers to choose that. 10:45.000 --> 10:49.000 But because of this, it calls some lag with packages being added, 10:49.000 --> 10:54.000 and that also the following effect was that people that were upgrading their OS version 10:55.000 --> 10:56.000 would wait. 10:56.000 --> 10:59.000 They said, you know, I'm not going to switch from, say, you know, 10:59.000 --> 11:05.000 send to a 6 to send to a 7 until all of the Apple 6 packages are in Apple 7 that I need. 11:05.000 --> 11:07.000 And it wasn't just on the send to a side. 11:07.000 --> 11:09.000 We also had that inside Red Hat. 11:09.000 --> 11:13.000 We had that feedback from customers, where it's harder to support customers 11:13.000 --> 11:15.000 on the oldest version of your product. 11:15.000 --> 11:18.000 You want to get them on the more modern versions that have more fixes. 11:18.000 --> 11:21.000 So when they push back and say, yeah, I'm not upgrading from, say, 11:21.000 --> 11:25.000 real 7 to real 8 until all of these packages are available on Apple. 11:25.000 --> 11:27.000 I know their community maintain packages. 11:27.000 --> 11:29.000 They're not, we can't file support cases for them, 11:29.000 --> 11:30.000 but I need them to exist. 11:30.000 --> 11:33.000 It was actively blocking our customers from upgrading. 11:33.000 --> 11:36.000 So we saw that as a problem as well. 11:36.000 --> 11:38.000 This feedback from customers. 11:38.000 --> 11:41.000 We didn't always know in this community side that it was a problem. 11:41.000 --> 11:45.000 But eventually the customer feedback led to the creation of the Apple team in 11:45.000 --> 11:48.000 SideRehat, which I now lead, started it. 11:48.000 --> 11:50.000 I was the first member of the team. 11:50.000 --> 11:54.000 The first thing we wanted to look at on our agenda was Apple 9. 11:54.000 --> 11:59.000 And we wanted to take a stab at trying to improve on that major version problem. 11:59.000 --> 12:03.000 And just, obviously, what we came up with originally was that we would start 12:03.000 --> 12:07.000 Apple next by itself for 9, build it against Sinto S9. 12:07.000 --> 12:12.000 And then quickly after the rail 9 launch, we would rebuild all the Apple 9 next packages 12:12.000 --> 12:14.000 and to populate the Apple 9 repo. 12:14.000 --> 12:16.000 Rebuild the against rail 9. 12:16.000 --> 12:19.000 We thought that would make sense when we designed the plan, 12:19.000 --> 12:22.000 but as we started trying to implement it, we realized this is a lot of extra work. 12:22.000 --> 12:23.000 It doesn't make sense. 12:23.000 --> 12:28.000 And so what we switched to was, we'd start Apple 9 just normal. 12:28.000 --> 12:31.000 But we would build it against Sinto S9 at first. 12:31.000 --> 12:35.000 We were telling people on the Sinto S9 that, you know, this is a preview of the next rail 12:35.000 --> 12:36.000 minor version. 12:36.000 --> 12:37.000 So this is our chance to prove it. 12:37.000 --> 12:39.000 You build it against Sinto S9 today. 12:39.000 --> 12:43.000 And in six months it's going to work on rail 9.0 because we said it's a preview. 12:43.000 --> 12:44.000 So it should work. 12:44.000 --> 12:46.000 And that's what we did. 12:46.000 --> 12:50.000 We started building and gave maintainer six extra months before the rail 9 launch 12:50.000 --> 12:52.000 to get their packages ready. 12:52.000 --> 12:57.000 And then at the rail 9 launch, once it was available, we switched the repo over two 12:57.000 --> 12:59.000 build against rail 9 proper. 12:59.000 --> 13:01.000 And the good news is that plan worked great. 13:01.000 --> 13:02.000 We did it as far as I know. 13:02.000 --> 13:06.000 We didn't get a single bug report related to compatibility problems 13:06.000 --> 13:07.000 between Sinto S and rail. 13:07.000 --> 13:09.000 What we said came to fruition. 13:09.000 --> 13:14.000 It worked as a preview and worked as a base to get things ready for the rail 9 launch. 13:14.000 --> 13:17.000 And I mentioned the amount of time. 13:17.000 --> 13:19.000 It was about five and a half, six months. 13:19.000 --> 13:23.000 So package maintainers had a lot of time to get their packages added. 13:23.000 --> 13:25.000 And the good news is that let us launch. 13:25.000 --> 13:31.000 When rail 9 came out, we had 5700 Apple packages available and ready for users 13:31.000 --> 13:33.000 in customers used just on day one. 13:33.000 --> 13:35.000 And that was a real good news. 13:35.000 --> 13:37.000 Lots of lots of applause. 13:37.000 --> 13:39.000 Everyone loved that inside the business at least. 13:39.000 --> 13:42.000 Made it look like I was doing my job on the Apple team at least. 13:43.000 --> 13:45.000 That's all context setting up where we're at. 13:45.000 --> 13:50.000 The main point of the talk, Apple 10. 13:50.000 --> 13:56.000 The big benefit of having an actual dedicated team to work on Apple is that not similar to Apple 9. 13:56.000 --> 13:59.000 We thought about it ahead of time about how we could start it early. 13:59.000 --> 14:02.000 And once we had that working, we're like, OK, well, let's think even further ahead, 14:02.000 --> 14:05.000 thanks strategically in the next step as Apple 10. 14:05.000 --> 14:08.000 What could we do to make that better and different? 14:09.000 --> 14:11.000 So it wasn't much longer after the Apple 9 launch. 14:11.000 --> 14:12.000 We started planning Apple 10. 14:12.000 --> 14:16.000 And some of the things we were realizing as we were planning this was that 14:16.000 --> 14:19.000 Apple next, I mentioned some of the downsides earlier. 14:19.000 --> 14:21.000 It was really a subpar solution. 14:21.000 --> 14:23.000 It was bolted on to the existing Apple workflows. 14:23.000 --> 14:26.000 It wasn't really a holistic solution. 14:26.000 --> 14:30.000 And the more meta point around it was that with Apple and Apple next, 14:30.000 --> 14:34.000 we were really just giving maintainers around about way to target two separate 14:34.000 --> 14:37.000 minor versions at the same time. 14:38.000 --> 14:41.000 And what we want to do is to really solve the problem, 14:41.000 --> 14:45.000 just lean into that approach and fully embrace minor versions in Apple. 14:45.000 --> 14:49.000 Historically, Apple had always been major version only and just kind of kept bouncing 14:49.000 --> 14:51.000 to the next minor versions of real as it went along. 14:51.000 --> 14:54.000 So we wanted to integrate with it better in a better way. 14:59.000 --> 15:03.000 So as we were looking at this, the more technical details of it, 15:03.000 --> 15:06.000 we wanted the branching process to be more intuitive for maintainers. 15:06.000 --> 15:09.000 I don't even have a diagram for how Apple next works because I don't even know how 15:09.000 --> 15:10.000 I would represent that. 15:10.000 --> 15:13.000 But we looked at how Fador relates to itself. 15:13.000 --> 15:15.000 We're all hiding the root release versions, how branch is off. 15:15.000 --> 15:20.000 And then also the new centOS model where real minor versions branch from the major version. 15:20.000 --> 15:24.000 So we tried to emulate that and make something that feels intuitive to maintainers. 15:24.000 --> 15:29.000 So we'll have an Apple 10 branch and then the Apple 10.0 branch comes off from that. 15:29.000 --> 15:32.000 This is right about the point where we're going to do that. 15:32.000 --> 15:35.000 And the Apple 10.0 branch comes off from that. 15:35.000 --> 15:38.000 This is right about the point where we're at right here right now. 15:38.000 --> 15:40.000 We have Apple 10. 15:40.000 --> 15:44.000 But it's technically Apple 10.0, but we don't have the Apple 10.0 branch yet. 15:44.000 --> 15:48.000 We'll create that shortly and then the Apple 10, the main branch, 15:48.000 --> 15:51.000 will become the preview of Apple 10.1. 15:51.000 --> 15:54.000 Very similar to house into us in real relates to each other now. 15:55.000 --> 16:07.000 In a similar way, I'll point out that Fador is in the same boat where if Fador 41 is the current release and raw hide, 16:07.000 --> 16:10.000 basically considers itself Fador 42 today. 16:10.000 --> 16:17.000 So it's like a preview of the next major version in the Fador space where it's a preview of the next minor version in the centOS real space. 16:25.000 --> 16:30.000 Really big point that I want to emphasize here is that the Apple next problem about the double rebuilds, 16:30.000 --> 16:36.000 where you have to rebuild for a library change in Apple next first and then Apple that goes away in this model. 16:36.000 --> 16:42.000 You start to rebuild for some libraries as they change, but that just happens once in the main branch. 16:42.000 --> 16:46.000 And then those changes will be prepared and ready for the next minor version. 16:46.000 --> 16:47.000 You don't have to do it twice. 16:47.000 --> 16:52.000 So we removed a lot of duplicate work there, or we will as this goes into the minor versions. 16:52.000 --> 16:58.000 We're just now starting at this all brand new territory for us. Very exciting. 16:58.000 --> 17:03.000 Aside from those branching mechanics, a higher level concept emerged during this planning. 17:03.000 --> 17:05.000 Apple is a community repo. 17:05.000 --> 17:12.000 So it makes sense for us to prioritize centOS, which stands for the community enterprise operating system. 17:12.000 --> 17:17.000 An Apple maintainer can somewhat to a degree, ignore rail, and just focus in community spaces. 17:17.000 --> 17:20.000 They can do Fador packages. They can do Apple packages. 17:20.000 --> 17:25.000 They can work with centOS, and they can just ignore that mostly ignore that rail exists as a product. 17:25.000 --> 17:30.000 They may get a bug report for something on rail that doesn't apply to centOS at some point. 17:30.000 --> 17:33.000 And we don't want them to ignore that entirely. 17:33.000 --> 17:43.000 But they can change their priorities and not necessarily have to worry about Red Hat products as much. 17:43.000 --> 17:46.000 We are going to have it set up where maintainers can still build. 17:46.000 --> 17:49.000 They can request older minor version branches. 17:49.000 --> 17:52.000 We'll have the automatic branch in similar to what Fador does. 17:52.000 --> 17:56.000 Any package with an Apple 10 branch will get a 10.0 branch automatically. 17:56.000 --> 18:00.000 And if you create the package after that point, you can go back and request the old ones, 18:00.000 --> 18:05.000 which will build against rail, but that won't be the default experience in workflow. 18:05.000 --> 18:07.000 We have a very similar thing on the Fador side. 18:07.000 --> 18:11.000 If you had a brand new package to Fador today, it won't go into Fador 41. 18:11.000 --> 18:15.000 You have to go backwards and ask for, hey, I also want a 41 branch and build it there too. 18:15.000 --> 18:20.000 If you just do it now, it'll go into 42 as it gets created. 18:20.000 --> 18:25.000 And coincidentally, that is also a very similar workflow for rail maintainers. 18:25.000 --> 18:27.000 I should really have this slide still a long time out this part. 18:27.000 --> 18:30.000 For rail maintainers, they're default experiences. 18:30.000 --> 18:33.000 They build it in centOS, and then it goes in the next rail minor version. 18:33.000 --> 18:38.000 If there's customer pressure to put something into a currently released minor version, 18:38.000 --> 18:45.000 maintainers have to go backwards and do a little extra work to put it in that one. 18:45.000 --> 18:50.000 All of these changes, though, aren't just beneficial for centOS, they also help rail quite a bit. 18:50.000 --> 18:55.000 Apple 9 used centOS 9 to start building packages for rail 9.0 early. 18:55.000 --> 19:01.000 And Apple 10 is kind of an extension of that philosophy, doing that for every minor version, not just the .0. 19:02.000 --> 19:05.000 And it avoids that double rebuild problem we talked about. 19:05.000 --> 19:11.000 So in the past, rail or currently, a new rail minor version comes out. 19:11.000 --> 19:13.000 And third party packages might not install on day one. 19:13.000 --> 19:17.000 I mentioned delaying upgrading minor versions until packages are ready. 19:17.000 --> 19:19.000 We can help get rid of that problem. 19:19.000 --> 19:23.000 Maybe not entirely, but for the most part, have those packages ready to go on day one. 19:23.000 --> 19:28.000 So nobody can say, well, I'm not upgrading yet to get all these security fixes because these packages broke. 19:28.000 --> 19:30.000 We want those to work right away. 19:31.000 --> 19:32.000 If we can. 19:36.000 --> 19:38.000 So enough planning. 19:38.000 --> 19:40.000 It's time for the rubber to meet the road. 19:40.000 --> 19:43.000 Last August, we held an Epipsin Hackfest at Flotk. 19:43.000 --> 19:46.000 This kicked off a soft launch period. 19:46.000 --> 19:48.000 Earlier than we ever had before for an Apple release, 19:48.000 --> 19:50.000 where maintainers could start getting to work. 19:50.000 --> 19:54.000 I mentioned with Apple 9 that giving them more time proof to be really popular, 19:54.000 --> 19:57.000 and helped us have a lot of packages ready quickly. 19:57.000 --> 20:01.000 And so the logical thing was we had this great plan all these minor versions, 20:01.000 --> 20:06.000 but let's let's just start when we, you know, let's pick an earlier time to start. 20:06.000 --> 20:08.000 Starting too early in that branching process, 20:08.000 --> 20:11.000 there may be some libraries still in flux that are changing. 20:11.000 --> 20:19.000 One thing I noticed was that we had Pearl got rebased from 538 to 540 in that bootstrap phase. 20:19.000 --> 20:22.000 Luckily, we didn't have that many Pearl packages in Epipsin yet. 20:22.000 --> 20:24.000 I think we only had to rebuild two of them. 20:24.000 --> 20:26.000 But that's the kind of thing that makes it where like, 20:26.000 --> 20:29.000 yeah, this isn't why we don't, this is why we don't start like a year earlier. 20:29.000 --> 20:34.000 We need a time to just write. 20:34.000 --> 20:40.000 But after a few months of this, Epipsin officially launched back in December. 20:40.000 --> 20:43.000 Thanks to the hard work of our community of Epipackagers, 20:43.000 --> 20:48.000 we were able to launch Epipsin with over 10,000 packages available. 20:48.000 --> 20:53.000 And remember, we had Epiply 9 up to 5,700 by the time of the Real and Mind launch, 20:53.000 --> 20:55.000 and we're not even at the Real 10 launch yet. 20:55.000 --> 20:58.000 But the way you had it where we were in the past. 20:58.000 --> 21:03.000 And since I made these slides, it's already gone over 11,000 packages. 21:03.000 --> 21:06.000 So I'm really curious to do the update version of this talking to the future. 21:06.000 --> 21:10.000 When we have that package count of compared to 5,700 at Real Mind launch, 21:10.000 --> 21:13.000 at Real 10 launch, we've got X number, whatever it is. 21:13.000 --> 21:15.000 So that's our story so far. 21:15.000 --> 21:17.000 It's far from complete though. 21:17.000 --> 21:20.000 There's all expected timelines, no guarantees, 21:20.000 --> 21:24.000 but next month we're going to automatically create those Epiply 10.0 branches I've talked about. 21:24.000 --> 21:27.000 For every package with an Epiply 10 branch. 21:27.000 --> 21:31.000 We'll temporarily build that branch against the snapshot of CentOS 10. 21:31.000 --> 21:34.000 To make sure we don't get any of the changes that are coming in last minute for, 21:34.000 --> 21:37.000 or the changes that start happening for 10.1. 21:37.000 --> 21:42.000 And then whenever Real 10.0 actually exist, we'll switch to building into that. 21:42.000 --> 21:47.000 And then the Epiply 10 branch will move on to be start populating Epiply 10.1. 21:47.000 --> 21:51.000 And that's real similar to what we did for Epiply 9 with that snapshot mechanics. 21:51.000 --> 21:54.000 A little bit of inside baseball down on the weed stuff. 21:54.000 --> 21:57.000 We'll just keep repeating that for every minor version. 21:57.000 --> 22:00.000 We'll rinse and repeat that. 22:00.000 --> 22:03.000 And that's what we have coming up, and that's what we've been. 22:03.000 --> 22:05.000 And that was the road to Epiply 10. 22:05.000 --> 22:08.000 And maybe I have a time for a question or two.