WEBVTT 00:00.000 --> 00:20.500 Okay. Hello everyone. Today I'm going to talk to you about Bluezy, Bluezy, I don't think 00:20.500 --> 00:27.000 Bluezy project for implementing Bluezy in production. 00:28.000 --> 00:34.000 First of all, where am I? I'm George. I'm a principal software engineer at Klabera. 00:34.000 --> 00:40.000 I have been working on multimedia things for the past 14 years. I worked with just your 00:40.000 --> 00:47.000 mirror, I'll show you a pipe wire, I'm the maintainer of wire plumber. I've contributed 00:47.000 --> 00:54.000 to automotive grade Linux, audio staff there, and I'm near to Bluezy. 00:58.000 --> 01:11.000 Now, in the past seven, eight months, I had a new adventure that stepped out of my audio 01:11.000 --> 01:17.000 zone, let's say. I teamed up with my awesome colleague Frederick Dines, and together we deployed 01:18.000 --> 01:26.000 Bluezy as the Blue to stack of real world vehicle in for payment system, which is currently 01:26.000 --> 01:33.000 under going testing for qualification, and as far as I know it's the first vehicle that is 01:33.000 --> 01:40.000 going to actually use Bluezy in production. 01:40.000 --> 01:45.000 The project involved actually switching away from a proprietary stack. Previously, that 01:45.000 --> 01:53.000 system had a proprietary Bluetooth stack that the customer that we worked with wanted to 01:53.000 --> 01:59.000 remove and replace with Bluezy, and the goal was to just implement the same features, 01:59.000 --> 02:08.000 no, nothing different. Along this process, we also had the chance to improve the code base, 02:08.000 --> 02:14.000 improve a lot of open source components, and I personally learned a lot. I learned about Blue Tooth, 02:14.000 --> 02:23.000 the audio part. I learned also about audio use cases on the automotive, like how the audio 02:23.000 --> 02:33.000 DSPs work and what kind of requirements they have there. Now, in this talk, my intention is to 02:33.000 --> 02:39.000 pass on what I learned for other people that might want to do the same thing later on, 02:39.000 --> 02:46.000 like a change from a proprietary Bluetooth stack over to Bluezy. What should you expect 02:46.000 --> 02:56.000 if you want to do that? First of all, let me say that it works. It works pretty well. 02:56.000 --> 03:02.000 There is no difference from the proprietary stack. We support media player, phone calls, text 03:02.000 --> 03:08.000 messaging, the contact list, access of course, and pairing controllers, like gaming and 03:08.000 --> 03:16.000 roles, for example. The architecture that we used looks like this. So inside the Linux 03:16.000 --> 03:22.000 kernels, there are some parts of Bluezy. There are drivers that talk to the Bluetooth 03:22.000 --> 03:28.000 adapter, and implement some protocols, and then on top of that, we have Bluetooth D, 03:28.000 --> 03:34.000 the main Bluetooth D, and then we have OBEXD that implement some of the profiles for 03:34.000 --> 03:43.000 objective change. For example, the contact list exchange works through OBEXD. On the other 03:43.000 --> 03:50.000 side, we have pipe wire and wire plumber, which was used to implement the audio profiles. 03:51.000 --> 03:57.000 Namely, the A2DP profile, which is the advanced audio distribution profile, which is used for 03:57.000 --> 04:04.000 transporting audio from the media player or the video player, and the HSP profile, which is the 04:04.000 --> 04:15.000 hands-free profile, which is used for phone calls. On top of that, there is some proprietary 04:15.000 --> 04:21.000 middleware and UI components that the customer already had, and they just wanted to 04:21.000 --> 04:26.000 switch that to talk to Bluezy instead of the proprietary stack that they were previously 04:26.000 --> 04:35.000 using. As part of this process, we added some things. First of all, we added 04:35.000 --> 04:42.000 telephony support in pipe wire in the hands-free profile implementation. Like the hands-free profile 04:42.000 --> 04:49.000 implementation was already there in pipe wire, which was being used for enabling headsets. 04:49.000 --> 04:56.000 So if you have a Bluetooth headset and you want to enable bi-directional audio with a microphone, 04:56.000 --> 05:03.000 if you are using the HSP profile to do that. That was working, so the audio part was working, 05:03.000 --> 05:11.000 but what we were missing was some control to be able to send commands to the phone to let's 05:11.000 --> 05:19.000 say make a call or answer a call or put a call on hold. Now this with this functionality 05:19.000 --> 05:25.000 pipe wire is now able to replace a phone, which was the only project that the only component 05:25.000 --> 05:35.000 that could do that before, and we decided to do that in pipe wire instead it was somewhat simpler 05:35.000 --> 05:43.000 to add that in pipe wire. We also added the basic imaging profile support in Bluezy. 05:43.000 --> 05:50.000 The basic imaging profile is a profile that controls for images to Bluetooth. We added a subset of that 05:50.000 --> 06:00.000 that is able to download the cover art from the media player, like for the media player 06:00.000 --> 06:07.000 that is on the car. If you're using something like Apple Music or Spotify on your phone, 06:07.000 --> 06:13.000 it's a requirement that you have the same information together with the album art 06:13.000 --> 06:20.000 on the infotainment system. That was Merrick's in Bluezy, while the pipe wire telephony support 06:20.000 --> 06:28.000 is not yet merged. It lives in a merger quest. It needs a little bit more attention from me. 06:28.000 --> 06:38.000 We also fixed some other issues, mostly related to the audio video remote control profile in Bluezy. 06:38.000 --> 06:50.000 Now what were the challenges? As you can understand, we're working with people that came from a world where 06:51.000 --> 06:59.000 they only knew about how the proprietary stacks work. For us, for me, at least, I didn't know 06:59.000 --> 07:06.000 how the proprietary stacks work. I lived in my Bluezy open source world and they lived in their 07:06.000 --> 07:16.000 proprietary world and we had to understand each other. As we moved along the project, we found 07:16.000 --> 07:23.000 this difference with what is different. On proprietary stacks, it is very common that the 07:23.000 --> 07:30.000 Bluezy controller is exposed as a serial device and then you have a user-space demon 07:30.000 --> 07:37.000 that which is the proprietary stack. This is the software, the proprietary software. 07:37.000 --> 07:43.000 That basically grabs the serial port and then talks with the controller using the 07:43.000 --> 07:50.000 HCI protocol. That's the host controller interface protocol. There's nothing in the 07:50.000 --> 08:01.000 kernel space except for the serial driver or something like that. It is probably a good 08:01.000 --> 08:08.000 part for the people that implement this demon because they can reuse the same thing 08:08.000 --> 08:14.000 in other operating systems. It doesn't matter if you're implementing this one Linux or 08:14.000 --> 08:20.000 something else. It's just a serial port as far as the operating system is concerned. 08:20.000 --> 08:27.000 Now with Bluezy, this is different. The serial port is attached directly inside the kernel. 08:27.000 --> 08:36.000 The kernel then exposes. The kernel actually implements internally the main Bluetooth 08:36.000 --> 08:44.000 protocols. It talks HCI with the controller. It exposes L2 cap which is a higher level 08:44.000 --> 08:51.000 protocol and score which is another higher level protocol. You can basically request 08:51.000 --> 08:58.000 rockets from the kernel. This rockets have the Bluetooth address family. Similar to 08:58.000 --> 09:08.000 how we have the internet address family and we have the CPP IP to DP sockets. 09:09.000 --> 09:18.000 So what makes sense? 09:18.000 --> 09:29.000 Yeah, that makes sense but it is different from what people are expecting. 09:29.000 --> 09:38.000 Now some other expectations that were different and things that we were looking at. 09:38.000 --> 09:45.000 One is that in Bluezy, if you're using Bluezy, the profiles are not really implemented 09:45.000 --> 09:52.000 in one place. You don't have one demon. You have multiple demons and multiple components in those 09:52.000 --> 10:00.000 multiple versions because they are different projects. You cannot really say that this version 10:00.000 --> 10:08.000 of Bluezy supports Bluetooth 5.2, for example, 5.4 or 6. There is no such thing. 10:08.000 --> 10:15.000 You have to collect the PCs based on the version of the Linux kernel that you have. 10:15.000 --> 10:22.000 Based on the version of Bluezy that you have, the version of pipe wire. Bring it all together. 10:22.000 --> 10:30.000 Bring in whatever UI that you have or other management software that you may have to make this all work. 10:30.000 --> 10:35.000 Then you have to assess what version you are supporting. You have to run actually the tests. 10:35.000 --> 10:39.000 You have to do the testing and say, okay, this supports this feature. 10:39.000 --> 10:46.000 Take the list from the Bluetooth qualification. 10:46.000 --> 10:52.000 Look at each test and say, okay, this should sound. So this is what we support. 10:52.000 --> 11:01.000 But yeah, that wasn't something that people were expecting. So they were asking, like, okay, which version of Bluetooth does this version of Bluezy support? 11:01.000 --> 11:06.000 And we're saying there's no such thing. 11:07.000 --> 11:15.000 Another thing is there's no straightforward way to filter the API packets, which sometimes can be a security requirement to implement firewalls, 11:15.000 --> 11:23.000 like to filter what is going on between the controller and the host. 11:23.000 --> 11:32.000 Unfortunately, there is no, even though the architecture looks similar to the internet address family, where we have net filter in the kernel, 11:32.000 --> 11:36.000 and filter the packets, there's no such thing for Bluetooth. 11:36.000 --> 11:43.000 There are some solutions, but they're not as straightforward as it is with internet packets. 11:43.000 --> 11:46.000 There's no support for containers as well. 11:46.000 --> 11:54.000 So right now, it is a requirement that you run Bluezy on the host operating system, on the host container. 11:54.000 --> 12:08.000 You cannot really isolate using network namespaces and run Bluetooth D in a different network namespace than the main one. 12:08.000 --> 12:13.000 These are fixable of course, but needs some work. 12:13.000 --> 12:22.000 Other differences that we saw is Bluezy hides complexity in general. It tries to be abstracted, tries to provide you with a nicer interface. 12:22.000 --> 12:33.000 But the customer codebase had been already developed using the proprietary stack, so they had access to low 11 information before. 12:33.000 --> 12:42.000 And that was clashing sometimes, like the logic that they were implementing didn't match exactly to the logic that Bluezy exposed. 12:42.000 --> 12:45.000 So we had to work around that. 12:46.000 --> 12:53.000 They were also using a lot of logging to monitor things, to monitor problems. 12:53.000 --> 13:01.000 And we had to add some logs also in Bluezy and pipe wire all around the place, just to make sure that they collect all the information. 13:01.000 --> 13:09.000 Also, they were expecting to collect the logs from one place, from one demon, and that was no longer the case. 13:09.000 --> 13:19.000 Depending on which profile you want it to debug, you have to look at different demons and different logs and correlate information, and that is more challenging. 13:19.000 --> 13:22.000 Sometimes there's not enough error granularity in Bluezy as well. 13:22.000 --> 13:31.000 So one use case that I can remember is you have a phone that disconnects and Bluezy says it disconnected, but why? 13:31.000 --> 13:38.000 There can't be a number of reasons why this connected. It could be because the user just disconnected manually. 13:38.000 --> 13:42.000 It could also be because the phone went out of range. 13:42.000 --> 13:52.000 And these are different use cases, but there is no such information given in Bluezy, while there was in the other stack. 13:52.000 --> 14:05.000 And also, Bluezy maintains state. So Blue Tuesday, for example, maintains the list of pair devices internally, and it has that in a directory structure. 14:05.000 --> 14:14.000 That wasn't the case with the proprietors stack, and there was another layer that was already doing that in the customer's middleware. 14:14.000 --> 14:36.000 And when we discovered that we realized there was some friction there, because the higher level middleware was already trying to do stuff that we were already doing, and sometimes mismatching with Bluezy. 14:36.000 --> 14:54.000 Another pain point is vendor specific HCI commands. So HCI, as I said earlier, is the host controller interface, is the protocol that you talk to between the host FFC and the controller. 14:55.000 --> 15:12.000 There are standardized HCI commands that talk to controllers, but there are some vendor specific as well, like vendors add stuff to make sure that they implement custom functionality. 15:12.000 --> 15:18.000 So many profiles, we actually found many profiles that need the vendor specific HCI commands, and that wasn't easy to handle. 15:18.000 --> 15:34.000 One particular that I personally handled was the HFP profile, the HHP profile, which typically on embedded systems, the audio that goes from the phone to the Bluetooth chip. 15:34.000 --> 15:52.000 It doesn't pass to the main FFC, but instead there is a hardware path between the controller and the audio DSP, where the audio travels directly through the hardware path, and the main FFC never gets the audio. 15:52.000 --> 16:02.000 And that's a little bit obscure, we don't exactly know how that works, every vendor has a different way of implementing that. 16:03.000 --> 16:25.000 And in the case of the particular chip that this customer is using, in order to make this hardware path work, we had to send vendor specific commands to configure the audio format, the sample rate, like the codec that was being used, and things like that. 16:25.000 --> 16:43.000 And we had to do that every time that the audio path was opened and then closed. So every time that you had a phone call, you had to send this commands to open the path and then as soon as the phone calls stops, send the commands again to close the path and continue. 16:43.000 --> 16:56.000 Which is pure unnecessary pain, this could have been part of the driver, like this is something that belongs in the kernel, really, and it's so static. 16:56.000 --> 17:10.000 Like the format, the sample rate, etc, are static information that depend on how the scholarships have been wired together, it doesn't depend on parameters that come from the phone or anything. 17:10.000 --> 17:20.000 It's a really information that could be encoded in the device tree or somewhere else to make this work, but it's not there. 17:20.000 --> 17:29.000 And yeah, that's a pain point, we had to patch you the space to do that, you the space sending HCI commands to enable that path. 17:29.000 --> 17:51.000 And with that comes the next problem which is security concerns of doing that. So if you allow you the space to send HCI commands, you need to give it capabilities and the first capability you need to do is to give the row the network row capability at allows sending row packets. 17:51.000 --> 17:58.000 With some commands, some functionality we also need this cabinet admin, so we need it to be root. 17:58.000 --> 18:20.000 On this particular system, nothing runs as roots, they are all isolated, they have, they are giving capabilities one by one, and the customer was not pleased, obviously, when we decided to add cabinet row to pipe wire in order to be able to send these commands to enable the audio path. 18:21.000 --> 18:29.000 I'm happy they were not pleased, that's actually a good thing they care about security, that's good. 18:29.000 --> 18:39.000 And well, eventually we found the solution to that, which was very specific to their system, and it's not something generic. 18:39.000 --> 18:49.000 We need to, for a generic solution, we need to really fix that somewhere else, I think the kernel is the better place. 18:49.000 --> 18:55.000 Also, there were questions about the music demon itself requiring those capabilities. 18:55.000 --> 19:02.000 Like they ask why does bluesy require cabinet admin to run? 19:02.000 --> 19:16.000 But the answer they are is that this is part of the bluesy design, so bluesy requires that these permissions, so that it can execute privileged operations, which is a security feature. 19:16.000 --> 19:29.000 Basically, the kernel only allows the bluesy demon to run those operations, and everybody else has to talk to bluesy with debas. 19:29.000 --> 19:40.000 Instead of having all the user space components, talk directly to the kernel and try to do privileged operations that would have been a disaster. 19:40.000 --> 19:47.000 And some non-challenges, something that we thought was a challenge, but it wasn't, eventually, the card parsing. 19:47.000 --> 19:59.000 So the card is the file that describes contacts, when you download contact list from the phone, the contacts cameras as the card files. 19:59.000 --> 20:09.000 And initially we thought, okay, we need to find the parser for this, we looked at parsers, we found parsers in GNOME and KDE, other places on GitHub, 20:09.000 --> 20:14.000 they were too hard to package to many dependencies. 20:14.000 --> 20:25.000 And then we looked at automotive red Linux, which had something else, it had actually a parser itself, like it was very simple parser there. 20:25.000 --> 20:28.000 And we adopted the solution. 20:28.000 --> 20:38.000 We implemented the VCAR parsing again, which is so simple, it's much easier to write a parser than to package any data. 20:38.000 --> 20:58.000 Now, next steps, this project concluded, of course, for the community, the things that have been contributed are there, all the open source bits have been upstream or have been pushed to merge requests. 20:59.000 --> 21:11.000 Something that I would like to do after that is to expose the telephony bits of pipe wire to the desktop, so have an application that you can make calls. 21:11.000 --> 21:21.000 I think GNOME calls exist, which is possibly an interesting target, maybe there are other desktop apps that I haven't looked at yet. 21:21.000 --> 21:37.000 But yeah, I was thinking that if functionality that you have on the Mac, for example, where if you receive a phone call, it automatically shows you on the laptop that you have a call and you can answer from the laptop directly, that's something useful. 21:37.000 --> 21:49.000 And hoping to get that also into what if great Linux, like get all the changes into automotive green Linux again and have a demonstrator again in 2025. 21:50.000 --> 21:52.000 Thank you very much. 22:06.000 --> 22:10.000 Do we have any questions? 22:11.000 --> 22:19.000 So you touched mostly upon all the challenges for implementing this. 22:19.000 --> 22:28.000 We're also some notable advantages over choosing this, this open source stack over the proprietary solution. 22:29.000 --> 22:47.000 No, the goal was to achieve future parity, the advantage is open source, right, so the customer wanted to use open source for, you know, for the future to to be to have access to the source code and be part of the community. 22:47.000 --> 22:51.000 That was the reason, no technical advantage. 22:59.000 --> 23:12.000 So I know most of the proprietary Bluetooth stack solutions, they, the vendors basically say, well, we've tested with this, you know, several hundred phones. 23:12.000 --> 23:23.000 Does that, is that something your OEM customer worries about or is there discussion about trying to enable more testing against lazy upstream. 23:24.000 --> 23:30.000 So in our case, no, we didn't care about this hundreds or thousands of phones. 23:30.000 --> 23:41.000 We mostly cared about five or six specific ones that are, you know, the 99% of phones that are being used by their customers because. 23:41.000 --> 23:50.000 They have the telemetry data to understand which phones are being used the most, so that's what we've tested with. 23:50.000 --> 23:58.000 How did you solve the who owns the, the data about pair devices, did you. 23:58.000 --> 24:02.000 I'm impressed about, did you change, Blue Z. 24:02.000 --> 24:16.000 We modified the customers logic there, so we kept Blue Z as it is, so Blue Z maintains the pair devices and the, the layer above adopt. 24:16.000 --> 24:18.000 Yep, I think that's the end of the time. 24:18.000 --> 24:20.000 Right, thank you very much.