WEBVTT 00:00.000 --> 00:17.400 So, last year, I spent some time interviewing a lot of developers who had just started building 00:17.400 --> 00:22.800 with AI and specifically started playing around with using large language models. 00:22.800 --> 00:27.000 And from those conversations, there was like a few different themes that came up. 00:28.000 --> 00:29.000 And you're in a picture. 00:29.000 --> 00:31.000 Ah, what are you? 00:31.000 --> 00:34.000 It's around here that you can hear that you can hear that. 00:34.000 --> 00:37.000 And yeah, there was kind of a few different themes that came up. 00:37.000 --> 00:39.000 Like, one of them was some developers, 00:39.000 --> 00:43.000 was, sometimes they'd find a really cool open source project on GitHub, 00:43.000 --> 00:46.000 but then the first thing that would come up as soon as they tried to start using it, 00:46.000 --> 00:49.000 was they'd have to enter that open AI API key. 00:49.000 --> 00:51.000 That was really annoying, especially if they didn't want to spend money, 00:51.000 --> 00:54.000 and they wanted to keep that data private. 00:55.000 --> 00:58.000 Another thing that came up, the second one you see there, 00:58.000 --> 01:00.000 and this one was probably the one that came up the most, 01:00.000 --> 01:02.000 it's just this feeling and sense of overwhelmed, 01:02.000 --> 01:05.000 I think Stephen touched upon it as well earlier in the space. 01:05.000 --> 01:08.000 And just how many different tools, how many different frameworks, 01:08.000 --> 01:11.000 how many different models you have to navigate when you're starting out. 01:11.000 --> 01:15.000 And not only is the difficulty come from having to pick the right things, 01:15.000 --> 01:19.000 but then even if you pick certain tools, certain frameworks, certain models, 01:19.000 --> 01:21.000 there were a lot of interoperability issues. 01:21.000 --> 01:23.000 And sometimes some of these projects are quite new, 01:23.000 --> 01:25.000 and a new release comes out, and it breaks everything. 01:25.000 --> 01:29.000 And that was causing a lot of headaches for developers just starting out. 01:29.000 --> 01:32.000 And then the third one that came up a little bit as well, 01:32.000 --> 01:34.000 is just like this feeling of pressure, 01:34.000 --> 01:38.000 and this need to kind of be able to experiment and prototype quickly. 01:38.000 --> 01:41.000 And in a lot of that comes down to this space, 01:41.000 --> 01:43.000 being and moving very quickly. 01:43.000 --> 01:48.000 So what a lot of developers kind of wanted was to feel like they could easily experiment, 01:48.000 --> 01:51.000 and swap out different, swap out data, swap out models, 01:51.000 --> 01:53.000 swap out tools, and that needs to be. 01:53.000 --> 01:57.000 So with Blueprints, we wanted to start trying to address some of these pain points 01:57.000 --> 01:59.000 that were coming up with coming up for developers, 01:59.000 --> 02:03.000 and really trying to make sure that using open source tools and open source models 02:03.000 --> 02:06.000 is a default for developers. 02:06.000 --> 02:08.000 So what are Blueprints? 02:08.000 --> 02:12.000 So Blueprints are essentially customizable workflows 02:12.000 --> 02:17.000 that allow developers to prototype AI applications using open source models and tools. 02:17.000 --> 02:22.000 When we're building Blueprints, there's kind of generally five principles 02:22.000 --> 02:24.000 that we're trying to stick to. 02:24.000 --> 02:28.000 The first one is that we only use open source models and tools, 02:28.000 --> 02:31.000 obviously the code itself is open source. 02:31.000 --> 02:34.000 Secondly, we're trying to make them really customizable, 02:34.000 --> 02:38.000 so that you can adapt the Blueprint to your needs with your own data for your own use case. 02:38.000 --> 02:41.000 Thirdly, we're trying to make them extensible. 02:41.000 --> 02:45.000 So we see the Blueprint basically as the foundation code that you can build on top of, 02:45.000 --> 02:49.000 and extend further, if your needs. 02:49.000 --> 02:53.000 Fourthly, we kind of want to build up a collection of different Blueprints 02:53.000 --> 02:55.000 that show you how to do things differently. 02:55.000 --> 02:59.000 And as part of that, we want to have some standardization across different Blueprints. 02:59.000 --> 03:01.000 So we're trying to build them in a consistent way, 03:01.000 --> 03:04.000 and we think that'll make them easier to leverage for people, 03:04.000 --> 03:07.000 especially people starting out in AI and starting to use algorithms. 03:07.000 --> 03:10.000 And then finally, I think this is the most important one. 03:10.000 --> 03:12.000 We want us to be community driven. 03:12.000 --> 03:16.000 So we've started building our own Blueprints that we've put out there, 03:16.000 --> 03:20.000 but really we want the community to also contribute and build some, 03:20.000 --> 03:22.000 as well, so that we can build up this collection of Blueprints 03:22.000 --> 03:28.000 to really show people how you can start with open source models and open source tools. 03:28.000 --> 03:33.000 So, AQR code, maybe not as many as Stephen, coming up. 03:33.000 --> 03:35.000 Oh, there is. 03:35.000 --> 03:38.000 So to kind of like kickstart this project, 03:38.000 --> 03:41.000 we've released something recently in a beta. 03:41.000 --> 03:42.000 It's called the Blueprints Hub. 03:42.000 --> 03:45.000 So please feel free to scan this QR code or take you to the Hub. 03:45.000 --> 03:50.000 Basically, this is going to be our website for collecting all the different Blueprints 03:50.000 --> 03:53.000 and kind of showcasing them in a user-friendly way. 03:53.000 --> 03:57.000 So people can kind of get started using them and testing them nice and easily. 03:57.000 --> 03:59.000 For now, this is a beta, so we've only got a couple so far. 03:59.000 --> 04:02.000 But over the next month or two, we'll be adding a lot more, 04:02.000 --> 04:07.000 and hopefully it's going to showcase some of the communities Blueprints as well. 04:07.000 --> 04:09.000 So now I'm going to hand over to David, 04:09.000 --> 04:13.000 he's going to talk us through on the first couple of Blueprints we made. 04:13.000 --> 04:14.000 Hello. 04:14.000 --> 04:19.000 So, the first one that we built is called Documentuposcast. 04:19.000 --> 04:24.000 As the name states, it's combatsa.commentuposcast. 04:24.000 --> 04:28.000 The idea is basically that pretty much everyone of you 04:28.000 --> 04:31.000 have probably heard about Notebook LLM by Google, 04:31.000 --> 04:33.000 and it has a postcard feature. 04:33.000 --> 04:38.000 So the question was, can we actually replicate this with open source tools? 04:38.000 --> 04:41.000 We did some research, find some implementations, 04:41.000 --> 04:46.000 that didn't fulfill the details that we, the bullet points that we said before, 04:46.000 --> 04:51.000 either they require an API call to some model or whatever. 04:51.000 --> 04:53.000 So we give it a try. 04:53.000 --> 04:54.000 I don't have a QR code. 04:54.000 --> 04:56.000 I trust in your ability to type. 04:56.000 --> 04:58.000 Yeah. 04:58.000 --> 04:59.000 Sir. 04:59.000 --> 05:02.000 Start Documentuposcast. 05:06.000 --> 05:07.000 Try again. 05:07.000 --> 05:12.000 I guess, image loading, not super optimized here. 05:12.000 --> 05:15.000 So here is just a screenshot of the GitHub rhythm. 05:15.000 --> 05:18.000 So if you go there, you are going to see the same. 05:18.000 --> 05:23.000 It's basically to state that there are three options to run it, 05:23.000 --> 05:25.000 depending on who you want to pay the hardware. 05:25.000 --> 05:28.000 And you can choose to Google to pay the hardware, 05:28.000 --> 05:33.000 and you go to a Google collaboratory and run the Blueprint there. 05:33.000 --> 05:38.000 It's probably the easiest option, and the one where you can customize, 05:38.000 --> 05:42.000 try to tweak the prompt, try a different number of speakers, 05:42.000 --> 05:45.000 try a different model, wherever you want. 05:45.000 --> 05:49.000 Then there's also high emphasis spaces, if you want high emphasis to pay, 05:49.000 --> 05:52.000 and then there's GitHub, if you want Microsoft to pay. 05:52.000 --> 05:57.000 If you want to run locally, there is a pipeline package that you can install, 05:57.000 --> 06:02.000 and try directly if not you can clone the GitHub repository. 06:02.000 --> 06:06.000 And the next one is a working progress in the school. 06:06.000 --> 06:12.000 A structure QA, and this might not be as obvious title as the previous one. 06:12.000 --> 06:15.000 The idea was basically that I was working on the podcast. 06:15.000 --> 06:17.000 I liked board games. 06:17.000 --> 06:18.000 It was Christmas. 06:18.000 --> 06:21.000 I was playing with my family, and the literary told me, 06:21.000 --> 06:23.000 why don't you do something useful? 06:23.000 --> 06:27.000 That can answer questions about the board game rules, 06:27.000 --> 06:31.000 because we're having a little bit of a discussion about the rule of a game. 06:32.000 --> 06:40.000 So a structure QA, the idea is that many of the question answering approaches out there, 06:40.000 --> 06:43.000 assume that the world is absolutely chaos. 06:43.000 --> 06:46.000 No one has the document well structured. 06:46.000 --> 06:51.000 Well, in my experience, for example, in board games, people actually think about 06:51.000 --> 06:57.000 a structure in the document into section, putting some reasonable section names, 06:57.000 --> 06:58.000 so you can navigate them. 06:58.000 --> 07:03.000 And this is also true, I believe, for many technical documentation and other places. 07:03.000 --> 07:11.000 So we wanted to give it a try, whether what of the existing approaches work the best. 07:11.000 --> 07:12.000 Yeah. 07:12.000 --> 07:15.000 So the first thing we did, we collected a small benchmark. 07:15.000 --> 07:17.000 We run different solutions. 07:17.000 --> 07:23.000 So the MENA is probably the largest, has the largest context window out there, 07:23.000 --> 07:26.000 and is the most easy to use for free. 07:27.000 --> 07:32.000 So as a benchmark of what you can achieve with open source, we run the benchmark on the MENA. 07:32.000 --> 07:38.000 These are the results, and then in comparison, we run it against QA and 2.57 billion. 07:38.000 --> 07:43.000 I actually was trying to run the benchmark against DeepCick just today, 07:43.000 --> 07:49.000 and over the weekend, but it keeps freaking out about trying to answer the question thinking, 07:49.000 --> 07:55.000 instead of just creating the document, like reasoning the, so it didn't work. 07:56.000 --> 07:59.000 So yeah, this is a working progress. 07:59.000 --> 08:03.000 We are trying to implement an alternative solutions to the long context, 08:03.000 --> 08:07.000 that is not feasible locally and to the rough implementations. 08:07.000 --> 08:13.000 Here I choose Raga Tool, because it's such a cool name that I have to use that. 08:13.000 --> 08:18.000 So I guess I'll just end my thing. 08:18.000 --> 08:21.000 These are kind of the first couple that we worked on. 08:21.000 --> 08:25.000 We're going to be working on some more, but we'd love to hear suggestions from you on what we think, 08:25.000 --> 08:27.000 what you think we should work on next. 08:27.000 --> 08:30.000 If you've seen something where maybe there's a good close source solution, 08:30.000 --> 08:32.000 or something that works well with a good close source model, 08:32.000 --> 08:35.000 but if you want to see an open source alternative, 08:35.000 --> 08:37.000 please feel free to submit an idea here, 08:37.000 --> 08:40.000 or if you think you can contribute to this project and build something yourself, 08:40.000 --> 08:43.000 feels of scan this QA code and submit something. 08:43.000 --> 08:46.000 We'd love to work with you. 08:46.000 --> 08:47.000 That's it. 08:47.000 --> 08:48.000 Thank you. 08:52.000 --> 08:55.000 Okay, we have questions. 08:55.000 --> 09:00.000 And we don't call the we have questions on the chat. 09:00.000 --> 09:03.000 No, I don't ask questions now. 09:03.000 --> 09:06.000 This is the room first. 09:06.000 --> 09:08.000 Should I put you? 09:08.000 --> 09:12.000 I'll put you. 09:12.000 --> 09:14.000 Quick question. 09:14.000 --> 09:18.000 Do you are you expected in the person who submitted the question 09:18.000 --> 09:20.000 to be a developer? 09:20.000 --> 09:23.000 Oh, I got to talk a lot louder about me too. 09:23.000 --> 09:28.000 So do you expect the person submitting the question to be a developer? 09:28.000 --> 09:33.000 Or is this open to designers and open to anyone? 09:33.000 --> 09:34.000 Yeah, literally anyone. 09:34.000 --> 09:37.000 If you've got something you think would be cool to work on, 09:37.000 --> 09:39.000 we'd love to hear about it. 09:39.000 --> 09:41.000 And then we can chat about your idea, 09:41.000 --> 09:42.000 and you don't have to develop it, 09:42.000 --> 09:44.000 maybe we can find someone else too. 09:45.000 --> 09:49.000 Team, do we have questions on the chat? 09:49.000 --> 09:50.000 No, we're done. 09:50.000 --> 09:51.000 Thank you very much. 09:51.000 --> 09:52.000 Thank you. 09:52.000 --> 09:58.000 Again, if you leave the room, do so with that door or that door? 09:58.000 --> 09:59.000 The...