WEBVTT 00:00.000 --> 00:11.560 I want to start today with a story, and we're going to hear lots of stories today, but I like 00:11.560 --> 00:12.560 signal. 00:12.560 --> 00:15.560 I use signal, maybe some of you, also use signal. 00:15.560 --> 00:22.760 And you might have noticed during the latest AWS East Outage, that's your signal stop 00:22.760 --> 00:23.760 working. 00:23.760 --> 00:26.320 Did anybody have similar experience? 00:26.320 --> 00:31.400 Yeah, and I live in Berlin, and I was messaging other people in Berlin, and I found 00:31.400 --> 00:38.400 this quite peculiar, of course, later Meredith Whitaker was asked, what happened, and it 00:38.400 --> 00:45.480 was like, well, there's no other choice, all right, there's no other choice. 00:45.480 --> 00:52.560 Okay, at least she also said that's a problem, right, there's no other choice. 00:52.560 --> 00:59.160 But what I'm going to ask us today repeatedly is, is it a story or is it a fact? 00:59.160 --> 01:04.760 I'm a scientist, I come from machine learning science, data science, and what I want 01:04.760 --> 01:11.000 us to keep asking around this sovereignty debate is, is it a story or is it a fact? 01:11.000 --> 01:15.840 And if we think it's a fact, I want us to ask, why do we believe that fact? 01:15.840 --> 01:20.440 I want us to assemble evidence, and I want to test it, like we would everything else 01:20.440 --> 01:21.440 with the scientific method. 01:21.440 --> 01:25.800 Okay, so who's with me, wants to talk about stories or facts? 01:25.800 --> 01:26.800 Yeah? 01:26.800 --> 01:27.800 Okay. 01:27.800 --> 01:31.640 All right, I'm going to tell you some other stories that are going around. 01:31.640 --> 01:37.800 This is the BSE, so if you live in Germany or familiar with the BSI, who does a lot of things 01:37.800 --> 01:43.200 around type of our infrastructure, security, and the policies around it, and they signed 01:43.200 --> 01:50.580 a deal with AWS because AWS is so advanced in cloud security that they're the only 01:50.580 --> 01:59.480 ones that the German government can learn from, right, is true, right, must be true, right? 01:59.480 --> 02:05.280 And this continues in a lot of the marketing that you look at around this, that says 02:05.280 --> 02:12.280 that sovereign clouds might be good for your legal department, but they lag behind in things 02:12.280 --> 02:15.280 like AI, it's just impossible. 02:15.280 --> 02:20.680 And this tends to be the story around U.S. technology companies with this idea that somehow 02:20.680 --> 02:26.920 there's something very special about U.S. technology companies or U.S. AI companies that means 02:26.920 --> 02:32.840 that nobody else in the entire world can understand how to do this. 02:32.840 --> 02:33.840 Okay? 02:33.840 --> 02:36.200 This is a story. 02:36.200 --> 02:40.840 Then there's another story going around, which is it's sovereign if it's controlled 02:40.840 --> 02:44.840 by us, astric for us, astric. 02:44.840 --> 02:51.240 And here's an example of T-Systems, and we see at the bottom it's trying to hide, right, 02:51.240 --> 02:57.520 like it's very secret, and T-Systems controls that, and that means it's sovereign regardless 02:57.520 --> 03:01.600 of what runs on top of it, right? 03:01.600 --> 03:09.160 And AWS went so far as to actually require EU citizenship to work on the AWS sovereign 03:09.160 --> 03:14.840 cloud, which means many persons in this room, including myself, who is not yet a citizen 03:14.840 --> 03:20.720 due to the long line in Berlin, would be unable to actually work on this regardless of 03:20.720 --> 03:30.120 our capabilities, right, because something about citizenship must bestow sovereignty. 03:30.120 --> 03:36.200 And then there's a third story, right, which is something something military and sovereign 03:36.200 --> 03:43.080 means, let's build military drones and police systems and all of these things. 03:43.080 --> 03:49.040 And we can see a lot of this push in the changes to the AI act that happened, which basically 03:49.040 --> 03:55.680 made an entire huge loophole for things like border control, military and police. 03:55.680 --> 04:02.760 So something, something sovereign T means, let's use all the AI that we can for such systems, 04:02.760 --> 04:03.760 right? 04:03.760 --> 04:05.760 At least you see this with T-S. 04:06.000 --> 04:11.800 Okay, so it's my opinion, and I hope you're with me that it's time to spread other stories. 04:11.800 --> 04:16.240 And I think I was very inspired earlier today, and the AI plumbers room, I think there's already 04:16.240 --> 04:19.840 some stories that I'm hearing that are echoed in this talk. 04:19.840 --> 04:24.480 So let's spread some different stories. 04:24.480 --> 04:30.200 First and foremost, I think very happily that I hear for many years here at Faustem is 04:30.280 --> 04:35.600 let's try thinking about building hardware and infrastructure differently, which coming from 04:35.600 --> 04:40.040 the machine learning side, so my background is in machine learning means we need to have 04:40.040 --> 04:43.960 new silicon chips and we need to have new designs for silicon chips. 04:43.960 --> 04:47.360 And how do we deploy this with any reasonable security? 04:47.360 --> 04:52.600 Well, open Titan, which I think there might be some contributors to the project already here, 04:52.600 --> 04:55.480 is one such way that we can start to think about these things. 04:55.480 --> 05:02.960 And we can start to open up silicon for different types of machine learning workflows. 05:02.960 --> 05:08.160 Alongside, there's also some pretty cool conferences growing around how do we actually 05:08.160 --> 05:11.120 do this in more efficient ways? 05:11.120 --> 05:16.800 And whatever we mean when we say green computing, how do we do that for machine learning? 05:16.800 --> 05:21.520 And the eco-compute conference this year in Berlin was sold out, but I'd be very happy 05:21.560 --> 05:26.280 to hear of other conferences that you might recommend, where people are talking about, 05:26.280 --> 05:31.080 how do we actually increase efficiency of the types of workflows that we're running? 05:31.080 --> 05:39.400 And use that instead of thinking what is the next biggest gigabyte center or whatever we can build? 05:39.400 --> 05:45.400 And I want us to take very Bernie energy when somebody comes at us and they start talking 05:45.400 --> 05:50.360 about how everybody flops they have or other metrics like this that the giga factor is 05:50.400 --> 05:57.760 use because at the end of the day, this is usually an extremely inefficient use of the resources 05:57.760 --> 06:03.160 that are available is also not possible with the energy density in most places. 06:03.160 --> 06:07.840 And thirdly, I've been doing machine learning before giga factories, so I'm happily here 06:07.840 --> 06:13.360 to tell you, and many of you probably too, that you actually don't need giga factories 06:13.360 --> 06:18.760 to do whatever it is that we call AI today, and we don't always need the next biggest foundation 06:18.760 --> 06:23.320 model for obvious reasons. 06:23.320 --> 06:28.480 And how might we build things differently, both from a green compute point of view, but also 06:28.480 --> 06:33.360 from changing the way that we think about AI communities or machine learning communities? 06:33.360 --> 06:40.000 Well, I come from a background of decentralized machine learning, and I don't know for those 06:40.000 --> 06:44.800 of you that are also infedited learning or decentralized machine learning, flower labs, 06:44.800 --> 06:52.760 which is also European startup built out the first pre-trained LLM of 1.3 billion with 06:52.760 --> 06:59.400 the same accuracy of its competitors, all via federated learning across multiple continents 06:59.400 --> 07:06.480 with normal data center links, and these by the way were separate types of data center structures, 07:06.480 --> 07:10.160 so they weren't all on AWS or Google Cloud or something like this, right? 07:10.160 --> 07:14.720 So you can check out, I think there'll be some more exciting announcements this year for 07:14.720 --> 07:17.600 other models that will be released the same way. 07:17.600 --> 07:23.200 I also sit on the jury of the Splend composite learning challenge. 07:23.200 --> 07:27.440 I don't know if you're familiar with Splend, but it's a part of the German government 07:27.440 --> 07:31.320 that is looking for what they call Splendent Innovation. 07:31.320 --> 07:36.360 So how do we take a leap with our innovation to go from where we are today to something 07:36.360 --> 07:41.400 totally new, and the composite learning challenge is about how do we actually learn 07:41.400 --> 07:48.000 across very disparate types of hardware, and when we're able to do that, we're maybe able 07:48.000 --> 07:55.280 to expand our idea of what we need in terms of compute to actually do machine learning 07:55.280 --> 08:00.960 at scale, and even to do machine learning at a variety of sizes, so not always training 08:00.960 --> 08:05.840 something with billions of tokens or billions of parameters. 08:05.840 --> 08:10.760 And I was here, if you're in Berlin, there was a few last year that will be a few more 08:10.760 --> 08:18.400 this year distributed AI hacks where teams get together, compute was donated by AMD, and a 08:18.400 --> 08:23.840 few of the teams from the composite learning provided the software, and some of the infrastructure 08:23.840 --> 08:25.360 layer for running these. 08:25.360 --> 08:30.440 So I encourage you to also do these types of things, and what we're moving there is 08:30.440 --> 08:33.120 to what we call participatory learning. 08:33.120 --> 08:38.200 So a place where you can show up with your own hardware, with your own data, and be able 08:38.200 --> 08:45.240 to contribute to building a model, which you can also take a copy home with you, right? 08:45.240 --> 08:50.400 And to do this, we also need to enable, kind of, how do you use that model, right? 08:50.400 --> 08:54.400 Once you have it, or how do we reason about the different open weight models that are 08:54.400 --> 08:56.400 already available today? 08:56.400 --> 09:01.120 Well, over in the other room right now, unfortunately, at the same time, there's some 09:01.120 --> 09:07.080 of the core contributors of GGML talking, which is basically, if you've ever run one of 09:07.080 --> 09:14.400 the LLMs on your computer, there is a good chance that some part of it was using LMSCPP, 09:14.400 --> 09:18.800 and then there's a good chance that that is also then using GGML. 09:18.800 --> 09:23.520 And this is essentially ways that we can take something like TensorFlow computations, and 09:23.520 --> 09:28.680 we can move them over to a variety of different hardware backends, and hopefully, eventually, 09:28.680 --> 09:33.280 over time, this becomes kind of seamless and something that people can use. 09:33.280 --> 09:39.000 But GGML, as far as I understand, still has like three core contributors, right? 09:39.000 --> 09:44.040 So we really are, again, in this precarious types of situations where we have a very small 09:44.040 --> 09:49.880 amount of libraries supporting, again, a very large or growing infrastructure around running 09:49.880 --> 09:52.560 models locally. 09:52.600 --> 09:58.480 We have a new act in the EU called the Data Act, it went into a effects last year, which 09:58.480 --> 10:03.480 basically gives you the rights, if you're an EU resident, that if you buy something that 10:03.480 --> 10:08.160 you have a rights to get the data from it, how this will actually play out. 10:08.160 --> 10:12.800 I hope it doesn't play out like the GDPR data portability, which basically, a lot of 10:12.800 --> 10:17.440 companies just hand-waved and give you your user form back. 10:17.440 --> 10:22.080 But what we really are trying to get is how do we get at the data that you can then take 10:22.160 --> 10:28.560 along with your local libraries, and you can run things locally for you and buy you 10:28.560 --> 10:32.000 of you being an individual in this case. 10:32.000 --> 10:35.640 Whoops. 10:35.640 --> 10:43.680 And I ran last year at Picon Germany, Pided Germany, the feminist AI land party. 10:43.680 --> 10:48.320 How many people are old enough to have attended a land party before? 10:48.720 --> 10:54.960 Yes, if you want to come to Domestat, if you want to bring your computer, come to our 10:54.960 --> 10:59.680 land party this year, we're going to host it again, it's an April, but we've basically 10:59.680 --> 11:07.120 had a few different computers, that's my little machine, it's got 32 GB GPU in it, and we hosted 11:07.120 --> 11:14.000 and served for inference, numerous models, we did a little bit of fine tuning experimentation, 11:14.080 --> 11:21.120 I ran a workshop on hacking LLMs for feminism, which is a very fun exercise, and yeah, 11:21.120 --> 11:24.720 if you want to donate computers, it's free, you don't have to have a ticket to Picon, 11:24.720 --> 11:30.400 you can just show up, so I want to see more events like this, we open source all our kits, 11:30.400 --> 11:32.160 so if you want to run your own, please do. 11:33.200 --> 11:38.880 But by this, I mean, bringing your data, bringing your compute, getting together as a community, 11:38.880 --> 11:44.320 and really having a participatory way, though it kind of also debunk some of these myths 11:44.320 --> 11:46.880 around how machine learning and AI works. 11:48.800 --> 11:54.080 On top of that, so most of my work is actually on deploying privacy technologies in deep learning 11:54.080 --> 11:59.360 systems, what does that mean? That is a lot of random words. What part of it means is different 11:59.360 --> 12:05.600 of privacy, has anybody here heard of different privacy? Use different privacy? 12:05.600 --> 12:14.640 Yes, I want to see more of that hand, so different privacy is basically a robust, mathematically-based 12:14.640 --> 12:20.000 probability-based method for us to try to measure what is the amount of information you're 12:20.000 --> 12:25.520 contributing, and how can I attach that amount of information you're contributing to the level 12:25.520 --> 12:32.320 of privacy that you might be leaking via your contribution. And by limiting this, we have 12:32.320 --> 12:38.720 variety of ways that we're using different privacy. We're able to then give you some reasonable 12:38.720 --> 12:44.560 sense of a guarantee. Now, of course, does this always translate? No, right? Technology is 12:44.560 --> 12:50.480 in magic, and privacy is a social phenomenon, right? So I don't want to say all that, but I do want 12:50.480 --> 12:55.920 to say, different privacy is available. There's lots of cool libraries. There's many amazing 12:55.920 --> 13:02.000 researchers here in Europe working on different privacy. And I'd like to see those 13:02.000 --> 13:10.000 logos grow beyond just Google and PyTorch and Harvard, right? I want to see the logos on 13:10.000 --> 13:14.800 the different privacy that we have available for us grow. And if you're interested in integrating 13:14.800 --> 13:21.200 different privacy and open source that you're working on, please feel free to come find me and 13:21.200 --> 13:27.280 let's talk or find that person and talk with them. Sorry, that was very not private to point 13:27.280 --> 13:33.920 at you, but yes. So let's build that. Let's make this normal, because when I think about 13:33.920 --> 13:40.240 sovereignty from EU perspective, maybe or even from a European perspective, we have an understanding 13:40.240 --> 13:45.600 here that privacy is a human right. And it's not necessarily every place in the world where 13:45.600 --> 13:50.800 that is a fundamental understanding. And so I think this is one way that we could really add 13:50.800 --> 13:56.640 to the diversity of projects and experiences out there when we think about AI and machine learning. 13:58.160 --> 14:03.280 I come from a background of working on encrypted machine learning. I worked on the TF encrypted 14:03.280 --> 14:09.760 library, which was the first library to do multiparty computation or encrypted computation for 14:09.760 --> 14:16.240 deep learning training. We've TensorFlow at that time. Most of the cryptographers, the project 14:16.240 --> 14:21.200 is kind of archive now. Unfortunately, most of the cryptographers that I worked with moved on to 14:21.200 --> 14:27.600 Tsama, which is in France, and they work on new protocols for developing fully homomorphic 14:27.600 --> 14:33.040 encryption, but also MPC protocols. You can prove these types of things. And they have a lot of 14:33.040 --> 14:39.280 cool open libraries. There's open FH, she is growing and really cool. And there's plenty of 14:39.280 --> 14:44.320 people that are starting to work on the compute problem that we have when we think about encrypted 14:44.320 --> 14:50.400 computation. And to just summarize it, it means that we need maybe new types of chip. We need new 14:50.400 --> 14:55.760 types of maybe even optical computing that allows us to do the expensive steps that we have 14:55.760 --> 15:03.200 in encrypted computation and allows us to do it faster. And encrypted learning in inference is 15:03.200 --> 15:09.680 real. It's certainly not comparable with what happens on plain text, but I would love to see 15:09.680 --> 15:15.920 this side of this slide grow and change. And even to hear from you all, how are you building 15:15.920 --> 15:22.400 maybe encrypted computation into parts of your projects? So, if anybody wants to dev room any of 15:22.400 --> 15:30.160 these together next year, please let me know. Finally, open source communities. I mean, I've talked 15:30.160 --> 15:37.520 a little bit here of geographies. You ask you these things, but open source communities, I think, 15:37.520 --> 15:45.520 are a really good example of diplomacy overall. And I think we need diplomacy right now in a big way, 15:45.520 --> 15:51.360 in many, many ways. And I personally think, and I don't know about you all, but I personally think 15:51.360 --> 15:58.160 that open source can be a way that we achieve this diplomacy when other geopolitical tensions 15:58.160 --> 16:06.400 are rising. And so, I point out, PyTorch, you know, this is one type of AI open source project. 16:06.400 --> 16:12.560 I love PyTorch. I've been using PyTorch now for many, many years, but many of you probably know, 16:12.640 --> 16:19.520 PyTorch is a project from meta and is mainly driven by core developers at meta, right? And so, 16:19.520 --> 16:24.880 it really is kind of driven by that road map and that decision making. Of course, there's many, 16:24.880 --> 16:30.000 many contributors who are not working at meta and many, many places that are improving other 16:30.000 --> 16:35.280 libraries at touch, PyTorch that aren't meta, but this is kind of one community that, again, 16:35.280 --> 16:40.400 is international and reaches across many different types of machine learning communities. 16:41.360 --> 16:46.560 And then we have Pykit Learn, which is a very different type of community. So, I get learn 16:46.560 --> 16:53.120 grew mainly through just open source contributors who for very long time were contributing 16:53.120 --> 17:00.400 on top of their day job, right? It was not their main project. And Pykit Learn has an even bigger 17:00.400 --> 17:07.280 amount of people using it. And currently, also finally got some funding and support from the 17:07.280 --> 17:13.360 French government, including most of the core contributors are now working at a company together. 17:13.360 --> 17:19.600 And yet, it still has many contributors across other places. So, I want to share this as two examples 17:19.600 --> 17:26.320 where if you use this definition of sovereignty like citizenship, these projects could never exist, 17:26.320 --> 17:32.560 right? Or if you use the definition that we don't ever use US tack or we only use this or that 17:32.560 --> 17:38.720 or whatever these communities might also not exist the same way that they do today. And so, 17:38.720 --> 17:44.320 I want to encourage again us to think through diplomacy as part of our work. And of course, 17:44.880 --> 17:50.320 more than anything we need to think about prioritization, funding, and we need to be imaginative. 17:51.200 --> 17:57.120 We need to think through conversations that are being have had right now around competitiveness 17:57.200 --> 18:03.360 and scale. And what I think around these is not only we need more collaboration, which I'll get 18:03.360 --> 18:08.640 to in a second, but we also need to decide like go the locks and the three bears. What is the 18:08.640 --> 18:15.520 scale that makes sense for us, right? Because not everything needs to be foundation model scale. 18:15.520 --> 18:22.240 There's many different types of machine learning beyond that and perhaps that is one way to counter 18:22.240 --> 18:28.800 this conversation around competitiveness. And then I think we have to continue the idea of collaboration 18:28.800 --> 18:35.680 and diplomacy maybe start building in how do we support open source as part of EU tenders and not just 18:35.680 --> 18:40.800 the same five companies that always get tenders, right? How do we combine that with investment? 18:41.680 --> 18:46.800 The composite learning is doing a lot of reaching across borders of how do we connect 18:46.800 --> 18:53.120 HPCs with EU clouds, but I think there needs to be more of this as well. And how do we do this 18:53.120 --> 18:59.440 in a composite learning or a federated learning type of setup? And obviously with ODS and supply chain, 18:59.440 --> 19:07.360 we need to collaborate and do diplomacy to build new types of chips, new types of open silicon, 19:07.360 --> 19:14.000 that allow us to run inference or training workloads, but also start to develop open standards 19:14.080 --> 19:18.640 that can compete with people that have two year waiting lists for purchasing. 19:20.480 --> 19:25.760 So I'm going to leave you with a few questions, but I want to hear yours. After this, 19:25.760 --> 19:30.560 I'll be around today and tomorrow, I'll be hanging out in the privacy and internet devroom mainly. 19:31.280 --> 19:37.680 But can I decide for what is sovereignty, AI, sovereignty, I mean to me as an individual? 19:37.680 --> 19:43.440 Can I decide how I interact with a model? Can I decide not to interact with a model? 19:44.320 --> 19:48.400 Can I choose what models I use based on how they work for me? 19:49.280 --> 19:55.200 And if I'm doing that, that means that I need to be able to test locally. So can I run models locally? 19:55.200 --> 20:02.400 Can I run them on my own data and on my own compute? And can I support? Can I take my portability? 20:02.400 --> 20:08.800 And can I support projects, organizations not only with code and contributions in touch, 20:08.800 --> 20:16.880 but also with my data, should I want to? And perhaps we can think about this from 20:16.880 --> 20:22.480 whatever it means to have a European perspective. I think we need to talk about and think about 20:22.480 --> 20:28.480 what does it mean to build human rights, like privacy and agency into AI ecosystems, 20:28.480 --> 20:35.120 machine learning ecosystems and model development? How can we discuss sovereignty without 20:35.200 --> 20:41.200 forgetting lessons learned from nationalism and from war, which I think maybe we have a unique 20:41.200 --> 20:48.720 perspective to contribute? What becomes available if we forget about whatever we mean when we say 20:48.720 --> 20:57.120 global scale? Your local park bench is not global, but it serves a purpose. Can we move beyond 20:57.120 --> 21:03.440 this idea of scale? And how do we use our diplomacy and collaboration to grow and support 21:03.520 --> 21:09.760 international open source communities? So I want you to challenge the sovereign status quo. 21:10.320 --> 21:15.200 I want you to gather real constraints and evidence. If somebody tells you nobody does that, 21:15.200 --> 21:23.680 I want you to reply yet. I want you to stay creative, get curious. I want to replace zero 21:23.680 --> 21:30.320 some conversation, which means we have to do either or or or no but. I want you to replace that 21:30.320 --> 21:39.520 with yes end and I want to nourish hope. So this is me deploying to EC2 in 2010. I nearly got fired 21:39.520 --> 21:46.480 for it because the app that I was running fell over at one point in time and I got called into the 21:46.480 --> 21:53.520 IT directors room and he told me you should forget about this Python thing and you should 21:53.520 --> 21:58.400 forget about this cloud thing and then maybe you'll have a future in technology. 22:00.400 --> 22:05.920 Thankfully I did not listen to this person. I left the company and I went on and did other things. 22:05.920 --> 22:12.080 But I want you to have the Hutspa that's little K jam had then. I want you to have Hutspa 22:12.080 --> 22:19.760 and courage to say no we don't have to do it the most obvious way. We can do it a different way. 22:19.760 --> 22:25.280 And I'm pretty sure that in 15 years if you start that journey now you will look back 22:25.280 --> 22:30.160 and you'll be proud of yourself. So thank you very much for your time today. I look forward 22:30.160 --> 22:36.160 to talking with you later. 22:46.160 --> 22:50.880 Thank you. We don't have time for a question.