Hello everybody, I am Cynthia, I'm P.M. at GitHub and today we're going to talk about universal
software which hoodie, indicators and different government open source readiness.
And here we have Pelin.
Hi, I'm Penin Salzimina, I'm from D.P.J., the digital public goods alliance.
I don't know if you know anything about it.
I can just shortly say what's D.P.J. and D.P.G. means.
First of all, how many D.P.J. are here?
Okay, very few.
So that's the reason we need to explain.
Digital public goods alliance is a multi-stakeholder organization that is helping open source projects
to be attained for the sustainable development goals achievement.
And we have more than like 50 members at the moment.
They are government organizations and UN organizations and development banks.
And we also have more than 230 D.P.J., I think 236 D.P.J. at the moment.
And D.P.J. are the open source products and solutions they can be standards, AI systems,
they can be softwares, datasets, and so on.
And they are just tested against D.P.J. standard.
And if they are complying with the standard, they can reach the D.P.J. status for a year.
And then they need to be renewed every year.
So yeah, shortly that.
And I'm working as the product and experiences lead at D.P.J., nice to meet you.
Awesome.
And also you have a stand in building H, right?
We can go chat more with the D.P.J.
So dive right into it.
We want to talk a little bit about universal maturity indicators for digital public goods.
So GitHub is part of the D.P.J., a number of different agencies.
Governments is also part of the D.P.J.
We have care code to go to the repo there.
We want to take a look.
And we got together as a community practice to really discuss and take a look at different
different universal maturity indicators for D.P.J.,
and this was built off of UNICEF's Django project.
Their framework, past framework as well.
And really pulling all that in to take a look at what's the best way to really support
maturity for different digital public goods, different projects that may have been built years ago
or something brand new.
So first, we also organized this by different pillars.
And when you scan there, you can go take a look at this.
We'd love your questions and feedback.
So to start, we first took a look at governance.
Being able to take a look at a digital public good.
What is the governance structure?
These are all these pillars are sorted by a low medium high type of ranking.
So we want to take a look at what governance means for this project.
We want to take a look at what data protection and security is like.
So being able to take a look at, is there a secure deployment?
What is the development set up like?
Are there regulatory data compliances that this project complies to like GDPR?
And, perhaps eventually, CRA as well.
These cyber resilience act.
And then also different security controls.
Being able to take a look at high wouldn't mean that this project has very robust security
controls from the very beginning.
It's regularly audited.
You have individuals.
People can reach out to and it's very clear.
Having open standards.
Having openness.
There's interoperability and different state data standards for that project.
Having a roadmap.
A different project.
Strategies.
A clear public roadmap that people can view, contribute to.
And it's very clear what's going to happen next for that project.
Having source code accessibility.
So documentation.
That's an unending sometimes a little bit thinkless type of role.
But it's very important for the community.
For contributors to take a look and be able to understand how to contribute.
Being able to look at a very clear contributor file is the read me really clear.
Something that the community can access and it's a living document.
Total cost of ownership.
Open source.
There is a cost.
So having a really clear budget that's indicated for the project.
Being able to see what costs what and what's going to happen in the next few years.
And then lastly, we have a flexible architecture.
Being able to take a look at the composability of that project.
So.
Next.
Here's a quick screenshot of what we mean with a low medium of high.
So you see here on the governance.
This is what that refers to.
And if you want to go to that report, it's going to take a look and go to the discussion.
Love your thoughts as well.
So that was a lot of information a lot of words.
But I'm sure you're more interested in a demo.
So we have here.
I think Helen is going to go through the demo that the digital public good alliance built.
Thank you.
So this is just the screenshot.
But now I can just show you the QR code.
We just invite everyone to test the tool.
It's brand new.
It's open for all scrutiny.
Please make sure that you are checking it and actually testing against all this indicators
that Cynthia just mentioned.
I'm just going to tell shortly what you're going to see when you do that.
This is the result page.
This is the summary page.
So when you actually answer all the questions that we have about all these indicators that we just mentioned.
Based on your answers, you're going to be computed with some result.
And then you will just see where you stand in terms of maturity.
The confidence ratio is based on the evidence that you actually provide for each answer.
Currently those answers, the evidence is not being reviewed.
But soon we are planning to have automatically collected evidence as well.
So that it can be more confident when we are calculating these results.
And the summary is just including like where you stand for each indicator pillar.
And then you can see the detailed calculation where you score better, where you score a little bit worse.
And then you can get some simple indication like what could be the next step for improvement.
And for the DPG product owners that are very few here, you can actually go to your portal entrance.
And you will now see that you have your DPG status.
But then you have an action item which is the maturity assessment.
And if you click on that one, you will be able to start your assessment.
The difference is that of course we are DPG organization.
And we have the famous registry.
If you are a DPG, you are in that registry listed.
And along with a lot of information, we are also planning to use the maturity assessment result.
But this is not decided fully yet.
So we would also like to receive opinions and comments about how we should be displayed.
And when it should be updated, how often would you like to have the assessment done.
So that would be really nice if you could actually share your thoughts in our discussion board on GitHub.
And yeah, this is the product owner screen.
Like when you come there, your name is going to be already pre-filled and you will continue.
So please, if you have any comments and any questions about it,
make sure that you use our GitHub boards.
We are open for all the feedback because it's the product in making.
And just before I'm leaving it to Cynthia again, just wanted to share why we are doing these ridden assessments.
Of course, this was coming out of a need when the implementers, the governments, the UN organizations and development banks,
are looking for open source projects to implement support, invest.
And then they are looking for certain factors.
And that was the indicators that Cynthia just mentioned.
And this is the reason we are doing the ridden assessment.
But during our last annual members meeting in Brazil, we saw that both depGs and the members are actually needing to take it a little bit further.
And they want to close the loop.
Now we are talking about government readiness, aka implementer readiness as well.
So this is something we started to look at recently.
And I know that there are a lot of frameworks out there by UN, by many organizations, even universities and some, even some local governments.
And I know that as open source product owners, you are probably facing many different cases.
So we are really open for hearing your thoughts about that as well.
At the end of the presentation, we're going to share QR code for a form.
So if you are informed about any framework that you are aware of, and you think like it's useful, please inform us about that.
And the reason we are involving the government readiness into the process now is that we are hearing that,
if the implementer, if the integrator is also not having a certain level of maturity, the software can so do so far.
So that's the reason that we want to close the loop to see like to take the current snapshot of the readiness and identify the strengths and gaps, and guide the governments also for the priority areas and readiness.
So that's all for my side, I think.
And now I just want to leave it to Cynthia to open for discussion.
All right.
So we answered this up as a roundtable because we really wanted to hear from the community.
We wanted to hear from you here at Boston.
So we have two questions.
Hopefully we have time growth.
And I think I have stickers to maybe pins for anybody who wants to contribute to this discussion.
So first we'd love to know from everybody our first question.
What governance or incentive mechanisms are needed to really ensure that such assessments like the ones we saw or different government assessments,
UNHNC assessments like Django's framework with UNICEF will actually be used and continuously use year after year.
Yes.
I think that perhaps what's needed is clearly understand where the data is going like looping community in actually.
Because I'm this is the first time I'm hearing about the FDA.
But I really look it up later.
And it seems like if we want this to get like on a government level and establish a connection between governments and open sort of projects that are invested by.
Yeah, this is something that should be on everyone's tongue.
Like that should be talking about it.
Yeah.
There should be local government initiatives spread awareness about the EPA, like general community.
Yeah.
That is why that's something that folks want.
For example, that's how they can so it's a funding.
Yeah.
That's a great point on discoverability of DPGs because there's a wide range of different projects from.
Once are created by the government created by the UN agencies on disaster response all the way to civil society or universities that create projects on.
Different libraries different banks.
Human essential banks that exist and so being able to make that connection.
Because we may already use DPGs a lot of times because it's already integrated in different infrastructure.
But a lot of times it's not as well known to be thoughts on discoverability.
Yeah.
We agree with you.
And I think the maturity tool is going to help with this discovery with the as well.
Because just right now, I said 236 DPGs and they are not doing the same thing.
They are answering different needs.
They are at different levels.
They are targeting different locations and audiences.
So with this maturity assessment tool, we are also hoping that this is going to help decision makers to make more informed decisions about what is the most suitable solution for them based on the assessment results.
I hope it answers.
Any more questions?
Yes.
If you consult, we are on that when you make a measure into a target.
Then it stops being a good measure.
It's called good odds law.
I mean.
So there are two issues.
One would be that genuine maintainers won't want to go through the bureaucracy.
That's involved in getting validated.
And the second would be like bad actors and publicity came that they could have before.
To be able to fulfill the criteria and getting included in the set.
So I guess that would be like a kind of reservations that I might have.
Could you summarize the questions of people online and do them?
Okay.
The question is like if the indicators and the measurement tool would be used to make things feeding certain targets, but not actually delivering a value.
I think it is something like that.
The second part, which is that the good maintainers won't have an extra check list of forms of phenomena.
It's not a checklist of form.
Actually, if you're a good maintainer, you're already doing these things.
So it's just about just giving the answers that are helping others.
Like if you want to be discoverable, if you want to be found by the right implementer, I think this is one of the easiest things that you do.
If you're a good maintainer already.
And what I'm seeing, it is not like making it a checklist and becoming like something that you kind of fulfill.
But I'm seeing it as an opportunity to see how you can accelerate.
And we in the DPJ are planning to use it like that by the way, because when I say about 200 and 3060PGs.
Because there is a community management around of it.
We have great community management initiative.
And our review team is also quite tightly connected to all the DPGs that they reviewed.
And they see, they don't need this tool, actually, to know who is mature, who is doing what.
They already know that, but this is more about transfer of the knowledge, let's say.
So this is going to help both the DPJ Alliance to actually help bring forward many DPGs to the next step.
And also help the implementers to find the best suitable solutions.
I don't see it as a checklist problem.
Like I don't think answering these ones and achieving high result is going to be just the thing to achieve.
Like I think this is actually going to deliver value at the end, because what we are asking is that is your governance model in place.
And how do you go on your product?
What kind of product roadmap practices you have?
Like are you engaging the community?
Do you hear from your implementers?
Those type of things.
And those you have to do anyway, I think.
I kind of see it like a hygiene measure, actually.
Yes?
Because as I said, you put better.
I might not have enough trust for mathematics.
Because for me, it feels like it's at 1 to 4 metric.
But it's not.
And for me, I think that's enough time if it is my particular case.
And if the mathematics are so general, I don't find them useful.
Then I can say this maybe.
By being a DPG, you are already not general.
So like if you're a DPG, and if you're getting the metric assessment,
you're kind of really narrowing it down if I understand what you meant correctly.
I think also to act about what?
Yes, sorry.
Just to act about the pillars that we have for the assessment, it's also open.
So we'd love your thoughts on that.
Because that is something we want to have the community's thoughts on those pillars.
The repo is on there.
Let me know if you need that repo again.
So that is something where we have the community to reach back out.
And let us know, are they, if it is to stringent on some of the pillars or some of the indicators,
being able to hear from the retailers, from the developers, from the product owners,
would definitely help on being able to mold it to fit a wider range of different projects.
Just wanted something more on that.
Cynthia was actually pointing out something great.
As you know, we prepared this together with UNICEF, FAO, and Digital Square.
And there were a lot of discussions by the way to find the universal indicators.
So you are kind of right about that that is kind of general.
But we were also seeing that coming back to your point, like another checklist,
another assessment, another form that you need to go through.
Like we saw that Digital Square, which is, for example, supporting DPGs in health area,
they have different type of requirements.
And FAO has different type of requirements.
They don't need to be even a DPG to get FAO support.
UNICEF also, like having a quite a large spectrum.
But when these organizations came together with GitHub and say like,
we need universal indicators to understand if a software is mature,
it was actually to find more general standard to bring everyone up to.
So for a specific implementation cases, of course there are going to be even more like detailed checks,
like for example in health.
There are standards that no other software needs to fit unless they deliver some health product.
So those those will stay, but very general maturity indicator is going to be here.
So basically they all don't want to help me pick, but we will help me probably there.
Yeah.
You can say that.
Yeah.
So sorry.
Yes.
They will help you validate what you are actually looking for.
So.
Yeah.
So.
I was interested in what's your feedback that judging your engagement and participation.
In terms of GitHub or for like this.
The pillars or the like because you're saying like what it's sensitive.
I'm assuming like.
For organizations that might want to use these metrics.
How would you where.
How would you update these new metrics.
Yeah.
Yes.
On the.
And for the maturity tool itself.
We will actually first launch it with the DPD for DPI collection.
So that they can they can start using it to assess themselves.
And then we have 51 members who will benefit from the results of this.
To start with.
Digital public infrastructure.
So all the DPDs are used to build different digital public infrastructure, which may be different ways.
Governments build.
Telecommunication networks.
An example of a DPI would be anything on how governments manage their.
Yeah.
Did she ideas very common.
Yeah.
So all those products are used to build that for digital payments.
Yeah.
But we had that time that I think first.
Sorry.
So first I can maybe summarize the question while you are formulating the answer.
The question is that how are we making sure that.
Regional implementation requirements are met with these indicators.
That is a great question because different governments and different countries.
States have their own assessments as well as for example Canada.
They also have a digital assessment tool that the government built for themselves.
That is then a little bit more geared towards the population and different rules and regulation.
So that is something on top of what we currently have a lot of times as you know open source products.
They can be used globally.
And so a local government or a different region would be able to also use their own assessment tools on top of that.
I believe candidates have just really stares last year recently.
And they just share that so that one is also an exciting one.
A good question.
And I know we had one on the side too.
So you showed that there are two points in the same month.
So the overall for the government.
While we were talking about it's most about the developers.
We can so parent them.
I think it's a thing.
But I mean you have to learn how to involve these governments or let's say,
or suppose we are growing in that potential.
So they can actually take use of or why they can use it.
They'll fund for that.
Thank you for the question.
The question is what kind of plan do we have to involve all spos or other governmental organizations to actually adopt and use this?
Thank you for the question.
And I have two answers to that.
One is that we do have our members which are from all around the world and from different different verticals.
Let's say we have governments.
We have UN organizations.
So that we are working with them to define how we can make sure that these measures are actually being used.
And second, we are coming here to hear from you like what kind of experience you have.
And where do you see the actual strength that we can leverage here to be part of?
So if you have any suggestions that's also like you can be part of this like how we can implement it.
Yeah.
Oh, I was working with the public sector.
So like, just because there was a lot of resources.
And so because then we have a company or somebody who is maintaining it.
And then the public sector comes and they have what they would like to have some kind of security.
So we will be maintaining for five years.
So I think that's the extra work.
And somehow, how much you let the public sector know is what has come.
And you would say, not a year or a level of trust or some discreet being measured.
So the companies can involve in that.
And maybe I wish there was one more mic so that everybody would hear what he said.
Well, I will try to summarize which was very nicely put.
Yes.
System integrators and implementers are a very key factor here.
As you said, like because they are the ones maintaining it or implementing it and making sure
that that software is functioning as it is supposed to for the next five years.
So we do also agree that they should be part of this assessment process as well.
And that is how we are working because as I mentioned in the beginning during our annual members meeting in Brazil,
this was a specific session by the way, how a size could be included into certain processes.
And how could we make sure that the maturity or the quality that we are requiring from the software is maintained throughout entire chain.
So there is the consensus around that.
And if you would like to add more, I would like to hear your thoughts after the session as well.
Thank you.
Yeah, very good suggestion.
I'm just going to summarize again.
You were telling that it's really nice that we measure the current status, but based on the result,
how can we assure that the improvements are made and which improvements are supposed to be made to reach the optimal level.
And we do actually think about that by the way, because when I said like accelerator,
like that's what I was actually meaning because after taking the snapshot and maybe hopefully,
like when more than the DPDs were DPI collection, maybe more DPDs are taking the assessment.
If we can see where we are standing, then we can collectively actually develop road maps to bring the level up.
However, that being said for individual cases, we were also discussing, like instead of, for example, publishing the results as they are,
what matters is what are the plans of that specific DPD to achieve the next level.
So that's going to be something that we're going to work on in the upcoming period, like how that is implemented and how it can be communicated.
So thank you for that.
And so we're going to find the next question.
If we solve time, I think we're okay.
So we have another question on what roles do you think should you different governments in the EU and the private sector and civil society,
play in shaping or validating a universal maturity indicator?
I know we had your hand up.
Yes.
Yes. Thank you.
So as a vendor ourselves, I was in the company that does the assessment point.
And since it's also implemented to the government and the government publicly,
it should also be certified there for the doors and make sure that on a tender level,
it would be a good idea to have tools that are already implemented to be certified.
And we, by example, are certainly with their tools to be certified.
Should we just summarize?
What was your name?
Helen.
Helen was just telling that maybe when the implementers are actually choosing their partners,
they would select among the ones which were already certified and which were already verified their maturity.
So is it something similar to open source first policy type of thing?
Great. We like that.
Yes.
Yeah.
Two is a great suggestion.
Actually, I wish our standards manager Amrino was here because she's working on making the DPG standards as one of these more generally available standard as well.
But at the moment we are maintaining our own standards and DPGs are tested against that standard.
Any other question?
Yeah.
Thank you for that suggestion.
It was suggested that why don't we collect all that data through an API?
Actually, we have a data scientist which is currently working on what kind of data is automatically collectible and kind of valid dateable, let's say.
So it's in the plan, but it's not the one that we are able to release at the moment, but we are working on that direction.
So I don't think that you might have to do that.
There are feedbacks as well.
I'm very fond of you.
And I will answer that comment and suggestion with the question if it is possible.
The suggestion was that the comment was that if there were any already collected indicators from the implementers regarding their general pain points when they are selecting and implementing open source solution.
I don't have that answer, but I'm going to ask that since you mentioned digital square, have you ever checked and benefited from their shelf fitness assessment for health products?
Do you have any experience with that that you could share with us if you did?
Like if that answers that need?
I see that we did a look at this all and we did not necessarily attempt from another device to share that with the government partner.
To share that as a resource and in order to in the long negotiations that went on around, how are you making a decision about you?
So if I understand correct, you used one existing tool to actually help a decision maker to make a decision about what you could provide potentially.
So the rest of the folks actually have to implement the solution and that was an entirely fundamental issue.
So these resources were very good point.
Two of them was sometimes trouble to think is in language.
Yeah, and one thing that I can tell about that is that as Cynthia mentioned, there is the Canadian example, like there are a lot of examples about how implementers and governments are assessing their readiness.
Some of them are from the point of view of the product owners themselves.
Like they are product owner governments and they do assess how they adopt those products within their organizations.
And that's the assessment type let's say it's different.
The one that you're talking about is a sector specific one, for example, and there might be a lot of regional differences as we were discussing.
So to find the universal one there, that's what we are working on right now with our members because it took a lot of time to to land on the universal indicators for software, which is kind of pretty much defined by the sector best practices anyway.
But when it comes to implementers side, we don't have that base level which is already good.
So that is what we're going to work on this year. We have a target of achieving some sort of indicator drafts until November, until our next and all members meeting.
I hope like we will learn from your experiences as well. So please do participate in that process. Thanks.
And I'd love to know a little bit more about the DBG product.
As well, if you're feeling comfortable sharing what the DBG?
So if we're data kind, it's a non-profit organization that works on data side data solutions for software.
The product is called the data observation tool, not is essentially a way to local, local way to check data quality or just, you know, you may, you may dimension with data quality,
how can you just check it? And then also can you write some custom testing to check whether you are, the workflow that you use in our case, how can you check it?
Okay, so the workflow is going to need to help workers and actually test those workflows also.
It actually takes sector agnostic tool.
It just happens to be right now being used in the process.
And it's basically in my colleagues, but it's basically the wrapper of course.
Thank you very much.
Thank you.
Any more?
Yeah.
Because I see that my purpose is using the lecture that material here, I was wondering, you get up, where you're working for,
and it's also a convenient part of your research project.
There are also many governments that are using.
And I was wondering if you can share all the feedback on the track, you come into the,
this is the public infrastructure.
And whether you're so first of all, do you or about,
or did you not get to be really critical?
Thank you.
So the question was whether more on GitHub or different software companies,
being a digital infrastructure,
or being able to be a digital public good, perhaps?
I think for me, digital public infrastructure are built on digital public goods.
I think it's a little bit more for a country or country state to be able to,
to build for the community for their constituents or beneficiaries.
So I think it's a little bit more on an infrastructure that is, for example,
internet or telecommunication or healthcare.
So I feel the product itself as a good, like GitHub for instance,
may not qualify for an infrastructure in the same way as a national ID system.
That is managed and delivered by a government,
but that could be interesting in terms of how that could work in terms of,
as an infrastructure.
I don't know if a kid would fit into that in virtually,
but that's interesting to think about.
Thank you.
And I think just to read directly to the quick,
I would love to know from here what role do you think,
governments or the private sector or society could play in shaping,
or validating a universal or universal maturity indicator.
I feel like that question does really, very well into that,
in terms of GitHub being able to really provide a platform for that.
And having it open, encouraging more open source and more contribution
to supporting the community, supporting developers on that.
The question.
And then I saw, yes.
Maybe I missed a very fundamental contribution here,
but I'll just like to know, for you,
also concerned with assigning things and what's the evaluation
for the product, or are you just concerned with establishing the metric,
and then everyone can do their own thing with it?
I think that would be interesting in terms of,
it's also much assigning.
I think it's more of this is what's available.
I think we mentioned here, this is something that different
implementers may be curious on being able to write guidance on
a maturity of a product, so it's not so much regulated.
It's more of this being able to provide additional resources.
Currently, there isn't any sets rule for it,
like if, for example, like what I understood from your question is,
if the DPGA members, 51 members,
if they're going to be requiring the DPGs that they may work with
to take this assessment, is it going to be an assignment?
Is it going to be a requirement by implementers to have this test?
It looks like they wanted, but currently,
there isn't a set requirement for that at the moment.
What we as DPGA and the DPG community management
want to achieve is that running this assessment with as many DPGs
as possible, so that we do have a snapshot of our ecosystem,
like where we are standing and understanding and defining
the current problems, like where is the pain point
about reaching a certain maturity, like in which indicator
we are struggling most worldwide,
or are there any regional, how can I say,
difficulties that we may find out?
The assignment is not going to be an assignment, let's say,
but we will encourage taking it as much as possible,
because it's going to be a very valuable data.
On the way, it's going to be updated as I was mentioning,
like the data is going to be automatically collected, hopefully.
So if they opt in for that, that can be done for their DPG,
there are different ways of going forward,
but currently we don't have an assignment for that.
Only requirement we're going to have is going to be the DPGs
for the IPI collection, it has currently 14 DPGs in it.
We will first knock their door to take the assignment actually,
so that might be the only one.
I think we have very limited time, so despite the last question.
And since you were already asked, I will...
Yeah, there's still a lot to do here,
because as I said, DPGA is a members organization.
So when someone here, I think it left already,
but he didn't hear about DPGA before.
Yeah, it's the organization which 50 members,
with quite a good footprint actually,
but it's still an organization and this was a close topic,
which we now have for the first time opening
to the rest of the audience, let's say.
So it's a process that is still developing,
and that's why we are asking everyone to participate.
I just want to share these QR codes.
This one is to share the best practices
that you know about government and implementation readiness, let's say.
And this is for the DPDs actually.
If you would like to test the...
Did everyone take it, sorry?
Like, especially not one.
And I will give the QR code for the one that is open for everyone.
And it is this one.
So thank you very much for all your engagements and the questions.
I was there a very burning question there.
Yeah, don't pick it up.
People are so far and I have a specific statistic news about RPGs
that they've heard about five.
They've heard that they've heard about that.
They've been offered as I.
But it's good.
Now, what happens when one of those digital critics,
or whatever, because from one to the other,
many more people will have a system greater
than when it gets a family.
My question.
My question is, do you know what happens?
If a new team becomes,
practically dead,
I'm active.
And it's not new.
So do you think any more,
do you think,
do you understand what happens?
What is your intention or what is your action
that will do you to keep it alive or offer it up as an orphan?
It looks like a simple question.
What is the very complicated one?
The question is,
if a DPG is becoming dormant,
what is happening with that one?
So the question is,
simple in a sense,
that if a DPG is dormant,
if the maintainers,
if the developers are no longer doing anything about that product,
the DPG status is anyone.
If they are not renewing their application,
if they are not renewing their standard fitness,
they are not a DPG anymore anyway.
But most likely,
they are not a software anymore anyway either.
So that's a different thing.
The second part of your question was that,
what do we do to not let that happen?
That's the complicated part.
We are in alliance with all these members,
with all the donors and everything,
and the time is up.
So we are doing our best to keep them alive,
but if they choose to die, they die.
Thank you.