Hi, this-is steve malarkey from as our government engineering team I’m joined today by this was a lately ceo of applied information sciences and we’re going to talk about building real-world solutions on casual government with machine learning welcome this was a receive so when we start out by you just kind of given …a high-level overview of the solution in question here that you…steve so, this is the solution that …the built for a government agency and I should say classified government agency and …later on in this talk we can look at some of the solution elements but at the high-level…interesting solution and then we met …customer this is a manual process so, you know, think of this organization that is comprised of a number of policy analysts that cover different policy areas as you can imagine a lot of documentation and articles and new stories …documents are fed to them. And,…is used to be a manual process that…needs to come and …in the morning and classify these documents in order to make it productive for these people and make it available to them and we said can be apply machine learning to this problem so we created this machine learning …solution which I’m here to talk about so if I can you know, get into the solution description foressentially the solution is really part of the involve three thinks the first thing we do is …a large number of documents that are coming in and it’s really impossible for someone to go read these documents so…wanted to use a machine learning algorithm to auto classify these documents and then once we have sort of auto classified these documents and I’ll talk about the algorithms involved in a moment we then recommend these documents based on so you are working on a certain policy desk and you’ve told us that these are your preferences will match those preferences with the auto classification and the third component of this is that let’s say we have made the recommendation to you and now you’ve read this article and you think that it is really a valuable article and maybe other members of your …may be interested in it so we take that feedback that you provide and then applied to the back to the machine learning algorithm so that, you know, others can benefit so in a sense of cleaning that …the algorithm constantly…on the user feedback I see so no matter how well the human how good the human is at their job…sort of scaleability limit I as a single human can only go through so many documents at once this enables us to get higher level of just throughput in terms of how many documents you can analyze as well as you come to me and say steve…your recommendation wasn’t that good or a I love that recommendation again as me as a single human if I have to now keep track of that …lots of feedback… that becomes difficult something computers are really good at night and there’s some really good algorithms …out there that can that can help for that so that description the document if I if I may go to the to the next slide steve really talk about the solution elements so this is and, you know, those little solution originated in larger commercial and in… pleasure gap and the elements that you see on the screen of the elements that make-up the solution severe management and we’ll get into why these elements came into play the piano management at service jobs …db and machine learning servers and…help to keep this in mind and then we look at the diagram and we’ll call out all the individual pieces, you know, one thing I love about just seeing this list is that a lot of these are paths services platform is a services so now you guys just have some application and third over to a bunch of e m in as you’re actually using these higher-level services to make the solution more efficient absolutely and, that’s the goal that wherever possible use the high-level services and if you need to drop down imply…absolutely, absolutely do that awesome so these are the elements and then we look at the diagram these elements will become clear here so let me bring up the solution architecture here and walk you through this on the top left corner is let’s start with there is an actual app service wherever api ultimately is whole …and this information this solution the way it is made available to the consumer says to sharepoint portal for the purpose is this discussion will not go into the internet and sharepoint but ultimately it’s a web part in sharepoint which…so as a web app is hostility…web app is also hosting something interesting is hosting …job which is responsible for running every night and, you know, …provided to the list of data sources and brings that data and stores that into…musty sort of…data every night into cosmos dp and even the simple fact that you guys surface…an api …in this case the sharepoint but it could have been a regular web app it could have been because you surfaces and…flexibility absolutely person so that’s one thing so once the data is …db then we apply a machine learning algorithm on it so you…talking about…can only read a certain number of documents so we’re using a tool called that microsoft machine learning server which you’re use you as the noise are server previously has been the name to microsoft machine learning server and the real our of that tool is it can either run in a standalone windows or linux server or it can run inside sequel server and it gives you the ability to run out and buy …of programs and what is really interesting is they give you a bunch of algorithms out-of-the-box but they’re also you can bring in open-source algorithms as you see fit so when we were building this experiment and see if I’m not a data scientist and …we started out with some algorithms and be tested it and the users slight some of the results did not like other results so we ended up with an algorithm that was developed by german named david bli at princeton I think he’s a columbia now and he did some really interesting work called topic modeling and the idea is that looks at the document and by looking at this document using this algorithm she comes up the probabilistic model of what this document entails what are the topics covered that part of the document and fortunately enough not only he published a paper about how he desktop it modeling …not us but he published an open-source packaged…package for that the algorithm file and we took that algorithm brought it inside microsoft machine learning server and then that algorithm …applies that on the …incoming documents…cost most…that’s amazing so let’s break this down a little bit …machine learning server…have your choice whether you want to use our python? Personal preference…this case I think you guys selected are you have your ability to bring in some out-of-the-box libraries even open-source libraries you can bring in fact that’s what you guys did yes by bringing this gentleman’s library yes …a lot of flexibility how you can approach this and solve the problems strike… very interesting this is the way the reason…one of the reasons this algorithm was developed flows david…supports digital historians so let’s say, you know, two million cable. Or telegrams got declassified now whose job …to read those two million cables and figure out what does being talked about so he developed this algorithm for that file and then and then, you know, of course we were able to use it in this instance so once you have this algorithm we make a call into this algorithm we figure out how to classify these documents and then we stored the results back into course db over and then we have a very api which supports, … getting the documents based on the user’s preference and then there’s one of the final component we have a number of moving parts here but the final component is the api management …just super critical in this case we did not want other callers to directly talked over…for multiple reasons they could have very well done that but api management of…the layer of indirection which …became important for us as I mentioned this is the classified agency using this project and api management because of that layer of indirection allowed us to inject some interesting policies, for example, we said only allow the ip address range from that intranet to come in and call… And, the replied that…put in place or the …policy that we put in place rather was, you know, the sharepoint that part goes ahead and gets a job to …and that just…needs to be sent into the…management for the call to be serviced so it allowed us without really…burdening the …pie that these kinds of details were able to external lies that to decimate game management and, you know, make it available to the shipment that box so these are all the components that we talked about very cool so …cosmos that was providing a lot of flexibility not only for storing the data after ingestion yet but also after machine learning runs store the results also in cosmos that’s nice that’s…and then with api we talked about the aps great because you having to you…but here you guys also used…management as our government to be able to apply these policies could be security policies like you guys …or…of other ones as well…as a developer of the web upiwhoever was task that responsibility they didn’t have to worry about all those policies they could just create the logic …web guy without being burdened with like you said knowing what the policies would be if I …that’s going to have to put that right and, you know, that examples would be cash …you know, if the content can be cache them we can improve the response time told users so there’s the number of benefits and of course, you know, you can now right policies yourself you don’t have to use of the …policy so the real our to that…putting that in place that’s awesome so, that’s sort of the high-level overview of the architecture …why the solution came into being…you do like to see a slow small flight…this application work so let’s just exit out of this our point here and let’s just go into the magic of borderline within our government portal …version actually coming…and I have a resource group called…demo and, you know, for the purpose of this demo btwo box of an application brought it here so we can share the users class of the powerpoint you had where you listed those cosmos db api management we see you have these resources so I can associate was on that are …either what we…this radio…the…that’s a good point steve because you know, let me just call out a few of them right so we talked about …management right here of course most db which is wherever ingestion is happening…jobs are coming in I’ll talk about the …demo servers remember I talked about the machine learning server which like now is a standalone server running on this virtual machine and then I can scale it out if I wanted to and then, you know,…see a bunch of disk storage accounts and things like that click you get so that’s the entirety of the solution and just to give you a flavor for a couple of these things here let me go The aps management portal here and this is as, you know, there are two portals that come…one is the publisher poodle and then the developer portal and publisher’s portal is I’m an author of an api want to set my policies absolute vulgar …portal is a consumer of the pie and at one of the kind of see what it’s available …things out yet so I’m in the…right now and I just you know, not all policies have been enabled I commented out a couple of policies but, very quickly to show you here that, you know, I have a joint policy…that we were talking about…so you have policies and as you said steve that, you know,I have you know, this is the dashboard gives me information about how many pies are being called and, you know, what is the most popular method I get a lot of information about that and then if I go to the developer portaland, let’s go to school up here and…these are the three methods…that steve we should be we should talk about here…moment here so the first method is …the recommended new stories based on the user right and the other method here that I want to like to show you is given the user please give me the categories that I’m interested in and, …what could categories apply to me so those are the two examples here in fact if you like if you have time we can discretely execute one of them…so, you know, that this is not just an empty demo so, give me the new stories for this users and, you know, the developer portal is nice that it allows me to exercise the cpi very quickly without having to write any code …the put some default values and just go ahead and…you know, run this so it is going to for the users let’s just say this is the endpoint that we were looking at earlier so I’ll demo api management api dock us of course and then this is the url and if you go ahead and run this we’ll come back with the stories that I have…I may be interested in based on the preferences that asset so these are all the stories that are coming back so I’m a developer this been tasked with the job of consuming this api and …stories back I don’t what api management is enabling force here is saving me from having to maybe go to documentation that may not be too good yes…how do I make this a …width doing nothing vacation token do I need this and…I can just go right is a going as mit fry it…in seconds I can actually be making calls to the api absolutely there’s a common term that…api community uses which is, you know, how much time does it take to make the full successful call and you’ll either …there you are both developers and we’ve seen that if that time is less anything more than five minutes to get my first…successfully executed it’s too long developers are impatient and they move on so this gives you come in exploding …and be able to invoke it very quickly…of wanted-to…show you this I just want toalso just very quickly go to the machine learning …steve let me just maximize this here and here is the …that is just…just a ring of the code here? In…I’ve some…open right here’s the controller right here and this is these are the two api methods and, you know, this is the one we call the get recommended model …api when we saw api management this is actually what’s in front of the code that were …us connect and thank you for pointing that out …management is essentially a facade so when you on board you…in a reply management you need to provide one back in the pie but be calling and that bacon the pi is the absence of us…and the code that…looking at is indeed the…I controllers for that…I just …out-of-the-box…without the…reducing all the time absolutely, absolutely this is the part that steve I want to talk about just for a moment so that if we talked about that you know, this machine learning server gives us the way to called …package and the python package and the and of course the hard parts of these algorithms are written in… And, the…of course have to execute those on the …server but in order to …server the decided to use as much csharp as possible because, you know, there’s some data manipulation that’s happening all done in csharp link but then …using a package here called …dot net which allows …csharp program called on package so. Going further down here I just want to point out the one that is really into ask interest here which is the process lda and this lda is that algorithm that the german david black had built and, it stands for latent they clay allocation some fancy dumb which may not be concerned about but…a complex of package so you can see here I’m in csharp I did some data manipulation and ultimately calling that are package which is this returning the data and then because I’m in csharp it just makes it easier for me to interact the cost…at this so, being a csharp developer myself I think we need to take a moment to bask in this what we’ve …here because I’m not a scientist either you…that earlier if I …I can write my csharp…a way…but now it’s time to call a machine learning algorithm I can just write…csharp code. And, coal directly into this machine learning algorithm on our service vast pretty powerful that’s and you those another reason I knew that you…to stop me …so, I can prepared there’s one of the thing to talk about right you’re talking to cosmos db and, you know, there’s no easy way to talk to customers db from harsh and maybe somebody will correct me and… and say that, you know, well, you can make a rest api call from a you can but there’s a lot of work in setting up the tokens and things like that so by having the code that we have in csharp it allows us to transform the code and then using this package called the our libraries and then and then interact… I should say that another approach if you don’t want to use this the dot net …package is expose that aren’t package itself as of that service…then it will be a seamless interaction so that may be a preferable option there is…so that’s essentially the code steve of how this all came about and the, you know, the solutions running …graph okay that’s very cool so not only are we running machine learning on measure government but we’re using all these past services cosmos db web jobs app service api management that’s really cool focusing a little bit more on the machine learning I knew that you have a lot of experience doing machine learning you’ve done this and as your ml and ml studio and measure commercial if with asher government hd insight jupiter notebook spark clusters here we’re looking at a machine learning server key just give us your thoughts from, you know, what the experiences like with these different tools and doing this on as your governor yes so that’s a really important question to consider because yes as you said we have done a lot of work with the actual machine learning andboth of them …studio? And, then we were converting the solution over to measure… …imagine the machine learning…prove to be a really useful …because not only could we bring the open-source libraries that I talked about and in fact there’s another library that we are also using which we didn’t get a chance to talk about which is called entity extraction. That is another …package that we found so this ability to take these open-source packages and then run them on a scalable or python engine. And, this ability to, you know, run it completely inside your virtual network where it can be sensitive data I think it opens up a real possibility for doing ml in this manner and measure government very cool okay well thanks for joining us this was this has been steven fly with…was lately ceo of applied information sciences we’ve been talking about building real-world solutions on as our government’s with machine learning thanks for joining us.