Transcript
Transcript: Innovate on Demand, Episode 22: The Black Box
Kaveh Afshar
And you guys cue me if I'm talking too fast or mumble stuff so people don't get discouraged listening.
Nathalie Crandall
That's Todd's job. (you get) the big eyes and then we both have to be like, oh, what's going on?
All
I'm Todd Lyons. I'm Natalie Crandall. My name is Kaveh Ashar. And this is the innovate on demand podcast.
Nathalie Crandall [0:33]
Welcome. So Kaveh, you work at ECCC? Tell us a little bit about what you do over there.
Kaveh Afshar
Well, I work in the Chief Data office. I am the manager responsible for data science and departmental data governance - which they go hand in hand and feed off each other. So those are the two things we do. In the Chief Data office we have no data, we produce no data, we consume no data, we are just solving data problems for others. So we have the computer science knowledge, we have the statistics and math knowledge, but we don't have the business knowledge. Our colleagues from different branches come and say, "I have this business problem, and this is the data, what can you do?" And then we kind of go about working together to solve those problems using data.
Nathalie Crandall [1:21]
So maybe you can give us a couple of examples of some of the sort of more relevant or interesting things.
Kaveh Afshar
Yeah, I think probably the most interesting or successful project we've had was with the regulatory enforcement branch. And before we arrived there, before we had any conversation directly, and in general, regulatory environments are highly regional (and) run regionally. And it's not always clear from the center how the forces are distributed; which parts are we aggregating data at the national level. To say, kind of like you know, I don't like heat maps, but heat maps are where the activities are happening, (and) all this. That historically didn't exist. A lot of these enforcement's were done on paper, not necessarily available, kind of some regional decision, and some central stuff. I kind of call it "let's do this year, what we did last year - with some changes", that's generally it. So, in our enforcement branch, and it's in the public, they got audited by auditor general number of times, and they say, "you need to be more risk based, you need more evidence based interviews with data effectively". It was very timely. When I had come when we arrived at enforcement and I had a little bit of knowledge about regulatory enforcement from Canadian Food Inspection Agency, where I used to work I was a manager of risk analytics. So, I met with our counterparts at the enforcement branch, and I mentioned, "here's what we could do, we could use all of this enforcement data that you've picked up over some 20 some years. And we could visualize it, we could show you what's happened - that would kind of tell you the story". That was very appealing to them, because if you asked them around 15 (to) 16 months ago, "What happened last week?, What happened last year? What happened any time in history?" There would be weeks' worth of Excel sheets, copy and pasting (with) a bunch of FTEs (would) have to get together and do that. And then they would ask that (same) question two months from now. And then you have to do the same work all over again. So it's very laborious, not repeatable. Even reporting on performance indicators, which is PIPS (Performance Information Profiles), which you have to do with TBS, weeks if not months of work to do, because to fix all of these within the system - that they're using a CRM system, which is client relations management systems,- are not great at data collection. They're (CRM) good at interactions, capturing interactions. So we got that data. And they were given this huge price tag, of like billions of dollars, to warehouse that data (and) to visualize it. I think somewhere in the private sector. We arrived there and warehouse all their data, visualize all our data. And within 15 months, we went from not knowing anything that had happened at its systematic level across Canada for 20 years, to a place where you can see all of this data, in one place, visualized and accessible to the business. (It is) updated every night (with) an automatic scheduler. And even if they continue to enter bad data, like if Shell Canada is misspelled, it corrects (the data) along the way, and puts into a warehouse, in a good condition that can be retracted with all of the performance indicator profiles. It can report it every day if they want to, because that data is updated. You don't have to lift a finger. We just use artificial intelligence and machine learning to produce a kind of top 100 risk areas, or risk entities, you need to look at going forward. This was unheard of. This is generally unheard of in regulatory enforcement, outside national security of course, and delete they made forward is unbelievable inside. But we were also kind of surprised at the pace at which we could execute this. A lot of analysts are super happy because they don't spend time dealing with Excel sheets. They are literally analyzing intelligence (that) is coming in, more or less, on a real time basis at this point. And that has enabled them to take those targets we're producing for next year, to operationalize it and field test it to see how successful they are and what we need to make adjustments.
I would venture a guess, that this is probably the most advanced data operations in the regulatory enforcement across Canada, again, outside national security issues. They have 300 officers in the entire country to enforce (regulations). Having that kind of system tell them where they were they need to pay attention to is invaluable. All of this took two FTEs and $10,000 worth of O and M.
Nathalie Crandall
Wow. So are you knocking on all the other regulators doors now, like, I got some names for you at Transport Canada?
Kaveh Afshar
We'd benefit from a bit of a unfair advantage, because our mandate is very focused and clear on environmental regulations and in wildlife regulations. I do feel for our colleagues at Transport Canada, because their mandate is much larger: Commercial planes, regular planes, cars, roads, trains, driverless cars, drones.
If you ever want to innovate and advance something really fast, you start where you have the most flexibility and the narrowest scope. So to be able to execute and see is this possible. And this was a good example. And I think we're trying to help elevate that into regular community federal regulators. And it's all about impact. We don't talk data, because it's not interesting to them. They are interested in the impact. And one of the things I find in enforcement, they're very operationally focused. That (makes) the alignments are tremendous. We don't have to convince people data is important. And that's really, really good. That culture piece (with regulators) really advances our cause.
Nathalie Crandall
So you said that not only was the client surprised at the pace of execution, but actually you guys internally was surprised by that. So what are some of the factors that enabled you to be able to actually execute effectively and efficiently?
Kaveh Afshar
Yeah, that's a very good question. I think, first things first, have a really good business question and not a data question. Not a specific prescription (but) have a really good business question. And it normally starts with it's "I don't know what happened in our business? Can you figure (it) out? Here's the data". That gives us a lot of room and a lot of flexibility to try to build that story. They didn't tell us what tools to use, what system to engage or what specific detailed deep question you want answered. They just wanted to know what's happened? And how can we use this data to be more evidence based.
So that process, the first iteration of that we had, like a proof of concept, took six weeks. They saw it, it was also that we worked in a very agile way. And like, we constantly say, we're only as good as our customers understanding of what we do. So everything we do is a pipeline. Everything they see is a dashboard. So if the dashboard doesn't really communicate to them, the pipeline is not showing its value. So we visualize it in BI, (business intelligence) tool. And they got it and said, "This is what I want". And so we had a really clear goal of what we wanted to do. We just needed to scale that.
One of the key things was that in all these projects, you need an executive champion, on the business side. They need to want to have this happen. So we've done multiple projects elsewhere, where everyone says, "Oh, that's nice. That's really good. Have a nice day". Because it's not I find if it's another PMA to do it, it's very hard for me to take those kind of risks. You mean, the Westminster system was not designed for its flexibility. It was designed to be rigorous, static, even when there's no governing party. They're running. And you're trying to innovate within that. So there's a lot of cultural issues there. But having that executive culturing of "I want this to happen" and go forward.
We also had some flexibility with tools, in the sense that we could use open source. Everything we use is open source, there's nothing we paid zero for software. And it's all on Python. And the visualization is actually a Power BI visualization. But we're using the free version because our department doesn't have the enterprise solution. It's good for, you know, initial steps. But ultimately, if you want to be data driven, you have to have the right tools. And that really enabled us to having ours in Python. Visualization today, where in our warehouse, we have a little stations, a little content - $10,000 worth of O and M as I mentioned - just a little computer that helps us warehouse the data in a secure way. We probably have more security measures around that station then the entire department. Because I have an audit log, I run security automated stuff. Like (if) anything goes wrong, anyone has a weak password, we let them know. And that's just a bad password, you have it, you know, so we've set up we have that bit of an infrastructure.
But one of the ideas, to me, is that it doesn't take a lot of money. It takes some courage, it takes some commitment. And it what it takes from our side, is make sure to communicate what it does. What's the impact. Often I find the ivory tower type approach to data is, Oh, the coefficient of this is...this. And as a result, you know, the R squared. Most of our executives won't understand an R squared of a regression model. So you have to say, what does this mean? So if every data scientist can learn a new skill, it is to learn how to communicate your results in an accessible way. One where your grandma or mom or dad, who's not a data scientist, will understand. If they don't, you're too complex. It's not distilled yet and that messaging is critical.
Nathalie Crandall
We'd call it the new three C's for data science: Courage, Commitment and Communication.
Kaveh Afshar
I like it, I'm going to use that.
Nathalie Crandall
Well, it's yours.
Kaveh Afshar
So yeah, that's a ton of stuff we do. And one of the things we do when we do these exercises, one very useful thing, is you open up the black box of data. Because everyone knows they have data problems, they just don't know what. So when you do these things, as you build your data pipelines, as you engineer the data for a purpose, you learn what the key issues are along the line that informs governance. If you want to have a data strategy, (you) need to address X, Y, and Z. What are those X, Y and Zs? There's an ocean of data problems. And you need to be able to focus on key elements and then dig deeper. And so that also adds clarity for our enforcement branch. What investments do they need to take or they need to make in their data governance? What areas do they need to tackle first, and how could they achieve that overall utopian vision of some kind of Minority Report or something close to it. And one of the things we did during enforcement (project), we brought in financial information on regulated entities. Because we know from research, companies who are going through financial duress are not paying their taxes. They are breaking laws and being sued. They have poor labour relations issues and guess what? If you're not compliant, you're not going to comply with federal regulations that are very low probability of getting caught. So instead of looking at the symptom of non-compliance, we're looking at possible causes (that) are coming in there. (Are) These companies are bad behaving companies, or are they going through financial difficulties. Maybe enforcement's not the right solution. Maybe a line of credit is the solution. So trying to better understand the industry and various companies and their behaviours and use those behaviours, to better understand where the risks are going to be. If you actually look at it as if you're in a financial space. You can look at people like Warren Buffett. They look at companies as a person. Is this person behaving? Are they doing the right thing? Are they borrowing too much money? Are they taking too many risks? That says something about the character of a company - just like a human? When a human passes a red light on purpose, they're probably not going to stop at a stop sign. Right? So if you have the red light information, you can deduce something and predict their behaviour at that stop sign too. And be able to better understand how to allocate your resources to focus your attention on areas that need attention. In the enforcement world, the most expensive way to check for compliance is to send an enforcement officer.
Nathalie Crandall
So how do you make sure they go where actually absolutely they need it?
Kaveh Afshar
So how do you find the find the spot where you need to send? Exactly. And this is handy to enforcement - for what they want (is) to be where they are the most useful. So we're very happy to have enabled that.
Nathalie Crandall
So if I kind of sum up a little bit your recipe for success really quickly you said the first biggest thing is have a clear business problem. You need to have executive sponsorship - you need to have a champion - and wants to do this.
So would you say that the sort of environment, or the background of government has changed to allow this to happen in the last maybe five or 10 years?
Kaveh Afshar
That's a pretty good question. I don't know if I can make a general statement on that. Frankly, there are pockets that have succeeded and those are good enough because they create FOMO -fear of missing out -for others, and that is much more powerful than me standing saying this is useful. Yeah, so those who have courage and are required to do and they do it and they succeed and often have the right combination in there, they tend to stand out. And others want to catch up. And that's a really good competition to have. It's a healthy competition to have inside the government. I don't know, if left unattended, the culture will change. I really don't think so. I think there's a lot of work to be done.
And there is a lot of work to be done at the executive level. There's a lot of work to be done with the people we hire! We are not hiring typists or mailers anymore. People mail stuff. We are hiring people with analytical capacity. And we have to give them the tools, the knowledge, the data to be able to do that work. We don't want bureaucrats we want brains. So it's hard to attract them if you're not doing the Brainiac stuff.
Nathalie Crandall
Yeah, and it's hard too. And we need to change how we manage our human resources and that we attract the talent, and then we can retain them and move them when we need them moving on.
Kaveh Afshar
And they want to go give them tools they need. Like in the 80s, if you went to work, you left pretty crappy technology and went to state of the art technology at work. Now we are leaving sometimes really state of the art technology at home and going to crappy technology at work at times. And that's a part of things we have to be careful about when we hire new people. We will provide them the tools that they're comfortable with, they can do things and trust them that they can deliver and give them space. I think that's very important. Just to add to that recipe, maybe one thing I would also say, from a data perspective, I'm not big on buzzwords, but agile to me and user centric is key. And the idea of "Oh, give me $2 million, I'll be back in five years" is not a really good recipe for success. That means you're too disconnected, (or) you may not have a good understanding. Also, "I'm going to take a year to figure out your business requirements" is not it. I mean things move on. So we need to be willing to create that room to be able to find prototypes. Or, you'll see proof of concepts that work, and scale them and build these with little technical debt. Which is (if) they can be modified down the line with little investment, and gives them business autonomy over how they manage their part of the equation. If they need to add or remove a fields on a form, you don't have to issue an IT ticket. You should have governance internally to say "I can add this (because this) is my business, I know what I'm adding or removing". In fact, IT will be happy to hear that. Because that's not a very gratifying thing to add a field - unless it's your business -than you probably care more about it. So that's very important to be able to have that agility in iterative nature of doing the work. By the time we deliver something to the customer -our clients -like I've seen this and I'm sick of it. I know it's good. So there's no surprises. They're not getting anything that they haven't seen before. And they go around and do a lot of the culture piece under operations there do it on their own. We kind of try to play a supporting role. But that's kind of like a more business side of things that they take care of. We try to say our role is: We did the statistics and data and show you the impact; and then you translate that impact into your workforce. You have the credibility. We don't. We don't know your business well enough.
Nathalie Crandall
But we all have to be the supporting cast to the frontline business. But, I really echo your sentiments and that that ability to work in a different way to deliver in a different way in different timelines is critical for us in the government because the world is happening. And the rate of change in the world is so fast right now. If we want to be relevant, we have to.
Kaveh Afshar
Absolutely. I mean, this is a little personal for me, too, because I come from Iran. It's not known for its democratic institutions. And I know when it's not there it's suffocating. And there's nowhere written in our history books that Canadian democracy is going to last for 30 years if left unattended. Part of our role I see is that to make sure we make democracy relevant. We make it smart. So people see value in it. So they they're willing to invest in it, they're willing to pay their taxes. Do they think that government is using what they have smartly, to protect their environment, to protect their privacy, to protect their freedoms? And if we are kind of falling behind? And I I would guess if you don't do stuff today, well, in 10 years, there might not be an opportunity for us to make up the time. Googles and Apples and Facebook, the world will show value to their customers - often at the price of their privacy. Government might not become as relevant because we are to detached and out of touch. So that's kind of my bit of a democratic imperative in my life. We can't not do these things. We can't just sit up to say status quo is okay.
Nathalie Crandall
It's not. So the days of a complacent public service are over.
Kaveh Afshar
But it's kind of like, we are not the Westminster system that's supposed to be static and not change. People expect change. And we just have to manage that change, communicate it and understand there is a possibility of failure. And if you want to be better at not failing, especially externally, you have to do (better) things internally. If you can't do HR in a fast way internally, we might struggle in AI. So if you don't have internal services that work well and have familiarity where there's more room for risk. You know, God help us when we go outside. So that's kind of a regard. We can take risks, calculated risks, have a backup plan, communicate it, and explain it that AI isn't going to solve their problem. It may help us direct our attention. But that's important because we have an ocean of problems. It's the idea of like, Where do I look now? Where do I put my resources? Just like a day. In a day you have a eight and a half hours. Which emails are going to answer? Which papers are going to read? And AI can really assist us in doing that. It will not solve our problems. But it would definitely make our questions smarter, make our operations smarter, and inform us to know what the next steps are. And I'm not sure always, as it stands, we know what our next steps are...like digital transformation, data work, all kinds of stuff.
Toddy Lyons
You've been listening to innovate on demand brought to you by the Canada School of Public Service. Our music is by Grapes. I'm Todd Lyons, producer of this series. Thank you for listening
