What is process mining?
Share
Summary
It’s one of the most important skills employers are looking for in the business world today: process mining. What is it and how can businesses use it to leverage data and become more efficient?
Guest: Vishal Shah, associate professor of Business Information Systems at Central Michigan University.
Summary
In this episode of The Search Bar, Vishal Shah, an associate professor of Business Information Systems at Central Michigan University, discusses the concept of process mining. Process mining is a set of techniques that allows organizations to analyze and optimize their business processes by examining the data generated by their enterprise resource planning (ERP) systems. It provides a comprehensive view of the entire process, from start to finish, and helps identify inefficiencies, bottlenecks, and areas for improvement. Process mining is applicable to various domains, including manufacturing, healthcare, supply chain management, and more. It is a growing field with a high demand for professionals who can analyze and leverage data to improve operational efficiency and achieve business objectives. Students and professionals from various disciplines can benefit from learning process mining skills, as it provides a valuable mindset and toolset for understanding and optimizing complex processes.
Transcript
Chapters
- 00:00 Introduction
- 01:07 What is process mining?
- 05:54 What are some examples of process mining applications?
- 10:52 How can I get into the field of process mining?
- 13:40 What is ERP and SAP, and how do these systems benefit businesses?
- 15:52 How does process mining build on data mining?
- 19:38 What makes quality data and why is that important for process mining?
- 22:48 How can process mining work in tandem with human judgement and decision making?
- 29:42 What career paths can benefit from studying process mining?
- 35:57 Why is the demand for process mining skills increasing?
- 38:48 How can process mining skills can help students succeed in the job market?
- 43:24 How can I apply process mining to my personal routine?
- 49:35 Are there limits to process mining?
- 51:27 Are there any ethical or legal concerns with process mining?
- 54:38 Review and wrap-up
Introduction
Vishal: So, if you can ask [the] right questions and you can see the right patterns, you'd be better off versus not seeing them. And that's where the tool — of a tool set of process mining allows you to do that. Because it's not one thing, it's kind of a set of things, process mining, right? So many different ways to look at information.
Adam: It's one of the most important skills employers are looking for in the business world today. Process mining. Welcome to The Search Bar. I'm your host, Adam Sparkes, and on today's episode we're talking with Vishal Shal, associate Professor of Business Information Systems at Central Michigan University. Thank you for coming today, Vishal. Thank you. It's great to see you again. I wanted to open up our conversation by offering a definition, at least in your opinion, of what process mining is. I know that prior to knowing that we were going to talk about this, I only had a really vague assumption about what it might be, and that it was adjacent to data mining. So, I was hoping maybe you could clear that up for us.
What is process mining?
Adam: The process mining might be like where I'm seeing different points of data being mined across either a duration of time or different steps in a process.
Vishal: Duration of times, locations, units, wherever the life cycle of the process takes from step A to step B to step C. Everywhere you're being followed by information, and if you mine that information in near real time, that's what the goal of process mining is. So, it's a lot more than data mining. Data mining is some of the techniques that are used in process mining. It depends on data mining under the hood in many cases.
Adam: Right, because you need to have that reliable data at those different places in the process for it to be beneficial for the business or the individual. What would a good example in your mind be, if you were telling a student about this, what's a good example process that could be data mined for benefit?
Vishal: Maybe there's a personal example and a corporate example. The personal example is you go and you buy coffee, you want to make sure that you only pay once for your coffee, you don't pay twice.
Adam: I'd rather not.
Vishal: I would rather not pay for coffee.
Adam: Yeah, I'd like free coffee and free dumplings.
Vishal: Me too. But the idea being that wherever there is wastage occurring or reputation occurring, which could be reduced and which reduces the efficiency of the outcome, the outcome that an individual in a corporation may be interested in. If you can see that at a visual sense and you can pinpoint that this is happening, then I can reduce it. So, for example, when I buy — a big company like Dow Chemicals, for example, they want to buy something from someone. And they have a vendor that is selling them, and if the payment process is automated, and maybe it's done twice or thrice without knowing it. And they might not know that that is happening unless they actually mine the entire chain, value chain, that goes through that, and then they will see that this is happening. “Okay, we can reduce that wastage here.” Maybe we want to track where the information is coming from. So that type of thing. And it's very useful for operational efficiencies, but that's not the only case. Especially things like carbon footprints, finding out where your sourcing is coming from, who is doing what in your supply chain. Sort of a transparency into that transparent nature of that, as transparent as you can make [it]. So that's just sort of an idea.
Adam: Right. So, that's interesting too. I guess I didn't really think about that either. It's not just maximizing profit too, it might be maximizing certain outcomes within the process, right?
Vishal: Right. And profit may be one of the outcomes that the company may be interested in. The others may be like the operational efficiency, reporting guidelines. For example, if you run a big corporation now you have to follow certain emission rules, and you want to know where your process is emitting the most, or where is the production is — where you should focus your resources so that you can reduce your emissions. I need to find that out. And all of the information is diffused in my procurement and production lines, for example. They buy something and make something, but all of this information is in chunks, and it's not unified. Process mining sort of lays the groundwork to make a unified view, of how to see that. And then if you find that this specific product or this specific plant is responsible for most product carbon emission, maybe you could do something. You can engage in carbon trading at that point, or maybe change the raw material that goes into it, or find an equal and raw material. So there is different objectives other than profit that might be served by this process.
What are some examples of process mining applications?
Adam: It was weird because, for me, as a person who's way off in other areas professionally, and I heard about it, I'm like, “Man, it feels like you got to be really into efficiency.” But as soon as you said that my brain started saying, “Well yeah, you have things like emissions or regulation requirements or safety.” There's probably, especially for really large operations, you're talking about Dow Chemical, or if you're Coca-Cola or General Motors or something, safety — where people are being hurt or being prevented from working might also be able to be identified in something like this, right?
Vishal: Right. And for example, even admissions, you said, for universities. Admission process and the student progression process. Because [a] student goes from freshman to sophomore, junior, senior, and universities would be most interested in improving their graduation rates, so on and so forth. See where delay’s happening, why that delay is happening, what is the cause of that? What unit is responsible or maybe related to the delay, and can we do something about that if we can map that journey out? So we have all these data that is sitting with us and process mining is a set of tools, which relies heavily on data mining as underlying algorithms, but it is to the next level of data mining because it's sort of a chain that is not done.
Adam: And before we get into some of the details of how you do this too, I think the thing that struck me as interesting, again, and just kind of thinking about it before I knew you were coming in, was that these processes, some of them that we engage with every day, they're massive, but they're generally invisible. Like a process — if you and I were able to sit here and define a process, it can be anything that has more than one step to happen. But you were talking about that coffee, right? There's so many processes in place before coffee shows up at a coffee shop, or before your box shows up from Amazon. For you or I, the process seems pretty simple or pretty invisible until it doesn't work. And this is helping people make those processes less likely to fail as well, I would assume.
Vishal: So, fault tolerances, yes, because it depends on what is being looked at. So, for example, if it is generally a normal structure, like order to cash in the same sales cycle. If a company is interested in how they have a product and they sell a product, and there's a generally defined [set of] steps for that. Are you deviating from that? That's not mission critical. But, getting the raw materials for your defense organization, for example, that's a critical thing. You don't want to break in that and that is important. So, there's a different view of that in terms of what key performance indicators you would look at that domain versus sales domain versus production domain. Those change. But the applicability of creating transparency from one point to another, because usually it has been so diffused so far. As a consumer, we don't really think about the process that goes behind it. But the company's competitive advantage, and also regulatory compliance, depends on how well they can execute these repeated transactions. And if they can do it better, they found something wrong, they can improve that and they can see that change in their — either in their compliance, either in their bottom line, either in their corporate social responsibility, KPIs, anything that has an objective function. So for example, I want to achieve something, what goes behind achieving that? And can I see all the data lines that go into achieving that? And I can see, well, this is not really that useful. Can I then change that? Can I impact that? And the next step [that] then happens at the big corporations, for example Dow or Walmart, is automation in terms of if something is repeated, repeated, repeated. And if it's not really value added, maybe that can be automated by something else, and it'll take some lower time or lower resources, or fewer resources, so that I can get to the same point.
Adam: And then I would assume you could data mine an automated process as well, too.
Vishal: Right, so, it kind of works both ways. One is input to what you need to automate. You have to find that out first. When you look at your own data with that level of granularity, you can start to understand what needs to be automated. And once that is automated, does it feed back and say, well, is it actually improving what we want it to improve? So, kind of like a two step....
How can I get into the field of process mining?
Adam: What attracted you to this type of work? I'm curious, do you see yourself — are you a very efficient person or is your brain very efficient?
Vishal: Not really. *laughs*
Adam: What got you here?
Vishal: So, one of the things is what we already have in place because, as a — CMU is a University Alliance School and we have a very good SAP program and ERP program. So, ERP is kind of a centralized warehouse of information. Everything goes inside that. So, if people are going to put their process data, which is sales, procurement, production, anything that has to do with running a business or running a unit — that goes into a centralized database like ERP, SAP. We have a very good program with that. So, if we can leverage that to mine all the information that ERP systems generate because around — Fortune 500, [around] 2000 companies, they all use SAP or similar systems and they have this treasure trove of data that is sitting with them, but it's not telling them a whole lot beyond what is the daily operations. If the next set of techniques called process mining comes along, and then taps into this data warehouse, and then it can tell us what was wrong with how we were doing a process. So, can you improve something or is it perfect or something like that? So, we already have the infrastructure to teach ERP, which is SAP, right? And Central is known for that. So, what is the next logical step? It’s to take all this information and understand how to reverse engineer that. So, if we were making any mistakes in actually doing this transaction in the first place, it would tell us. So, that got really started — me to think about… Since this came along, and actually this started in Netherlands and Germany, process mining as a field, and it's [a] perfect pairing with a data warehouse like ERP or big data store. And we are teaching these things at college of business anyway. So, if you can put it on top of what we are teaching, then people will be more prepared for the jobs. And the same companies that are using ERP also will use — are going to use process mining and using process mining.
Adam: So, for you it was an educational motivation then, right?
Vishal: And research and education, yes. And fun.
Adam: Yeah, that’s right, it's fun. I want to take a tiny, little bit of a step back just so that we don't leave anybody in the dust on this because I know that if I didn't work at Central Michigan University, I would have no idea what ERP and SAP is. Can you give me a real quick definition?
What is ERP and SAP, and how do these systems benefit businesses?
Vishal: Everything that we do in our daily life, like buying a product or selling something or going to Walmart and getting something off the shelf, that supply chain efficiency, you said, has to be coming from somewhere, right?
Adam: Yeah.
Vishal: Somebody has to store all this information in unified format. And the thousand-foot view of this is there is a centralized system called ERP, which is broken into many parts like production, sales, procurement, and manufacturing warehouses, and all of this data is talking to each other. So, we as a consumer get our products on time. We as a vendor can get our invoices paid in time, bills are paid on time, things of that nature. We never really think about that. But all of that has to come from somewhere, right? And ERP is one such concept that brings everything together so people can run it. But at the same time, there have been lots of inefficiencies, because even if it is automated or stored in electronic format, mistakes happen such as paying things twice, or buying more than what you need, or then you are to dispose of what is not needed, or you don't have enough product and you lose sales. Where is this happening? If we can see that ‘where’, we can interject and change these things. And then a set of technology comes along, process mining, which plugs into ERP and says, well, we see that you're doing something wrong here, or this can be improved, or this is perfect. And ERP is sort of like a big box central information system which powers our everyday life. Like Central uses SAP for administration, for students' lifecycle. They come and register the classes. All of that's done through SAP. So, we get all this in the center and it makes our life easier. But there is something which can be improved if we actually looked at the data that it produces using something more powerful than just generalized data mining.
How does process mining build on data mining?
Adam: Right. And that's where that process mining comes in. So, I'm going to take a crack at it, a really bad analogy and tell me — and don't worry, because if it's really bad, Kyle's going to cut it. Going back to that, kind of the, big box retailer example, something like SAP is telling that retailer — it's not a clerk that goes around and goes, “Man, we sell this many boxes of Frosted Flakes a month. I'm going to count them, and I know that every month I have to replace 45 boxes of Frosted Flakes.” It's a software that tells them that, that's SAP. But that same big box retailer might realize that, “Man, every year we're selling 350…,” I mean it’s probably bigger than that. “We're selling 3,000 super soaker water guns, and if we keep restocking those at the same rate throughout the whole year, we might not be doing…,” and I'm using an obvious example, but what process mining would tell us is that that only happens between these certain dates because it's summer, right? I mean that might be too obvious for process mining to need to step in, but it seems to me like the cereal sells all year round, but you suddenly are overstocking on water guns in October because people stopped buying them in September. Is that sort of the difference in how the data's being looked at?
Vishal: Right. So, for example, in this case you would have something like inventory turnover as a KPI, or a key performance indicator, like how fast your inventory turns. And you manage that, as a big company they should manage that; okay, this is my optimal level of inventory. But even then, shocks in supply chain, shocks in geopolitical situation, all of these will change that calculus. And if we know that information earlier than that happens — and how would we know that if we only look at the data that is actually coming into our system? If we just sit back and let the data come into the centralized database and then we work on that, that's not going to help us a whole lot. So, what this is doing is trying to see what is coming in this big central database and is it according to what we wanted, and say, “Well, okay, now we are sensing that, okay, this country in Africa is not going to produce this much. Okay, then we need to shift our production to here. But how do I know that? Because my vendors in Africa, or my suppliers in Africa, are not really sending me this stuff on time. But that would take a long time to understand within ERP because for that to surface at the higher level, I have to wait. And if I had something that is plugged into this data flows where I can see what is happening earlier, then I could shift, so I could manage this more effectively, if you will. So — and ERP is a great improvement over what we used to do using pen and paper, and SAP is that way good. But even SAP has wrong rules that are coded into it. What if somebody has coded the process wrong and is following that same process because it's automated and then this person is getting paid, instead of once, twice? And, well, of course he is not complaining because he's getting paid twice.
Adam: I wouldn't say anything.
Vishal: But we wouldn't know that if we really didn't do the proper audit. So, it's kind of like a very high-level auditing help, also, to understand what we are doing. And it can be applied to internal operations, operations outside your company, interfacing operations with partner companies. So, depending on where you apply these set of techniques, you can get different results. Does that make sense?
What makes quality data and why is that important for process mining?
Adam: It does. In a process like this, I have to imagine that making sure that your data is accurate, that the data that is being observed through the process mining, the process mining process — that feels really weird to say — but through the process mining, as you're doing process mining, making sure that those data points are accurate probably has a lot of value. So, if a company is preparing to make a cultural shift from being more like, “Our success is based on the performance of individuals versus the efficiency of a process.” I guess two part question. One, that must be hard for some organizations to swallow, because it must feel scary to do that. And then two, how difficult is it for them to make sure that the places that they're about to measure are actually giving accurate data, that their data mining is good enough to be process mined?
Vishal: Yeah, I think it's a great question. I think data quality is one of the most important thing that goes into the process. Like garbage in, garbage out situation. If it is not a proper representation of reality, then there's no point in mining that, right? And so, there are steps in place to, and where the interface happens, to make sure that this is actually good, or at least you start with a good sample that is representative of what is going on. And this is where it connects to a deeper area of, you might call, data engineering where the data is massaged in a way that you can actually use it. And there's lots of infrastructure already developed for real time things, like streaming, Netflix. It's a real time streaming platform, and then imagine that but for a business level. Instead of Netflix, we are talking about sales and procurement, same thing, but different end product. But the same data pipeline should exist, even if it doesn't exist to a hundred percent, it should exist to an extent where you can start to apply process mining. If the data is not clean, then obviously mining the unclean data is not going to tell us a whole lot. So for that thing, there is three real important information that is needed, like a timestamp. When an activity happens, what is an activity and how do you identify that? So, a unique identifier of step A, what is step A and when did it happen? If you have these three [pieces of] information as data point, you can then start thinking about process mining. You don't have to have all the data in the world, but anything that happens, you have to have an identifier what that is. And then when that happened, because timing is important in process mining. So, if you have a timestamp, the description of what is happening and a unique way to say this is a different event than other event, an identifier — once you have these three things, you can think about applying process mining to that data.
How can process mining work in tandem with human judgement and decision making?
Adam: Kind of creating those values for those different data sets. I imagine that's probably a really important step, too, because some stuff is going to be more heterogeneous than other stuff. I was talking with Aaron, the producer, about this. Where the thing that struck me was that it's probably a little bit easier to look at manufacturing efficiency than healthcare efficiency. Because, what a physician or a nurse does, there are certain things that are super measurable. I gave a patient this many CCs of this, we did a blood draw, they were in at this time, out at that time, went to billing, rescheduled a new appointment. But when a doctor spends 15 minutes with one patient and an hour and a half with another, it's not always that measurable. It's a little bit more mushy, because they're doing so many human processes in there. Is that something that process mining can address, or is getting better at addressing? I'm just curious.
Vishal: Yeah, I think so. There is different domains that is applied to, and this is — bring back to your earlier question [of] why we thought it would be [a] fit here? Because ERP produces output of ERP's data in the format that the process mining wants. So, the SAP ERP has timestamp, has an activity, or you could at least convert it into a format which would be digestible for a process mining tool, like Celonis or Signavio, any one of these. But to your point, it is applicable to hospitals and healthcare domain, but it depends on which specific site. You cannot apply to human processes where judgment is involved. But something like the payroll for healthcare, because that's a big data producing pipeline there. Also, with the patient journeys. For example, if the patient is being transferred from one to another, sometimes if they just keep on going from one department to other department, and they don't know what they're doing, and they really don't have a bird's eye view of what's going on, then actually it's counterproductive for the patient and wastes time. So, there have been a couple of research papers to [the] application of process mining in healthcare. But it is not cut and dry to say that. It cannot, obviously, because if one doctor is spending 15 minutes versus one hour with one different patient, you have a different problem, you can't generalize that. But for the more regular processes, like going to radiology, or going to take a sample, or going to have a CT scan — if this person is going to transfer from one department to the other department, is he doing it in the fashion that hospital imagined or is he not doing in the fashion that the hospital imagined? And that's where the process mining can be helpful.
Adam: You can at least flag that. So, you can go, “Well, why?” Is it an individual reason, or is it a process reason that that's happening?
Vishal: And then, it can apply to machines. For example, MRI machines will have timestamp data, and if the machine lifecycle is — it can tell you that it's going to break down. And if it is predictive, you could predict, maybe, when the machine is breaking down, you'd be better off ordering one in advance.
Adam: Yeah, you got three months before it finally breaks or something.
Vishal: So, something of that nature. If you keep looking at the data that the machine produces, and you see the aberration in the results and you say, okay then, well that machine is not doing its job. Maybe we need to reduce it, or change it, or… So, where the human judgment is not involved, especially with — you can't say, “Well somebody has a cancer, and somebody has fever.” That’s a different type of consultation.
Adam: I was thinking even, it would probably be super quick for somebody who's looking at a dataset and is trying to process mine that dataset to say — again, I'm making up examples here. You have a product that you're wholesaling, and it comes from a certain vendor that is overseas, and when it shows up, the label on the package is not regionalized, it's in another language. But through whatever ERP system you had previously, nobody took for account that somebody in the warehouse actually puts a new sticker on it, and that at the end of the day adds this many hours of work to add this regionalized sticker. Until you process mine and go, “Hey, why are we doing rework right here at this step after receiving the product?” Probably pretty easy. But if a physician — I’ll just go back to that example, because I think people are familiar with it. If a physician lives in a certain area, in a certain city and people come in and [say], “Oh, my stomach hurts, and I have this bowel problem,” and they know there's a restaurant in town that serves the spiciest dumplings, right? It's a spicy broth. And they go, “Wait, did you go over to have the spicy dumplings over next door?” and whatever it is. And they say, “Yeah.” I go, “Ah, that's what it is. You'll be fine tomorrow.” All of a sudden, the patient with the burning stomach pain ate the spicy dumpling; they're out of there in 30 minutes because the physician’s — that knowledge, it's not as transferable, I guess, is what I'm saying. So that's a harder thing to account for.
Vishal: Yes, exactly.
Adam: But, process mining could probably still see that, “Oh, when [a] patient comes in and it’s a stomach pain, some of them are much quicker.” And you could probably determine that the physician does know that at some point
Vishal: Or you can say, well, they come to this clinic and they're being ordered to do this test and this test and this test. Is the protocol generally being followed? For example, where the human judgment is not, like testing. Testing is — you have to do these tests, because doctors have to do these tests before prescribing something and there is a certain protocol to that. And if this protocol is being, “Well, somebody is doing one test again and again and again,” or somebody is not getting an appointment to a specific clinic because it's overloaded for whatever reason. So, what is — those issues can be fixed where there is no human judgment about illness involved. So, those issues, issues of machine life, x-ray machines, MRI machines and that type of thing, payrolls for healthcare. So, where there is not, I would say, subjective evaluation of patient's health, which obviously is not something that — I cannot codify that, so I cannot, obviously…
Adam: Not yet. We want to make sure that we're not going into the hellscape of the software’s listening to you and knows whether you're being honest with yourself.
Vishal: Software is always listening to us.
What career paths can benefit from studying process mining?
Adam: If I'm a business student and I'm in that ERP/SAP ecosystem, what are the first steps? What are the first skills-sets that I should be looking for if process mining's in my future? Or what makes me, maybe, a good candidate for these types of positions? This is new, right? This isn't something that's really entrenched and established yet. It's kind of hot, it's a new thing.
Vishal: So, the industry I think is over a billion dollars.
Adam: So, it's already huge.
Vishal: And it's growing at, I think, 40% compounded growth a year. So, it's growing very rapidly, especially the established base of companies using ERPs, because they need to then get efficient and better about their own operations and their own processes. So, it's growing very quickly. But to your point, the process mining is kind of applied to all disciplines, mostly. It's not something that [just] an information systems person will do, or finance will do, or a healthcare person will do, because what is the core of this is that if you can see what is happening under the hood, you can obviously improve that. So far we didn't have that visibility. Even with ERP, we had some visibility, we had ability to collect it, but with process mining, now we have visibility into that. So, people, and the person who invented process mining, like to say is kind of like an x-ray of organizations. So, you can see what is the nuts and bolts of what is going on. So, in that case, it's discipline agnostic, but it's a very good skill to have because there are so many jobs out there which require this. And ERP jobs, they’re not necessarily information systems jobs — they’re accounting jobs, they’re finance job, they’re management jobs, they’re marketing job. Because, you end up using ERP sometimes without knowing it. And sometimes when you go to roles like analyst or data analyst or any of those roles which will eventually interface with these systems. So, if you can ask [the] right questions, and you can see the right patterns, you'd be better off versus not seeing them. And that's where the tool of… tool set of process mining allows you to do that, because it's not a one thing, it's kind of a set of things, process mining. So, many different ways to look at information. That's what it is. And Central Michigan is a good place to do that because we've been chosen three times now for Center of Excellence in process mining, in addition to being a University Alliance SAP school — and there's only 18 schools worldwide.
Adam: We're doing it.
Vishal: And the other schools are — the previous iteration at William and Mary, London School of Economics.
Adam: We do it better than them, though.
Vishal: So, think, like, we are in good company. So, it's a really good way to, then, start looking at information because at the world we live in, as everybody notices, getting more and more data heavy. There's no escape from — but you have to use data to your advantage. And obviously we can't use all the data, but if we have some tool set, and one of them being process mining, and data mining is obviously part of it, but it is a much bigger view than data mining. Data mining is very localized for application. This is a lot more generalized and can be applied to many places. And companies like Celonis, which has supported us, and SAP, they have great tools. So, if you go to market with saying that I know these things and these are growing, your demand, obviously, is going to increase.
Adam: So, you're saying that you're seeing those students, kind of, from all over the college of business?
Vishal: Yeah, the first class we taught with, we had photography or cinematography majors. Biology majors. So, statistics majors, computer science majors. So, it's not really anything that has to do with information systems, per se.
Vishal: It’s kind of…
Adam: It works nice there.
Vishal: Yeah. It's kind of like a word processor. Everybody uses it. And if it is in the phase where you can master it, it's taking off. So, the currency is much higher because it's applicable to a general — a lot, many domains, then just a specific algorithm in data mining.
Adam: So, really for anybody who's interested in going into a field or profession and being able to improve it, or research even, being able to improve the efficacy of research. I didn't even think about that when we were talking earlier, you mentioned the biology students, and I was thinking that would work, too. So, this is a set of classes that you can attach it to any area of study, really.
Vishal: And I think it's more of a mindset, and once you have this mindset you can pick up the right tools, like Celonis or SAP, and then you can certify in them and then your value will increase. But the idea of, “How do I look at what's going to happen tomorrow?” I have to look at what happened before that. And this makes it somewhat easier, not easy, but gives you a vocabulary, gives you some sort of a translation mechanism, technical mechanism to look at all these so that you can then say, “Okay, I have done this much, and this is going to be…” So, process mining is usually becoming part of many jobs, and there is certain jobs like Process Mining Analysts, Process Architects, which are separately just doing process work, but it is also becoming part of many other jobs. It's embedded into job roles, that type of work.
Why is the demand for process mining skills increasing?
Adam: Are those specific roles that have process mining in the title, are those the type of positions that you see young people getting the most, or is it other areas?
Vishal: We have some students getting those, but more often than not they ask for this in addition to something else. Because this is an up-and-coming area where companies and corporations, basically, are also figuring out how to use that. So, they would like somebody to have this knowledge and use the data, because a lot of times [within a] company data it’s just not being used. But if you do have the skill, you can use it to advantage — your advantage, to the company's advantage, to organizational advantage, too.
Adam: So, they're looking for that professional in their area to come in and go, “Hey, we have all this data sitting here,” and we kind of know what it does in bucket A, B, C, and D, but we don't know how it gets across.
Vishal: And it's getting more and more traction because, as geopolitics change as — well, there's talks of recession. Okay, but what do companies want to do in recession? They want to save money. So, how do you save money? You look at what you're doing and saying where is the wastage? Well, how do you look at where is the waste? And so, this is sort of a skill which is applicable in different, I would, say time pockets, whether it's expansion of economy or reduction of economy, because either way you want to do better than what you did yesterday, and the only way to do that is to kind of look into what you're doing. This, kind of, like, “Use your window into that.” So, that really....
Adam: Yeah, if you're going to — I mean, to be a little bit grim about it — it's a field where, if you're going to make a reduction in process or reduction in employment, you want to make sure that you're doing it in a way that makes sense and is ethical, right? It's not just — it's not Office Space where the Bobs interview people and they just fire who they don't like, kind of.
Vishal: Yeah, and I think it's kind of, like, the conception is it can come across like that, but it is not — it's been used as, not as a reduction of workforce situation but you can allocate your capital better. So, for example, if a certain thing is not getting done and your people have more slack in another area, you can assign them to that area. This is how it's been used, as far — to my knowledge, it's not been used as cost cutting by cutting heads.
Adam: So, there's not going to be a cool movie with George Clooney flying around firing people, and he's not the axe man anymore, he's a process mining terminator.
Vishal: It's more like somebody looking at the chart and asking what's going on.
Adam: They probably won't be as handsome as George Clooney. Some of our students are as handsome as George Clooney, some of 'em. What do you feel like our students are doing to stand out here? What makes them set apart?
How can process mining skills help students succeed in the job market?
Vishal: So, I think one thing is we are certifying. We have certified, I mean, roughly 1500 to 2000 students, I think — more than that — including this year, into these certificates. And they take classes in this area. They've also — once they take classes, what we have done so far is, we have pair them up with the companies. So we have a set of companies that are interested in process mining, some of them like Dow, Infosys, Celonis, PWC, SAP. And so, we have these students go through these certifications and then assign them to these companies where these companies will give them a data set, and then they will work on that data set and present their finding back to the company. And if they do well, they can get scholarship money. And we've had two of these events and, the company behind process mining — well, it's a leader in process mining — Celonis has supported us with scholarships. So the people who are doing well, they can get scholarships, they can get interviews, that type of thing. And we have done that now twice in [the] last two years. So, it's [the] second year we have done [it], next year we'll do it again in November. So, students are — and this is, I think, one of the very few schools, given there is only 18 of these centers worldwide, is trying to do this. And we are sort of a leader in this space, and we are backed by a good infrastructure with ERP. So, it, kind of, fits in. That, and then the process automation and all of that. So, students have been doing well and they've been recognized. Some of them have gotten jobs through these events. They have got money through these events and they go out and say, “Well, I won.” It's not a typical case competition because they actually work for much longer period with the company than just one day. It is not like we are giving them data set and saying give the result tomorrow. So we train them, we let them work with a company for anywhere between two to four months, and they can work their mechanisms that we are devising to make that happen. And at the end of every year, they come back and present to the companies. And they'll be anywhere between five to 10 teams and then they can gain money, scholarship, and recognition. So, our students are doing well. We are hoping to expand this more.
Adam: And it's a great investment for that company too, I imagine, because they're helping mold a workspace that — ultimately, they will profit from more people being functional in this workspace. But these students are going to come out of here, and they're doing something that they know is tangible before they ever walk across the dais here.
Vishal: And also, if that company at Celonis has helped us, as we make a name in this space, other partners will start helping us and it's kind of win-win for everybody. So, it's not about the money, but what the money can do for us for the students and the research, which is good development for us.
Adam: For everyone. And I imagine — I mean, like you said, it's growing like 40%... Was it annually you said earlier?
Vishal: It's a compounded annual growth rate of 40%.
Adam: Which is wild, for any professional area or area of study. I mean, that's nuts.
Vishal: So, it's really good to be first in the space, or at least among the first few in the space, to do this and establish our name in this because just like we established our name in SAP, we want to establish our name in process mining and how they talk to each other and that it gives you a good visibility into companies. And basically it's all about creating this ecosystem of students, research, corporations; they work together to sort of elevate the standard of what we are doing.
Adam: And don't forget, if you're a student or you're a researcher and you want to bring this into your area too, it's not just for business admin or finance people or SAP people either, too.
Vishal: No, not at all. I mean, there's lots of research. It's also happening now in America. It used to primarily happen in Europe because they started in Europe, they started in Netherlands and Germany. But there's a lot more, and there's a lot more scope because companies are all — headquarters, a lot of them, big companies are here.
Adam: Yeah, that's awesome to hear.
How can I apply process mining to my personal routine?
Adam: Let's create a couple examples for people here that they can really relate to. So, if you've listened with us so far and you're like, “I still don't get it,” because, I mean, in a weird way, it's a concept. Us sitting here and talking about the particulars of what you might read in the software isn't as valuable as, I guess, the ethos of being able to do this big thing [and] wrap your head around it. So, if I’m — let’s think of an activity that we can apply it to. What do you guys feel? Like, a workout regimen, right? Could you process mine your workout regimen? If you had a fitness goal…?
Vishal: I think workout regimen is okay, but [it’s] one activity in your day, so I think your timeline should be longer than just one activity.
Adam: So, you should measure all the things you do for your wellness.
Vishal: For example, one day, the unit of one day. And then in the unit of one day, how many hours do you spend working, how many you are working out, and if there is something that you have, you know, time off, okay, you are… So now you're looking at, well, I just want to reduce my onscreen time. But then there's an app for that. But you do that, but once you do that, maybe you spend more time somewhere else to compensate for that unconsciously. But you might not realize that because the app is telling you that, "Hey, I block your screen and you're good," but you may be doing something else in lieu of your screen blocking, which you may not even register. And if you have the information about that, for that unit timeline from start of the day to the end of the day, and then say, “Okay, I'm using less screen, that is good, but I'm also doing this and I didn't really realize that I'm doing this," unless I take a step back and say, well, in its entirety. So, it's a sequence of activities that are being repeated. Are they good for you? So that's where your objective comes in. What objective do you want to have? Want to be healthy? You don't — you want to go on vacation and don't do anything, then it's obviously a good thing to not do anything. But yeah, so everyday life, you can think of it as a longer timeline, a day timeline, and a different task in a day where instead of optimizing one task, you're optimizing the entire day.
Adam: So maybe the difference between, even broader than that — tell me if I'm doing it right because you’re the expert.
Vishal: I’m claiming.
Adam: He's definitely the expert. If I'm hearing what you're saying, it might be the difference in — if I'm trying to have a more efficient… or more health, whatever my goal might be — I want to sleep better at the end of the week. If I want to look at what I'm doing and I wanted to kind of process mine my own process, my own day (my day being a process), over the course of the week, if I write a to-do list, it's not really process mining if I only look at what I did or didn't do the to-do list. I really need to be writing down everything that I'm doing, to some degree. It's not about here's my goals, I met these goals or I didn't. I got less screen time or I didn't. It’s evaluating it all. It's, kind of, going, I ate this, I woke up at this time, I did this many miles on the treadmill, I took my lunch at this time. It's kind of doing that for a certain duration before you can really evaluate it.
Vishal: Right, so, for example, the smart watch. The data from the smart watch, that's not just one point data, but it's going you the whole day's information. And then, maybe, add something else to it and then say this is happening, this happening. And then you look at all of that, not just your to-do list at the end and say, “Well, I did this, I didn't do this.” But if you look at — if you're not doing something repeatedly for one week, maybe you want to find out a reason why that is not happening, other than just I'm not doing it. I'm frustrated, I'm supposed to do it, but I'm not doing it. I find out every end of the day, well, I'm supposed to do it, I'm not doing it well. There has to be more nuanced understanding of what's going on in my day other than just one activity where I am blocking my screen time. Okay, that's the happy activity and I can see that, but I cannot really see anything else that I haven't done, and I cannot see that just from the result. I have to see the entire chain. And so, that information already exists, as far as many organizations are concerned, in the ERP systems. And then if you can just plug that and look at that, then you can say, well, and then the same goes for the day. But it's hard to automate it, as far as the day is concerned.
Adam: I just think that's an interesting way… I think it's an interesting way to look at it because probably these young professionals, as they're getting into this, too, like you were talking about, that quality of data is so important, right? If you think about it for yourself, and then apply that to a company that's made up of many people and many smaller processes, there's probably a lot of things that are more the checklist than the actually recording it as you go and you sort of have to replace the quality of — the checklist is so binary, right?
Vishal: Right.
Adam: And I'm sure there's steps where binary data is valuable, but we want more quantitative stuff than that, right?
Vishal: More granular information. But this information already exists in the electronic systems that capture. The idea is, so far, it’s not being used as much as it should have been used. If we can tap into that, and one of the set of techniques is process mining that taps into it, and then say, “Well, I can look at it, just not from a one short point of view, but I can look at it from a longer time point, and say I can define my endpoint and start point. And I can say, “Well, I want to look at this.” And that way I have a better handle of what I'm looking at, as opposed to this is just a bunch of data and we don't know what to do with it. So it doesn't really minimize the value of data engineering, if you will, or quality of data. It's very important. But at the same time, it gives people a mindset to say, well, it's more than just a binary yes or no decision at the end impacting all of this. Let's start looking at what we are doing at a longer timeline. So…
Are there limits to process mining?
Adam: Do you feel like you could process mine anything then? Like, just....
Vishal: It can be a kind of dystopian thought to think about that, but I don't think that can be done because we are so fragmented. It's mostly suited for big organizations. We have a centralized systems which we can tap into, because although we talk about mining the data, we don't log our activities every day 24/7 into some kind of a spreadsheet.
Adam: No, you could probably barely get most people to do that. I wouldn't do it.
Vishal: I wouldn't do it. But to get the concept across, you know?
Adam: Yeah, so really, the personal analogy is actually, even though it might make it easy to understand, it's probably very flawed because most people would probably struggle with that honest with themselves for a very long time. And I include myself in that. I'm not coming from…
Vishal: Yeah, I mean, if that does happen then it's kind of a Black Mirror episode. You don't want this.
Adam: You don't want to live in it where every interaction you have is rated by the person you’re interacting with.
Vishal: But definitely there are some optimizations that — I mean, there's a degree of granularity, and what you want to know and how much you want to know, even within your own data logging. I mean, this obviously brings into this privacy issues and all of that. Cybersecurity is a big issue with the companies, and that's where data privacy is also a big issue, because you can't just mine data we want. Because if it coming from a vendor, then you'll take their permission, you write contracts to make sure that you can use that data. So, there is a whole sort of a regulatory framework that also is sort of evolving with this area.
Are there any ethical or legal concerns with process mining?
Adam: Right, because people aren't just mining these kinds of internal processes, too.
Vishal: They're finding interactions…
Adam: …that we're having externally.
Vishal: …externally, yeah.
Adam: What are you mining that's ethical at that point?
Vishal: Right, yeah.
Adam: Yeah, that gets really wild, too, because — again, for those listening — sometimes digital intellectual property, digital processes, what's shared online, the rules around those aren't as solid as things were 20 or 30 years ago.
Vishal: Exactly.
Adam: Especially in the US where we don't always function to update laws very well to match technology at a rate.
Vishal: It's kind of after the fact.
Adam: Right? It's one of those things where — I mean, listen, we've all saw Congress try to interview the owners of tech companies. They don't usually come off looking too savvy.
Vishal: Or look like an alien.
Adam: Yeah, right. And I say that, I'm being a little bit cynical saying it, but it also, that's part of the reason that there's probably gray areas to how these things are regulated when it comes to what constitutes intellectual property, right?
Vishal: So, from an analytics standpoint, the technique is matured a lot, but from, I think, as you go along, there's going to be questions of how to use the data. Because before you can use the data, it should belong to you, you should have agreements of those type of things. And there have been some use cases and success stories of this happening, especially in EU where they have a proper framework and process mining is applied to many things, and it has produced very good results so maybe we can learn from...
Adam: …what they're doing in Europe.
Vishal: Yeah.
Adam: The EU seems to have taken a little bit more of a stance on whether you own your own digital footprint to any degree or not. And in the US, it's way more up in the air. Am I correct on that?
Vishal: Right. Yeah. But as far as the devices are concerned, within the organization, daily work footprint, and that belongs to the organization. Right now, that's not being used as much as it can be used. And [that] can come to process mining. It can be used. And there's no questions of data ownership in that case because it's within the organization. And also, as we go and go to supply chain issues, emission issues, there are standards which will force people to share that information, because if you are procuring from Africa or Asia, if child labor is involved, or if the emissions are off the roof and you don't want that. So that is mandatory audits, but that data is usable because we are going to then have to be responsible for our reporting to [the] government and say, “Well, we didn't source from environmentally destructive supplier versus environmentally…” So, there is — it’s an evolving thing.
Adam: And just in those specialized areas, there's going to probably become more and more jobs available, too. So, to kind of bring it back into that educational sphere, I mean, it's really wild. It feels like it could be touching a little bit of everything at some point.
Review and wrap-up
Vishal: And you can get jobs in many areas, including these emerging interactions between nations and organizations and setting up the rules. It doesn't have to be technical process mining, but the process mining follows a standard, standards of how the data should look like. When the data goes from point A to point B, it should look like a certain way and there is an XCA standard that is a technical standard that people should follow. But beyond that, there is a lot of other issues like social and political issues that needs to be, kind of, juggled and morphed into something that can be usable. So, there's lots of room to grow, and then you don't have to be technical to... That's another thing that I would like to point out. You can be technical, it helps, but you don't have to be just technical to benefit from this set of techniques.
Adam: Yeah. This is not always math genius computer programmer. Because I know, honestly, before we started the conversation a little while ago, I probably would've been like, this is something I could never do. Because I don't do math. I only work in increments of 10. I still have 10 fingers. So yeah, we're good. That's somehow it. But I mean, it sounds like that depending on the organization that you're doing and their needs, this could be something that's pretty accessible to you because you don't necessarily have to go in and also be programming that SAP software to meet the needs of your organization. That's a different job. It could be both, but it's a different job.
Vishal: Yeah. And companies have made it accessible like Celonis and now even SAP. They've made tools which can make this accessible. You don't have to know how the algorithm works to use it. And the fact is you're using it to generate something [that] can give you better results. And having that mindset helps. And you don't require to be a math major to be having a process mining information included in your curriculum. You could be a marketing major, you could be a statistics major, you could be any major. You could still benefit from this development, as long as you understand some basics steps really.
Adam: Yeah. You heard it here, apply. It's for everybody.
Vishal: It's true.
Adam: Come on in, visit Vishal at the College of Business Administration at Central Michigan University where they have great sandwiches in the building. Right? It's true.
Vishal: That is true.
Adam: Well, Vishal, it's been great to talk to you.
Vishal: Thank you. Thank you.
Adam: And hopefully we can do this again on something in the future.
Vishal: I appreciate your time and all of this arrangement for having me here. Thank you.
Adam: No, we're glad that you came, and we hope we'll have you again soon.
Vishal: Thank you.
Adam: Awesome. Great to meet you, or meet you again.
Vishal: Meet you again, yes. Thank you.
Adam: Thanks for stopping by The Search Bar. Make sure that you like and subscribe so that you don't have to search for the next episode.