Cybersecurity & Interoperability

March 15, 2023

About the speakers

Dr. Sunil Sinha

Dr. Sunil Sinha's research, teaching, and consulting are in the areas of infrastructure management, sustainability, pattern recognition, sensor informatics, and resilience, especially water systems. His research includes the development of an integrated infrastructure management system with recent sensor technologies, life-cycle cost/benefit analysis, and condition/performance deterioration modeling. Critical civil infrastructure systems face a multitude of hazards that must be identified, assessed, communicated, and managed appropriately.

Zonetta English

Strategic Initiatives and Project Delivery Director
at
Louisville and Jefferson County Metropolitan Sewer District

Zonetta English oversees strategic initiatives and partnerships for innovation, implementing company-wide innovation through review and implementation of advanced technologies, best in class process improvements, development of key strategic partnerships, and collaboration with university and research experts on wastewater and storm water emerging trends and regulatory advancements. She works along-side peer directors within organization to support their operations while reporting to the Chief Innovation Officer, and pursues special projects and initiatives championed by the Executive Director.

Michael Thompson

Critical Infrastructure & ICS Cyber Outcome Lead
at
MITRE Corporation

Michael Thompas is a curious optimist with a wealth of experience and extensive background in engineering and operations, spanning more than 20 years across several industries—Oil and Gas, Nuclear, Chemical, Biotech-Pharma, and Water Treatment. In addition to an interesting blend of engineering, process control, Process Safety, and cybersecurity experience, he possesses a strong foundational understanding of the Recognized and Generally Accepted Good Engineering Practices (RAGAGEP) needed in the application of Functional Safety Engineering and have interpreted and applied the aforementioned to deliver millions in savings within the past five years.

Dr. Zdenko (Cello) Vitasovic

Project Team Principal
at
WEF WISE Program

Zdenko Vitasovic has more than 30 years of experience in water and wastewater industry. He has managed many projects and teams in the US, Asia, and Europe, and is a recognized expert in mathematical modeling, decision support systems, real time control, automation, and application of technology to enhance process control.

Elkin Hernandez

Director Maintenance Services
at
DC Water

Mr. Elkin Hernandez currently serves as DC Water Director of Maintenance Services, in this role, he oversees the maintenance of the over 40,000 assets used at Blue Plains AWTP.  He has over 20 years of experience in the design, construction, commissioning and maintenance of water treatment and power utilities. For the past 9 years, he has worked at DC Water Blue Plains AWTP. Prior to joining DC Water, he worked in the consulting and construction engineering fields as a design, development and commissioning engineer and manager.

Mr. Hernandez is the immediate past chair of WEF’s Intelligent Water Technology Committee, his background includes work on telecom cation, automation, process control and cybersecurity. Currently his work is focused in the fields of smart water, management, and maintenance optimization. Mr. Hernandez holds a Bachelor’s and a Master’s engineering degree in Electrical and Computer science.

[00:00:00] Max Herzog: Welcome, everyone. Thanks so much for taking the time to join this month's edition of the Water Data For. Water Data For is a virtual web panel series. Presented by Cleveland Water Alliance, Water Environment Federation, and the Midwest Big Data Innovation Hub. It's my great pleasure to welcome you today to, I think, the first session of the new year.

[00:00:42] My name is Max Herzog. I'm a program manager with Cleveland Water Alliance. And we at CWA at WEF and the Midwest Big Data Innovation Hub are really excited to have you here today. Water Data For is a series that these three organizations put on, and have been putting on now. This is entering our third year I belive. 

[00:01:00] And the purpose of this series is really to engage with and demystify a variety of topics in the sphere of water and data. So really excited to engage in our conversation today about cyber security and interoperability in the water space. A little bit of housekeeping before we dive in today.

[00:01:22] Please do feel free to submit questions throughout today's session. We're actually not using the webinar format today. So contrary to this slide, you will be able to enter your questions directly into the chat. Depending on how Elkin wants to facilitate things, we may also allow questions verbally ,but if you want to enter your question into the chat, during the panel discussion, you won't forget them.

[00:01:53] It'll also help us sort of curate the flow a little more smoothly throughout. So with that, it's my great pleasure to introduce today. Our facilitator for this session, Elkin Hernandez, who's a director of maintenance services at DC water as well as a member of the WEF intelligent water technology committee. So, now I'll hand it over to Elkin to introduce our panelists

[00:02:18] Elkin Hernandez: Thank you, Max and good afternoon. Good morning for some today. Thank you for being here. And well, before I introduce our panelists today, I'd like to make a small introduction of the topic.

[00:02:38] And make reference to the fact that the US water and wastewater sectors are currently undergoing a digital transformation. So for some more than others, but we're all going in that direction. So many sector stakeholders are utilizing data enabled capabilities to improve utility management, operations, and service delivery.

[00:02:58] This undergoing process adds the need for adoption of more automation, sensors, data collection, network interfaces, analytical software, and as one of the collaterals of this effort is that we are increasing the cyber risk and exposure of our systems to different threats. You have to consider that at the same time that we are improving our detailed capabilities, there's a state nation as attackers are becoming more aggressive.

[00:03:34] The criminals are more sophisticated. We have AI enabled hacking tools and cybercrime as a service that intensifies the competition between attackers and defenders in what can be described as a never ending arms race. So today we have a panel of SMEs experts from different backgrounds, utility academic backgrounds, and consulting sectors that will help us to understand how the industry sees the relation between interoperability, what the needs are, and how that can be create new vectors of exposure for cyber threats, and how the industry as a whole is trying to prepare itself to better handle the challenge that that represents.

[00:04:22] So at this point, I would say we're going to go just to understand the structure of the panel. We're going to have four very short presentations. And after that, we're going to open the panel for discussion with questions that we have prepared. But also, we want to encourage you guys as Max mentioned to send your questions to the chat.

[00:04:45] And we'll try to organize those questions to make sure we get the best out of this hour. So with that said i'm going to introduce my first panelist is Dr. Sunil Sinha. Dr. Sinha is a professor of civil and environmental engineering and is the director of the SWIM Virginia Tech. The Swim Center is the Sustainable Water Infrastructure Management Center of Excellence.

[00:05:12] Dr. Sinha is a National Science Foundation Career Award recipient in the area of Sustainable Water Infrastructure and Management System. He has performed extensive research in the areas of infrastructure, sustainability, pattern recognition, sensors, informatics, resiliency, especially with water systems.

[00:05:31] He has participated in over 4450 publications and referred in journals and reports. He's also featured as one of the participants of the PBS documentary titled Liquid Assets is the story for water infrastructure. And he's also been a president in different other sectors and NPR, the historic channel, and so on. He's truly an expert and a leader in the U. S. when it comes to water infrastructure. With that said, welcome Dr. Sinha. He's going to take over control with his presentation.

[00:06:14] Sunil Sinha: Great. Thanks, Elkin. Let me share my screen.

[00:06:23] Okay, so let me ask Elkin, can you see my screen? Yes, we can see you. Okay, perfect. So thanks Elkin again for a very good introduction. And I would like to thank the Water Environment Federation, also Cleveland Water Alliance and Midwest Big Data Hub for giving this opportunity. So again, my name is Sunil Sinha.

[00:06:46] I'm a professor and director here at the Swim Center, Virginia Tech. So I'm going to talk about this topic briefly before we go into the open panel session. So I have a few slides to share. So I will start with my first question: what's the grand challenge for society? So if you carefully look at the first revolution, it came 1700, where a steam engine and all those came.

[00:07:09] Then the second revolution was more mass production of electricity and other stuff. Then the third came is IT. Now we are in the fourth revolution. What does that mean? That means the fusion of the digital physical world. And that's why this topic is very important. So my background, I'm a civil engineer, but I did my PhD in computer science.

[00:07:29] So I still relate to the physical world very well. And my PhD was 20 years back. From University of Waterloo in Canada in artificial intelligence. So I will touch on that a little bit. Also, the ai, the family of ai and big data today in the context of cybersecurity and interoperability issues. So having said that, this is just one of the projects we are doing with the Water Research Foundation, mainly to look into as a system of systems, right?

[00:08:00] When we talk about a distal twin or collecting data. It's a complex system. It's not just one pump or pipe. If you carefully look at the whole system, it's a natural built on social. So that's why we call it big data, right? Because it's a lot of data and then the interaction, interdependencies and dependencies of this data play a big role.

[00:08:23] So this framework I'm just showing you for the context, because today what we are going to talk is digital. So if you look at the top one, we are developing an intelligent water framework. So why intelligent? Why is it not intelligent now? All right. And what makes it intelligent? People use these terms intelligent, smart, kind of, but what does it really mean?

[00:08:45] So what it means is, better decisions, right? Better service delivery. That's all it is. So what makes it smarter and what makes it better? So it's a technology people process and technology. So all the building blocks you are seeing. Most of them are related with data, which is what we are going to talk about today. So one block I will just talk about today for this context, not each and every block.

[00:09:10] But this report should be out by the end of this year or early next year. So these are the building blocks for intelligent water. Most of them are related with data. Cyber security and all those. So I will not use the word cyber security because that's what I'm going to talk about. So if you look at the smart electric grid or transportation, they talk about distal resilience.

[00:09:33] So what really is distal resilience? It's the same as physical resilience, right? When we talk about how we can deliver the service, when hurricanes and all those come, or even the kind of a disaster caused by human. So delivering the service is the key. So that's kind of a. My discussion will be in the digital resilience framework.

[00:09:55] So if you carefully look at the bottom one, that's where the distal technologies, then the distal workforce, because we are not talking about a fully autonomous system. At least in my lifetime, I feel that the water sector will be a human in the loop kind of a concept, right? Even if we bring AI and all those, some we can automate.

[00:10:15] But most of them will be human in the loop, will be there. So humans are also important. And then comes the whole cyber, which includes cyber infrastructure. But that's the bottom layer. But look at the middle layer. That is also important. The level two, the data and the whole interoperability and all that data can be biased if QA QC is not done.

[00:10:37] So that will also affect the service delivery. So all the bottom part is kind of a distal resilience, which brings to the whole water sector resilience or intelligent water, whatever term we use. There are many frameworks you have seen in the literature from AWWA from many organizations including NIST, Department of Homeland Security.

[00:11:00] So we have also looked into all, but they all talk kind of the same threat assessment and all those and other speakers are going to talk in the panel. So I'm not going to spend a lot of my time in the cyber resilience point of view, but I will focus more on the data side and give you some example, what challenges we faced by doing some project.

[00:11:20] So what is really data? So data, big data, we say it's not just volume. Big data is defined by five Vs. Velocity, volume, veracity, variety, and all those. And also data fair, which is a big research going on from National Science Foundation, not just in the water sector, but in medical and other. Fair. You may have heard this term, data fair, FAIR findable, accessible, interoperable, and reusable.

[00:11:48] Right. So otherwise you cannot convert data, the pyramid you are seeing at the bottom, data to information, knowledge to wisdom, that will not come if your data is not interoperable or QA, QC and all those are not done in the right way. So other sectors I feel are advanced. I worked in the transportation and smart electric grid.

[00:12:09] I talked to NIST all the time. I feel they have developed better. Water is behind. And I will give you a real example why I say that. So ours is very siloed, even within the utility we worked with in one of the project, which I'm going to talk about, we face that GIS, most of the utility said Sunil we have a GIS database.

[00:12:32] Okay, so we said we will take all this GIS and create a national database. The challenge came and that's what I'm talking about, mainly the interoperability for the PipeID project. So most of my talk is, and then we can open it for discussion, is mainly related to data interoperability.

[00:12:50] There are frameworks out there. This is just one I cut and pasted. But there are many frameworks out there from Australia, from Europe, and all those. In some way, I feel Australians and Europeans there are kind of advanced than us in developing some kind of a standards or interoperability model. So I feel that the water sector in the US also needs that.

[00:13:14] And why? Because of the pipe ID project. Some of you may have heard about this project after Flint Congress decided to have it. So this was a congressional allocation to develop a national database of water pipes. When we started this project way back in 2016, the biggest challenge came. This is not based on a survey.

[00:13:35] This is data. So we have to collect all the data from the utilities all across the U. S. because that's what Congress wanted to see the statistical confidence and all those. So GIS data, we tried to get GIS data. Most of the utility. They gave us GIS, not all, some smaller utility gave us CAD files and all the, so we had to digitize that.

[00:13:57] So even GIS data units were not the same, coding was not the same. So now you think about analysis at the scale. If you have to do so, this is a very heavy slide, but the right hand side is the architecture of this database, and the left hand side is the sources. So data comes in many shapes and sizes, even from the utilities somewhere.

[00:14:17] Excel spreadsheets, somewhere GIS database, some was coming from their SCADA, some from proprietary software. Now we had to go and buy different softwares to read those files. And that was, some was even written in Fortran 4. I don't know if you guys know Fortran. I know because in India, when I started my undergrad, the Fortran 4 language.

[00:14:38] So that is the real challenge. And especially the challenge is when you want to do analysis at a scale. Utilities are political boundaries. So I will give you one example, Portland. Portland is way advanced in pipe analysis and all those, but Just Portland data was not enough. Why? Because when you stratify these data like a cast iron pipe, a cast iron pipe is not just one data set.

[00:15:04] If you divide cast iron into line on line, different corrosive environment, pre 1970s, your data set reduces. And then you are not going to get a statistical confidence of, 80% or 90%. So that's why it is so important to have. And now we are talking about AI application. We can't use AI and all those if data interoperability standards and all those are not there.

[00:15:28] Otherwise, orange apple you are mixing, right? And it's a lot of manual work. So what I'm showing you in this slide is. that really to develop at the scale of U. S., you want to compare the data with one utility, Portland, because no utility will have all the data they need. All right. And so you need to bring in the data set from other utilities, of course, with the same features and all those, but you can do it if there is no standard.

[00:15:52] So that's kind of my overall. Is like why we need in the water sector both and my term is more digital resilience where we have to look into all from the technology software side and then analytic side. So I will stop here. I'll come back to you.

[00:16:12] Elkin Hernandez: Thank you, Sunil. Thanks for the presentation. And then we'll get back to you during the panel.

[00:16:18] When the panel opens. So let me quickly introduce our next panelist. This is Zonetta English. She's the Strategic Initiatives and Project Delivery Director for the Louisville and Jefferson County Metropolitan Sewer District. Ms. English has 30 years in the wastewater industry and various capacities.

[00:16:39] Lab Management and Certification, Operations Support, Research and Capital Project Management. She was selected as one of the 20 professionals among five disciplines. Chemists, Wastewater Industry Experts, State Regulators, and Environmentalists. Was selected by the U. S. EPA Office of Water to serve on the Federal Advisory Committee for Detection and Quantitation to propose a new method for a new method of detection limit.

[00:17:09] She's currently leading the research for new processing and advanced technologies that could enhance any aspect of the operation for MSD, and she works under the direct, as per assignments from the executive leadership of the water utility. That's among others you know, that kind of  description and covers a lot of interactions and initiatives with different organizations across the public and private sector.

[00:17:46] Miss Zonetta will go on with her presentation starting now. So we say thank you, Zonetta, for joining us. Thank you for your time. And I'll turn it to you. Thanks.

[00:18:00] Zonetta English: Thank you, Mr Hernandez. I want to try to share my screen. Let's see, here. Is it sharing?

[00:18:17]  Elkin Hernandez: Not yet. It's working now. 

[00:18:25] Zonetta English: Okay, all right. As he mentioned, I just want to thank our host WF and all of our esteemed panelists. I've really feel privileged to have the opportunity to participate. As he mentioned, I am the strategic initiatives project delivery director for MSD and I also am also the chair for utility management committee for Weft and this is one of the topics that came up in our leadership roundtable at Weftec and so my presentation is basically bringing it down to the part at the project.

[00:18:57] Management level, because this is what we're supposed to do in terms of some of my responsibilities. And I just wanna try to give you an overview about how we get there. And, and so just to kind of give you just a, a couple of sites about what we do at MSD and how we get to the point of where this comes into play about trying to to implement new technologies as well as dealing with the concerns for cybersecurity.

[00:19:26] So, we just want to just kind of just give you some overview about our core services. We provide wastewater stormwater flood water flood protection. So, our mission is to provide quality wastewater stormwater flood protection. And our goal is, off course, with everybody to protect public health and safety through sustainable solutions.

[00:19:45] And the key thing that leads us today is our fiscal stewardship and strategic partnerships. Louisville, MSD is located in Louisville, Kentucky, and this is just as a map of our service area, and so we have two treatment plans to outfall to the Ohio River, as well as we have some at that outfall to local streams.

[00:20:08] And that'll be important when you talk about some of the decisions that we've made. Going forward. Here's some of the statistics to kind of let you know how busy we are. We have a service area of 376 square miles. We have stormwater drainage. We have 73, 000 catch bases and 675 detention basins.

[00:20:30] Flood protection. We protect more than 200,000 people, 137,000 homes, and the most important is $34 billion in property. We have 27.1 miles of flood wall and levy and 16 pump stations. We have a dedicated staff of 700 plus employees. In the area that I work with in terms of capital projects is wastewater treatment and we roughly treat 156M gallons of water, wastewater per day. Part of the other initiatives that we have going is that we've also extended our service due to regionalization. So, with the amount of staff, we just recently acquired 17 other treatment facilities outside of our county. So it's very, very, very important for us to be able to use technology which involves software in order for us to maintain the services that we have and to expand.

[00:21:32] So, our current model is that MSD and to our ratepayers and stakeholders that we're actually 3 utilities in 1. We also expect over the next 5 years approximately to do about a half a billion dollars in construction.

[00:21:52] So one of the things that we have done in terms of our mission is talk about 1 of our critical success factors and that is to realize operational efficiencies and revenue generation through strategic partnerships and innovation and one of that innovation at work has been artificial intelligence.

[00:22:09] Our executive director, Mr Tony Parrot, Who is very forward thinking, introduced this concept to us. And so we began a pilot project back in 2020. And it's currently in progress at 2 regional facilities one that falls to the Ohio River, which is Derek Guthrie, which is a 60 by design of 60 plant that has average flow about 32 MGD per day.

[00:22:36] And then also, which we and that one has been going for a year. And then I have another project in place at Cedar Creek, which is a smaller plant by designers, about a seven and a half MTD plant, and about an average flow of five. Our largest wastewater treatment plant is forming, which we will leave for last which treats about 120 MGD, which is the largest treatment facility in the state of Kentucky.

[00:22:59] And we have a wet weather flow, about 350 MGD. The overall goal for us to implement artificial intelligence at MSD is that we are wanting to utilize this as a training tool for our staff. I think we're facing all the same challenges. Other utilities are in terms of attrition in terms of retirement.

[00:23:22] We're also looking for cost efficiency. So looking at predictive maintenance for pumps and motors. This is what this allows us to do and also operational costs in terms of monitoring our energy and chemical uses. We're also utilizing a to optimize our operational efficiencies. And we're also, which has been very, very helpful is using this data to guide capital decisions.

[00:23:47] So, often our operational staff do this every day. That we may need some new equipment. We also are all of us are faced with those challenges about how we're going to do our capital will spend. But what this has allowed us to do is to verify our capital needs.

[00:24:05] So, how do we minimize our cyber security risk? One thing that we have done is we have a dedicated team that are core project team members. And as you know, that's a very, very challenge, with everybody who already has a full day with all of the current workload, but to pull those that core team we have a vetted agreement that involves our legal team and then our, we also have an information security agreement that our chief IT officer has put into place. It's very, very vigorous that we sign with all of our people who provide us software and applications. We have firewall restrictions and monitoring and then we also have very critical input protection and then we also on a regular annual basis have ongoing security, cyber security training and we also have efficient exercises that are regular. 

[00:25:07] So that we unexpectedly have opportunities to test our staff and grade KPIs about how well we're doing as a staff. So we are constantly doing this. One thing I just wanted to share with you is this is our current process mapping for the project that we have right now, in terms of implementing, this is very, very important because as you all probably all occur is a struggle with maintaining our staff.

[00:25:35] So, we wanted to have a template that we know, regardless of who's involved and how it's going to happen. I got permission from Mr. Bagley to share this with you. So this could kind of tell you that we have two firewalls set up because that's the most concern is how far you're gonna let the systems that you have, get into your actual network.

[00:25:59] And so there's something that may be here that we limit we have to where we limit what our technology we have is an aqua site about going to our historian. So we do limit that and there's some other activities to go in between off to your left of your screen between the firewall and go in there in terms of our lab data.

[00:26:21] So we uploaded that through CVS files. So, my last slide is just really wanting to just talk to you about a lot of lessons learned. One thing I would say, if any technology that you're implementing does make sure you get your IT team involved at the project concept.

[00:26:42]  that's very important, for them to basically understand what you're trying to do and for them to simulate the proper team. Make sure you include all your stakeholders. One thing we found out that we didn't even realize would be affected by the projects that we were implementing and the systems that we need to allocate contingency funds for unexpected project costs.

[00:27:04] One thing that we had to do is increase our number of skater tags. And so those weren't typically funded. We were able to come up with those, but those are the things that you want to do as far as your project management and mapping that out. Also develop a vigorous training program and user guide to optimize.

[00:27:21] That's one thing that we are very proud of. We've also allowed that. This is part of the training that allowed us to give continued education units to our operators and so that's going to be very important because you have this learning curve and you don't want to implement this very expensive.

[00:27:37] Yes. Technology without having ongoing support and to make sure that the people that's intended to use it have the opportunity to use it and make it user friendly. And then also answer this question. Do you have the appropriate devices to deliver the technology and make sure that you have those as soon as possible?

[00:27:57] So, that's kind of just what I want to just give you at that project management level. Wanted to share that with you today and I'll be standby to answer any questions on the panel and I'll turn it back over to Mr. Hernandez and again, thanks everyone for allowing me the opportunity to participate today. Thank you.

[00:28:18] Elkin Hernandez: Thank you, Miss English. Thank you for bringing the utility perspective to the discussion. It's highly appreciated. I move quickly to introduce our next presenter, Michael Thompson. Michael is the critical infrastructure and ICS Cyber Alchemy Lead at the MITRE Corporation, ICS is for whoever doesn't know it stands for Industrial Control System.

[00:28:42] Michael joined MITRE in 2021, after spending more than 25 years in the process control, functional safety engineering, and industrial control systems cybersecurity fields. Michael has deep knowledge and experience in the chemical, nuclear, water, and oil and gas industries. He holds a bachelor's degree in Information Systems Science from the University of Maryland. And a master's degree in systems engineering from Johns Hopkins University. Welcome, Michael. 

[00:29:11] Michael Thompson: Great. Thank you very much. Glad to be here. I'm hoping everybody can see my NCCoE slide, right? With the water plant in front there. Good to go. All right. Yes. Perfect. So I spend the bulk of my day working with the National Cybersecurity Center of Excellence, the NCCoE.

[00:29:32] It's a function of NIST and MITRE together, you know, working as a partnership. And when I heard about this opportunity to speak, I wanted to come on here and talk about one of the things we're doing in the water industry right now. We've got a community of interest standing up and we are always looking forward to looking to the industry to help provide us guidance with some of the work that we're doing.

[00:29:56] And so this right here is the latest community of interest around securing water and wastewater utilities. And so I'll make this presentation, of course, available to anybody that wants it, but we would absolutely love for anyone in the organizations represented on this call to come and participate in this community.

[00:30:15] So right now, we understand water and wastewater. It's an extremely important part of our civilization. And so I know I'm preaching to the choir here, but this slide might give you a little bit more of an understanding about how water ties into every other critical infrastructure dependency sector out there.

[00:30:35] And so one of my other functions is I'm the critical infrastructure principal engineer in charge of industrial control systems, cybersecurity. And so I'm doing a job right now where I'm looking at the various critical functions. So the government has identified 55 national critical functions, and those 55 national critical functions are tied to these 16 critical infrastructure sectors, and so I put a little red line around the water sector so you can see how important it is.

[00:31:07] Pretty much every other sector going down, chemical sector, commercial facilities, communications, all the way down. They all have some dependency on the water sector. And so we think that that makes the water sector incredibly important for the way we live life in our country. And so because of that, we find this to be an incredibly important industry to safeguard.

[00:31:30] And so what we're doing currently at this time is we're standing up real world applicable security guidance around securing water and wastewater utilities. So, yeah, the project right here is securing water and wastewater utilities.

[00:31:57] And so the idea is we're going to profile several areas that will strengthen the cyber security posture within the operational environment of the water and wastewater facilities. And so we're looking to focus primarily on the asset management, the data integrity component, remote access and network segmentation.

[00:32:16] Many of the organizations that are on this call today have already contributed in some way to the initial draft of this project description, and so we'll be rolling this out here shortly. By rolling it out, I mean we're going to be starting on the actual project work to create this docent. And so again, we're looking to make this into a real world way to apply cyber security controls to the water and wastewater industry.

[00:32:43] And so we're again looking for anyone that wants to participate. We welcome you. The idea here is to make this something that organizations can use and not just a theoretical research docent. We don't want to create another piece of shelfware where you're like, yeah, this is great and put it on the shelf and never look at it.

[00:33:02] We want this to be something that an organization, both large and medi, even small, rural water treatment facilities could look to to leverage the information within. And so, you know, when we hear a lot about some of the challenges faced and one of the things we've seen a lot is oftentimes you see a misalignment of business needs and cyber security requirements.

[00:33:30] And I'm sure we're going to probably talk about something along those lines in our Q&A session. But documents like these and documents like the NIST cybersecurity profile are great opportunities to help align those expectations between the business and cybersecurity requirements. And so if anyone's interested, by all means, please check out the NIST profiles that are already out there.

[00:33:54] There's a couple of good ones that are probably, could be leveraged to, for the water and wastewater treatment industries. We could always customize it to be more reflective of that. But if anyone's familiar with the NIST cybersecurity framework, identify, detect, protect, respond, and recover these are all applicable areas that can be leveraged to to support the water industry.

[00:34:19] And so with that, I'm always short on slides and long on words. So I don't want to bore you guys with the whole long slide deck presentation here. If you have any interest in what we're doing in the NCCOE, please reach out to this email. We would love to have you, and we're looking forward to working with you and with that, I'll turn it over.

[00:34:38] Elkin Hernandez: Thank you, Michael. Thanks for the presentation and we'll get back to you after the next presenter. So let me introduce our next panelist, Mr. Cello. Vitasovic. Cello is a known SME in the industry he has for his focus has been on mathematical mod modeling, operational technology, and real time controls and information systems.

[00:35:09] He has executed a number of projects for water sector utilities in North America, UK, China, Hong Kong, Singapore, and Australia. We can describe him as a principal investigator. For a worth 88 06 project. A co principal investigator for project 50 39 smart utilities and intelligent water systems.

[00:35:34] He's been past chair of the intelligent water system committee and wealth. He has been recipient of the web, Harrison Prescott, Eddie metal metal of our standard research, and he has been the project team lead for the web twice program.  With that says welcome Cello and thank you for being here. Turning to you.

[00:35:57] Cello Vitasovic: Thank you very much. Welcome. So both cyber security vulnerability and lack of system integration or interoperability are significant challenges for most organizations. I will offer a systems thinking view of these issues. The title of today's for conveys a conventional thought that interoperability may be impeded by and lacking because of the concerns about cyber viral vulnerability with adverse impact on different aspects of utility management and performance that benefit from system integration.

[00:36:34] So if our primary or sole focus is on defending our technology systems, concerns about cyber security can result in data being locked up in different systems inaccessible and remain in solitary confinement. So for interoperability, I'll use the word integration from now on because it's easier to pronounce and system integration can improve the capabilities and performance of organizations.

[00:37:11] However, the significant investments their organizations have made in technology have often failed to produce full value because data in different systems is not easily accessible. Or shared. There is a tendency to view both cyber security and system interoperation as technology issues. I will propose that people, organizations and processes are integral parts of both the challenge and the solution here.

[00:37:44] Organizations create value when members of the workforce interact within an organization to execute processes that may be enabled by technology. For quite a long time, even before cyber security emerged a major issue. Interoperability and system integration have been broadly recognized to be both beneficial and lacking.

[00:38:06] And this has been viewed traditionally as a primarily technology issue. So, let's look at an example during a 4 day water research foundation workshop that was attended by roughly 40 utility representatives. Participants examined possible improvements in these 4 areas. of core business and identified key challenges that they may be facing.

[00:38:37] They mapped each challenge to a three by three matrix, describing whether the challenge was related to people, processes, or technology. And if it was on the strategic tactical or operational level across these four business areas, utilities identified that only 11 percent of their challenges were related to technology.

[00:39:02] 46 percent of them were related to people. 43 percent to processes and the majority of the challenges across these 4 areas were on a strategic level. This was self reported by utilities. This underscored the need for an approach that would consider all 3 of these key elements. This approach starts by determining how well we're doing the maturity of an organization's current capabilities to manage people, processes and technology.

[00:39:30] Our approach includes different models for levels of maturity. The maturity model for processes defines five levels. On the initial level, the knowledge about the processes and people's heads. On the managed level knowledge is shared among team members.  On the standardized level processes have been well defined and documented. On a predictable level, processes are managed quantitatively based on measurements and feedback and metrics and KPIs. And on an innovating level organizations practice continuous improvement. About technology, the technology maturity model examines are we collecting the data that we need to successfully execute a business process?

[00:40:15] Is data managed effectively? Do individual systems have adequate analytical capabilities? And if we can access the data across different systems and perform enterprise level analytics and system integration and interoperability are the key required for the enterprise analytics.

[00:40:40] So The maturity model and tool for organizational culture and workforce was developed over more than a year of engagement and collaboration of 18 utilities, and it helps us assess the maturity in the people's aspects of value creation, and the different components of that model are shown on the slide.

[00:41:06] Our suite also includes a maturity model and a tool to assess an organization's readiness to implement the changes that are required to achieve improvement. So we can assess this maturity based on these categories. So what do all these maturity models tell us? Let's say that you are asked to develop a decision support system or automation for a specific process.

[00:41:35] It might be useful for you to know that the process is not well documented. That is complex and the impact of failure is significant. That the data required to perform the business process is not collected or generated in claim that the employment engagement is low and that the teamwork and collaboration among different business units is lacking.

[00:41:56] You might want to know these things. So, in summary, the barriers to system integration and cyber security are determined by maturity of our technology, maturity of our processes, workforce and organizational culture and our capability to implement changes and focusing on a simple aspect that creates risks.

[00:42:21] So, methods based on systems thinking, allow us to consider all key aspects of utility management and to think holistically, like the Buddha said to the hot dog vendor, make me one with everything. So thank you very much for the opportunity to share my thoughts.  This is a list of references that I provide the background for my presentation, and I will also put them into the chat line for our zoom meeting. And thank you again very much.

[00:42:57] Elkin Hernandez: Thank you, for bringing the perspective of your project and reminding us that there is a human element that we cannot underestimate. So the next we're gonna move into a panel mode Q&A session. If anybody has a question, please to feel free to send your questions through the chat.

[00:43:18] And other than that, I'm going to start putting questions out to the panel and let them answer it and provide their opinions to my questions. So the first one is please if you can define what the intersection of cybersecurity and interoperability mean for you, please share your thoughts with us Michael perhaps you have something to share with us in that regard.

[00:43:44] Michael Thompson: Yeah, no, this is great. In fact, it's very much tied into what the gentleman was just talking about on that spot right there. And you know, I see so much value in it when you look through the proper lens in this regard. So you know, I'd say interoperability is a kind of a crucial element when it comes to cybersecurity.

[00:44:00] Poor interoperability, to me, translates to gaps. You know, gaps translate the ball or vulnerabilities. Sso it's less of one versus the other. That's kind of what I'm getting at. It's less of a, you know, interoperability versus cyber security. They should be looked at as more complimentary, and one could even argue that an increase in operability could lead to even better cyber security. We could use data historians, for example, to help better detect anomalous behavior. So that's kind of how I look at it when I, when we're talking about the intersection of cyber security and interoperability. That's all I got.

[00:44:48] Cello Vitasovic: Elkin I think you may be on mute.

[00:44:53] Elkin Hernandez: Thank you. Thank you Cello. So I was going to say if anybody has any thoughts on that question that I wanted to share with us on what is their point of view or perception on that intersection between the two fields.

[00:45:09] Sunil Sinha: Yeah, I will just add, I agree with. I'm getting some feedback. Yeah, no, I agree with Michael that interoperability shouldn't be taken as a kind of a hurdle or versus cybersecurity interoperability, a good interoperability, what exactly I mean, we are talking interoperability, especially I mean, interoperability also has many layers.

[00:45:34] But if we are talking about the whole data and all those in the water sector, so many vendors are there. We don't have standards. Each vendor comes with their own software and now you need to talk to each other. I know Houston is trying to develop in-house data interoperability mainly because they work with like 10, 15 different vendors for CCTV pipe inspection for the pump, so I feel that having a good interoperability standard will definitely enhance cybersecurity.

[00:46:11] Elkin Hernandez: Thank you, Sunil. We have limited time. We have only about 10 minutes left. So thinking about using the time as well as we can, what can we do to improve moving down the road? What, what are the things that you think needs to be considered to make sure we are on a path of improvement, in regard to having supporting that interoperability that the digital transformation requires? How to do it in a safe manner.

[00:46:47] Cello Vitasovic:  I have something to offer. What I was trying to communicate is that although technology is a big part, when we talk about things, I think my focus has been on utility management and data management is a part of it. I walked, you know, about a couple of years ago, I walked into a process control center and there was a sticky note at one of the monitors that said, well, this is the account and the password that you use to log into the system. So I don't care who the vendor is. I don't care who configured the system, but you got to get rid of these sticky notes on our project that we're doing right now. One of the initiatives that Great Lakes is doing is managing gas cylinders.

[00:47:42] Let's start from the basics. We're in the business of moving water, cleaning it and stuff like that. So let's start with the basics and, you know, making sure that we're focused on the business.

[00:48:00] Michael Thompson: You know, Iheard earlier and I thought that was a great response, but I heard earlier the mention of a good applied systems engineering approach.

[00:48:06] And this holds true in so many fashions and so many facets of it. Right. I always tell people, Why do systems fail? And there was a big study done of 800 or 1000 different processes by a man named Trevor Kletz. He's the father of functional safety engineering, and he found out that about 65 percent of all failures can be attributed to improper specification and design.

[00:48:30] And so leveraging a good applied systems engineering approach will clearly define those requirements that need to be satisfied in order for the system to operate. And if you apply a good systems engineering approach, you'll have good interfaces between the different systems leading to interoperability.

[00:48:47] And so when I heard that earlier, my ears perked up and I was like, wow, this person gets it, you know, leveraging that systems engineering mindset and applying that to the system. A lot of people will say, Hey, it's too costly or it takes too much time and I would counter, it's if you don't do it, it'll cost you even more in the long run. So that's kind of my thoughts on what could make it better going forward.

[00:49:10] Sunil Sinha: Yeah. So Michael, I will add on your systems approach. The one project we are doing at the Water Research Foundation, a sewer set system. The challenge is How you define a sewer surge system in the water is very problematic because these utility operate on the political boundary, right?

[00:49:30] And that boundary is not really your system boundary. And how you define dependencies, interdependencies, because the river doesn't stop, starts from Houston boundary and ends at the Houston boundary. So that's, I feel, is the biggest challenge. in the water sector and I, I saw one question was there that how do you develop the standard interoperability.

[00:49:54] I feel each utility cannot develop a standard and being an academic institution Virginia Tech cannot develop a standard. So that's where this professional associations need to come in. Somebody has to take a lead in the water sector to develop some standards.

[00:50:12] Cello Vitasovic:  I would like to bring in the voice of utilities. We, when we met on our initiative, and we asked utilities, what are the most likely causes of problems or failure and ranking? They said that it was workforce and organizational culture and acquisition implementation technology was 3rd. When we asked them, when we asked them about what actually would help most, it's a common methodology, not improved technology, but a common methodology.

[00:50:54] And I think that maybe the previous Speaker Max was kind of alluding to that. And system thinking is all about interactions. It's not about, you know, the queen and the different pieces of the chess sport. It's about playing chess.

[00:51:21] Elkin Hernandez: Thank you Cello. That's certainly the question that was the first question that was posted in the, in the chat is certainly a very profound one. And, and certainly one of the, the challenge, the bigger challenge that, that we have in the industry I guess goes as Sunil was saying. The challenge comes along with the fact that we have political boundaries and that's the nature of the industry.

[00:51:46]  Another question in the chat is more on the technical side, maybe for Michael, why does our industry not expand the use of biometrics? Any thoughts there?

[00:52:01] Michael Thompson: Well, I mean, yeah, biometrics is great for like you know, identification and everything, right? I mean, I don't think we should limit it to just that component.

[00:52:13] Any good cybersecurity program should consist of layers of protection or defense and depth. And so biometrics is just one, one means to identify a person, right? So, I mean, yeah, it can definitely be a technology that can be further embraced, but I would never like to put blinders on and just kind of focus on one thing again, using a good systems engineering approach, we should be looking at using the technology that best suits the requirement that has been, you know, decided upon about it by the users.

[00:52:43] So, you know, it's a matter of making sure that you know, we're using the right tool for the job. Essentially, biometrics could absolutely be that right tool in certain conditions. So it's all that matters in a, you know, typical fashion. It depends.

[00:53:01] Zdenko Vitasovic: I'd like to address the question about developing interoperability standards. We gotta get rid of this word. It's too tough for me. How many of you have seen consultants presentations that have a vendor that looks like this? There's a bubble in the middle with their name, and there are bubbles all around. With other systems and arrows going into the central bubble. How many times have you seen that?

[00:53:40] Elkin Hernandez: I have a follow up. We have a follow up question regarding that question. Shall I say, is anyone currently specifically taking on the challenge of developing their ability and their probability standards? I mean, we, and the question has a full out saying, yeah, it seems everybody knows that they're needed and I agree.

[00:53:58] We all know that is needed. I don't know that there is an effort. That comes over my head. I know there's been several going on.

[00:54:05] Sunil Sinha: Yeah, one I will tell you for the pipe, we are partnering with both AWWA and ASCE because we as an academic institution cannot create a standard. So we are partnering for but this is just specific to pipelines.

[00:54:19] Cello Vitasovic: I want to say one thing. Clemenceau said that war is too important to be left to the generals. Standards are too important to be left to the vendors. 

[00:54:32] Michael Thompson: I would definitely you know, exclamation point behind what he just said. You definitely, that's like letting the fox run the chicken house.

[00:54:39] You know, if you let the vendors write the standards, it should, when I work heavily in the IEC 62443 standard development community, and we always limit the amount of influence the vendors themselves have, right? We want more end users. We want the end users to help us define what the standard should be.

[00:54:58] And then the vendors should adapt that. So that's an approach I think has worked for us for years in the ISA and the IEC 62443 committee space. But yeah, I completely agree with that. Yeah, let's let the end users develop the standards. , I was going to say that NIST often have a lot of.

[00:55:22] Zonetta English:  No, that's what I was going to say for our information security agreement. That's the baseline that we view this NIST because they have a standard for cyber security. So our agreements are all based on that standard.

[00:55:37] Michael Thompson: And the NIST docents, I always like to say, it's like if you look at the NIST cybersecurity framework, and to use a reference here from our childhood, like, it's like the Voltron robot, the Black Lion.

[00:55:49] So that the NIST docent is that center thing that all the other things are attached to, right? And so that's a good jumping off point. Again, I think the use of the trades organizations would also be very beneficial in.

[00:56:02] Sunil Sinha: But I will add, Michael, that the NIST cannot create a standard, right.  It's still a NIST recommendation, but the problem with the water sector is someone has to take the leadership role. And try to make that a standard workable standard.

[00:56:19] Elkin Hernandez: Well, thank you. We're coming to the end of the hour. I think it's very interesting discussion and we'll have to figure out a way to follow it up.

[00:56:28] But very interesting. I wanna thank the panelists for giving us their time and sharing their expertise with us today. I wanna thank every attendees and especially the ones that share with us questions. And I want to invite you guys to continue attending these series of webinars.

[00:56:51] And remember this is the Water Data Forum. And the next session is coming on Wastewater Surveillance for Public Health on June 16th, 2023, we invite you to join us and continue to be part of these knowledge sharing sessions. Thanks to everybody and hope you guys have a wonderful rest of the day.