SCADA Data and Water/Wastewater Compliance – Risks and Rewards
Why dealing with previous day, current day, and aligning operational, SCADA, and compliance lab data correctly can make your head hurt, but it is incredibly important.
Background
Technology should save resources, reduce risk, or make us happy; otherwise, I would not recommend it. For the utility industry, technology helps in a variety of ways. For instance, Supervisory Control and Data Acquisition (SCADA) systems are often used to provide monitoring and control oversight for a complete facility with various operations. For compliance purposes, information is collected from the SCADA system and placed into downstream reports and spreadsheets, often transcribed by an operator in the control room swiveling the office chair from one screen to another. This process is less than efficient by today’s technology standards and can expose a utility to data discrepancies. For systems seeking to reduce compliance risk and improve their operational efficiency, they need to create what we call “Operational Data Alignment” or ODA for short.
ODA is often neglected or poorly implemented leading to ambiguity and time-consuming data mining. Proper implementation of ODA requires that the manager, or the “chief water data steward” understands how, where, and when data is created, stored, and used. This article seeks to convince you that ODA matters, then define, explain, and suggest ways that SCADA data can gain ODA with other data, resulting in reduced environmental risk and saved time. Regulatory agencies across North America do this differently from each other and we would all benefit if there were better standards, which we are optimistically suggesting with foresight in this article.
What does “Current Day” and “Previous Day” Represent?
Certain operational measurements are often aggregated to some time-based value, often daily in utility operations. This aggregation typically applies to simple usage information such as the volume of water measured by a flow meter, the mass of chemical used from the measurement of weight by a scale, or the time a pump was online measured by the time the pump was energized. With totalized data, sometimes we have the privilege of having a SCADA system totalize these items from midnight to midnight. Often times however, operators are making “rounds” during the daytime, recording “daily totals” that often reflect something like 8 AM one day to 8 AM the next day for flows and usages. When we look at yesterday from 8a to today at 8a, we are really looking at roughly 2/3 of the flow or usage from yesterday and roughly 1/3 from today (maybe it’s more usage in the morning and might be closer to 60/40). The calendar date on which the totalized value is applied defines whether the method is “current day” or “previous day”.
There is no standard that the USEPA enforces that we are aware of. Similarly, there is no “totally” right or wrong way to account for this type of reading, although being consistent in your operations and across different data streams (aligning data) becomes VERY IMPORTANT if you are using the data to make decisions. You don’t need to understand how all this data is aggregated, but you should understand how to define and align the data properly. Following are some ODA improvement concepts and questions for system owners, data (software & hardware) providers, systems integrators, analytical and other staff that are aggregating the data for reporting and decision making.
“Ideal” SCADA data
SCADA systems typically connect to a network of disparate unit operations. Each operation or set of operations is controlled by a localized control system, typically by a Programmable Logic Controller (PLC). Some operations may be controlled by a mass-engineered control system that allows for a data interface over a standardized protocol to the SCADA system.
Here’s how ideal Operational Data Alignment (ODA) should work with your SCADA data:
SCADA systems should ideally record daily totalized data from slightly (seconds) after local time midnight to slightly after midnight (24 hours), so that you can correctly record totals for chemical usage, flow, or min/max/averages according to a calendar day in the local time zone. PLCs ideally should be responsible for either accumulating pulses or integrating flow over that day. In a future article, we will address other aggregate (min/max/average) similar functions.
Totalized data for the previous day is ideally stored in a “PLC register” (called “Finished Flow Previous Day,” or similar, for example). Waterly recommends these registers are populated slightly after midnight so that their time stamps are stamped properly as the previous day. Some systems will totalize in their SCADA software or historian. This can be problematic when the computer is rebooted, patched, cannot see the instrument, or if its clock drifts. Best practice is to keep the totalization within the PLC, which is more of an appliance.
The SCADA server usually then receives the totalized or aggregated data from the PLC or PLCs every few seconds, so that number on the SCADA screen you see should match the PLC register for previous day. You now have what we usually informally call a “Totalized P-Day” value. This is the tag value you should then map into your favorite data management app to help you report on this and other similar P-Day data (e.g. lbs used, gallons consumed, hours ran, etc.).
Depending on your particular installation, your favorite reporting app will receive those Previous Day values about every few minutes (you could do it once per day, but Waterly likes to sync more often in case we miss one), but since the previous day totals don’t change but once per day, as long as your favorite reporting app receives the correct previous day’s totals after midnight but also before 11:59:59 PM, it will represent the date correct. If your Previous Day totals are sent with a timestamp before midnight the same day, the reporting software could interpret that “Previous Day” as the wrong day.
As a last note, when Waterly imports SCADA (and other) data, we pull in the Updated timestamp from the data source, which allows us to get data for yesterday, or even two days ago when everything comes back from a break in communication. This concept is sometimes called “store and forward” where a data source keeps track of its own timestamps, then forwards them onto a receiving application along with the date and timestamp of the original data. Knowing how and where your P-Day data is timestamped is critical to proper ODA.
How should we account for time zones?
For many investor-owned utilities and contract operations, we understand that sometimes data is collected right around midnight but reported in a different time zone. Waterly recommends pulling data from SCADA after midnight local to the data. If your local SCADA system is in Pacific Time and your daily totals change on or after midnight, then someone in the Eastern Time zone won’t see yesterday’s flow totals updated till 3a their time – which should be by design. If your system doesn’t pull yesterday’s flow totals until 1a local time, that’s fine too…it just means that if you run your operating reports BEFORE yesterday’s flow totals are imported, you might not have the most current data. Bottom line is that no matter what time zone you are in, wait until after the local device has shifted its P-Day values for you, so you can see yesterday’s numbers.
How should we report our daily values?
We see differences even within some of the largest investor-owned utilities on what best practices should be regarding reporting data to regulators. Some systems do rounds first thing and report the difference between today minus yesterday’s readings as yesterday’s flow and some report it as the current day; it’s just not consistent and regulations are usually silent as to what is right. With over 150 combined years of operations and engineering experience in-house, Watery recommends organizations favor calling data where the majority of the data was captured. So, if you are collecting 8a – 8a, the majority of that flow volume normally happens between 8a yesterday and 11:59 yesterday, so we would call that the previous day’s flow, which seems to conflict with many systems’ practices. We recommend thinking carefully about this as an organization, because sometimes SCADA will give you midnight to midnight, but your operators will give you “rounds” done in the morning, creating a mismatch. Outside of automating everything, which may not be economically feasible, you just have to decide intentionally how to do the math and be consistent when using both SCADA and manual data. You are also best to note your assumptions on regulatory and water reports, so there is no question about what you are doing. Just be careful not to make operational decisions based on a concentration or other downstream calculations that are based on data sets from two different timeframes.
One thing we usually see agreement on is water quality samples, which should always be reported on the date/time the sample was taken. Whether it is chlorine residual, hardness, pH, TSS, etc. – that data should be reported on that day and not shifted to the previous day. Sometimes a lab will take days to process data, but we should also be careful to report the results on the same day (and time) that the sample was taken. ODA is simple when it comes to reporting daily lab results.
Integrating SCADA Flow Data: Flow Meters Matters to ODA
As mentioned above, typically, most SCADA systems will record totalized data from midnight to midnight, which is preferred. However, not all flow meters will give you data in the same level of accuracy and it’s important to understand the difference in data quality. This article will stop short of a full explanation on meter accuracy but suffice it to say there are many factors that go into getting an accurate reading that you can use when accounting for non-revenue water (NRW), real losses (i.e. leaks), and overall water supplied to the distribution system or processed by a treatment plant. For simplicity in this article, we will assume you have flow meter accurate enough to use for reliable reporting.
Most larger facility flow meters (ultrasonic, magnetic, and turbine) are not positive displacement meters. They generally provide a variable signal that linearly correlates to a velocity in the pipe. The meter electronics can then use that velocity, combined with its knowledge about pipe diameter and the fluid to calculate a flow rate (volume per time). The flow rate is then integrated over a period of time (hopefully once or more per second) to give you a volume that has passed through for that time period. This math can be done by the meter manufacturer’s instrumentation (which is usually preferred), or it can be done in a PLC (second best), or it also can be done in the SCADA or other software (this is least desirable but still can be accurate). Regardless of what is chosen, the PLC will usually accumulate these readings of volume per time for the day (again, hopefully once or more per second) and give you a total accumulating volume for the day.
For some lucky facilities that have positive displacement (e.g. nutating disc, rotary piston, diaphragm) meters, each “pulse” corresponds to a given volume. For these positive displacement meters, all the PLC or software has to do (provided the meter stays accurate) is count pulses and multiply by a fixed volume per pulse. Here again, the PLC will ideally accumulate the readings of volume per time for the day and you’ll have an accurate current day totalized flow. In both cases, the current day is tracked until just after midnight, when the PLC should “shift” that data register to a P-Day register for the SCADA system (or reporting app) to see.
If flow data is not integrated every second or so (ask your integrator!), your system could be introducing significant error by “assuming” a constant flow between integration period measurements. For very constant flow applications, we wouldn’t expect this to be a significant error, but for applications where flow varies more often than the meter integration period, you will have inaccurate flow data, which then propagates to all of your other calculations. Water Industry Stewards should ask questions of your programmer/integrator to understand your specific application. I have seen very inaccurate totalized flow readings from systems that have trusted the data for decades only to find out that 5-15% or daily flow errors have been (mis)reported, so asking your integrator/programmer how your flows are totaled is absolutely important for proper ODA and hence, for accurate reporting.
Figure 1- Three trends showing the relationship of flow rate, accumulated flow volume, and previous day's flow for two days
The following charts illustrate this concept in a real SCADA system. The first chart shows two days of data showing real-time Flow Rate (top line). You’ll see the pump station turn on, ramp up the pump, pump against increasing head (which reduces flow rate a bit), then either kick off, or in the first and last examples, it looks like the pump started up at one speed, then increased it/decreased it to better match demand.
The second chart shows the Accumulated Total volume (kGal) for that day. It does this by using the flow rate in the first trend to integrate and calculate the increase in volume for that current day until just before midnight.
The last chart then transfers the accumulated total volume (current day) into a register that locks in that Previous Day Total (P-Day) flow and appears as a square wave, where the value only changes once per day (slightly after midnight).
Figure 2 – An additional set of trends showing the relationship of flow rate, accumulated flow volume, and previous day's flow for a week
For the math lovers out there, the Y-axis value of the top trend (Flow Rate) becomes the slope of the next trend (Accumulated Total) for that time slot. Then the max high point value of the second trend (Accumulated Total) becomes the value for the next day (P-Day flow) of the bottom trend. Meter meets PLC meets PDay. You may have an ODA headache, but this is important to get it right.
Now that you know how to read these trends, here’s another example zooming out to show seven days at once. Note the relationship between the value in the top trend and the slope in the middle trend, as well as the very clear “P-Day” square wave in the bottom trend. The Y axes of the bottom two trends are not the same scale, but you should see the day lag to the middle trend peaks.
What if I’ve been reporting Current vs Previous Day wrong?
If you’re in an organization that stewards more than one system, you should ensure everyone uses these same principles to report consistently. Many of you, regardless of if you work in one or many systems, will find that you might have been inconsistent in your application of Current vs. P-Day. You can fix it. Pick a day that you want to cut over to be more accurate based on this reading; most systems decide to change in the first of a month (the readings you take on the 2nd will apply to the first). On that day, you’ll report current day as previous day, so you’ll have one day where you essentially “skip” as you shift. It’s a one time event, but do note it in your operating report comments so your stakeholders and regulators understand what you are doing and why. Many states have seen these changes over the years and most should understand the reasoning (for our customers, let us know if you’d like some help in communicating, as we have helped in this respect before). You may want to have your responsible operator talk with the management or their inspector to ensure there aren’t any questions about the change.
Conclusions
An argument about water is an argument about data. Accounting for water loss correctly will only gain in importance as the world learns to account for and charge properly for drinking and treating water. While we can all wish we didn’t have to think this hard about how to get our data aligned right and improve our Operational Data Alignment (ODA), as we want to highlight the value of water accurately, it is essential that we understand concepts that impact how we communicate water data accurately. Whether you are accumulating flow, chemical use, or other data, it is important you know what questions to ask your engineers, integrators, programmers, and software providers, to understand how you can and should be reporting data to interested parties. Waterly works with water professionals to understand and apply these complicated principles through simple software designed to reduce environmental risk, improve efficiency, and give time back to the stewards that can then use the saved time to apply themselves to other important problems. Visit waterly.com to learn more.
