Weighing the Case for Piloting Proven Technologies

Written by Jeff Casey

Utilities understand the concept of prudence. In a capital-intensive industry, mistakes and errors can cost utilities thousands — sometimes millions — of dollars. They can also present setbacks in achieving goals such as improving reliability, lowering the cost of service, or enabling commitments to a renewable future.

That’s why utilities historically have benefited from pilot programs and system testing. As the industry increasingly moves toward large-scale deployments of Private LTE (PLTE) as a foundational and unified wireless network for all of its applications, the concept of piloting is being examined like never before.

It is prudent to ask: What does “doing a pilot” really mean, and how should it be defined?

Making decisions about deploying PLTE is complex, and such decisions are further confused by the noise and hype generated by the 5G advertising engine of commercial carriers and industry. Pilots for Private LTE are understandable, but the nuance in what gets tested and why is essential. Much of this technology is mature — 4G for example, has more than ten years of use and over a billion connected devices — but using it for private utility networks is new. Everybody has seen that 4G works, and many utilities have applications already deployed on public cellular networks, but differences in how it works in a private environment and for utilities are worth exploring.

The Case for Pilot Testing Has Nuance

There are several valid reasons to test PLTE:

• Demonstrating and proving how private 4G performs for utility applications.
• Gaining a better understanding of the specific benefits of new 3GPP releases, including 5G.
• Conducting head-to-head comparisons of various vendor solutions in a controlled environment.
• Validating RF coverage estimations with actual equipment used in the field and utility design criteria for fixed devices.
• Building consensus among technical teams and executives.
• Testing utility-specific failure and failover scenarios.
• Identifying new bands and new power limits relative to coverage, including potential interference concerns.
• Validating end-to-end security provisions.

Success Factors

A robust testing methodology is a strong way to achieve meaningful conclusions with well-documented results and learnings.

A common failing of pilot tests is that too often they become check-the-box exercises that do not include validation in real-world environments. Results in these cases can be misleading and damaging to the success of the ultimate build-out.

If done correctly, PLTE pilot testing can provide real insights and value. These are only a few of the benefits the industry has recognized:

• Building readiness for full-scale project rollout: With early planning and limited deployments as a step beyond and a replacement to a typical pilot, the next phase should run significantly smoother. This additional work provides a pause before the actual deployment to allow fine-tuning and corrective actions where challenges arose.
• Demonstrating coverage in complex areas: Some utility service territory areas will not have coverage for one reason or another. They could have dense vegetation or hard surfaces that do not model well. Device positioning may also vary from what was modeled (ex: height). A pilot will help identify complex RF conditions sooner and potentially de-risk the deployment as it scales out.
• Validating multi-network interference: It is important to verify that the land mobile radio (LMR) systems deployed in the narrowband 900 MHz spectrum will work alongside the broadband LTE channel. By studying the impacts of LTE 900 MHz interference on existing 900 MHz narrowband LMR systems or 900 MHz mesh systems (specific to Anterix), we can head off future issues, as both networks are critical.
• Demonstrating abstract utility-specific use cases: System performance expectations should be validated. This is especially true for latency-sensitive applications. Full end-to-end application performance should be tested, including the failure and adverse network performance scenarios.
• Getting consensus and overall buy-in: This is especially important for those in the organization who don’t fully understand the transformation, including executive team members. Never underestimate the importance of consensus-building. Simply showing a device bearing the utility as the “carrier name” and making a first call could be key.
• Training and planning: PLTE is new for most utilities. Bringing engineering and operations together early for training and long-term operations planning will help key personnel understand new operational complexity.
• Testing new feature sets: Many 5G-specific features like ultra-reliable low latency communications (URLLC) or side link haven’t been proven within the carrier industry in real-world deployments, let alone for utilities. Testing of devices and chipsets can reveal potential use cases, as well as obstacles that need to be addressed.

Are Pilots Always Needed?

It depends. Here are some arguments against PLTE testing and why it may not be the best investment in time and money:

• 4G is a standards-based, mature, proven technology deployed at scale worldwide: 4G technology is utilized by all major carriers, and many utilities already use public carrier 4G networks for distribution assets or other noncritical use cases.
• The components to be piloted are not the same as those deployed: If this is the case, it is important to define what you hope to learn and be clear about this upfront. If the utility is spending significant money to deploy something for a test only to rip it out later, the key learnings may not have enough value to justify this significant investment. Additionally, if you aren’t testing the production-scale equipment, are the learnings valuable and scalable? For example, using a ‘core in a box’ solution to complete a data or voice session isn’t interesting or new. Using off-the-shelf equipment for ease of deployment in the pilot will not demonstrate all the value the new network could bring.
• Not testing a multi-vendor solution: Testing equipment from one vendor could imply or result in vendor lock-in and risk failing to get the architecture or functionality that is needed. Head-to-head tests can add significant complexity and cost if different equipment is being tested, and extra analysis may be required to understand the results. The nuances between results aren’t for the technically weary.
• Technology is only a small part of the story: The organizational changes often needed for the rollout are several orders of magnitude larger than the technology pilot. The simplicity of a pilot may underrepresent the overall complexity. 

Standardized Testing Needed

There are a number of potential pathways forward, but there is a strong case for the industry to coalesce around single-purpose testing in which vendors set up common testing environments in a common facility. All utilities would have equal access to test their specific use cases. Allowing utilities to share in the effort and cost to address the common check box applications and go directly to the utility specific complex cases is where value is created.

The main caution is not to take a technology-first approach to change and business transformation. Change sounds much simpler than it is. The pilot should not be the only source of validation for your transformation.

Strategies should be aligned with goals and outcomes, and you must manage against them relentlessly. With thorough communication, everyone can become aligned. Remember, the pilot is only a step along the way in your network and business transformation journey. Getting it right the first time must be the priority.

This article edited by Doug Houseman

For a downloadable copy of the September 2021 eNewsletter which includes this article, please visit the IEEE Smart Grid Resource Center.

Jeff Casey, MEIT, is a strategy and consulting manager at Burns & McDonnell. With over a decade of experience in the transmission and distribution industry, Casey’s diverse background in substation networks, IEC 61850, distribution substation automation, program management, and cybersecurity standards has helped him deliver energy projects for clients worldwide. He leads the development and growth of market opportunities within the firm’s Networks, Integration & Automation Group, currently focusing on private 4G LTE and 5G for utilities, commercial fiber networks, closing the digital divide with rural broadband, and data analytics business lines.