Building trust in ML applications also requires such ML routines that respect power grid physics and provide guarantees on feasibility, optimality, and computational stability. Despite recent advances, there is a gap in fidelity (e.g., modeling alternating current power flow models) and scalability (e.g., the ability to solve stochastic management problems for larger network instances with countless DERs and multi-period couplings). Moreover, when filling these gaps, it is critical to augment these ML routines to convey an intuition behind each solution to grid operators (e.g., if a decision is made, it is important to understand what causes this decision and how this decision fares against plausible alternatives).

Our approach to resolve these challenges is to embed representation of the IBRs and power grid constraints into ML training procedures and to develop efficient “learning-to-optimize” methods. To scale to a large number of DERs, the embedded representations of DERs will leverage inference and aggregation. For tractability, ML training will be pushed from real-time to off-line, with only light optimization routines solved live. To enhance the robustness of these methods under uncertainty, we will work with Project Affiliates NREL and EPRI to leverage recent advances in end-to-end learning to integrate forecasting and decision-making (optimization) and to align their objectives. This RD will deliver novel decentralized stochastic optimization algorithms that capture a massive number of DERs, storage and demand-side management options, accelerate them using dual prediction methods, and learn them using decentralized optimization proxies as in. This decentralization will enable efficient coordination between transmission and distribution power grids, thus delivering the DER benefits across large geographical areas.

The ability to operationalize the T100RE models and ML routines described above assumes that data is available and shared among all participants. In practice, however, data sharing is obstructed by valid power grid integrity and security concerns, motivated by risks of malicious interventions (e.g., cyber and physical attacks, loss of competitive advantages (e.g., power producers, utilities, retailers) and/or invasion of privacy (e.g., households participating in demand response). Therefore, we will develop a data sharing framework that incentivizes stakeholders to allow their data to be used in power management, while allowing for control of their data risk. While such privacy-preserving analytics exists for simple problems (e.g., unconstrained optimization to estimate regression parameters), constrained, mixed-integer and ML problems still lack foundational privacy-by-design approaches. The T100RE goal is to develop privacy-preserving analytics that allows for data valuation within power management routines, thus taking into account downstream implications, and provides verifiable guarantees. We will enhance the derivation and analysis of data sharing incentives by blending ML and algorithmic game theory. Building on the incentive-based data acquisition approach, we will explore methods to assess data value from data to both operationalize and valorize digitalization of electric power grids.

The proposed data sharing with high-fidelity models in RD and trustworthy ML in RD will enhance the degree of coordination between power producers, grid operators, and consumers, and become the scientific backbone of the next-generation decision support enhancing efficiency, reliability and resiliency of power management. This will shape computational methods for future T100RE power grids, form the basis of standards for validating explainable and trustworthy forecasting methods and ML systems, and update the current decision-making pipeline (e.g., day-ahead and real-time power grid scheduling) in partnership with G-PST and FPFM. We will collaborate with the NSF AI Institute for Advances in Optimization (AI4OPT), led by EPICS-US co-PI Van Hentenryck, in the area of ML/optimization use for decentralized DER integration.