Pacific Wave Supporting FABRIC Coast-to-Coast Terabit Testbed with Connections in Los Angeles and Seattle

The global expansion of the Internet has revolutionized nearly every aspect of human society, and even greater transformations lie ahead.  Furthermore, many of these coming transformations will change not only how we do things but networking itself.  To ensure that the US’s K-20 research and education communities in California and beyond continue to contribute their leadership to these transformations, those communities will need access to a Terabit network capable of functioning as a testbed for creating tomorrow’s networking revolutions.

In pursuit of this goal, Pacific Wave is pleased to support the NSF-funded FABRIC testbed (Awards #1935966, #2330891, and #2029261) with high-bandwidth connections ranging from 100 to 400 Gbps in Los Angeles and Seattle. FABRIC is an international infrastructure that enables cutting-edge experimentation and research at-scale in the areas of networking, cybersecurity, distributed computing, storage, virtual reality, 5G, machine learning, and science applications.

The FABRIC infrastructure is a distributed set of equipment at commercial collocation spaces, national labs, and campuses underpinned by a coast-to-coast 1.2 Terabit per second ring, referred to as TeraCore and shown below. It is not intended to be an isolated testbed facility but instead interconnects with other specialized testbeds, production facilities, HPC facilities, and the Internet to create an environment for a wide variety of experimental activities where experimenters are allowed to pick and choose portions of the infrastructure for their experiments.

Additionally, while previously federated facilities were connected to FABRIC at 100 Gigabits per second (Gbps), thanks to TeraCore becoming operational in October 2023, the project team is now working to connect several federated facilities including SDSC at 400 Gbps.  Thanks to its connections to other international research and education networks, FABRIC also contributes to global network-enabled collaboration.

FABRIC provides access to cutting-edge programmable network technologies such as Programming Protocol-independent Packet Processors (P4), an open source, domain-specific programming language for network devices, and enables novel approaches to integrating artificial intelligence and machine learning (AI/ML) into distributed and network systems control and management. Its diverse environment combining programmable and large computational resources also makes it a perfect fit for helping to train the next generation of computer science and network researchers.

A Testbed for New Network Architectures: the InterEdge project

The original Internet was designed to treat all packets of data equally and simply pass them along as quickly as possible between two endpoints, crossing boundaries between smaller networks as needed.  For many years, this functionality was sufficient and indeed was enough to spur the expansion of the Internet until it has become the backbone of the modern global economy.

However, these two design and operating principles, known as end-to-end simplicity and interconnection, are challenged by the need for network performance, security, and privacy.  These additional functions involve the network not only moving data from one place to another but also processing it while it is still traveling between its point of origin and destination.  Moreover, the devices that carry out these functions (caching, load balancing, firewalls, authentication) are typically at the network edge, isolated from one another and not interconnected at all.

Investigating how these inconsistencies in architecture can be reconciled and a new, consistent network architecture developed which is capable of carrying out these important functions is the purpose of the InterEdge project.

As core team member Scott Shenker of UC Berkeley explains it, “The InterEdge project aims to [provide] an edge architecture that allows [...] interconnection. A prototype of the InterEdge has been built by researchers at (in alphabetical order) ICSI, LBL, Mount Holyoke College, NYU, UC Berkeley, ICSI, and U Washington and will be soon deployed on the FABRIC testbed. The FABRIC testbed is ideally suited to the needs of the InterEdge, because all of the testbed nodes are equipped with the general-purpose compute that edge services need.”

A paper on the InterEdge project, titled “An Architecture for Edge Networking Services,” was written by Shenker and colleagues and accepted for the ACM SIGCOMM 2024 conference to be held in Sydney, Australia from August 4-8, 2024, after which a public link to the PDF will be available at the SIGCOMM 2024 website.

An Intelligent Testbed Enables At-Scale Network Security and Privacy Research

Cybersecurity and privacy threats increasingly impact our daily lives, our national infrastructures, and our industry. Moreover, as new technologies are released, threat actors improve their own capabilities through experience and close collaboration while defenders often work in isolation, using private data and facilities, and producing defenses that are quickly outpaced by new threats.

As with InterEdge, FABRIC’s ability to interconnect compute-capable edge devices at Terabit speeds makes it an excellent testbed for the Security and Privacy Heterogeneous Environment for Reproducible Experimentation (SPHERE) project (Award #2330066), the participants of which include USC’s Information Science Institute (ISI) and Northeastern University.  This project aims to transform cybersecurity and privacy research into a highly integrated, community-wide effort by providing all researchers with a common, rich, representative research infrastructure that meets their needs and facilitates reproducible science.

As Lead PI for the project USC ISI’s Jelena Mirkovic states, a large share of the SPHERE testbed is currently at ISI, while Internet-of-Things devices will be housed at Northeastern University.

Mirkovic further explains, “FABRIC allows for the control of bandwidth and delay between various parts of the SPHERE testbed and doesn’t mix experimental data traffic with other data traffic on the public Internet.  Also, because FABRIC’s participants include other major research facilities across the US, FABRIC opens up possibilities where a researcher can combine resources from these facilities with SPHERE resources, using dedicated, user-controlled connections from FABRIC.”

Currently, the SPHERE project seeks to connect its constituent parts in a way that guarantees reliable interconnection, but in the future FABRIC will also enable the SPHERE project to explore network security and privacy via experimenting with in-network computation that FABRIC provides.

“So far, we’ve focused on dedicated connections,” Mirkovic states, “but in the future we’d like to explore intelligent connections, capable of processing data in-network.  As an intelligent testbed, FABRIC will allow us to explore that.”

Building Productive Connections between Scientists and Network Engineers

At UC San Diego, Frank Wuerthwein and his colleagues have their eyes on another form of network redesign: building new interconnections between disciplines.  Wuerthwein seeks to offer hardware and other resources to FABRIC users through the NSF-funded Prototype National Research Platform (PNRP) (Award #2112167), as much of what they already have available fits well with FABRIC’s existing architecture.  Their connectivity supports this goal through a variety of network paths, with 100 and 400 Gbps connections through CENIC to both Pacific Wave and ESnet, both of which provide paths to FABRIC’s TeraCore ring.

Wuerthwein’s ultimate aim however extends beyond the interconnection of hardware.  In his discipline of high-energy physics, bespoke compute, storage, and networking technology is a must.

“The High-Luminosity LHC project is coming online in 2029 and is expected to produce up to an exabyte of data every year,” Wuerthwein states.  “Processing and moving that volume of data around will be an enormous challenge, and we can’t assume anymore that compute power will increase sufficiently if we just wait.  We need to change how scientific networking development is done.  Part of this is getting the scientists who use the network and those who create it on the same testbeds.”

Thus a major goal of making PNRP resources available to FABRIC is the creation of a blended social environment that includes researchers in both the physical and computer sciences on the same testbed. This shared space will help speed the development of bespoke technologies and their integration into the many big-data disciplines that use them.

Creating the Professionals that will Create the Transformations

With its high performance, convenient access, and large geographical span, FABRIC is not only a resource for the research community but can be used for non-networking and non-research applications, including classroom instruction.  FABRIC provides a hosted JupyterHub environment for experimenters, which comes with Jupyter notebooks automatically preloaded with FABRIC libraries and CLI tools. The portal and JupyterHub use federated identity through CI Logon and support any institutional identity provider that is part of InCommon Research and Scholarship (R&S) federation. This includes the majority of US educational institutions. Efforts are underway to make FABRIC easily available for use in the classroom, and more information about how to use FABRIC for instruction is available at the FABRIC website.

If you are interested in learning more about FABRIC or have any questions on how to apply it to your institution’s research and education projects, please visit the FABRIC website.

About Pacific Wave | pacificwave.net

Pacific Wave is a distributed, Research and Education (R&E)-focused, open Internet exchange. It provides for very high-performance Internet connectivity among US Science and Engineering R&E institutions and their international partners and is critical infrastructure for access to internationally supported instruments and large-scale data sources and repositories.

Pacific Wave enables large-scale scientific workflows to accelerate discovery in all areas of science and engineering, including high-energy physics, Earth sciences, astronomy and astrophysics, biology, and biomedical engineering, as well as scalable visualization, virtual reality, machine learning, and artificial intelligence.

Pacific Wave is a joint project of the Corporation for Education Network Initiatives in California (CENIC) and the Pacific Northwest Gigapop (PNWGP) and is funded in part by a grant from the National Science Foundation.

About FABRIC | portal.fabric-testbed.net

FABRIC (FABRIC is Adaptive ProgrammaBle Research Infrastructure for Computer Science and Science Applications) is an International infrastructure that enables cutting-edge experimentation and research at-scale in the areas of networking, cybersecurity, distributed computing, storage, virtual reality, 5G, machine learning, and science applications.

The FABRIC infrastructure is a distributed set of equipment at commercial collocation spaces, national labs and campuses. Each of the 29 FABRIC sites has large amounts of compute and storage, interconnected by high speed, dedicated optical links. It also connects to specialized testbeds (5G/IoT PAWR, NSF Clouds), the Internet and high-performance computing facilities to create a rich environment for a wide variety of experimental activities.

FABRIC Across Borders (FAB) extends the network to 4 additional nodes in Asia and Europe.