Running SCION Locally

SCION is an Internet architecture and SCION networks are composed of many hosts running the SCION control-plane services, routers, and SCION-enabled applications. To simplify development, testing, and tinkering, we provide a tool that generates test topologies and runs an entire SCION “Internet” on a single machine. Packets are only sent between different processes on the same host, not over the network.

Running SCION in this development setup, is also called running a local topology.

There are three different ways to run a local topology:

  • containerlab (recommended). Each SCION host runs in its own container. The inter-AS links are dedicated point-to-point links between the containers, so the network closely mimics real-world connectivity. See containerlab.

  • supervisor, using scion.sh. This is a bit more light-weight. Packets are sent over the loopback interface.

  • docker compose, using scion.sh. Runs individual processes in separate containers connected with docker network bridges. Only this mode supports running a “SCION-IP gateway”.

    Hint

    Before attempting to use the docker compose mode, be sure to build the necessary docker images with make docker-images

Quickstart

Prerequisites

  1. Build the SCION node image and load it into docker. It contains the SCION services, the scion CLI and the test applications.

    bazel run //testing/clab:clab_node.load
    

    Re-run this whenever you change the code, so that the next deployment picks up the changes.

  2. Generate the control-plane PKI keys and certificates, configuration files, and the containerlab topology gen/scion.clab.yml:

    bazel run --run_in_cwd //testing/clab/cmd/testgen -- --topo topology/tiny.topo
    

    See testgen for the available options.

    Attention

    This deletes the existing gen/ directory. The certificates created by this command expire after 365 days if the infrastructure is not running for automatic renewal.

  3. Deploy the topology:

    clab deploy -t gen/scion.clab.yml
    

    If a lab of the same name is already running, add --reconfigure to replace it.

  4. Wait until the control services have beaconed and all ASes can reach each other:

    bazel run --run_in_cwd //testing/clab/cmd/await_connectivity
    
  5. To verify that your topology is running correctly, you can run the end to end tests. e2e_scion runs scion ping and scion showpaths between all pairs of ASes, e2e_http runs an HTTP/3 server in each AS and fetches from it from all other ASes:

    bazel run --run_in_cwd //testing/clab/cmd/e2e_scion
    bazel run --run_in_cwd //testing/clab/cmd/e2e_http run
    
  6. This local infrastructure runs one SCION daemon per AS. We need to specify which instance is used when running applications that rely on the SCION daemon, e.g. to query paths. The daemons are reachable from the machine running the lab:

    # show paths from 1-ff00:0:112 to 1-ff00:0:110
    bazel run //scion/cmd/scion -- showpaths --sciond $(./scion.sh sciond-addr 112) 1-ff00:0:110
    
  7. To stop the infrastructure, run:

    clab destroy -t gen/scion.clab.yml --cleanup
    

Quickstart with scion.sh

Prerequisites

  1. Generate the control-plane PKI keys and certificates, configuration files, and process orchestrator (supervisor or docker compose) configuration.

    ./scion.sh topology -c topology/tiny.topo
    

    Attention

    The certificates created by this command expire after 365 days if the infrastructure is not running for automatic renewal.

  2. To start the infrastructure we just generated, run:

    ./scion.sh run
    
  3. To verify that your topology is running correctly, you can run an end to end reachability test using:

    bin/end2end_integration
    
  4. This local infrastructure runs multiple SCION daemons, one in each AS. We need to specify which instance is used when running applications that rely on the SCION daemon, e.g. to query paths.

    # show paths from 1-ff00:0:112 to 1-ff00:0:110
    bin/scion showpaths --sciond $(./scion.sh sciond-addr 112) 1-ff00:0:110
    
  5. To stop the infrastructure, run:

    ./scion.sh stop
    

Local Topology Environment

containerlab

The topology is generated by testgen into the gen/ directory:

  • gen/scion.clab.yml is the containerlab topology.

  • There is a subdirectory for each AS (e.g. gen/ASff00_0_110 for AS ff00:0:110), with the AS’s crypto material, the shared topology.json, and a subdirectory per host (e.g. gen/ASff00_0_110/host-1) that is mounted into that host’s container.

  • gen/sciond_addresses.json maps each AS to the address of its scion daemon.

  • gen/network-allocations.yml lists the subnets and addresses assigned to the ASes, the hosts and the links.

Each SCION host is a container named clab-<lab>-<ISD-AS>-host-<N>, e.g. clab-scion-1-ff00_0_110-host-1. Every AS has at least host-1, which runs the Control Service and the Daemon of the AS, and a Router. Additional border routers of the AS run on additional hosts. Inside the container, a controller process starts the SCION services according to the mounted configuration.

All hosts are attached to the containerlab management network, which also serves as the internal network of each AS. The inter-AS links are point-to-point links between the containers.

Commands accessing the SCION network can be run inside the containers. The SCION binaries are located in /app. For example:

# show paths from 1-ff00:0:112 to 1-ff00:0:110
docker exec clab-scion-1-ff00_0_112-host-1 /app/scion showpaths \
   --sciond $(./scion.sh sciond-addr 112) 1-ff00:0:110

They can also be run directly on the machine running the lab, as it is attached to the management network. The local AS is determined by the SCION daemon instance that the command connects to:

# show paths from 1-ff00:0:112 to 1-ff00:0:110
bazel run //scion/cmd/scion -- showpaths --sciond $(./scion.sh sciond-addr 112) 1-ff00:0:110

By default, an application running on the machine uses the machine’s address on the management network. To give the machine a dedicated address in each AS, e.g. to run a server there, generate the topology with testgen --host-links. The addresses are listed under host_links in gen/network-allocations.yml. For example, to run an HTTP/3 server on the machine in 1-ff00:0:112 and fetch from it on the machine in 1-ff00:0:110:

bazel run //testing/clab/cmd/e2e_http -- server \
   --sciond $(./scion.sh sciond-addr 112) --listen <host_addr of 1-ff00:0:112>:31000

bazel run //testing/clab/cmd/e2e_http -- client \
   --sciond $(./scion.sh sciond-addr 110) --remote '1-ff00:0:112,<host_addr of 1-ff00:0:112>:31000'

IPv6 addresses must be enclosed in brackets, e.g. [fd00:f00d:cafe:9000::2]:31000.

scion.sh (supervisor and docker compose)

The scion.sh topology command generates configuration in the gen/ directory in the repository.

There is a subdirectory for each AS (e.g. gen/ASff00_0_110 for AS ff00:0:110), containing the configuration for the services and routers of that AS. Specifically, these contain the .toml configuration files for the individual components and a shared topology.json configuration.

Various helper files are also generated for the benefit of scripts and tooling of the test infrastructure, for example, gen/sciond_addresses.json is a simple mapping from AS number to the address of the corresponding scion daemon instance.

If scion.sh topology -d command is used, configuration files are created to enable running the SCION services in docker containers (see docker). Otherwise, a configuration file is created to enable running the SCION services as plain processes (see supervisor)

supervisor

The gen/supervisord.conf configuration defines the programs that make up the local topology.

All the SCION traffic goes via the loopback interface, the separation of the internal networks of the simulated ASes is not enforced in any way.

There is a single Dispatcher instance, receiving and sending the SCION packets for the Control Service instances and SCION applications/tools in all simulated ASes.

There is one Daemon instance running for each simulated AS. Commands accessing the SCION network can be run directly from the host. The information about the local AS in which the command is running, is determined by the SCION daemon instance that the command connects to. For example:

# show paths from 1-ff00:0:112 to 1-ff00:0:110
bin/scion showpaths --sciond $(./scion.sh sciond-addr 112) 1-ff00:0:110

# reveal the full SCION address of a simulated host in in 1-ff00:0:111
bin/scion address --sciond $(./scion.sh sciond-addr 111)

# and now ping this host from inside AS 1-ff00:0:110, with interactive path prompt
bin/scion ping --sciond $(./scion.sh sciond-addr 110) 1-ff00:0:111,127.0.0.1 --interactive

docker

The main docker compose file is gen/scion-dc.yml.

Each SCION service or router runs in a separate container, and the network access of the individual containers is configured to mimic real-world connectivity.

There are “tester” containers configured in each AS to mimic end hosts in a SCION AS. These tester containers can be used to run commands accessing the SCION network. As a shorthand for the somewhat unwieldy docker compose invocation, the tools/dc script can be used. For example:

# show paths from 1-ff00:0:112 to 1-ff00:0:110
tools/dc exec_tester 1-ff00_0_112 bin/scion showpaths 1-ff00:0:110

# reveal the full SCION address of the tester container in 1-ff00:0:111
tools/dc exec_tester 1-ff00_0_111 bin/scion address

# and now ping this host from inside AS 1-ff00:0:110, with interactive path prompt
tools/dc exec_tester 1-ff00_0_110 bin/scion ping 1-ff00:0:111,172.20.0.29

Note that the --sciond flag does not need to be provided in this setup, as it’s preconfigured in the tester containers via the environment variable SCION_DAEMON.

testgen

testgen generates a containerlab topology from a *.topo configuration file. Run it from the repository root with:

bazel run --run_in_cwd //testing/clab/cmd/testgen -- <options>
--topo <FILE.topo>

Path to the *.topo configuration file. Default: topology/default.topo.

-o <DIR>, --out <DIR>

Output directory. It is deleted before generating. Default: gen.

--name <NAME>

Name of the containerlab lab. It is part of the container names and of the topology file name (gen/<NAME>.clab.yml). Default: scion.

--network <PREFIX>, --network-v6 <PREFIX>

Base IPv4 and IPv6 networks from which the AS-internal networks and the link networks are allocated. Defaults: 10.0.0.0/8 and fd00:f00d:cafe::/48. They must not overlap with any other network on the machine.

Connect host-1 of every AS to the machine running the lab with a dedicated link. This gives the machine an address in each AS, which applications running on the machine can bind to.

--isd-dir

Group the AS directories in per-ISD directories.

--as-validity <DURATION>

Validity of the generated AS certificates. Default: 1y.

The whole lifecycle (generate, deploy, run the end to end tests, destroy) is also available as a bazel test:

bazel test //testing/clab/system_test:go_default_test --test_output=streamed

scion.sh

scion.sh is the developer script to setup and run a local topology.

Note

The SCION tools and services need to be built before running these commands, using make or make docker-images (when using the docker compose configuration).

The basic usage is ./scion.sh <subcommand> <options>. The main subcommands are:

topology

Generate the control-plane PKI keys and certificates, configuration files, and process orchestrator (supervisor or docker compose) configuration for a given network topology defined in a *.topo configuration file.

-c <FILE.topo>, --topo-config <FILE.topo>

Path to the *.topo configuration file.

-d, --docker

Create a docker compose configuration (instead of default supervisord).

--sig

Generate a SIG (SCION-IP Gateway) for each AS. Only available with -d.

-h, --help

Display help text, list all options

run, start

Start the local SCION topology.

stop

Terminate this run of the local SCION topology.

start-monitoring

Start the monitoring services (jaeger and prometheus).

stop-monitoring

Stop the monitoring services.

sciond-addr <ISD-AS>

Return the address for the scion daemon for the matching ISD-AS by consulting gen/sciond_addresses.json. The ISD-AS parameter can be a substring of the full ISD-AS (e.g. last three digits), as long as there is a unique match.

help

Describe all available subcommands

end2end_integration

bin/end2end_integration is a basic functional test.

The basic usage is ./end2end_integration <options>.

-d

Assume the SCION services are dockerized. Must be consistent with the last invocation of scion.sh topology.