6 Describing a profile with python and geni-lib

geni-lib is a tool that allows users to generate RSpec files from Python code. PhantomNet offers the ability to use geni-lib scripts as the definition of a profile, rather then the more primitive RSpec format. When you supply a geni-lib script on the Create Profile page, your script is uploaded to the server so that it can be executed in the geni-lib environment. This allows the script to be verified for correctness, and also produces the equivalent RSpec representation that you can view if you so desire.

When you provide a geni-lib script, you will see a slightly different set of buttons on the Create Profile page; next to the “Source” button there is an “XML” button that will pop up the RSpec XML for you to look at. The XML is read-only; if you want to change the profile, you will need to change the python source code that is displayed when you click on the “Source” button. Each time you change the python source code, the script is uploaded to the server and processed. Be sure to save your changes if you are updating an existing profile.

The following examples demonstrate basic geni-lib usage. More information about geni-lib and additional examples, can be found in the geni-lib repository. The current version of geni-lib used by PhantomNet can be found in the 0.9-EMULAB branch. Its full documentation is online as part of this manual.

6.1 A single XEN VM node

"""An example of constructing a profile with a single Xen VM. Instructions: Wait for the profile instance to start, and then log in to the VM via the ssh port specified below. (Note that in this case, you will need to access the VM through a high port on the physical host, since we have not requested a public IP address for the VM itself.) """ # Import the Portal object. import geni.portal as portal import geni.rspec.pg as rspec # Create a Request object to start building the RSpec. request = portal.context.makeRequestRSpec() # Add a XenVM (named "node") to the request node = request.XenVM("node") # Write the request in RSpec format portal.context.printRequestRSpec() Open this profile on PhantomNet"""An example of constructing a profile with a single Xen VM. Instructions: Wait for the profile instance to start, and then log in to the VM via the ssh port specified below. (Note that in this case, you will need to access the VM through a high port on the physical host, since we have not requested a public IP address for the VM itself.) """ # Import the Portal object. import geni.portal as portal import geni.rspec.pg as rspec # Create a Request object to start building the RSpec. request = portal.context.makeRequestRSpec() # Add a XenVM (named "node") to the request node = request.XenVM("node") # Write the request in RSpec format portal.context.printRequestRSpec()

This example demonstrates the two most important objects: the portal context (accessed through the portal.context object in the geni.portal module), and the request RSpec created by calling makeRequestRSpec() on it. These fundamental objects are central to essentially all PhantomNet geni-lib profiles.

Another way to create a Request RSpec object is to call its constructuor, geni.rspec.pg.Request directly. We ask the Context to create it for us so it it is "bound" to the context and does not need to be explicitly passed to other functions on the context

Once the request object has been created, resources may be added to it by calling methods on it like RawPC() or rspec.pg.LAN. In this example, just a single node (created with the XenVM() constructor, asking for a single VM identified by the name "node") is requested.

Most functions called on Request objects are not directly members of that class. Rather, they are loaded as "extensions" by modules such as geni.rspec.emulab.

The final action the geni-lib script performs is to generate the XML representation of the request RSpec, with the printRequestRSpec() call on the last line. This has the effect of communicating the description of all the resources requested by the profile back to PhantomNet.

You will also notice that the profile begins with a string literal (to be precise, it is a Python docstring). The initial text will also be used as the profile description; the text following the Instructions: line will be used as the corresponding instructions. This documentation is so important that adding the description to the profile is mandatory. (Using a docstring like this is not the only way to produce the description and instructions, although it is the most convenient.)

This simple example has now demonstrated all the important elements of a geni-lib profile. The portal context and request RSpec objects, the final printRequestRSpec() call, and the docstring description and instructions are “boilerplate” constructions, and you will probably include similar or identical versions of them in every geni-lib profile you create unless you are doing something quite unusual.

6.2 A single physical host

"""An example of constructing a profile with a single raw PC. Instructions: Wait for the profile instance to start, and then log in to the host via the ssh port specified below. """ import geni.portal as portal import geni.rspec.pg as rspec # Create a Request object to start building the RSpec. request = portal.context.makeRequestRSpec() # Create a raw PC node = request.RawPC("node") # Print the RSpec to the enclosing page. portal.context.printRequestRSpec() Open this profile on PhantomNet"""An example of constructing a profile with a single raw PC. Instructions: Wait for the profile instance to start, and then log in to the host via the ssh port specified below. """ import geni.portal as portal import geni.rspec.pg as rspec # Create a Request object to start building the RSpec. request = portal.context.makeRequestRSpec() # Create a raw PC node = request.RawPC("node") # Print the RSpec to the enclosing page. portal.context.printRequestRSpec()

As mentioned above, most of these simple examples consist of boilerplate geni-lib fragments, and indeed the portal context and request RSpec operations are unchanged from the previous script. The big difference, though (other than the updated documentation) is that in this case the RawPC() method is invoked on the Request object instead of XenVM(). As you might expect, the new profile will request a physical host instead of a virtual one. (A side effect of using a real machine is that it automatically comes with a unique public IP address, where the VM used in the earlier example did not. Profiles can request public IP addresses for VMs too, though it does not happen by default.)

6.3 Two XenVM nodes with a link between them

"""An example of constructing a profile with two VMs connected by a LAN. Instructions: Wait for the profile instance to start, and then log in to either VM via the ssh ports specified below. """ import geni.portal as portal import geni.rspec.pg as rspec request = portal.context.makeRequestRSpec() # Create two XenVM nodes. node1 = request.XenVM("node1") node2 = request.XenVM("node2") # Create a link between them link1 = request.Link(members = [node1,node2]) portal.context.printRequestRSpec() Open this profile on PhantomNet"""An example of constructing a profile with two VMs connected by a LAN. Instructions: Wait for the profile instance to start, and then log in to either VM via the ssh ports specified below. """ import geni.portal as portal import geni.rspec.pg as rspec request = portal.context.makeRequestRSpec() # Create two XenVM nodes. node1 = request.XenVM("node1") node2 = request.XenVM("node2") # Create a link between them link1 = request.Link(members = [node1,node2]) portal.context.printRequestRSpec()

This example demonstrates two important geni-lib concepts: first, adding more than a single node to the request (which is a relatively straightforward matter of calling more than one node object constructor, being careful to use a different name each time). It also shows how to add links between nodes. It is possible to construct links and LANs in a more complicated manner (such as explicitly creating Interface objects to control interfaces), but the simplest case is to supply the member nodes at the time the link is created.

6.4 Two ARM64 servers in a LAN

"""An example of constructing a profile with two ARM64 nodes connected by a LAN. Instructions: Wait for the profile instance to start, and then log in to either host via the ssh ports specified below. """ import geni.portal as portal import geni.rspec.pg as rspec request = portal.context.makeRequestRSpec() # Create two raw "PC" nodes node1 = request.RawPC("node1") node2 = request.RawPC("node2") # Set each of the two to specifically request "m400" nodes, which in CloudLab, are ARM node1.hardware_type = "m400" node2.hardware_type = "m400" # Create a link between them link1 = request.Link(members = [node1, node2]) portal.context.printRequestRSpec() Open this profile on PhantomNet"""An example of constructing a profile with two ARM64 nodes connected by a LAN. Instructions: Wait for the profile instance to start, and then log in to either host via the ssh ports specified below. """ import geni.portal as portal import geni.rspec.pg as rspec request = portal.context.makeRequestRSpec() # Create two raw "PC" nodes node1 = request.RawPC("node1") node2 = request.RawPC("node2") # Set each of the two to specifically request "m400" nodes, which in CloudLab, are ARM node1.hardware_type = "m400" node2.hardware_type = "m400" # Create a link between them link1 = request.Link(members = [node1, node2]) portal.context.printRequestRSpec()

We now come to demonstrate requesting particular properties of nodes—until now, all nodes had been either XenVMs or RawPCs and nothing further was said about them. geni-lib allows the user to specify various details about the nodes, and this example makes use of the hardware_type property. The hardware_type can be set to a string describing the type of physical machine onto which the logical node can be mapped: in this case, the string is "m400", which means a ProLiant Moonshot m400 host (an ARM64 server). Obviously, such a profile cannot be instantiated on a cluster without a sufficient quantity of appropriate machines! (This profile was written with the Utah CloudLab cluster in mind.) PhantomNet will indicate a list of suitable clusters when the user attempts to instantiate the profile, so he or she is not forced to find one by trial and error.

6.5 A VM with a custom size

"""An example of constructing a profile with a single Xen VM. Instructions: Wait for the profile instance to start, and then log in to the VM via the ssh port specified below. (Note that in this case, you will need to access the VM through a high port on the physical host, since we have not requested a public IP address for the VM itself.) """ import geni.portal as portal import geni.rspec.pg as rspec # Create a Request object to start building the RSpec. request = portal.context.makeRequestRSpec() # Create a XenVM node = request.XenVM("node") # Ask for two cores node.cores = 2 # Ask for 2GB of ram node.ram = 2048 # Add an extra 8GB of space on the primary disk. # NOTE: Use fdisk, the extra space is in the 4th DOS partition, # you will need to create a filesystem and mount it. node.disk = 8 # Alternate method; request an ephemeral blockstore mounted at /mydata. # NOTE: Comment out the above line (node.disk) if you do it this way. #bs = node.Blockstore("bs", "/mydata") #bs.size = "8GB" #bs.placement = "nonsysvol" # Print the RSpec to the enclosing page. portal.context.printRequestRSpec() Open this profile on PhantomNet"""An example of constructing a profile with a single Xen VM. Instructions: Wait for the profile instance to start, and then log in to the VM via the ssh port specified below. (Note that in this case, you will need to access the VM through a high port on the physical host, since we have not requested a public IP address for the VM itself.) """ import geni.portal as portal import geni.rspec.pg as rspec # Create a Request object to start building the RSpec. request = portal.context.makeRequestRSpec() # Create a XenVM node = request.XenVM("node") # Ask for two cores node.cores = 2 # Ask for 2GB of ram node.ram = 2048 # Add an extra 8GB of space on the primary disk. # NOTE: Use fdisk, the extra space is in the 4th DOS partition, # you will need to create a filesystem and mount it. node.disk = 8 # Alternate method; request an ephemeral blockstore mounted at /mydata. # NOTE: Comment out the above line (node.disk) if you do it this way. #bs = node.Blockstore("bs", "/mydata") #bs.size = "8GB" #bs.placement = "nonsysvol" # Print the RSpec to the enclosing page. portal.context.printRequestRSpec()

The earlier examples requesting VMs used the default number of cores, quantity of RAM, and disk size. It’s also possible to customize these value, as this example does by setting the cores, ram, and disk properties of the XenVM class (which is a subclass of rspec.pg.Node.)

6.6 Set a specific IP address on each node

"""An example of constructing a profile with node IP addresses specified manually. Instructions: Wait for the profile instance to start, and then log in to either VM via the ssh ports specified below. (Note that even though the EXPERIMENTAL data plane interfaces will use the addresses given in the profile, you will still connect over the control plane interfaces using addresses given by the testbed. The data plane addresses are for intra-experiment communication only.) """ import geni.portal as portal import geni.rspec.pg as rspec request = portal.context.makeRequestRSpec() node1 = request.XenVM("node1") iface1 = node1.addInterface("if1") # Specify the component id and the IPv4 address iface1.component_id = "eth1" iface1.addAddress(rspec.IPv4Address("192.168.1.1", "255.255.255.0")) node2 = request.XenVM("node2") iface2 = node2.addInterface("if2") # Specify the component id and the IPv4 address iface2.component_id = "eth2" iface2.addAddress(rspec.IPv4Address("192.168.1.2", "255.255.255.0")) link = request.LAN("lan") link.addInterface(iface1) link.addInterface(iface2) portal.context.printRequestRSpec() Open this profile on PhantomNet"""An example of constructing a profile with node IP addresses specified manually. Instructions: Wait for the profile instance to start, and then log in to either VM via the ssh ports specified below. (Note that even though the EXPERIMENTAL data plane interfaces will use the addresses given in the profile, you will still connect over the control plane interfaces using addresses given by the testbed. The data plane addresses are for intra-experiment communication only.) """ import geni.portal as portal import geni.rspec.pg as rspec request = portal.context.makeRequestRSpec() node1 = request.XenVM("node1") iface1 = node1.addInterface("if1") # Specify the component id and the IPv4 address iface1.component_id = "eth1" iface1.addAddress(rspec.IPv4Address("192.168.1.1", "255.255.255.0")) node2 = request.XenVM("node2") iface2 = node2.addInterface("if2") # Specify the component id and the IPv4 address iface2.component_id = "eth2" iface2.addAddress(rspec.IPv4Address("192.168.1.2", "255.255.255.0")) link = request.LAN("lan") link.addInterface(iface1) link.addInterface(iface2) portal.context.printRequestRSpec()

This code sample assigns specific IP addresses to interfaces on the nodes it requests.

Some of the available qualifiers on requested nodes are specified by manipulating attributes within the node (or interface) object directly. The hardware_type in the previous example is one such case, as is the component_id here. (Note that the component_id in this example is applied to an interface, although it is also possible to specify component_ids on nodes, too, to request a particular physical host.)

Other modifications to requests require dedicated methods. For instance, see the addAddress() calls made on each of the two interfaces above. In each case, an IPv4Address object is obtained from the appropriate constructor (the parameters are the address and the netmask, respectively), and then added to the corresponding interface.

6.7 OpenEPC extensions

geni-lib includes convenience functions to assist constructing profiles including OpenEPC core networks. Although it is possible to instantiate OpenEPC using geni-lib primitives only, the geni.rspec.emulab.pnext" module hides some of the OpenEPC details, and allows more concise profile scripts. An example of a profile using geni.rspec.emulab.pnext is given below.

#!/usr/bin/env python import geni.portal as portal import geni.rspec.pg as PG import geni.rspec.igext as IG import geni.rspec.emulab as EL import geni.rspec.emulab.pnext as PN # # Global variables that should remain relatively static # class GLOBALS(object): EPCIMG = PN.PNDEFS.DEF_BINOEPC_IMG VM_COLLOCATE_FACTOR = 10 # Number of VMs to pack onto a phys node. # # Create our in-memory model of the RSpec -- the resources we're going # to request in our experiment, and their configuration. # rspec = PG.Request() # # This geni-lib script is designed to run in the PhantomNet Portal. # pc = portal.Context() # # Describe profile. Will be rendered in Markdown on Portal. # tourDescription = \ """ ###*<center>Use this profile to instantiate a basic EPC topology.</center>* --- The following functions are included: * AAA * HSS * MME * SGW * PGW * eNodeB * UE OpenEPC Release 5 software running on top of Ubuntu 12.04 LTS is used to implement this functionality. Please note that the disk image used does not allow root access, and only the OpenEPC binaries are present. Circumventing root restrictions and/or copying the OpenEPC binaries is prohibited. Such actions constitute a violation of the PhantomNet/OpenEPC sublicense agreement. """ tourInstructions = \ """ This profile makes use of user-supplied parameters. You can use these parameters to tune the number of clients (emulated UEs), request additional emulated eNodeBs, and to choose the hardware used. An advanced parameter allows you to set the default LAN bandwidth. This is a parameterized profile implemented via a [geni-lib](http://geni-lib.readthedocs.org "geni-lib documentation") script. You may make a copy of the script to use in your own profile where you can modify the script to suit your needs. """ # # Setup the Tour info with the above description and instructions. # tour = IG.Tour() tour.Description(IG.Tour.MARKDOWN,tourDescription) tour.Instructions(IG.Tour.MARKDOWN,tourInstructions) rspec.addTour(tour) # # Define some parameters for OpenEPC experiments. # pc.defineParameter("NUMCLI", "Number of clients (UEs)", portal.ParameterType.INTEGER, 1, longDescription="Specify the number of emulated client (User Equipment) resources to allocate. This number must be between 1 and 32 currently.") pc.defineParameter("NUMENB", "Number of eNodeB nodes.", portal.ParameterType.INTEGER, 1, [1,2,3], longDescription="Number of emulated eNodeB (LTE base station) nodes to allocate. May be from 1 to 3 (inclusive).") pc.defineParameter("HWTYPE","Node Hardware Type", portal.ParameterType.STRING, "pcvm", [("pc","Any available (compatible) physical machine type"), ("pc3000","Emulab pc3000 nodes"), ("d710","Emulab d710 nodes"), ("d430","Emulab d430 nodes"), ("pcvm","Any available (compatible) virtual machine type"), ("pc3000vm","Virtual machines on top of pc3000 nodes."), ("d710vm","Virtual machines on top of d710 nodes."), ("d430vm","Virtual machines on top of d430 nodes.")], longDescription="Specify which node resource type to use for OpenEPC nodes. Note that only those types that are compatible with the OpenEPC image(s) are listed.") pc.defineParameter("LINKBW","Default Link Bandwidth (Mbps)", portal.ParameterType.INTEGER, 0, longDescription="Specify the default LAN bandwidth in Mbps for all EPC LANs. Leave at \"0\" to indicate \"best effort\". Values that do not line up with common physical interface speeds (e.g. 10, 100, 1000, 10000) WILL cause the insertion of link shaping elements. Additionally, above 1Gbps, such shaping elements are unlikely to be accurate.", advanced=True) # # Get any input parameter values that will override our defaults. # params = pc.bindParameters() # # Verify parameters and setup errors/warnings to be reported back. # if params.NUMCLI > 32 or params.NUMCLI < 1: perr = portal.ParameterError("You cannot ask for fewer than one or more than 32 client nodes!", ['NUMCLI']) pc.reportError(perr) pass if params.NUMENB < 1 or params.NUMENB > 3: perr = portal.ParameterError("You cannot ask for fewer than one or more than three eNodeB nodes!", ['NUMENB']) pc.reportError(perr) pass if int(params.LINKBW) not in [0, 10, 100, 1000, 10000]: pwarn = portal.ParameterWarning("You are asking for a default link bandwidth that is NOT a standard physical link speed. Link shaping resources WILL be inserted!", ['LINKBW']) pc.reportWarning(pwarn) pass # # Give the library a chance to return nice JSON-formatted exception(s) and/or # warnings; this might sys.exit(). # pc.verifyParameters() # # Scale link bandwidth parameter to kbps # params.LINKBW *= 1000 # # Switch up some settings if VMs were requested. # usevms = 0 if params.HWTYPE.find("vm") >= 0: usevms = 1 PN.EPCNodeFactorySettings.use_vm_nodes = True params.HWTYPE = params.HWTYPE.replace("vm","") rspec.setCollocateFactor(GLOBALS.VM_COLLOCATE_FACTOR) rspec.setPackingStrategy("pack") # # If the generic hardware type "pc" was requested, don't set the # type at all. # if params.HWTYPE == "pc": params.HWTYPE = None # # Force gigabit speed for d430 nodes when no bandwidth is requested. # if params.HWTYPE == "d430": if params.LINKBW == 0: params.LINKBW = 1000 * 1000 # # Set the hardware and image for the epc node factory function # PN.EPCNodeFactorySettings.hardware_type = params.HWTYPE PN.EPCNodeFactorySettings.disk_image = GLOBALS.EPCIMG # # Create the lans we need # mgmt = rspec.EPClan(PN.EPCLANS.MGMT, vmlan = usevms) mgmt.bandwidth = params.LINKBW # Hack for d430 node type ... if params.LINKBW != 0: mgmt.best_effort = False net_a = rspec.EPClan(PN.EPCLANS.NET_A, vmlan = usevms) net_a.bandwidth = params.LINKBW net_b = rspec.EPClan(PN.EPCLANS.NET_B, vmlan = usevms) net_b.bandwidth = params.LINKBW net_d = rspec.EPClan(PN.EPCLANS.NET_D, vmlan = usevms) net_d.bandwidth = params.LINKBW an_lte = rspec.EPClan(PN.EPCLANS.AN_LTE, vmlan = usevms) an_lte.bandwidth = params.LINKBW # # Add the core EPC nodes # # epc-enablers node epcen = PN.mkepcnode("epc", PN.EPCROLES.ENABLERS) epcen.exclusive = True rspec.addResource(epcen) mgmt.addMember(epcen) net_a.addMember(epcen) # pgw node pgw = PN.mkepcnode("pgw", PN.EPCROLES.PGW) pgw.exclusive = True rspec.addResource(pgw) mgmt.addMember(pgw) net_a.addMember(pgw) net_b.addMember(pgw) # sgw-mme-sgsn node sgw = PN.mkepcnode("sgw", PN.EPCROLES.SGW_MME_SGSN) sgw.exclusive = True rspec.addResource(sgw) mgmt.addMember(sgw) net_b.addMember(sgw) net_d.addMember(sgw) # # Create the requested number of eNodeB nodes # for i in range(1, params.NUMENB + 1): ename = "enb%d" % i enb = PN.mkepcnode(ename, PN.EPCROLES.ENODEB, hname = ename) enb.exclusive = True rspec.addResource(enb) mgmt.addMember(enb) net_d.addMember(enb) an_lte.addMember(enb) # # Now pop in the requested number of emulated clients (UEs). # for i in range(1, params.NUMCLI + 1): cname = "client%d" % i client = PN.mkepcnode(cname, PN.EPCROLES.CLIENT, hname = cname) client.exclusive = True rspec.addResource(client) mgmt.addMember(client) an_lte.addMember(client) # # Print and go! # pc.printRequestRSpec(rspec) Open this profile on PhantomNet#!/usr/bin/env python import geni.portal as portal import geni.rspec.pg as PG import geni.rspec.igext as IG import geni.rspec.emulab as EL import geni.rspec.emulab.pnext as PN # # Global variables that should remain relatively static # class GLOBALS(object): EPCIMG = PN.PNDEFS.DEF_BINOEPC_IMG VM_COLLOCATE_FACTOR = 10 # Number of VMs to pack onto a phys node. # # Create our in-memory model of the RSpec -- the resources we're going # to request in our experiment, and their configuration. # rspec = PG.Request() # # This geni-lib script is designed to run in the PhantomNet Portal. # pc = portal.Context() # # Describe profile. Will be rendered in Markdown on Portal. # tourDescription = \ """ ###*<center>Use this profile to instantiate a basic EPC topology.</center>* --- The following functions are included: * AAA * HSS * MME * SGW * PGW * eNodeB * UE OpenEPC Release 5 software running on top of Ubuntu 12.04 LTS is used to implement this functionality. Please note that the disk image used does not allow root access, and only the OpenEPC binaries are present. Circumventing root restrictions and/or copying the OpenEPC binaries is prohibited. Such actions constitute a violation of the PhantomNet/OpenEPC sublicense agreement. """ tourInstructions = \ """ This profile makes use of user-supplied parameters. You can use these parameters to tune the number of clients (emulated UEs), request additional emulated eNodeBs, and to choose the hardware used. An advanced parameter allows you to set the default LAN bandwidth. This is a parameterized profile implemented via a [geni-lib](http://geni-lib.readthedocs.org "geni-lib documentation") script. You may make a copy of the script to use in your own profile where you can modify the script to suit your needs. """ # # Setup the Tour info with the above description and instructions. # tour = IG.Tour() tour.Description(IG.Tour.MARKDOWN,tourDescription) tour.Instructions(IG.Tour.MARKDOWN,tourInstructions) rspec.addTour(tour) # # Define some parameters for OpenEPC experiments. # pc.defineParameter("NUMCLI", "Number of clients (UEs)", portal.ParameterType.INTEGER, 1, longDescription="Specify the number of emulated client (User Equipment) resources to allocate. This number must be between 1 and 32 currently.") pc.defineParameter("NUMENB", "Number of eNodeB nodes.", portal.ParameterType.INTEGER, 1, [1,2,3], longDescription="Number of emulated eNodeB (LTE base station) nodes to allocate. May be from 1 to 3 (inclusive).") pc.defineParameter("HWTYPE","Node Hardware Type", portal.ParameterType.STRING, "pcvm", [("pc","Any available (compatible) physical machine type"), ("pc3000","Emulab pc3000 nodes"), ("d710","Emulab d710 nodes"), ("d430","Emulab d430 nodes"), ("pcvm","Any available (compatible) virtual machine type"), ("pc3000vm","Virtual machines on top of pc3000 nodes."), ("d710vm","Virtual machines on top of d710 nodes."), ("d430vm","Virtual machines on top of d430 nodes.")], longDescription="Specify which node resource type to use for OpenEPC nodes. Note that only those types that are compatible with the OpenEPC image(s) are listed.") pc.defineParameter("LINKBW","Default Link Bandwidth (Mbps)", portal.ParameterType.INTEGER, 0, longDescription="Specify the default LAN bandwidth in Mbps for all EPC LANs. Leave at \"0\" to indicate \"best effort\". Values that do not line up with common physical interface speeds (e.g. 10, 100, 1000, 10000) WILL cause the insertion of link shaping elements. Additionally, above 1Gbps, such shaping elements are unlikely to be accurate.", advanced=True) # # Get any input parameter values that will override our defaults. # params = pc.bindParameters() # # Verify parameters and setup errors/warnings to be reported back. # if params.NUMCLI > 32 or params.NUMCLI < 1: perr = portal.ParameterError("You cannot ask for fewer than one or more than 32 client nodes!", ['NUMCLI']) pc.reportError(perr) pass if params.NUMENB < 1 or params.NUMENB > 3: perr = portal.ParameterError("You cannot ask for fewer than one or more than three eNodeB nodes!", ['NUMENB']) pc.reportError(perr) pass if int(params.LINKBW) not in [0, 10, 100, 1000, 10000]: pwarn = portal.ParameterWarning("You are asking for a default link bandwidth that is NOT a standard physical link speed. Link shaping resources WILL be inserted!", ['LINKBW']) pc.reportWarning(pwarn) pass # # Give the library a chance to return nice JSON-formatted exception(s) and/or # warnings; this might sys.exit(). # pc.verifyParameters() # # Scale link bandwidth parameter to kbps # params.LINKBW *= 1000 # # Switch up some settings if VMs were requested. # usevms = 0 if params.HWTYPE.find("vm") >= 0: usevms = 1 PN.EPCNodeFactorySettings.use_vm_nodes = True params.HWTYPE = params.HWTYPE.replace("vm","") rspec.setCollocateFactor(GLOBALS.VM_COLLOCATE_FACTOR) rspec.setPackingStrategy("pack") # # If the generic hardware type "pc" was requested, don't set the # type at all. # if params.HWTYPE == "pc": params.HWTYPE = None # # Force gigabit speed for d430 nodes when no bandwidth is requested. # if params.HWTYPE == "d430": if params.LINKBW == 0: params.LINKBW = 1000 * 1000 # # Set the hardware and image for the epc node factory function # PN.EPCNodeFactorySettings.hardware_type = params.HWTYPE PN.EPCNodeFactorySettings.disk_image = GLOBALS.EPCIMG # # Create the lans we need # mgmt = rspec.EPClan(PN.EPCLANS.MGMT, vmlan = usevms) mgmt.bandwidth = params.LINKBW # Hack for d430 node type ... if params.LINKBW != 0: mgmt.best_effort = False net_a = rspec.EPClan(PN.EPCLANS.NET_A, vmlan = usevms) net_a.bandwidth = params.LINKBW net_b = rspec.EPClan(PN.EPCLANS.NET_B, vmlan = usevms) net_b.bandwidth = params.LINKBW net_d = rspec.EPClan(PN.EPCLANS.NET_D, vmlan = usevms) net_d.bandwidth = params.LINKBW an_lte = rspec.EPClan(PN.EPCLANS.AN_LTE, vmlan = usevms) an_lte.bandwidth = params.LINKBW # # Add the core EPC nodes # # epc-enablers node epcen = PN.mkepcnode("epc", PN.EPCROLES.ENABLERS) epcen.exclusive = True rspec.addResource(epcen) mgmt.addMember(epcen) net_a.addMember(epcen) # pgw node pgw = PN.mkepcnode("pgw", PN.EPCROLES.PGW) pgw.exclusive = True rspec.addResource(pgw) mgmt.addMember(pgw) net_a.addMember(pgw) net_b.addMember(pgw) # sgw-mme-sgsn node sgw = PN.mkepcnode("sgw", PN.EPCROLES.SGW_MME_SGSN) sgw.exclusive = True rspec.addResource(sgw) mgmt.addMember(sgw) net_b.addMember(sgw) net_d.addMember(sgw) # # Create the requested number of eNodeB nodes # for i in range(1, params.NUMENB + 1): ename = "enb%d" % i enb = PN.mkepcnode(ename, PN.EPCROLES.ENODEB, hname = ename) enb.exclusive = True rspec.addResource(enb) mgmt.addMember(enb) net_d.addMember(enb) an_lte.addMember(enb) # # Now pop in the requested number of emulated clients (UEs). # for i in range(1, params.NUMCLI + 1): cname = "client%d" % i client = PN.mkepcnode(cname, PN.EPCROLES.CLIENT, hname = cname) client.exclusive = True rspec.addResource(client) mgmt.addMember(client) an_lte.addMember(client) # # Print and go! # pc.printRequestRSpec(rspec)

While the geni.portal and geni.rspec.pg modules will seem familiar from their use in general geni-lib profiles, the pnext module is new, and provides two main classes: EPCNode and EPClan. These convenience classes provide facilities for adding OpenEPC nodes and networks, respectively. Both classes are also added as extensions to Request, so that (assuming rspec is a valid Request object) a simple call like mgmt = rspec.EPClan( PN.EPCLANS.MGMT ) will both create a LAN and add it to the request.

For the EPCNode class only, a factory method mkepcnode is also defined. (The advantage of using the factory method instead of invoking the EPCNode constructor directly is that it allows specifying default hardware types and disk images to be used by all EPC nodes in the profile. By default, mkepcnode will use a binary OpenEPC disk image.)

The mkepcnode call has two mandatory parameters: the node identifier (just as seen in previous generic geni-lib examples), and a new role, specific to OpenEPC nodes. The role must be chosen from the following, each defined within geni.rspec.pnext.EPCROLES:

Please consult the OpenEPC documentation for details about specific OpenEPC nodes.

Once nodes are assigned, they should be connected with appropriate core network links. This is the job of the EPClan class, and each LAN should be created by invoking the EPClan constructor with a single mandatory parameter chosen from the following list of network identifiers:

The OpenEPC Tutorial gives an overview of the purpose and topology of each network. Once the LAN has been created, its addMember method may be invoked to describe the nodes it should connect.

6.8 RF communication

PhantomNet provides facilities for radio frequency links well suited for wireless protocols, and geni-lib support is provided for requesting them in the form of the pnext module RFLink class. RFLink behaves much like standard LANs, except that all connections are made point-to-point (and so any RFLink must connect exactly two interfaces).

#!/usr/bin/python """ An example of a profile including RF links. Instructions: This is an example profile to demonstrate the description of RF links. It is not particularly useful to instantiate directly. """ import geni.portal as portal import geni.rspec.pg as rspec import geni.rspec.emulab.pnext as pn request = portal.context.makeRequestRSpec() node0 = request.RawPC( "node0" ) iface0 = node0.addInterface( "rf0" ) node1 = request.RawPC( "node1" ) iface1 = node1.addInterface( "rf1" ) rflink = request.RFLink( "rflink" ) rflink.addInterface( iface0 ) rflink.addInterface( iface1 ) portal.context.printRequestRSpec() Open this profile on PhantomNet#!/usr/bin/python """ An example of a profile including RF links. Instructions: This is an example profile to demonstrate the description of RF links. It is not particularly useful to instantiate directly. """ import geni.portal as portal import geni.rspec.pg as rspec import geni.rspec.emulab.pnext as pn request = portal.context.makeRequestRSpec() node0 = request.RawPC( "node0" ) iface0 = node0.addInterface( "rf0" ) node1 = request.RawPC( "node1" ) iface1 = node1.addInterface( "rf1" ) rflink = request.RFLink( "rflink" ) rflink.addInterface( iface0 ) rflink.addInterface( iface1 ) portal.context.printRequestRSpec()

6.9 Specify an operating system and set install and execute scripts

"""An example of constructing a profile with install and execute services. Instructions: Wait for the profile instance to start, then click on the node in the topology and choose the `shell` menu item. The install and execute services are handled automatically during profile instantiation, with no manual intervention required. """ # Import the Portal object. import geni.portal as portal # Import the ProtoGENI library. import geni.rspec.pg as rspec # Create a Request object to start building the RSpec. request = portal.context.makeRequestRSpec() # Add a raw PC to the request. node = request.RawPC("node") # Install and execute scripts on the node. THIS TAR FILE DOES NOT ACTUALLY EXIST! node.addService(rspec.Install(url="http://example.org/sample.tar.gz", path="/local")) node.addService(rspec.Execute(shell="bash", command="/local/example.sh")) portal.context.printRequestRSpec()Open this profile on PhantomNet"""An example of constructing a profile with install and execute services. Instructions: Wait for the profile instance to start, then click on the node in the topology and choose the `shell` menu item. The install and execute services are handled automatically during profile instantiation, with no manual intervention required. """ # Import the Portal object. import geni.portal as portal # Import the ProtoGENI library. import geni.rspec.pg as rspec # Create a Request object to start building the RSpec. request = portal.context.makeRequestRSpec() # Add a raw PC to the request. node = request.RawPC("node") # Install and execute scripts on the node. THIS TAR FILE DOES NOT ACTUALLY EXIST! node.addService(rspec.Install(url="http://example.org/sample.tar.gz", path="/local")) node.addService(rspec.Execute(shell="bash", command="/local/example.sh")) portal.context.printRequestRSpec()

This example demonstrates how to request services for a node, where PhantomNet will automate some task as part of the profile instance setup procedure. In this case, two services are described (an install and an execute). This is a very common pair of services to request together: the Install object describes a service which retrieves a tarball from the location given in the url parameter, and installs it into the local filesystem as specified by path. (The installation occurs during node setup, upon the first boot after the disk image has been loaded.) The second service, described by the Execute object, invokes a shell process to run the given command. In this example (as is common), the command refers directly to a file saved by the immediately preceding Install service. This behaviour works, because PhantomNet guarantees that all Install services complete before any Execute services are started. The command executes every time the node boots, so you can use it start daemons, etc. that are necessary for your experiment.

6.10 Profiles with user-specified parameters

"""An example of using parameters to construct a profile with a variable number of nodes. Instructions: Wait for the profile instance to start, and then log in to one or more of the VMs via the ssh port(s) specified below. """ # Import the Portal object. import geni.portal as portal # Import the ProtoGENI library. import geni.rspec.pg as rspec # Describe the parameter(s) this profile script can accept. portal.context.defineParameter( "n", "Number of VMs", portal.ParameterType.INTEGER, 1 ) # Retrieve the values the user specifies during instantiation. params = portal.context.bindParameters() # Create a Request object to start building the RSpec. request = portal.context.makeRequestRSpec() # Check parameter validity. if params.n < 1 or params.n > 8: portal.context.reportError( portal.ParameterError( "You must choose at least 1 and no more than 8 VMs." ) ) for i in range( params.n ): # Create a XenVM and add it to the RSpec. node = request.XenVM( "node" + str( i ) ) # Print the RSpec to the enclosing page. portal.context.printRequestRSpec() Open this profile on PhantomNet"""An example of using parameters to construct a profile with a variable number of nodes. Instructions: Wait for the profile instance to start, and then log in to one or more of the VMs via the ssh port(s) specified below. """ # Import the Portal object. import geni.portal as portal # Import the ProtoGENI library. import geni.rspec.pg as rspec # Describe the parameter(s) this profile script can accept. portal.context.defineParameter( "n", "Number of VMs", portal.ParameterType.INTEGER, 1 ) # Retrieve the values the user specifies during instantiation. params = portal.context.bindParameters() # Create a Request object to start building the RSpec. request = portal.context.makeRequestRSpec() # Check parameter validity. if params.n < 1 or params.n > 8: portal.context.reportError( portal.ParameterError( "You must choose at least 1 and no more than 8 VMs." ) ) for i in range( params.n ): # Create a XenVM and add it to the RSpec. node = request.XenVM( "node" + str( i ) ) # Print the RSpec to the enclosing page. portal.context.printRequestRSpec()

Until now, all of the geni-lib scripts have described profiles which could also have been generated with the Jacks GUI, or even by writing a raw XML RSpec directly. However, geni-lib profiles offer an important feature unavailable by the other methods: the ability to describe not a static request, but a request “template” which is dynamically constructed based on a user’s choices at the time the profile is instantiated. The mechanism for constructing such profiles relies on profile parameters; the geni-lib script describes the set of parameters it will accept, and then retrieves the corresponding values at instantiation time and is free to respond by constructing arbitrarily different resource requests based on that input.

The profile above accepts exactly one parameter—the number of VMs it will instantiate. You can see that the parameter is described via the portal portal.context object, using the defineParameter() call shown for the first time in this example. defineParameter() must be invoked once per profile parameter, and requires the parameter symbol, parameter description, type, and default value respectively. The parameter symbol ("n" in this example) must be unique within the profile, and is used to retrieve the parameter’s value during script execution. The description ("Number of VMs", in this case) will be shown to prompt the user to supply a corresponding value when the the profile is instantiated. The type is used partly to constrain the parameters to valid values, and partly to assist the instantiating user by suggesting appropriate choices. The list of valid types is:

The last field is the default value of the parameter, and is required: not only must the field itself contain a valid value, but the set of all parameters must be valid when each of them assumes the default value. (This is partly so that the portal can construct a default topology for the profile without any manual intervention, and partly so that unprivileged users, who may lack permission to supply their own values, might still be able to instantiate the profile.)

After all parameters have been defined, the profile script may retrieve the runtime values with the bindParameters() method. This will return a Python class instance with one attribute for each parameter (with the name supplied during the appropriate defineParameter() call). In the example, the instance was assigned to params, and therefore the only parameter (which was called "n") is accessible as params.n.

Of course, it may be possible for the user to specify nonsensical values for a parameter, or perhaps give a set of parameters whose combination is invalid. A profile should detect error cases like these, and respond by constructing a portal.ParameterError object, which can be passed to the portal context’s reportError() method to abort generation of the RSpec.

6.11 Add temporary local disk space to a node

"""This profile demonstrates how to add some extra *local* disk space on your node. In general nodes have much more disk space then what you see with `df` when you log in. That extra space is in unallocated partitions or additional disk drives. An *ephemeral blockstore* is how you ask for some of that space to be allocated and mounted as a **temporary** filesystem (temporary means it will be lost when you terminate your experiment). Instructions: Log into your node, your **temporary** file system in mounted at `/mydata`. """ # Import the Portal object. import geni.portal as portal # Import the ProtoGENI library. import geni.rspec.pg as rspec # Import the emulab extensions library. import geni.rspec.emulab # Create a Request object to start building the RSpec. request = portal.context.makeRequestRSpec() # Allocate a node and ask for a 30GB file system mounted at /mydata node = request.RawPC("node") bs = node.Blockstore("bs", "/mydata") bs.size = "30GB" # Print the RSpec to the enclosing page. portal.context.printRequestRSpec() Open this profile on PhantomNet"""This profile demonstrates how to add some extra *local* disk space on your node. In general nodes have much more disk space then what you see with `df` when you log in. That extra space is in unallocated partitions or additional disk drives. An *ephemeral blockstore* is how you ask for some of that space to be allocated and mounted as a **temporary** filesystem (temporary means it will be lost when you terminate your experiment). Instructions: Log into your node, your **temporary** file system in mounted at `/mydata`. """ # Import the Portal object. import geni.portal as portal # Import the ProtoGENI library. import geni.rspec.pg as rspec # Import the emulab extensions library. import geni.rspec.emulab # Create a Request object to start building the RSpec. request = portal.context.makeRequestRSpec() # Allocate a node and ask for a 30GB file system mounted at /mydata node = request.RawPC("node") bs = node.Blockstore("bs", "/mydata") bs.size = "30GB" # Print the RSpec to the enclosing page. portal.context.printRequestRSpec()

This example demonstrates how to request extra temporary diskspace on a node. The extra disk space is allocated from unused disk partitions, and is mounted at a directory of your choosing. In the example code above, we are asking for a 30GB file system mounted at "/mydata". Anything you store in this file system is temporary, and will be lost when your experiment is terminated.

The total size of the file systems you can ask for on a node, is obviously limited to the amount of unused disk space available. The system does its best to find nodes with enough space to fulfill the request, but in general you are limited to temporary file systems in the 10s of, or a few hundred GB.

6.12 Creating a reusable dataset

"""This profile demonstrates how to use a remote dataset on your node, either a long term dataset or a short term dataset, created via the Portal. Instructions: Log into your node, your dataset file system in mounted at `/mydata`. """ # Import the Portal object. import geni.portal as portal # Import the ProtoGENI library. import geni.rspec.pg as rspec # Import the emulab extensions library. import geni.rspec.emulab # Create a Request object to start building the RSpec. request = portal.context.makeRequestRSpec() # Add a node to the request. node = request.RawPC("node") # We need a link to talk to the remote file system, so make an interface. iface = node.addInterface() # The remote file system is represented by special node. fsnode = request.RemoteBlockstore("fsnode", "/mydata") # This URN is displayed in the web interfaace for your dataset. fsnode.dataset = "urn:publicid:IDN+emulab.net:portalprofiles+ltdataset+DemoDataset" # # The "rwclone" attribute allows you to map a writable copy of the # indicated SAN-based dataset. In this way, multiple nodes can map # the same dataset simultaneously. In many situations, this is more # useful than a "readonly" mapping. For example, a dataset # containing a Linux source tree could be mapped into multiple # nodes, each of which could do its own independent, # non-conflicting configure and build in their respective copies. # Currently, rwclones are "ephemeral" in that any changes made are # lost when the experiment mapping the clone is terminated. # #fsnode.rwclone = True # # The "readonly" attribute, like the rwclone attribute, allows you to # map a dataset onto multiple nodes simultaneously. But with readonly, # those mappings will only allow read access (duh!) and any filesystem # (/mydata in this example) will thus be mounted read-only. Currently, # readonly mappings are implemented as clones that are exported # allowing just read access, so there are minimal efficiency reasons to # use a readonly mapping rather than a clone. The main reason to use a # readonly mapping is to avoid a situation in which you forget that # changes to a clone dataset are ephemeral, and then lose some # important changes when you terminate the experiment. # #fsnode.readonly = True # Now we add the link between the node and the special node fslink = request.Link("fslink") fslink.addInterface(iface) fslink.addInterface(fsnode.interface) # Special attributes for this link that we must use. fslink.best_effort = True fslink.vlan_tagging = True # Print the RSpec to the enclosing page. portal.context.printRequestRSpec()Open this profile on PhantomNet"""This profile demonstrates how to use a remote dataset on your node, either a long term dataset or a short term dataset, created via the Portal. Instructions: Log into your node, your dataset file system in mounted at `/mydata`. """ # Import the Portal object. import geni.portal as portal # Import the ProtoGENI library. import geni.rspec.pg as rspec # Import the emulab extensions library. import geni.rspec.emulab # Create a Request object to start building the RSpec. request = portal.context.makeRequestRSpec() # Add a node to the request. node = request.RawPC("node") # We need a link to talk to the remote file system, so make an interface. iface = node.addInterface() # The remote file system is represented by special node. fsnode = request.RemoteBlockstore("fsnode", "/mydata") # This URN is displayed in the web interfaace for your dataset. fsnode.dataset = "urn:publicid:IDN+emulab.net:portalprofiles+ltdataset+DemoDataset" # # The "rwclone" attribute allows you to map a writable copy of the # indicated SAN-based dataset. In this way, multiple nodes can map # the same dataset simultaneously. In many situations, this is more # useful than a "readonly" mapping. For example, a dataset # containing a Linux source tree could be mapped into multiple # nodes, each of which could do its own independent, # non-conflicting configure and build in their respective copies. # Currently, rwclones are "ephemeral" in that any changes made are # lost when the experiment mapping the clone is terminated. # #fsnode.rwclone = True # # The "readonly" attribute, like the rwclone attribute, allows you to # map a dataset onto multiple nodes simultaneously. But with readonly, # those mappings will only allow read access (duh!) and any filesystem # (/mydata in this example) will thus be mounted read-only. Currently, # readonly mappings are implemented as clones that are exported # allowing just read access, so there are minimal efficiency reasons to # use a readonly mapping rather than a clone. The main reason to use a # readonly mapping is to avoid a situation in which you forget that # changes to a clone dataset are ephemeral, and then lose some # important changes when you terminate the experiment. # #fsnode.readonly = True # Now we add the link between the node and the special node fslink = request.Link("fslink") fslink.addInterface(iface) fslink.addInterface(fsnode.interface) # Special attributes for this link that we must use. fslink.best_effort = True fslink.vlan_tagging = True # Print the RSpec to the enclosing page. portal.context.printRequestRSpec()

In this example, we demonstrate how to create and use a dataset. A dataset is simply a snapshot of a temporary file system (see the previous example) that has been saved to permanent storage, and reloaded on a node (or nodes) in a different experiment. This type of dataset must be explicitly saved (more on this below) in order to make changes permanent (and available later). In the example code above, the temporary file system will be loaded with the dataset specified by the URN.

But before you can use a dataset, you first have to create one using the following steps:

1. Create an experiment
   
   Create an experiment using the local diskspace example above.

2. Add your data
   
   Populate the file system mounted at /mydata with the data you wish to use in other experiments.

3. Fill out the Create Dataset form
   
   Click on the "Create Dataset" option in the Actions menu. This will bring up the form to create a new dataset. Choose a name for your dataset and optionally the project the dataset should be associated with. Be sure to select Image Backed for the type. Then choose the the experiment, which node in the experiment, and which blockstore on the node.
   
   

4. Click “Create
   
   When you click the “Create” button, the file system on your node will be unmounted so that we can take a consistent snapshot of the contents. This process can take several minutes or longer, depending on the size of the file system. You can watch the progress on this page. When the progress bar reaches the “Ready” stage, your new dataset is ready! It will now show up in your “List Datasets” list, and can be used in new experiments.
   
   

5. Use your dataset
   
   To use your new dataset, you will need to reference it in your geni lib script (see the example code above). The name of your dataset is a URN, and can be found on the information page for the dataset. From the Actions menu, click on "List Datasets", find the name of your dataset in the list, and click on it.
   
   

6. Update your dataset
   
   If you need to make changes to your dataset, simply start an experiment that uses your dataset. Make the changes you need to the file system mounted at /mydata, and then use the "Modify" button as shown in the previous step.

6.13 Debugging geni-lib profile scripts

It is not necessary to instantiate the profile via the portal web interface to test it. Properly written profile scripts should work perfectly well independent of the normal portal—the same geni-lib objects will behave sensibly when invoked from the command line. As long as geni-lib is installed, then invoking the Python interpreter on the profile script should simply write the corresponding RSpec to standard output. (Parameters, if any, will assume their default values.) For instance, if the script in the previous example is saved as geni-lib-parameters.py, then the command:

python geni-lib-parameters.py

will produce an RSpec containing three nodes (the default value for n). It is also possible to override the defaults on the command line by giving the parameter name as an option, followed by the desired value:

python geni-lib-parameters.py –n 4

The option –help will list the available parameters and their descriptions.