IPDB guide¶
Warning
The IPDB module has design issues that may not be fixed. It is recommended to switch to NDB wherever it's possible.
Basically, IPDB is a transactional database, containing records, that represent network stack objects. Any change in the database is not reflected immediately in OS, but waits until commit() is called. One failed operation during commit() rolls back all the changes, has been made so far. Moreover, IPDB has commit hooks API, that allows you to roll back changes depending on your own function calls, e.g. when a host or a network becomes unreachable.
Limitations¶
One of the major issues with IPDB is its memory footprint. It proved not to be suitable for environments with thousands of routes or neighbours. Being a design issue, it could not be fixed, so a new module was started, NDB, that aims to replace IPDB. IPDB is still more feature rich, but NDB is already more fast and stable.
IPDB, NDB, IPRoute¶
These modules use different approaches.
IPRoute just forwards requests to the kernel, and doesn't wait for the system state. So it's up to developer to check, whether the requested object is really set up or not.
IPDB is an asynchronously updated database, that starts several additional threads by default. If your project's policy doesn't allow implicit threads, keep it in mind. But unlike IPRoute, the IPDB ensures the changes to be reflected in the system.
NDB is like IPDB, and will obsolete it in the future. The difference is that IPDB creates Python object for every RTNL object, while NDB stores everything in an SQL DB, and creates objects on demand.
Being asynchronously updated, IPDB does sync on commit:
with IPDB() as ipdb:
with ipdb.interfaces['eth0'] as i:
i.up()
i.add_ip('192.168.0.2/24')
i.add_ip('192.168.0.3/24')
# ---> <--- here you can expect `eth0` is up
# and has these two addresses, so
# the following code can rely on that
NB: In the example above `commit()` is implied with the `__exit__()` of the `with` statement.
IPDB and other software¶
IPDB is designed to be a non-exclusive network settings database. There may be several IPDB instances on the same OS, as well as other network management software, such as NetworkManager etc.
The IPDB transactions should not interfere with other software settings, unless they touch the same objects. E.g., if IPDB brings an interface up, while NM shuts it down, there will be a race condition.
An example:
# IPDB code # NetworkManager at the same time:
ipdb.interfaces['eth0'].up() #
ipdb.interfaces['eth0'].commit() # $ sudo nmcli con down eth0
# ---> <---
# The eth0 state here is undefined. Some of the commands
# above will fail
But as long as the software doesn't touch the same objects, there will be no conflicts. Another example:
# IPDB code # At the same time, NetworkManager
with ipdb.interfaces['eth0'] as i: # adds addresses:
i.add_ip('172.16.254.2/24') # * 10.0.0.2/24
i.add_ip('172.16.254.3/24') # * 10.0.0.3/24
# ---> <---
# At this point the eth0 interface will have all four addresses.
# If the IPDB transaction fails by some reason, only IPDB addresses
# will be rolled back.
There may be a need to prevent other software from changing the network settings. There is no locking at the kernel level, but IPDB can revert all the changes as soon as they appear on the interface:
# IPDB code
ipdb.interfaces['eth0'].freeze()
# Here some other software tries to
# add an address, or to remove the old
# one
# ---> <---
# At this point the eth0 interface will have all the same settings as
# at the `freeze()` call moment. Newly added addresses will be removed,
# all the deleted addresses will be restored.
#
# Please notice, that an address removal may cause also a routes removal,
# and that is the thing that IPDB can not neither prevent, nor revert.
ipdb.interfaces['eth0'].unfreeze()
Quickstart¶
Simple tutorial:
from pyroute2 import IPDB
# several IPDB instances are supported within on process
ipdb = IPDB()
# commit is called automatically upon the exit from `with`
# statement
with ipdb.interfaces.eth0 as i:
i.address = '00:11:22:33:44:55'
i.ifname = 'bala'
i.txqlen = 2000
# basic routing support
ipdb.routes.add({'dst': 'default',
'gateway': '10.0.0.1'}).commit()
# do not forget to shutdown IPDB
ipdb.release()
Please, notice ip.release() call in the end. Though it is not forced in an interactive python session for the better user experience, it is required in the scripts to sync the IPDB state before exit.
IPDB supports functional-like syntax also:
from pyroute2 import IPDB
with IPDB() as ipdb:
intf = (ipdb.interfaces['eth0']
.add_ip('10.0.0.2/24')
.add_ip('10.0.0.3/24')
.set_address('00:11:22:33:44:55')
.set_mtu(1460)
.set_name('external')
.commit())
# ---> <--- here you have the interface reference with
# all the changes applied: renamed, added ipaddr,
# changed macaddr and mtu.
... # some code
# pls notice, that the interface reference will not work
# outside of `with IPDB() ...`
Transaction modes¶
IPDB has several operating modes:
- 'implicit' (default) -- the first change starts an implicit
transaction, that have to be committed
- 'explicit' -- you have to begin() a transaction prior to
make any change
The default is to use implicit transaction. This behaviour can be changed in the future, so use 'mode' argument when creating IPDB instances.
The sample session with explicit transactions:
In [1]: from pyroute2 import IPDB
In [2]: ip = IPDB(mode='explicit')
In [3]: ifdb = ip.interfaces
In [4]: ifdb.tap0.begin()
Out[3]: UUID('7a637a44-8935-4395-b5e7-0ce40d31d937')
In [5]: ifdb.tap0.up()
In [6]: ifdb.tap0.address = '00:11:22:33:44:55'
In [7]: ifdb.tap0.add_ip('10.0.0.1', 24)
In [8]: ifdb.tap0.add_ip('10.0.0.2', 24)
In [9]: ifdb.tap0.review()
Out[8]:
{'+ipaddr': set([('10.0.0.2', 24), ('10.0.0.1', 24)]),
'-ipaddr': set([]),
'address': '00:11:22:33:44:55',
'flags': 4099}
In [10]: ifdb.tap0.commit()
Note, that you can review() the current_tx transaction, and commit() or drop() it. Also, multiple transactions are supported, use uuid returned by begin() to identify them.
Actually, the form like 'ip.tap0.address' is an eye-candy. The IPDB objects are dictionaries, so you can write the code above as that:
ipdb.interfaces['tap0'].down()
ipdb.interfaces['tap0']['address'] = '00:11:22:33:44:55'
...
Context managers¶
Transactional objects (interfaces, routes) can act as context managers in the same way as IPDB does itself:
with ipdb.interfaces.tap0 as i:
i.address = '00:11:22:33:44:55'
i.ifname = 'vpn'
i.add_ip('10.0.0.1', 24)
i.add_ip('10.0.0.1', 24)
On exit, the context manager will automatically commit() the transaction.
Read-only interface views¶
Using an interface as a context manager will start a transaction. Sometimes it is not what one needs. To avoid unnecessary transactions, and to avoid the risk to occasionally change interface attributes, one can use read-only views:
with ipdb.interfaces[1].ro as iface:
print(iface.ifname)
print(iface.address)
The .ro view neither starts transactions, nor allows to change anything, raising the RuntimeError exception.
The same read-only views are available for routes and rules.
Create interfaces¶
IPDB can also create virtual interfaces:
with ipdb.create(kind='bridge', ifname='control') as i:
i.add_port(ip.interfaces.eth1)
i.add_port(ip.interfaces.eth2)
i.add_ip('10.0.0.1/24')
The IPDB.create() call has the same syntax as IPRoute.link('add', ...), except you shouldn't specify the 'add' command. Refer to IPRoute docs for details.
Please notice, that the interface object stays in the database even if there was an error during the interface creation. It is done so to make it possible to fix the interface object and try to run commit() again. Or you can drop the interface object with the .remove().commit() call.
IP address management¶
IP addresses on interfaces may be managed using add_ip() and del_ip():
with ipdb.interfaces['eth0'] as eth:
eth.add_ip('10.0.0.1/24')
eth.add_ip('10.0.0.2/24')
eth.add_ip('2001:4c8:1023:108::39/64')
eth.del_ip('172.16.12.5/24')
The address format may be either a string with 'address/mask' notation, or a pair of 'address', mask:
with ipdb.interfaces['eth0'] as eth:
eth.add_ip('10.0.0.1', 24)
eth.del_ip('172.16.12.5', 24)
The ipaddr attribute contains all the IP addresses of the interface, which are accessible in different ways. Getting an iterator from ipaddr gives you a sequence of tuples ('address', mask):
>>> for addr in ipdb.interfaces['eth0'].ipaddr:
... print(ipaddr)
...
('10.0.0.2', 24)
('10.0.0.1', 24)
Getting one IP from ipaddr returns a dict object with full spec:
>>> ipdb.interfaces['eth0'].ipaddr[0]
{'family': 2,
'broadcast': None,
'flags': 128,
'address': '10.0.0.2',
'prefixlen': 24,
'local': '10.0.0.2'}
>>> ipdb.intefaces['eth0'].ipaddr['10.0.0.2/24']
{'family': 2,
'broadcast': None,
'flags': 128,
'address': '10.0.0.2',
'prefixlen': 24,
'local': '10.0.0.2'}
The API is a bit weird, but it's because of historical reasons. In the future it may be changed.
Another feature of the ipaddr attribute is views:
>>> ipdb.interfaces['eth0'].ipaddr.ipv4:
(('10.0.0.2', 24), ('10.0.0.1', 24))
>>> ipdb.interfaces['eth0'].ipaddr.ipv6:
(('2001:4c8:1023:108::39', 64),)
The views, as well as the ipaddr attribute itself are not supposed to be changed by user, but only by the internal API.
Bridge interfaces¶
Modern kernels provide possibility to manage bridge interface properties such as STP, forward delay, ageing time etc. Names of these properties start with br_, like br_ageing_time, br_forward_delay e.g.:
[x for x in dir(ipdb.interfaces.virbr0) if x.startswith('br_')]
Bridge ports¶
IPDB supports specific bridge port parameters, such as proxyarp, unicast/multicast flood, cost etc.:
with ipdb.interfaces['br-port0'] as p:
p.brport_cost = 200
p.brport_unicast_flood = 0
p.brport_proxyarp = 0
Ports management¶
IPDB provides a uniform API to manage bridge, bond and vrf ports:
with ipdb.interfaces['br-int'] as br:
br.add_port('veth0')
br.add_port(ipdb.interfaces.veth1)
br.add_port(700)
br.del_port('veth2')
Both add_port() and del_port() accept three types of arguments:
'veth0' -- interface name as a string
ipdb.interfaces.veth1 -- IPDB interface object
700 -- interface index, an integer
Routes management¶
IPDB has a simple yet useful routing management interface.
Create a route¶
To add a route, there is an easy to use syntax:
# spec as a dictionary
spec = {'dst': '172.16.1.0/24',
'oif': 4,
'gateway': '192.168.122.60',
'metrics': {'mtu': 1400,
'advmss': 500}}
# pass spec as is
ipdb.routes.add(spec).commit()
# pass spec as kwargs
ipdb.routes.add(**spec).commit()
# use keyword arguments explicitly
ipdb.routes.add(dst='172.16.1.0/24', oif=4, ...).commit()
Please notice, that the device can be specified with oif (output interface) or iif (input interface), the device keyword is not supported anymore.
More examples:
# specify table and priority
(ipdb.routes
.add(dst='172.16.1.0/24',
gateway='192.168.0.1',
table=100,
priority=10)
.commit())
The priority field is what the iproute2 utility calls metric -- see also below.
Get a route¶
To access and change the routes, one can use notations as follows:
# default table (254)
#
# change the route gateway and mtu
#
with ipdb.routes['172.16.1.0/24'] as route:
route.gateway = '192.168.122.60'
route.metrics.mtu = 1500
# access the default route
print(ipdb.routes['default'])
# change the default gateway
with ipdb.routes['default'] as route:
route.gateway = '10.0.0.1'
By default, the path ipdb.routes reflects only the main routing table (254). But Linux supports much more routing tables, so does IPDB:
In [1]: ipdb.routes.tables.keys()
Out[1]: [0, 254, 255]
In [2]: len(ipdb.routes.tables[255])
Out[2]: 11 # => 11 automatic routes in the table local
It is important to understand, that routing tables keys in IPDB are not only the destination prefix. The key consists of 'prefix/mask' string and the route priority (if any):
In [1]: ipdb.routes.tables[254].idx.keys()
Out[1]:
[RouteKey(dst='default', table=254, family=2, ...),
RouteKey(dst='172.17.0.0/16', table=254, ...),
RouteKey(dst='172.16.254.0/24', table=254, ...),
RouteKey(dst='192.168.122.0/24', table=254, ...),
RouteKey(dst='fe80::/64', table=254, family=10, ...)]
But a routing table in IPDB allows several variants of the route spec. The simplest case is to retrieve a route by prefix, if there is only one match:
# get route by prefix
ipdb.routes['172.16.1.0/24']
# get route by a special name
ipdb.routes['default']
If there are more than one route that matches the spec, only the first one will be retrieved. One should iterate all the records and filter by a key to retrieve all matches:
# only one route will be retrieved
ipdb.routes['fe80::/64']
# get all routes by this prefix
[ x for x in ipdb.routes if x['dst'] == 'fe80::/64' ]
It is also possible to use dicts as specs:
# get IPv4 default route
ipdb.routes[{'dst': 'default', 'family': AF_INET}]
# get IPv6 default route
ipdb.routes[{'dst': 'default', 'family': AF_INET6}]
# get route by priority
ipdb.routes.table[100][{'dst': '10.0.0.0/24', 'priority': 10}]
While this notation returns one route, there is a method to get all the routes matching the spec:
# get all the routes from all the tables via some interface
ipdb.routes.filter({'oif': idx})
# get all IPv6 routes from some table
ipdb.routes.table[tnum].filter({'family': AF_INET6})
Route metrics¶
A special object is dedicated to route metrics, one can access it via route.metrics or route['metrics']:
# these two statements are equal:
with ipdb.routes['172.16.1.0/24'] as route:
route['metrics']['mtu'] = 1400
with ipdb.routes['172.16.1.0/24'] as route:
route.metrics.mtu = 1400
Possible metrics are defined in rtmsg.py:rtmsg.metrics, e.g. RTAX_HOPLIMIT means hoplimit metric etc.
Multipath routing¶
Multipath nexthops are managed via route.add_nh() and route.del_nh() methods. They are available to review via route.multipath, but one should not directly add/remove/modify nexthops in route.multipath, as the changes will not be committed correctly.
To create a multipath route:
ipdb.routes.add({'dst': '172.16.232.0/24',
'multipath': [{'gateway': '172.16.231.2',
'hops': 2},
{'gateway': '172.16.231.3',
'hops': 1},
{'gateway': '172.16.231.4'}]}).commit()
To change a multipath route:
with ipdb.routes['172.16.232.0/24'] as r:
r.add_nh({'gateway': '172.16.231.5'})
r.del_nh({'gateway': '172.16.231.4'})
Another possible way is to create a normal route and turn it into multipath by add_nh():
# create a non-MP route with one gateway:
(ipdb
.routes
.add({'dst': '172.16.232.0/24',
'gateway': '172.16.231.2'})
.commit())
# turn it to become a MP route:
(ipdb
.routes['172.16.232.0/24']
.add_nh({'gateway': '172.16.231.3'})
.commit())
# here the route will contain two NH records, with
# gateways 172.16.231.2 and 172.16.231.3
# remove one NH and turn the route to be a normal one
(ipdb
.routes['172.16.232.0/24']
.del_nh({'gateway': '172.16.231.2'})
.commit())
# thereafter the traffic to 172.16.232.0/24 will go only
# via 172.16.231.3
Differences from the iproute2 syntax¶
By historical reasons, iproute2 uses names that differs from what the kernel uses. E.g., iproute2 uses weight for multipath route hops instead of hops, where weight == (hops + 1). Thus, a route created with hops == 2 will be listed by iproute2 as weight 3.
Another significant difference is metrics. The pyroute2 library uses the kernel naming scheme, where metrics means mtu, rtt, window etc. The iproute2 utility uses metric (not metrics) as a name for the priority field.
In examples:
# -------------------------------------------------------
# iproute2 command:
$ ip route add default \
nexthop via 172.16.0.1 weight 2 \
nexthop via 172.16.0.2 weight 9
# pyroute2 code:
(ipdb
.routes
.add({'dst': 'default',
'multipath': [{'gateway': '172.16.0.1', 'hops': 1},
{'gateway': '172.16.0.2', 'hops': 8}])
.commit())
# -------------------------------------------------------
# iproute2 command:
$ ip route add default via 172.16.0.2 metric 200
# pyroute2 code:
(ipdb
.routes
.add({'dst': 'default',
'gateway': '172.16.0.2',
'priority': 200})
.commit())
# -------------------------------------------------------
# iproute2 command:
$ ip route add default via 172.16.0.2 mtu 1460
# pyroute2 code:
(ipdb
.routes
.add({'dst': 'default',
'gateway': '172.16.0.2',
'metrics': {'mtu': 1460}})
.commit())
Multipath default routes¶
Warning
As of the merge of kill_rtcache into the kernel, and it's release in ~3.6, weighted default routes no longer work in Linux.
Please refer to https://github.com/svinota/pyroute2/issues/171#issuecomment-149297244 for details.
Rules management¶
IPDB provides a basic IP rules management system.
Create a rule¶
Syntax is almost the same as for routes:
# rule spec
spec = {'src': '172.16.1.0/24',
'table': 200,
'priority': 15000}
ipdb.rules.add(spec).commit()
Get a rule¶
The way IPDB handles IP rules is almost the same as routes, but rule keys are more complicated -- the Linux kernel doesn't use keys for rules, but instead iterates all the records until the first one w/o any attribute mismatch.
The fields that the kernel uses to compare rules, IPDB uses as the key fields (see pyroute2/ipdb/rule.py:RuleKey)
There are also more ways to find a record, as with routes:
# 1. iterate all the records
for record in ipdb.rules:
match(record)
# 2. an integer as the key matches the first
# rule with that priority
ipdb.rules[32565]
# 3. a dict as the key returns the first match
# for all the specified attrs
ipdb.rules[{'dst': '10.0.0.0/24', 'table': 200}]
Priorities¶
Thus, the rule priority is not a key, neither in the kernel, nor in IPDB. One should not rely on priorities as on keys, there may be several rules with the same priority, and it often happens, e.g. on Android systems.
Persistence¶
There is no change operation for the rule records in the kernel, so only add/del work. When IPDB changes a record, it effectively deletes the old one and creates the new with new parameters, but the object, referring the record, stays the same. Also that means, that IPDB can not recognize the situation, when someone else does the same. So if there is another program changing records by del/add operations, even another IPDB instance, referring objects in the IPDB will be recreated.
Performance issues¶
In the case of bursts of Netlink broadcast messages, all the activity of the pyroute2-based code in the async mode becomes suppressed to leave more CPU resources to the packet reader thread. So please be ready to cope with delays in the case of Netlink broadcast storms. It means also, that IPDB state will be synchronized with OS also after some delay.
The class API¶
- class pyroute2.ipdb.main.IPDB(nl=None, mode='implicit', restart_on_error=None, nl_async=None, sndbuf=1048576, rcvbuf=1048576, nl_bind_groups=67372509, ignore_rtables=None, callbacks=None, sort_addresses=False, plugins=None, deprecation_warning=True)¶
The class that maintains information about network setup of the host. Monitoring netlink events allows it to react immediately. It uses no polling.
- register_callback(callback, mode='post')¶
IPDB callbacks are routines executed on a RT netlink message arrival. There are two types of callbacks: "post" and "pre" callbacks.
...
"Post" callbacks are executed after the message is processed by IPDB and all corresponding objects are created or deleted. Using ipdb reference in "post" callbacks you will access the most up-to-date state of the IP database.
"Post" callbacks are executed asynchronously in separate threads. These threads can work as long as you want them to. Callback threads are joined occasionally, so for a short time there can exist stopped threads.
...
"Pre" callbacks are synchronous routines, executed before the message gets processed by IPDB. It gives you the way to patch arriving messages, but also places a restriction: until the callback exits, the main event IPDB loop is blocked.
Normally, only "post" callbacks are required. But in some specific cases "pre" also can be useful.
...
- The routine, register_callback(), takes two arguments:
callback function
mode (optional, default="post")
The callback should be a routine, that accepts three arguments:
cb(ipdb, msg, action)
Arguments are:
- ipdb is a reference to IPDB instance, that invokes
the callback.
msg is a message arrived
action is just a msg['event'] field
E.g., to work on a new interface, you should catch action == 'RTM_NEWLINK' and with the interface index (arrived in msg['index']) get it from IPDB:
index = msg['index'] interface = ipdb.interfaces[index]
- eventqueue(qsize=8192, block=True, timeout=None)¶
Initializes event queue and returns event queue context manager. Once the context manager is initialized, events start to be collected, so it is possible to read initial state from the system without losing last moment changes, and once that is done, start processing events.
Example:
ipdb = IPDB() with ipdb.eventqueue() as evq: my_state = ipdb.<needed_attribute>... for msg in evq: update_state_by_msg(my_state, msg)
- eventloop(qsize=8192, block=True, timeout=None)¶
Event generator for simple cases when there is no need for initial state setup. Initialize event queue and yield events as they happen.
- release()¶
Shutdown IPDB instance and sync the state. Since IPDB is asyncronous, some operations continue in the background, e.g. callbacks. So, prior to exit the script, it is required to properly shutdown IPDB.
The shutdown sequence is not forced in an interactive python session, since it is easier for users and there is enough time to sync the state. But for the scripts the release() call is required.