FRnOG 39 - Alan Barnett : Automatic Provisioning with IP and Console Access on a Dedicated Management Network
Category
🤖
TechnologieTranscription
00:00 So yes, good afternoon. My name is Alan Barnett from OpenGear. I'm going to talk about out-of-band
00:06 management networks and how they can be used to help you with managing the network and
00:14 provisioning new systems. I need my glasses. There we go. So I'm going to talk about some
00:22 of the current challenges that network operators face and then really look at how you manage
00:30 IP and console connected devices today and then how a modern out-of-band management solution
00:41 can really help combine multiple devices into a more secure combined solution that can be
00:50 used for access, management and secure provisioning of new equipment. So just to set the scene,
01:01 I talk to a lot of customers and I talk to service providers, enterprise customers and
01:08 typically I guess we're all network people here and network operators and what do those
01:15 people have in common? Typically they have distributed networks. Nearly everyone obviously
01:21 has a wide distributed network. You typically have one or more data centres. Some of those
01:28 data centres may be lights out. There might not be anybody there physically from the IT
01:35 or networking teams. You probably have a variety of different network technologies, variety
01:43 of different network vendors and remote sites, remote locations and they can vary enormously
01:49 depending on what kind of company you work for. But again the common challenge there
01:55 is remote sites typically don't have anyone technical on site. So you need remote connectivity
02:03 to all this equipment. There are a lot of challenges. The networks are typically becoming
02:09 more complex and network operations staff face a whole variety of challenges. Don't
02:16 worry I'm not going to read all these out because we're a little short of time today.
02:24 Network outages cost money. If you have to send someone to site to do something, it takes
02:29 a long time. So an out of band management or access system provides secure remote connectivity
02:37 so that your network, admin, firewall admin, security teams, server admins can all get
02:45 secure remote access to that equipment if something goes wrong or to provision new sites
02:55 or just to do that kind of day to day management of the network. That's really how an out of
02:59 band management network can help a network operations team. So if we look at that in
03:06 a little more detail, if we break it down there are three scenarios. We tend to talk
03:15 about the first day, every day and then the worst day. The first day is when you actually
03:23 deploy something at a new site. We talk to our customers and they told us that one of
03:30 their biggest challenges is turning up a new site or a new location. The traditional way
03:37 is you send the equipment there and you send some people there and they spend some time
03:41 on site getting it up and working. You can save some of that time on site by pre-provisioning
03:47 equipment but there is a better way. If you can ship the equipment there and provision
03:54 it securely and remotely, then you can save a lot of time and cost. So that's one way
04:02 an out of band system can help you with deployment of new equipment, especially in a new site
04:08 where there is no network yet. That's the biggest challenge, right, because a lot of
04:12 provisioning requires network connectivity but if you're actually provisioning the network,
04:20 you haven't got that connectivity yet. We'll talk about that in a moment. The same out
04:25 of band management system can help you manage your network every day and that's fine. Our
04:33 CTO says you should not manage the network using the network. What he means by that is
04:40 you shouldn't just manage things in band because the challenge with that is if something breaks
04:45 then you've lost connectivity and then that moves you into the worst day scenario where
04:51 you're looking to remediate something, you're looking to fix something that's unexpectedly
04:57 happened and that could be anything from a circuit failing, a piece of equipment failing
05:04 or, you know, human error, somebody could push a configuration that breaks something,
05:10 somebody could type in the wrong config, break a VLAN, stop a firewall rule and suddenly
05:17 you're blocked, your in band network is blocked somehow or broken, so an out of band solution
05:23 will enable you to get to that equipment to fix that problem, to roll back the last change.
05:30 It could be anything. We had a customer that had a pair of highly redundant firewalls and
05:38 the link between them broke and so they both went into primary operation and that caused
05:43 a lot of problems and they told me it took an hour for them to get someone on site to
05:48 unplug one of the firewalls so the other one just took over and the guy said to me if I
05:54 had out of band access, one command I could have taken one of the firewalls out and carried
06:00 on. So it's scenarios like that where out of band access for what you might call emergency
06:06 access is really, really useful.
06:10 So if I look at the customers that I speak to, some of them have some kind of management
06:21 network today, not all of them. I guess those of you out there, I guess you know whether
06:27 you've got a dedicated management network or not. We often see people using the production
06:33 network to access remote equipment. Some people have put in often some kind of organically
06:41 grown or deployed solution, possibly a different solution for IP connectivity. We see people
06:48 deploying firewalls with VPN tunnels going back to sites so they get remote IP access
06:57 and then you might have maybe some legacy terminal server or console servers for the
07:02 equipment that you want to get to the console ports of. So whether you've got a dedicated
07:11 system for that, whether you run over the production network, is there a better way?
07:18 What I want to talk about today is what the state of the art kind of current out of band
07:26 management solution might look like. So what would that look like? What we do today is
07:33 we tend to have a combined out of band management appliance. So we have one device that provides
07:41 both console access and IP management and it supports a variety of different connectivity
07:49 and dynamic routing. So effectively we're providing one appliance at each location provides
07:56 an overlay out of band management connection that's highly available with dynamic routing.
08:09 That's kind of the common theme here if you look on the left of this slide. A lot of network
08:15 management and provisioning tools, whether they're open source, whether they're vendor
08:20 specific, the common theme is that they nearly all run over IP. IPv4 actually, typically,
08:28 although the production network might be carrying IPv6 as well. At least in my experience most
08:34 management runs over v4 still. I don't know whether it has to, but that's just my experience.
08:42 So by combining all that functionality into a single dedicated out of band management
08:52 appliance you're simplifying the out of band management network and you're saving cost,
08:58 you're saving power and it's inherently more secure because it's just one device that's
09:04 connected via secure typically tunnel connectivity. So what does that kind of modern smart out
09:14 of band appliance look like? I've got some pictures here. It typically looks like this,
09:21 it's typically a 1RU device, it's got a combination of serial ports and Ethernet ports so you
09:28 can plug in to your target devices over console connectivity, provide Ethernet connectivity.
09:35 We typically have Ethernet connectivity for the WAN as well and optionally built in cellular
09:42 or typically 4G, I guess soon coming 5G. 4G has got much more coverage today, but I guess
09:50 we'll be seeing more 5G connectivity in the future. So this appliance effectively is a
09:59 combined cellular router, VPN device, firewall, terminal server and a LAN switch, all in one
10:10 box. So you deploy this, it makes a secure tunnel back to your central site and you've
10:16 got secure remote IP and console access to that equipment. So if you remember, we can
10:25 use that for first day, every day and worst day connectivity. So if you're looking for
10:35 an out of band management solution, here's a list of sort of features and functions that
10:42 you probably should be looking for. You should be looking for redundant connectivity, some
10:50 way to fail over or load balance or even dynamic routing protocols that the device should be
10:57 designed for high MTBF. Of course it's got to be very secure. So you need a lot of security
11:05 features, user authentication, firewall policies, secure tunneling protocols. The encrypted
11:18 storage is quite important as well. If you've got your own config files that you're going
11:23 to push down to other vendors' equipment, if that's stored on the flash or SSD of that
11:29 open -- out of band appliance, then it really needs to be encrypted for security reasons
11:38 and if you want to ship that device to site with those files on, then that's another reason
11:42 for the encryption on the box, just in case it gets delivered to the wrong address or
11:47 something like that. And of course you need logging and audit trails, a lot of industries,
11:55 finance industry and government and other industries are regulated, so you need a full
12:02 audit trial of what's going on. So just to switch gears a little, what does the secure
12:11 provisioning look like? This is the day one deployment of a new site. As I mentioned,
12:19 the out of band management appliance can provide console and IP connectivity with secure remote
12:27 access and you can use that in a variety of different ways. Kind of the traditional way
12:33 is it still enables manual configuration. So a human operator can use that solution
12:40 to connect to the console port or to browse into the web interface of an iLO processor
12:46 or a firewall or something like that. That's kind of a given. If something goes wrong,
12:52 it's useful for a human to get on there to do some troubleshooting. But of course for
12:57 secure provisioning, you really want to automate that. And depending on where you are in your
13:03 automation journey, you might be using some scripts, you might be using some open source
13:08 tools, you might be using a vendor specific provisioning solution. Like I said, most of
13:16 those tend to run over IP, although scripts can connect or SSH to console ports to put
13:23 on a few initial commands just to get an IP address on the box or something like that.
13:28 I guess the end goal typically is to go to a fully automated zero touch provisioning
13:35 and an out of band management solution can provide the connectivity for that solution
13:43 as well. We'll just go into a couple of these. I think we're running out of time here a little.
13:51 But local secure provisioning is really where we use DHCP based zero touch provisioning.
14:01 Move on to the next slide really, because this has got the kind of flow. So you power
14:08 on a new device on the remote site, it asks for a DHCP, broadcasts for a DHCP request.
14:16 If the out of band appliance responds as a DHCP server, if it recognises the MAC address,
14:24 it can serve up a vendor specific options to specify a config file and/or a boot image
14:31 and then the out of band appliance can be a TFTP or HTTP server and actually provide
14:37 those files as well locally. So that's one way of doing this just locally on site. You
14:44 can drop an out of band management appliance into the site and use it for this purpose.
14:51 However, it's actually in some ways better if you can do this over the wide area, because
14:58 it scales better if you can connect kind of over a temporary or permanent WAN connection.
15:05 So what we tend to do here is the out of band management appliance again can act as a DHCP
15:11 server, give the target device an IP address, a gateway and a DNS server address. And then
15:20 that will allow the device that you just powered on basically to get IP connectivity on its
15:26 management interface and then it can call home to its own maybe dedicated management
15:32 device. Or you can provision by pushing the other way. And finally, so last slide, the
15:41 out of band management device itself needs to auto provision, right? Otherwise it's chicken
15:47 and egg. So what you typically want to look for is for a solution that can call home to
15:53 a configuration server, get its basic config and then securely enrol to a central management
15:59 platform and over the red tunnel effectively then you've got that secure remote access
16:05 to the site and then you can use your IP based provisioning and management tools shown on
16:11 the right to connect to the target equipment on the left and turn up and provision all
16:19 the equipment on the new site. So that's about it. I mean, obviously, sorry, one more thing.
16:26 Once that solution is in place for the day one provisioning, you can use the same solution
16:31 for daily management of the network and of course more importantly, that solution, you
16:37 leave it in place for the emergency access if something goes wrong on the worst day.
16:46 So I hope that's given you a flavour. We didn't have much time. I hope that's given you a
16:49 flavour for what a current modern out of band management solution can do. Some of my colleagues
16:56 who are from locally here in France will be around, I'll be around actually during the
17:02 coffee break if you want to ask any questions. My colleagues are staying later for the events
17:07 this evening as well. So if you have any questions, please come and ask. Thank you very much.
17:12 Thank you.
17:13 [Applause]
17:13 Thank you.