CCIE v5 INE Home Lab - Part 3 - Console to Routers / Automating scripts

Don't you love it when it all comes together and all the puzzle pieces fall into place. Well I hope that this blogtorial does exactly that. In part 1 and part 2, we laid the groundwork and the foundation to successfully get a lab up and running and in this blogtorial we are going to finish it off with:

• How to console into the virtual routers? 
• How to get all initial configs prep'd so we can have concentrate on doing the labs rather than setting up the labs?
• How to automate the loading of the initial configs? 

I know it's a lot to cover in one post, but I also didn't want to breakup this series into too many parts. So without further adieu, let's get started and I will try to streamline it and post as much screenshots as possible. Here is the overview and where we will be spending most of the blogtorial.

I normally would do it for free but I have had tons of requests and questions regarding the lab setup and scripts. So for a nominal fee I will configure your entire VMWare ESXi server / all the routers / the Linux VMs / auto loading scripts. More importantly it includes an easy to use WEB GUI to load the config files. Contact me via arwinr@gmail.com if you are interested. 

Screenshot of the WEB GUI. 

Note: Thomas Kjær pointed out in one of the comments below that ESXi version 6 recently released does not limit 4 serial per Linux VM. Therefore, instead of creating 4 linux VM you could just do it all in one Linux VM, but you would have to edit the scripts accordingly. 


How to console into the virtual routers?

Console access to the routers is accomplished by installing Linux virtual machines on the ESXi, adding serial adapters to the Linux VMs and redirecting the PIPEs (see part 2 on how to create the PIPE) to the Linux VM's serial adapters. Now this might sound complicated to some folks but even the novice Linux users can follow this tutorial and accomplish the task. Again, the reason I am doing it this way is because VMware ESXi "Serial-Over-Network" that most tutorials on the Internet talks about is only valid for 60 days as a demo or you have to get the enterprise license for ESXi which is over $2000.

Only draw back with PIPEs and Serial Adapters is that you can only add 4 Serial adapters to any particular Linux VM so you will have to create [ ceil ( total number of routers / 4) ] number of Linux VMs. So in our case ceil (10 routers / 4) is 3, so we would need a total of 3 Linux VMs to console into 10 routers. I'll briefly walk through on how to install a Linux machine as this is not very hard to do. As a side note you could use Windows VM and use HyperTerm or putty to serial into your routers, I am using Linux because it gives me more flexibility.

First download your favorite linux distro. I used CentOS 7 minimal install which can be downloaded from http://isoredirect.centos.org/centos/7/isos/x86_64/

Just go into any one of those those directories listed above and download the minimal install.

Create a new virtual machine.

Right click on the newly created Linux Machine and alter the settings.

• Ram only needs to be 512MB
• Name the VMs sniffer1, sniffer2, sniffer3 and so on as it will be important later-on in "automating scripts".

Add the serial adapters
This is how we connect to the PIPEs we created in PART 2 from the Linux VMs. So when you are all done you will have CSR1.pipe, CSR2.pipe, CSR3.pipe, CSR4.pipe connected to Linux VM sniffer1.  CSR5.pipe, CSR6.pipe, CSR7.pipe, and CSR8.pipe connected to Linux VM sniffer2. And finally CSR9.pipe, CSR10.pipe connected to Linux VM sniffer3. 

Screenshot below as an example on how to connect PIPEs to Linux VMs. 

Right click on the appropriate Linux VM and click Edit settings. For example, here I am adding CSR4.pipe to Linux VM sniffer1. 

Installing w/ Linux ISO

You need to SCP the Linux ISO file you downloaded to the VMware ESXi so you can install it as a Linux VM. I used WINSCP and connected to the ESXi and uploaded the ISO to the vmimages folder. Now I can use it to boot from it and install the VM. See below.

Right click on the Linux VM and go to Edit settings.

Now power on the Linux and follow the on-screen instructions to install Linux on the VM. 

Once the installation completes you can login into the first Linux VM as root/password you created during install and use minicom to console into routers.

First install minicom and expect by 'yum install -y minicom expect'.

minicom -D /dev/ttyS0 -- This will get you console to R1
minicom -D /dev/ttyS1 -- This will get you console to R2
minicom -D /dev/ttyS2 -- This will get you console to R3
minicom -D /dev/ttyS3 -- This will get you console to R4

To get out of console you can do CTRL A then let go of CTRL and press Z press X and Enter.

You can repeat the process above to create the rest of the Linux VM and make sure to modify things like the Serial Adapter pipe name for example CSR5.pipe, CSR6.pipe and so on.

Prepping the Config files

Why do we need to prep the config files? Well the initial config files from INE are just a template and it's not configured to fit exactly to what we are doing so we need to edit all the config files at once to get it working in our environment.

First thing we need to do is download the INE initial config files. If you have access to INE then you should know how to do this.

Then create a folder in the first Linux VM machine you created. I'll call my first Linux machine "sniffer 1".

mkdir /root/ine/

Copy all the INE initial config files into this directory using winscp or some other scp software. Since the files I downloaded and extracted are in the windows format I had to get rid of the ^M in all the .txt files. We can accomplish this in a sed one liner.

 cd /root/ine  
 find . -type f -exec sed -i 's/^M//' {} \;  

To enter ^M, type CTRL-V, then CTRL-M. That is, hold down the CTRL key then press V and M in succession.

Now we need to add one more thing to the config files and that's the 'platform console serial' command. So if we happen to load INE initial config file as a startup-configuration and reload the routers we won't loose console connection. Again, we can accomplish this in a sed one liner. I am also going to TAR up all the files to be SCP'd into the routers.

 cd /root/ine/  
 find . -type f -exec sed -i '/hostname*/a platform console serial' {} \;  
 tar -cvf ine.tar /root/ine/ 

Now the INE initial configuration are prepped to be loaded into the routers.

In order to get all the INE initial config files on the routers, we need to connect GigabitEthernet2 of all the routers into the management virtual switch and TFTP or SCP all the config files.

Right click on all the routers, click on 'Edit Settings' and change Network Adapter 2 to VM Network.

Now SCP or TFTP the entire the INE files (sniffer1) to bootflash of all the routers. I used SCP in my lab and on R1 - R10 do the command below. 172.20.37.233 is the IP of my SNIFFER1 Linux VM. 

R1# copy scp://root@172.20.37.233/root/ine/ine.tar/ bootflash:ine.tar

Now you have all the INE initial config files ready to load into the running config as needed.

Extract the tar file on all the routers. 

Router#archive tar /xtract bootflash:ine.tar bootflash:ine

Save a blank.cfg on all routers

When the routers initially come up, you should have no config on them. Save this state in the bootflash by typing "copy run bootflash:blank.cfg". For example,

Router#copy run bootflash:blank.cfg

Make sure to do this on ALL ROUTERS. 

Snapshot all the routers

CSR license level is only good for 60 days. After 60 days you won't have access to advanced features like crypto map (needed for DMVPN etc). So after you load/extract the ine files on the routers take a snapshot of your router.

Right click on the router on VMware -> Snapshot -> Take Snapshot.

When you are passed 60 days, you can just revert back to this initial snapshot.

Automatic Loading Initial Configs

As you are going through INE or any other workbooks you will have to load different initial config files are or reset the running config so you can work on a different task. Loading up the initial config files, reloading the routers and waiting for 5 minutes doesn't make much sense, so the best option I've found is to use "configure [replace initial config file] force". With these scripts you can fully automate the loading of the initial config files on all routers.

To accomplish this we are going to use a little bit of PHP, parallel processing scripts, and bash scripting using expect. Again, even a novice user can get this accomplished by following my steps and if you have any questions leave comments.

First install the packages needed.

 yum install -y php.x86_64 php-cli.x86_64 php-common.x86_64            

Next add these entries in the sniffer1 host files. Make sure to edit the IPs accordingly. You can do a ifconfig -a on the Linux VMs to figure out the machines IP.

 root@sniffer1# vim /etc/hosts  

 172.20.37.201 sniffer1  
 172.20.37.202 sniffer2  
 172.20.37.203 sniffer3  

Next create the PHP script to parallel process and run multiple scripts simultaneously.

 root@sniffer1# vim /root/autoload.php 
  
 <?php  

 #NAME::INE AUTO LOAD by Arwin R.  
 #DESC::Expect script to auto load INE initial config files.  
 #Version 1.1

 if ($argv[1] == "exec") {  
     $configfile = $argv[2];  
     $server = $argv[3];  
     if ($server == "sniffer1") {  
         $consolenum = 0;  
         for ($i = 1; $i <= 4; $i++) {  
             shell_exec ("ssh root@$server /root/minicom.sh $configfile" . "R$i.txt $consolenum");  
             $consolenum++;  
         }  
     }  
     if ($server == "sniffer2") {  
         $consolenum = 0;  
         for ($i = 5; $i <= 8; $i++) {  
             shell_exec ("ssh root@$server /root/minicom.sh $configfile" . "R$i.txt $consolenum");  
             $consolenum++;  
         }  
     }  
     if ($server == "sniffer3") {  
         $consolenum = 0;  
         for ($i = 9; $i <= 10; $i++) {  
             shell_exec ("ssh root@$server /root/minicom.sh $configfile" . "R$i.txt $consolenum");  
             $consolenum++;  
         }  
     }  
 }  
 if ($argv[1] == "load") {  
     $configfile = $argv[2];  
     shell_exec ("php autoload.php exec $configfile sniffer1 >> switch.log &");  
     shell_exec ("php autoload.php exec $configfile sniffer2 >> switch.log &");  
     shell_exec ("php autoload.php exec $configfile sniffer3 >> switch.log &");  
 }  
 ?>  

Save the file using :wq!

Put this script on all 3 Linux VMs (sniffer1, sniffer2, sniffer3) under /root/

 root@sniffer1: vim /root/minicom.sh

 #!/usr/bin/expect  
 #NAME::INE AUTO LOAD by Arwin R.  
 #DESC::Expect script to auto load INE initial config files.  
 #Version 1.1  
 set timeout 1  
 set configfile [lindex $argv 0];  
 set serialnumber [lindex $argv 1];  
 set fd [ open /dev/ttyS$serialnumber { RDWR NONBLOCK } ]  
 fconfigure $fd  
 spawn -open $fd  
 send "\x1A";  
 sleep 1;  
 send "\r";  
 sleep 1;  
 send "\032";  
 sleep 1;  
 send "\r";  
 sleep 1;  
 send "configure replace bootflash:blank.cfg force\r";  
 sleep 4;  
 send "\r";  
 sleep 1;  
 send "\x1A";  
 sleep 1;  
 send "\r";  
 sleep 1;  
 send "\032";  
 sleep 1;  
 send "configure replace bootflash:$configfile force\r";  
 send "\r";  
 sleep 2  
 send "\x1A";  
 sleep 1;  
 send "\r";  
 sleep 1;  
 send "\032";  
 sleep 1;  

Save the file :wq!

Sniffer 1 needs to be able to SSH without password to sniffer1, sniffer2, and sniffer3 to run commands so we need to setup 'SSH without password'.

 root@sniffer1# ssh-keygen -t rsa  
 Generating public/private rsa key pair.  
 Enter file in which to save the key (/home/sniffer1/.ssh/id_rsa):   
 Created directory '/home/sniffer1/.ssh'.  
 
 Make sure you just hit enter twice here 

 Enter passphrase (empty for no passphrase):   
 Enter same passphrase again:   
 Your identification has been saved in /home/sniffer1/.ssh/id_rsa.  
 Your public key has been saved in /home/sniffer1/.ssh/id_rsa.pub.  
 The key fingerprint is:  
 3e:41:01:71:31:92:91:7c:3b:ad:e9:58:37:bc:37:e4 root@sniffer1  
 
 Now we need to create a directory ~/.ssh as user root on sniffer2 and sniffer3. If the directory already exists don't worry about creating it.   

 root@sniffer1# ssh root@sniffer2 mkdir -p .ssh  
 root@sniffer2 password:   
 root@sniffer1# ssh root@sniffer3 mkdir -p .ssh  
 root@sniffer3 password:   
 
 Let's now append the public key to sniffer1, sniffer2 and sniffer3.   
 
 root@sniffer1# cat .ssh/id_rsa.pub | ssh root@sniffer1 'cat >> .ssh/authorized_keys'  
 root@sniffer1 password:   

 root@sniffer1# cat .ssh/id_rsa.pub | ssh root@sniffer2 'cat >> .ssh/authorized_keys'  
 root@sniffer2 password:   

 root@sniffer1# cat .ssh/id_rsa.pub | ssh root@sniffer3 'cat >> .ssh/authorized_keys'  
 root@sniffer3 password:   

Sniffer1 now should be able to SSH to sniffer2 and sniffer3 without having to enter a password so the scripts can run seamlessly.

On sniffer1 type

root@sniffer1# php /root/autoload.php load ine/advanced.technology.labs/basic.rip.routing/

will load basic.rip.routing initial config on R1 - R10.

Obviously if you want to change the number of routers or on what Linux machines to run the scripts and such you can edit the script.

There you have it ... a very customized CCIE lab with easy console into the multiple routers without restrictions or licenses, and easy loading of INE initial configs. If you follow part 1 - part 3 series, it might save you a few hours of searching the Internet on how to setup the lab for CCIE v5. I hope you've enjoyed the series, and as always if you have any questions just leave comments.

Here is my first youtube video ever showing you what the end result should look like ...

https://www.youtube.com/watch?v=zvLx_-ufEUk&feature=youtu.be

Many more articles to come so ....

Please subscribe/comment/+1 if you like my posts as it keeps me motivated to write more and spread the knowledge.

Happy labbing!!!