Re: Templating/automating configuration

[Brian Knight <ml () knight-networks com>]

---- On Wed, 07 Jun 2017 04:23:33 -0500 <tim () pelican org> wrote ----




Hi Brian,



On Tuesday, 6 June, 2017 21:48, "Brian Knight" <ml () knight-networks com> said:



> Because we had different sources of truth which were written in-house, we wound up

> rolling our own template engine in Python. It took about 3 weeks to write the

> engine and adapt existing templates. Given a circuit ID, it generates the full

> config for copy and paste into a terminal session. It also hooks into a

> configuration parser tool, written in-house, that tracks configured interfaces, so

> it is easy to see whether the template would overwrite an existing interface.



Interesting. I'm going through much the same process at the moment, due to similar requirements - multiple sources of 
truth, validation that there's no clash with existing configs, but also with a requirement for network-wide atomic 
operations. The latter has been a strong driver for a custom tool - it's now grabbing an exclusive lock on all the 
devices, making all the checks, pushing all the config, commit check everywhere, commit everywhere, and only once all 
the commits succeed, release the locks. If any of those steps fail anywhere, we get to roll back everywhere. (Obviously 
with appropriate timeouts / back-offs / deadlock prevention, and specific to platforms with sane config management - no 
vanilla IOS).





Did you find anything to give you a leg-up on config parsing, or did you have to do that completely from scratch? At 
the moment, I'm working with PyEZ (I know, vendor lock-in, but we're firmly a Juniper shop, and going in eyes-open to 
the lock-in) to build a limited model of just the parts of the config I'm interested in validating, and it seems to be 
working.





The import process to the database runs directly on our rancid server, reading the downloaded configs out of the 
appropriate directory within rancid. Most of our gear is Cisco, so the ciscoconfparse module for Python helps a lot 
with organizing and querying the config.  From there, the config is parsed for key items like interface name, 
description, configured bandwidth, etc., and that info is then added or updated as necessary in the database.



Because it's dependent on rancid, there is some lag time between when a change is made and when the database gets 
updated, so we still strongly encourage running the pre-config checks for new circuits.  But with PyEZ, it looks like 
you easily could, after grabbing that lock, validate the existing config before pushing down new config.  Lots of 
possibilities there.  I'm envious that you have a vendor-written Python module as a choice!





> If I had a free hand and unlimited budget, I would find a single app that

> functions as a source of truth for all circuits and products, which includes a

> templating engine that hooks in easily.



Plus the business buy-in and the resource to go back and standardise all the existing configs, so the application can 
fully parse and understand the network before it starts. That, and a pony :)





Or, at least, rebuild the existing configs based on the new source of truth, so that subsequent config parsing conforms 
to a single standard.



Regards,

Tim.









Cheers,



-Brian