Wednesday, August 14, 2013

Post 1: Building a cross platform network switch automation system

Note: I have had this info in a pending publish state for a while and decided to finally post it. 

When I was a team member at the Enterprise Engineer Center ( we use a mix of network equipment (and Power PDUs, KVM, Servers and SAN).  The EEC hosts customer “engagements” to test/validate scale, proof of concept and features.  The facility has 7 customer labs and security is a number 1 priority. The EEC has multiple levels of security to isolate customer environments from each other, our team systems, and the Microsoft corporate network.  Vlans are used extensively as part of the isolation system and as part of testing customer scenarios.  Some test environments may only have 1 Vlan others may have 25.  Vlans are created and deleted as required for the scenarios being tested. As “the networking guy” I was called to action regularly to help resolve connectivity issues with Vlans that spanned multiple switches.  Many times the issues were caused by a single interface missing its Vlan configuration (tagged or untagged).   While the fix for issues was very easy finding which interface was incorrectly configured proved to be very time consuming. 

To increase the complexity the EEC uses a mix of network switch vendors depending on our partnership and features offered by each.  We also have a large variety of server equipment from multiple partners. These servers can be pre-release systems or a single system that fits a custom need.

As the EEC started its major remodel in 2008 we viewed this as a great time to improve our systems and processes.

In 2008 I started work on a project codenamed Green Monster (GM).  The goals of Green Monster were

1. Reduce the manual process required to perform network vlan and Layer 3 changes by creating automation framework
2. Support a diverse list of equipment manufacturers (network, power, kvm, servers)
3. Create a database of equipment inventory to replace excel spreadsheets
4. Create a database of network and power port to server mappings
5. Provide an automated system to power systems on/off
6. Create a near zero touch OS deployment environment for Engagement build out/setup
7. Abstract hardware vendor and network design from users (UI or script interface)

Development of Green Monster was to be done as a side project.  I was the PM, Dev and for the most part the tester.  Many of my teammates helped with feature feedback, DB design ideas and testing. They also provided a large list of features and ideas for future versions.

Green Monster consists of a WCF Service that performs all the automation logic, DB interaction and device integration. The frontend is a C# UI that consumes the WCF service.  OS imaging was done with WDS and some in-house scripts written by a co-worker.

I will be posting information about specific features and provide code snips on how I interact with devices over the network.

No comments:

Post a Comment

10 Years from last post

 Well world!   After the last almost 10 years I have been up to a few things in life and work.  Most recently I was working at Microsoft on ...