Friday, June 23, 2023

10 Years from last post

 Well world!  

After the last almost 10 years I have been up to a few things in life and work. 

Most recently I was working at Microsoft on the Mixed Reality team.  I joined the team when it was internally called Analog under Alex Kipman. The team had just finishing the HoloLens 1 and was getting ready to actually get it out the door and into the community.  

In 2018 I began working on the Microsoft IVAS project with the U.S. Army.  The IVAS device is a HoloLens for our war fighters and has the capability to significantly increase the over-match capability of our soldiers.  

I am moving all my future plans and posts over to https://msft.pro


Tuesday, April 8, 2014

Enumerating TFS Permissions for items within the TPC for audit and logging

Below is a powershell cmdlet that I threw together to enumerate data out of our TFS instance.  This is used to keep track of permissions in the event we need to re-create them and to allow us to audit it to make sure things don't change.

This can also be used to keep a pre-production environment's permissions in sync with Production.

In 2014 I was not the best at Powershell but it works :)

param ($serverName = $(throw 'please specify a TFS server name'))

function GetGroupMembership {
 [CmdletBinding()]
 [OutputType([System.Data.Datatable])]
 PARAM (
  [Parameter(Mandatory=$true, Position = 0)]
  [Microsoft.TeamFoundation.Framework.Client.TeamFoundationIdentity]
  $TFIdentity
 )
 
 $tabName = $TFIdentity.DisplayName + "Membership"
 $table = New-Object system.Data.DataTable “$tabName”
 $col1 = New-Object system.Data.DataColumn GroupName,([string])
 $col2 = New-Object system.Data.DataColumn DisplayName,([string])
 $col3 = New-Object system.Data.DataColumn UniqueName,([string])

 #Add the Columns
 $table.columns.add($col1)
 $table.columns.add($col2)
 $table.columns.add($col3)
  
 #Membership
 $GroupIdentity = $ims.ReadIdentity($TFIdentity.Descriptor,[Microsoft.TeamFoundation.Framework.Common.MembershipQuery]::Direct,[Microsoft.TeamFoundation.Framework.Common.ReadIdentityOptions]::TrueSid)
 $members = $ims.ReadIdentities($GroupIdentity.Members,[Microsoft.TeamFoundation.Framework.Common.MembershipQuery]::Direct,[Microsoft.TeamFoundation.Framework.Common.ReadIdentityOptions]::TrueSid)
 foreach($member in $members)
 {
  $row = $table.NewRow()

  #Enter data in the row
  $row.GroupName = $TFIdentity.DisplayName
  $row.DisplayName = $member.DisplayName
  $row.UniqueName = $member.UniqueName
  
  #Add the row to the table
  $table.Rows.Add($row)
 }
 
 return @(,$table )
}

function GetPermissions {
 [CmdletBinding()]
 [OutputType([System.Data.Datatable])]
 PARAM (
  [Parameter(Mandatory=$true, Position = 0)]
  $QueryName,
  [Parameter(Mandatory=$true, Position = 1)]
  [Microsoft.TeamFoundation.Framework.Client.AccessControlList]
  $acl,
  [Parameter(Mandatory=$true, Position = 2)]
  $namespace,
  [Parameter(Mandatory=$true, Position = 3)]
  $objectType,
  [Parameter(Mandatory=$true, Position = 4)]
  $objectPath
 )
 
 $PermissionstabName = $QueryName
 
 $table = New-Object system.Data.DataTable “$PermissionstabName”
 $col1 = New-Object system.Data.DataColumn ObjectPath,([string])
 $col2 = New-Object system.Data.DataColumn ObjectType,([string])
 $col3 = New-Object system.Data.DataColumn Name,([string])
 $col4 = New-Object system.Data.DataColumn Permission,([string])
 $col5 = New-Object system.Data.DataColumn Value,([string])

 #Add the Columns
 $table.columns.add($col1)
 $table.columns.add($col2)
 $table.columns.add($col3)
 $table.columns.add($col4)
 $table.columns.add($col5)

 foreach ($ace in $acl.AccessControlEntries)
 {
  if ($ace.IsEmpty)
  {
   continue
  }
  
  $haspermission = $false
  $DenyPermissions = @{}
  $CalculatedPermissions = @{}
  $AllowPermissions = @{}
  foreach ($action in $namespace.description.Actions)
  {
   
   $allowed = ($action.bit -band $ace.Allow) 
   $denied =  ($action.bit -band $ace.Deny)
   if (-not $allowed -and -not $denied)
   {
    continue
   }
   $haspermission  =$true
   if ($allowed)
   { 
    $CalculatedPermissions.Add($action.Name,"Allow")
    $AllowPermissions.Add($action.Name,$action.Name)
   }
   else
   {
    $CalculatedPermissions.Add($action.Name,"Deny")
    $DenyPermissions.Add($action.Name,$action.Name)
   }
   
  }
  
  if ($haspermission)
  {
   
   $identity = $ims.ReadIdentity($ace.Descriptor,[Microsoft.TeamFoundation.Framework.Common.MembershipQuery]::None,[Microsoft.TeamFoundation.Framework.Common.ReadIdentityOptions]::IncludeReadFromSource)
   
   $name = $identity.DisplayName
   
   Foreach($permission in $CalculatedPermissions.GetEnumerator())
   {
    $row = $table.NewRow()

    #Enter data in the row
    $row.ObjectPath = $objectPath 
    $row.ObjectType = $objectType
    $row.Name = $name
    $row.Permission = $permission.Name
    $row.Value = $permission.value

    #Add the row to the table
    $table.Rows.Add($row)
   }
  }
 }
 
 return @(,$table )
}

[void][Reflection.Assembly]::LoadWithPartialName('Microsoft.TeamFoundation.Client')
[void][Reflection.Assembly]::LoadWithPartialName('Microsoft.TeamFoundation.VersionControl.Client')
[void][Reflection.Assembly]::LoadWithPartialName('Microsoft.TeamFoundation.WorkItemTracking.Client')
[void][Reflection.Assembly]::LoadWithPartialName('Microsoft.TeamFoundation.Build.Client')
[void][Reflection.Assembly]::LoadWithPartialName('Microsoft.TeamFoundation.Build.Common')
[void][Reflection.Assembly]::LoadWithPartialName('Microsoft.TeamFoundation')

$tfs = [Microsoft.TeamFoundation.Client.TeamFoundationServerFactory]::GetServer($serverName)

$css = $tfs.GetService([Microsoft.TeamFoundation.Server.ICommonStructureService])
$auth = $tfs.GetService([Microsoft.TeamFoundation.Server.IAuthorizationService])
$gss = $tfs.GetService([Microsoft.TeamFoundation.Server.IGroupSecurityService])
$ss = $tfs.GetService([Microsoft.TeamFoundation.Framework.Client.ISecurityService])
$vcs = $tfs.GetService([Microsoft.TeamFoundation.VersionControl.Client.VersionControlServer])
$ims = $tfs.GetService([Microsoft.TeamFoundation.Framework.Client.IIdentityManagementService])
$wis = $tfs.GetService([Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemStore]) 
$cssNamespace = $ss.GetSecurityNamespace([Microsoft.TeamFoundation.Server.AuthorizationSecurityConstants]::CommonStructureNodeSecurityGuid)

$membershipds = New-Object System.Data.DataSet
$permissiondt = New-Object System.Data.DataSet
$Objectpermissiondt = New-Object System.Data.DataSet

#get all TPC Groups
$tpcGroups = $IMS.ListApplicationGroups($null, [Microsoft.TeamFoundation.Framework.Common.ReadIdentityOptions]::None)

foreach ($tpcgroup in $tpcGroups)
{
 
 $members = GetGroupMembership $tpcgroup
 if ($members.rows.count -gt 0)
 {
  $membershipds.Tables.Add($members)
 }
}
$Namespaces = $ss.GetSecurityNamespaces()
#Get all the permissions for top level NameSpaces
foreach($namespace in $Namespaces)
{
 
 $NameSpaceToken = ""
 
    switch ($namespace.Description.DisplayName)
 {
  "Server" { $NameSpaceToken =  [Microsoft.TeamFoundation.Framework.Common.FrameworkSecurity]::FrameworkNamespaceToken}
  "Build" { $NameSpaceToken =  [Microsoft.TeamFoundation.Build.Common.BuildSecurity]::PrivilegesToken}
  "BuildAdministration" { $NameSpaceToken =  [Microsoft.TeamFoundation.Build.Common.BuildSecurity]::PrivilegesToken}
  "Workspaces" {  $NameSpaceToken =  [Microsoft.TeamFoundation.VersionControl.Common.SecurityConstants]::GlobalSecurityResource}
  "Collection" {}#Namespace:
  "WorkItemTrackingAdministration" {} #
  "CSS" {  }
  "Iteration" {}
  "VersionControlPrivileges" {$NameSpaceToken =  [Microsoft.TeamFoundation.VersionControl.Common.SecurityConstants]::GlobalSecurityResource} 
  "WorkItemQueryFolders" {}
  "Project" {  $NameSpaceToken = [Microsoft.TeamFoundation.PermissionNamespaces]::Project}
  default
  {
  continue
  }
 }
 try
 {
  $groupAcl = $namespace.QueryAccessControlList($NameSpaceToken,$null,$false)
  $table = GetPermissions $namespace.Discription.Name $groupAcl $namespace "Group" $namespace.Discription.Name
  $permissiondt.Tables.Add($table)
 }
 Catch
 {
 
 }
}

foreach ($project in $css.ListProjects())
{
 $projectGroups = $IMS.ListApplicationGroups($project.Uri,[Microsoft.TeamFoundation.Framework.Common.ReadIdentityOptions]::TrueSid)
  
 foreach($group in $projectGroups)
 {
 
  #only get the groups we care about
  if ($group.DisplayName -eq "[$($project.Name)]\Build Administrators" -or $group.DisplayName -eq "[$($project.Name)]\Project Administrators" -or $group.DisplayName -eq "[$($project.Name)]\Contributors" -or $group.DisplayName -eq "[$($project.Name)]\Readers")
  {
   
  }
  else
  {
   continue;
  }
  
  $members = GetGroupMembership $group
  if ($members.rows.count -gt 0)
  {
   $membershipds.Tables.Add($members)
  }
  
  $projectsecNameSpace = $ss.GetSecurityNamespace([Microsoft.TeamFoundation.Server.AuthorizationSecurityConstants]::ProjectSecurityGuid)
  
  #Get Project Permissions
  $ProjectSecurityToken = [Microsoft.TeamFoundation.Server.AuthorizationSecurityConstants]::ProjectSecurityPrefix + $project.Uri
  $groupacl = $projectsecNameSpace.QueryAccessControlList($ProjectSecurityToken,  [Microsoft.TeamFoundation.Framework.Client.IdentityDescriptor[]]@($group.descriptor), $false)
   
  $table =  GetPermissions $group.DisplayName  $groupAcl $projectsecNameSpace "Project" $project.Name
  $permissiondt.Tables.Add($table)
  
  #Get Top Level Build Permissions for the Project
  $BuildsecNameSpace = $ss.GetSecurityNamespace([Microsoft.TeamFoundation.Build.Common.BuildSecurity]::BuildNamespaceId)
  
  $teamProjectGuid = [Microsoft.TeamFoundation.LinkingUtilities]::DecodeUri($project.Uri).ToolSpecificId
  $groupacl = $BuildsecNameSpace.QueryAccessControlList($teamProjectGuid,  $null, $false)
  $tablename = $group.DisplayName + "Build Permissions"
  $table = GetPermissions $tablename $groupAcl $BuildsecNameSpace "Build" $project.Name
  
  $permissiondt.Tables.Add($table)
 }
 
 #Get Root Area Path Permissions
 $rootAreaNodeACL = $cssNamespace.QueryAccessControlList($wis.Projects[$project.Name].AreaRootNodeUri,$null,$false)
 $table = GetPermissions "$($project.Name) Root AreaPath Permissions" $rootAreaNodeACL  $cssNamespace "Area Path" $wis.Projects[$project.Name].Name
 $Objectpermissiondt.Tables.Add($table)

  
 #Get Child Area Path Permissions 
 foreach ($area in $wis.Projects[$project.Name].AreaRootNodes)
 {
  $areapath = $area.Path
  
  $areaseclist = $cssNamespace.QueryAccessControlList($area.uri, $null, $true)
  
  $table = GetPermissions "$areapath AreaPath Permissions" $areaseclist  $cssNamespace "Area Path" $areapath
  $Objectpermissiondt.Tables.Add($table)
 }
 
 try
 {
  $vcproject = $vcs.TryGetTeamProject($project.Name)
 }
 catch
 {
  continue
 }
 
 #Get Version Control permissions on all VC objects
 $vcAcls = $vcs.GetPermissions(@($vcproject.ServerItem), [Microsoft.TeamFoundation.VersionControl.Client.RecursionType]::Full)
  
 $table = New-Object system.Data.DataTable "$($project.Name)VCPermissions"
 $col1 = New-Object system.Data.DataColumn ObjectPath,([string])
 $col2 = New-Object system.Data.DataColumn ObjectType,([string])
 $col3 = New-Object system.Data.DataColumn Name,([string])
 $col4 = New-Object system.Data.DataColumn Permission,([string])
 $col5 = New-Object system.Data.DataColumn Value,([string])

 #Add the Columns
 $table.columns.add($col1)
 $table.columns.add($col2)
 $table.columns.add($col3)
 $table.columns.add($col4)
 $table.columns.add($col5)
 
 foreach($perm in $vcAcls)
 {
  foreach($entry in $perm.Entries)
  {
   foreach($allow in $entry.Allow)
   {
    
    $row = $table.NewRow()
    $row.ObjectPath = $perm.ServerItem
    $row.ObjectType = "VC"
    $row.Name = $entry.IdentityName
    $row.Permission = $allow
    $row.Value = "allow"
    
    #Add the row to the table
    $table.Rows.Add($row)
   }
   foreach($deny in $entry.Deny)
   {
    
    $row = $table.NewRow()
    $row.ObjectPath = $perm.ServerItem
    $row.ObjectType = "VC"
    $row.Name = $entry.IdentityName
    $row.Permission = $deny
    $row.Value = "deny"
    
    #Add the row to the table
    $table.Rows.Add($row)
   }
  }
 }
 
 $Objectpermissiondt.Tables.Add($table)
}

$membershipds.Tables | Format-Table -AutoSize 
$permissiondt.Tables | Format-Table -AutoSize 
$Objectpermissiondt.Tables | Format-Table -AutoSize

Wednesday, March 26, 2014

A new pediatric oncology camp in Chicago

I have been working with a new pediatric oncology camp that is getting started in Chicago that will be at the Ritz Carlton Hotel, Chicago called Camp Kids are Kids.

How cool is it that there will be a pediatric oncology camp in a hotel in the city.

Take a look at their website at http://www.campkidsarekids.org.  They are getting ready for the camp to start in Aug 2014. 

Monday, November 11, 2013

1 year of Honey Bees

Its been about 7 months now from when we first got our bee package. 
 
During that time I have learned a lot about mostly the bee anatomy and parasites and diseases that affect them.  I also learned how much I love watching and being around bees. I will regularly spend time standing near the hive to allow the bees to fly around me and land on me.  While I have a small allergy to them I have not had a problem with this hive.  The Italian honey bees that are in the hive are very calm and do not seem to have any problems when I tend to their home. 
 
In the time that we have had the hive we have had to relocate it 2 separate times.  Each move was ~5 and 20 feet from the original location.  The first was due to a construction worker that was working on a house in the lot next to our house threating lawsuit because "some bees" swarmed him while he was operating the back-hoe. 
 
 
 
Long story sort this guy was a dick!  He had some incident with wasps in the past and he got all freaked out because some bees were hanging out around his equipment (obviously not the reproductive kind).  Without any knowledge and without visibly watching each of the bees return to the hive on my property he assumed it was from my hive box.  When I asked if he carried an EPIpen in the even he was stung he said "no".  So we will just go down the path of him being a dumb ass.
 
"Your bees swarmed me..." - Dumb Ass (2013)
 
While I was not concerned about losing a lawsuit (said bees that were bothering him were not even on my property or on public land).  However I was concerned because the hive was not in a location that technically following the regulations on placement.  We ended up moving the hive to a location which placed it 8 vertical feet above the neighbor property lines to satisfy the placement requirements.  I wish I could see the douche bag construction workers face when he came to the site the next day to see the hive was in a legal placement and not just removed from his baseless threats. 
 
 
The second move was due to the attack on the hive by a "neighborhood flower thief".  At ~2AM one September morning we heard some screaming ("Wake UP, Wake UP") and the sound of the hive falling off the retaining wall (falling wood).  After working with SPD it was determined that the suspect ignored our "no trespassing" and bee warning signs and continued to pick flowers that we had near the hive.  Something then caused her to knock the hive off the wall onto the ground below. After that the hive survived and still managed to rebuild and increase honey reserves. 
 
The lack spare parts was one of the biggest things that stressed me out.  After the hive attack we had 3 broken boxes, ~10 damaged frames and a damaged lid.  We managed to get things put together good-enough after about 2 hours of the attack.  The undamaged equipment allowed us time to get replacement parts without the hive swarming.

 
Due to consistent fear of the event re-occurring we decided to move the hive inside our fenced yard.  While not an ideal location due to blockage of direct sun light the bees seem to be happy with it.  And we have not had any additional damage to the hive. 
 
 

How we did the moves 

After talking with friends and Corky from Ballard Bee Company we found the best way to move the hive was a few simple steps.
  1. Wait until all the foragers make it back into the hive at night. 
  2. Use an entrance reducer with a handful of grass to seal the entrance
  3. Move the hive (we use tie-down straps to keep the hive together)
  4. Place a large amount of branches and limbs to cover the entire front of the hive
  5. The next morning we check to make sure the bees have cleared the grass
 
This move process has worked well so far.  Few days after each move we noticed a large number of the bees go to the old location first then make their way over to the new location.  After the first move we placed a spare box, frames, lid and entrance at the old location to collect lost bees.  Each night and in early morning the box was empty.  We didn't put any temporary home out during the second move. 
 

Liquid GOLD == RENT

We managed to get those bees to pay ~2.2 gallons of honey in their first year.  We learned a lot around this mostly that if you are going to give honey away get the person to commit to eating it.  Nothing upset me more than to find out I gave someone honey and they have never ate it.  Well except for seeing the hive smashed across the ground. 




The big lessons over the last year were

  • Have spare hive parts in the event needed
  • Have some good bright flash lights in the event you need to work late
  • Be ready to sprint barefoot through the neighborhood after your hive attackers
  • LED lights really do mess with bees at night - We noticed once dark the bees would go to LED light bulbs but not standard incandescent bulbs
  • be educated on how to calm the fears of bees and be ready to fall in love with each bee.
We were very lucky this year where we had minimal non-human caused issues with the hive.  I never did expect that I would feel as bad as I do every time I accidently kill a bee but sometimes it is inevitable. 

Friday, October 18, 2013

TFS 2012 and 2013: Changing bug WIT states breaks the ALM backlog and board with TF400917: The current configuration is not valid of this feature

Some teams have found the default bug states don't work well with their teams.  Moving to the Active, Resolved, Closed workflow for the bug WIT has been the most popular with those I have worked with.  I think part of this is because many teams (or their managers) are stuck in old-school waterfall mode while trying to claim they are embracing SCRUM or AGILE.

Either way if you change the BUG states from the default for the SCRUM or AGILE process template you will find the Backlog and board pages on web view for TFS 2012 or 2013 will break.  If you revert your WIT change the ALM features will work again. 

Broken ALM features will show: TF400917: The current configuration is not valid of this feature.  This feature cannot be used until you correct the configuration.

The link to the "Learn more about configuring features" page will take you to MSDN (http://msdn.microsoft.com/en-us/library/vstudio/jj159364.aspx) which does not have any info on this. 

http://blog.nwcadence.com/configure-features-in-team-foundation-server-2013/ has good info on how to setup the features if they never were setup before.  But in our situation it use to work and is now BROKE!

So what's up?
A few posts will tell you to use
witadmin exportcommonprocessconfig then
witadmin importcommonprocessconfig without making any change to the .xml from the export. 

This will result in an error tf400536.  It will say the "requirementworkitems" entries are wrong. 

So how the hell do you fix it?

You need to run
  1. witadmin exportcategories /p:<PROJECT> /collection:<URL TO TPC> /f:catagories.xml
  2. Edit the file and remove <WORKITEMTYPE name="Bug" /> from <CATEGORY refname="Microsoft.RequirementCategory" name="Requirement Category">
  3. witadmin importcategories /p:<PROJECT> /collection:<URL TO TPC> /f:catagories.xml

 

Saturday, September 7, 2013

First 3 months of Solar Panels in Seattle and we are making MONEY!

It has been ~3 months from when Puget Sound Solar installed my 4.3kW solar system. 

In that time we have generated 1,910kW.
Overall I am very happy with our Solar install and we have a nice credit with Seattle City light on our power bill.  I pushed and rushed Puget Sound Solar to get the install completed by July 1st 2013 so we could get the WA state sales tax exception.  After all that the state extended the Sales Tax exemption for some time longer.  Either way it worked out and we don't appear to have any water leaks or issues on the roof.
 
Now the APS interface and "software" that works with the Blue Frog inverters is not the best.  The web interface is hosted on a server in China and is real slow.  I personally have issues having a Linux based device (the APS Communicator) sitting on my home network. 
 
The APS Monitor (EMA Software) is the China hosted web interface that I'm not a huge fan of.  I was expecting software that was going to be installed on a home computer. The web interface does not allow easy access to the data nor does it perform well when attempting to view the detailed information on each inverter.  
 
It seems the Blue Frog inverters are branded as Made in WA but they are really manufactured by some company in China.  Then they come to WA where Blue Frog does some minor change to them that makes them be classified as made in WA.  Seems a little strange but hey I get .54/kWh generated because of it.
 
We have not yet received the paper work from the state of WA for us to provide our generation information so we can get our Generation Credit.  At the current amount we are looking to get 1910 * .54 = $1031.4 from the STATE!  Woot  Woot.
 
Everyone should get solar. 

Monday, August 26, 2013

AD attributes to fully Automating OS deployment with WDS (Windows Deployment Services)

Microsoft provides many tools for deploying an Operating System (Windows 7, Windows 8, Windows Server 2008, Windows Server 2012).  Each has their pros and cons.  Some are easy to setup some a little harder.

One thing I found over the years at Microsoft was when setting up a test system that is physical hardware it is best to do a clean install, thus using WDS. 

The info below was used as part of the Automation system I developed while at the Microsoft Enterprise Engineering Center.  By using WDS with the automation system we were able to simply click on a server in the UI and select Image.  This would then kick off the OS Imaging workflow.
Deployment Steps:
  1. Power Server off (via Raritan Switched PDUs and SNMP)
  2. Move Network interfaces for server into correct Vlan (via Network Vlan plugin system)
  3. Set Active Directory Attributes below for WDS
  4. Set KVM interface Name (via Raritan Command Center APIs)
  5. Power Server ON (via Raritan Switches PDUs)
 Advantages of using WDS
  • No image to maintain for each model
  • No images to patch each month
  • System is clean and pure
  • Bases (install) images are easy to setup (just need the .WIM from the CD)
  • Does not require agents to be installed on everything
  • Allows for an easy system audit script to be run at the same time
  • Don't have to worry about a "safe OS" being installed on some drive

I am not going to go into detail on how to setup WDS or add boot or install images.  This is well documented on MSDN. http://technet.microsoft.com/en-us/library/jj648426.aspx.

I don't recommend using Stand-alone mode if you need a system that needs some resiliency.  The AD integrated option works the best!

A few requirements
We will be working with 2 AD attributes

netbootMachineFilePath

This attribute specifies what PXE client should be used when the computer boots.  You can use wdsutil /Set-Device /Device:<name> /BootProgram:<path> to set this.

3 common values
  • <WDS Server FQDN>\boot\x86\pxeboot.n12 - Tells the PXE client to NOT required F12. If you used the N12 option be sure to run "wdsutil /set-server /resetbootprogram:yes" on the WDS server. If you don't the clients will always be in reboot loop.
  • <WDS Server FQDN>\boot\x86\abortpxe.com - Tells the PXE client to abort any PXE boot attempt on that NIC and move to next device in boot order
  • <WDS Server FQDN>\boot\x86\pxeboot.com - Tells PXE to request the user to push F12.  Will move to next boot device if they don't.

netbootMirrorDataFile


This attribute holds the following items for a ZERO Touch deployment.
  • What boot file (WIM) to use - BootImagePath
  • Path to unattend XML file (provides path to install WIM) - WdsUnattendFilePath
  • If you want the computer joined to the domain - JoinDomain
As MSDN provides (in link above) the info is in Key=value; format.

Example: to use boot image foo, unattend bar and NOT join the domain

netbootMirrorDataFile=JoinDomain=0;BootImagePath=Boot\x86\Images\foo.wim;WdsUnattendFilePath=wdsclientunattend\bar.xml;

 Note the trailing ;.  See http://www.mikepoulson.com/2013/08/bindlsvc-error-522-from-wds.html for more info.

These items can also be set by running the following WDSUtil commands
  • WDSUTIL /Set-Device /Device:<name> /WDSClientUnattend:<path>
  • WDSUTIL /Set-Device /Device:<name> /BootImagePath:<path>
  • WDSUTIL /Set-Device /Device:<name> /JoinDomain:No

Friday, August 23, 2013

Getting detailed TFS build log ActivityLog.xml information via APIs

The TFS Build Activity logs have lots of useful information.  Unfortunately this information is not real easy to get to.  Using the TFS 2012 APIs we can pull the log info out of the TFS SQL DB without having access to the DB.  This makes it nice because we also don't have to try to parse the activitylog.xml files.

Once you have the information you can start to track and trend how long each TFS build definition takes at each part of the build.  Once you have that piece of data you can start to determine where the best place to optimize to further reduce build times.

The sample code below looks up the LKG build based off the build Definition.  The LKG (Last Known Good) build will be set based on the last build for that build definition that passed/succeeded. It will do a simple output of the activity name and the time it took to complete.  You can take this data along with the buildDetail and create a gantt chart that shows time for each step.

The build may be old (aka not a current LKG) depending on how your system works.  But I found it is the best place to start.  It might be useful for some teams to import all build (success and fail) so you can determine if there is a trend area of where things are failing.


Usings and references
using Microsoft.TeamFoundation.Build.Client;
using Microsoft.TeamFoundation.Client;
logic
 

            Uri tfsCollectionURL = new Uri("http://TFSSERVER:8080/tfs/TFSTPC");
            string tfsProjectName = "TFSPROJECTNAME";

            string buildDefName = "BUILDDEFINITIONNAME";
            var tfs = TfsTeamProjectCollectionFactory.GetTeamProjectCollection(tfsCollectionURL);


            IBuildServer buildServer = (IBuildServer)tfs.GetService(typeof(IBuildServer));
            var _buildDefinitions = buildServer.QueryBuildDefinitions(tfsProjectName);
            foreach (var buildDefinition in _buildDefinitions)
            {
                if (buildDefinition.LastGoodBuildUri == null)
                    continue;

                var build = buildServer.GetBuild(buildDefinition.LastGoodBuildUri);
                IBuildDetail buildDetail = buildServer.GetAllBuildDetails(build.Uri);

                var activityTrackingNodes = InformationNodeConverters.GetActivityTrackingNodes(buildDetail);

                foreach (var activity in activityTrackingNodes)
                {
                    if (activity.State != "Canceled" && (activity.Node.Children.Nodes.Count() == 0 || (activity.Node.Children.Nodes.Any(x => x.Type == "BuildMessage") && activity.DisplayName != "Sequence")))
                    {
                        if (activity.FinishTime.ToString() == "1/1/0001 12:00:00 AM")
                            continue;

                        Console.WriteLine(activity.DisplayName + ":" + (activity.FinishTime - activity.StartTime));
                    }

                }
            }
            

Thursday, August 15, 2013

BINDLSVC error 522 from WDS

If you are getting BINDLSVC error 522 errors in the event log when attempting to boot systems into WDS you may have a bad config in AD.

Normally this is because when setting values to the netbootMirrorDataFile attribute on the computer object the ; (semicolon) at the end of line may be missing.

From MSDN:

netbootMirrorDataFile: This attribute is used to store multiple values in the following format:
<key-1>=<value-1>; <key-2>=<value-2>;...; <key-N>=<value-N>;


Wednesday, August 14, 2013

Using C# to manage a Brocade TurboIron switch via SNMP

This is to provide an example of a plug-in I wrote for the Green Monster System to manage a Brocade TurboIron switch.

The SNMP commands are performed via a base class that uses Nsoftware's SSNMP library.  That class is not included because of licensing.

If you need help or have questions feel free to send me mail.


public bool createVlan(int tag, string name)
        {
            SNMPDataCollection request = new SNMPDataCollection();
            request.Add(new SNMPData("1.3.6.1.2.1.17.7.1.4.3.1.1." + tag, name, datatypes.str)); //Set Vlan Name
            request.Add(new SNMPData("1.3.6.1.2.1.17.7.1.4.3.1.2." + tag, new Byte[1], datatypes.str)); //Egress Members
            request.Add(new SNMPData("1.3.6.1.2.1.17.7.1.4.3.1.4." + tag, new Byte[1], datatypes.str)); //Untagged Members
            request.Add(new SNMPData("1.3.6.1.2.1.17.7.1.4.3.1.5." + tag, 4, datatypes.integer)); //Create and Go

            return sendSNMP(request);
        }
        public bool deleteVlan(int tag)
        {
            try
            {
                return sendSNMP("1.3.6.1.2.1.17.7.1.4.3.1.5." + tag, 6);
            }
            catch (Exception)
            {
                //return false;
                throw;
            }
        }

        public bool addPortToVlan(int tag, int ifIndex, bool isTagged)
        {
            try
            {
                Byte[] currentMembers;
                if (GetQBridgeVlanMembers(tag, isTagged, out currentMembers) == false)
                    return false;

                if (currentMembers.Length == 0)
                    return false; //vlan needs to be created first

                currentMembers = this.GeneratePortByteStream(ifIndex, currentMembers, true);
                //Due to bug in TI code
                //Issue is that QBridge Add port to vlan logic does not check if target port is tagged 
                //So if target port is already member of another vlan as tagged it will fail on all other adds

                //Workaround is to use 1.3.6.1.4.1.1991.1.1.3.2.6.1.3.. i 4 to add the port to the vlan after it is added via qbridge to first

                //If we are doing tagged we need to do workaround
                if (isTagged)
                {
                    //check to see if target port is already member of another vlan
                    if (isPortTaggedinAnotherVlan(ifIndex) == true)//add with private OID
                        return sendSNMP("1.3.6.1.4.1.1991.1.1.3.2.6.1.3." + tag + "." + ifIndex, 4);

                    //if this did not find and return fall through to qbridge
                }

                return SetQBridgeVlanMembers(currentMembers, tag, isTagged);

            }
            catch (Exception e)
            {
                // return false;
                throw e;
            }
        }
        public bool removePortFromVlan(int tag, int ifIndex)
        {
            try
            {
                return sendSNMP("1.3.6.1.4.1.1991.1.1.3.2.6.1.3." + tag + "." + ifIndex, 3);

            }
            catch (Exception e)
            {
                //return false;
                throw e;
            }
        }
 public bool? isPortInVlan(int tag, int ifIndex, bool tagged)
        {
            Raw_VlanMemberCollection members = GetVlanMembers(tag);
            if (members.isErrorState == true)
                return null;

            if (members.Find(v => v.PortifIndex == ifIndex && v.isTagged == tagged) != null)
                return true;
            else
                return false;

        }

        public VlanCollection getVlans()
        {
            VlanCollection Vlans = new VlanCollection();
            SNMPDataCollection data = walk("1.3.6.1.4.1.1991.1.1.3.2.7.1.1");
            if (data.isErrorState == true)
            {
                Vlans.isErrorState = true;
                return Vlans;
            }

            foreach (SNMPData obj in data)
            {
                string vlanTag = obj.value.ToString(); //each value will be a vlan
                string oid = obj.oid;
                int ifIndex = Convert.ToInt32(vlanTag); //Foundry uses the Tag as the ifIndex

                //Will include 4095 which is the Management Vlan
                Vlan vlan = new Vlan();
                vlan.tag = Convert.ToInt32(vlanTag);
                vlan.ifIndex = Convert.ToInt32(ifIndex);
                vlan.name = getVlanName(vlan.tag);
                Vlans.Add(vlan);
            }

            return Vlans;
        }

        public Raw_VlanMemberCollection GetVlanMembers(int tag)
        {
            Raw_VlanMemberCollection data = new Raw_VlanMemberCollection();

            //Get untagged port members
            byte[] vlanMembers;
            if (GetQBridgeVlanMembers(tag, false, out vlanMembers) == false)
            {
                data.isErrorState = true;
                return data;
            }

            ArrayList members = GetMemberPorts(vlanMembers);
            foreach (int p in members)
            {
                Raw_VlanMember member = new Raw_VlanMember();
                member.isTagged = false;
                member.tag = tag;
                member.port = p;
                member.slot = 1; //only supports 1 slot
                member.PortifIndex = p;
                member.VlanifIndex = tag;
                data.Add(member);
            }

            //Get Tagged members

            if (GetQBridgeVlanMembers(tag, true, out vlanMembers) == false)
            {
                data.isErrorState = true;
                return data;
            }
            members = GetMemberPorts(vlanMembers);
            foreach (int p in members)
            {
                Raw_VlanMember member = new Raw_VlanMember();
                member.isTagged = true;
                member.tag = tag;
                member.port = p;
                member.slot = 1; //only supports 1 slot
                member.PortifIndex = p;
                member.VlanifIndex = tag;
                data.Add(member);
            }

            return data;
        }

        public NetworkPortCollection getPorts()
        {
            NetworkPortCollection Ports = new NetworkPortCollection();
            SNMPDataCollection data = walk("1.3.6.1.4.1.1991.1.1.3.3.5.1.18");
            if (data.isErrorState == true)
            {
                Ports.isErrorState = true;
                return Ports;
            }

            foreach (SNMPData obj in data) //Get all our ports
            {
                string value = obj.value.ToString();

                //static devices dont return slot info
                if (value.IndexOf(@"/") == -1) //we are not doing a moduler device
                {
                    value = @"1/" + value; //set default slot to 1
                }


                //Value returned looks like /
                //Lets split it out
                string[] slotport = value.Split(Convert.ToChar(@"/"));
                int id = Convert.ToInt32(obj.oid.ToString().Substring(obj.oid.ToString().LastIndexOf(".") + 1));


                NetworkPort port = new NetworkPort(id, Convert.ToInt32(slotport[0]), Convert.ToInt32(slotport[1]));
                port.name = getPortDescription(port.ifIndex);
                port.adminStatus = getPortAdminStatus(port.ifIndex);
                port.operationStatus = getPortOperationStatus(port.ifIndex);
                if (port.unTaggedVlan == null)
                {
                    port.unTaggedVlan = new VlanMember();
                }
                port.unTaggedVlan.tag = getPortUntaggedVlan(port.ifIndex);

                Ports.Add(port);
            }

            return Ports;
        }
        public bool addFirstPorttoVlan(int tag, int ifIndex, bool isTagged, string vlanName)
        {
            try
            {
                SNMPDataCollection request = new SNMPDataCollection();

                request.Add(new SNMPData("1.3.6.1.2.1.17.7.1.4.3.1.1." + tag, vlanName, SNMPBase.datatypes.str)); //Vlan Name
                request.Add(new SNMPData("1.3.6.1.2.1.17.7.1.4.3.1.2." + tag, new byte[1], SNMPBase.datatypes.str)); //Egress Ports

                if (isTagged)
                    request.Add(new SNMPData("1.3.6.1.2.1.17.7.1.4.3.1.4." + tag, new byte[1], SNMPBase.datatypes.str)); //Untagged ports

                request.Add(new SNMPData("1.3.6.1.2.1.17.7.1.4.3.1.5." + tag, 4, SNMPBase.datatypes.integer)); //CreateAndGo
                bool response = sendSNMP(request);

                return addPortToVlan(tag, ifIndex, isTagged);
            }
            catch (Exception)
            {
                return false;
                throw;
            }
        }
        private int getPortUntaggedVlan(int ifIndex)
        {
            string data;
            if (getSNMP("1.3.6.1.4.1.1991.1.1.3.3.5.1.24." + ifIndex.ToString(), SNMPBase.datatypes.integer, out data) == false)
                return -1;
            return Convert.ToInt32(data);
        }
  #region "Private"
        private bool isPortTaggedinAnotherVlan(int ifIndex)
        {
            VlanCollection vlans = getVlans();
            foreach (Vlan vlan in vlans)
            {
                Raw_VlanMemberCollection members = GetVlanMembers(vlan.tag);
                Raw_VlanMember member = members.Find(m => m.PortifIndex == ifIndex && m.isTagged == true);
                if (member != null)
                    return true; //found a member
            }

            return false; //port is not any other vlans

        }
        private static readonly Byte[] PORTMASKARRAY = { 128, 64, 32, 16, 8, 4, 2, 1 };

        private ArrayList GetMemberPorts(Byte[] memberbytes)
        {
            //Find out which bit positions are set in a byte.  Based off which position and byte we are in we can determine the port number
            //ie bit 7 in byte 0 = port 1
            //ie bit 0 in byte 0 = port 8
            ArrayList members = new ArrayList();

            int bytecoute = 0;
            int portNumber = 0;
            int result = 0;

            foreach (Byte b in memberbytes)
            {
                if (memberbytes[bytecoute] == 0)
                {
                    bytecoute++; // No ports where active in the Byte
                }
                else // if we have port membership in the Byte lets see which ports
                {
                    for (int i = 0; i < 8; i++) // Loop through each bit 
                    {
                        result = memberbytes[bytecoute] & PORTMASKARRAY[i]; // Is each bit value (port) in the array?
                        if (result == PORTMASKARRAY[i])
                        {
                            portNumber = i + 1 + bytecoute * 8;
                            members.Add(portNumber); // Add the portnumber to our returned list
                        }
                    }
                    bytecoute++;
                }
            }
            return members;
        }

        private static bool isPortMember(int portnumber, Byte[] membershipstream)
        {
            //Determine the port number we are working with for the given slot
            //Mod by 1000 to remove slot number; set remainder to portnumber
            portnumber %= 1000;

            //Byte[] PORTMASKARRAY = { 128, 64, 32, 16, 8, 4, 2, 1 };
            return (membershipstream[(portnumber - 1) / 8] & PORTMASKARRAY[(portnumber - 1) % 8]) != 0;
        }
        private bool GetQBridgeVlanMembers(int tag, bool isTagged, out  Byte[] members)
        {
            members = new Byte[0];
            if (isTagged)
            {

                byte[] allmembers;
                getSNMP("1.3.6.1.2.1.17.7.1.4.2.1.4.0." + tag, out allmembers); //dot1qVlanStaticEgressPorts
                //dot1qVlanStaticEgressPorts also includes untagged ports.  So we have to remove the untagged ports from the array.
                //What is left will be the tagged only ports
                byte[] untaggedMembers;
                if (GetQBridgeVlanMembers(tag, false, out untaggedMembers) == false)
                    return false;

                byte[] taggedports = new byte[allmembers.Length];
                for (int i = 0; i < allmembers.Length; i++)
                {
                    taggedports[i] = Convert.ToByte(allmembers[i] ^ untaggedMembers[i]);
                }

                members = taggedports;
                return true;
            }
            else
            {
                byte[] untaggedMembers;
                if (getSNMP("1.3.6.1.2.1.17.7.1.4.2.1.5.0." + tag, out untaggedMembers) == false) //dot1qVlanStaticUntaggedPorts
                    return false;

                members = untaggedMembers;
                return true;
            }
        }

        private bool SetQBridgeVlanMembers(Byte[] members, int tag, bool isTagged)
        {
            //Allways add Vlan member to Egress port.  By default is tagged.  If you want untagged then also add to untaggedPorts
            bool result = false;

            if (isTagged)
            {
                return sendSNMP("1.3.6.1.2.1.17.7.1.4.3.1.2." + tag, members, datatypes.str); //dot1qVlanStaticEgressPorts
            }
            else
            {
                //for egreess ports we need to get all current egress ports to append the current port
                Byte[] currentMembers;
                if (GetQBridgeVlanMembers(tag, true, out currentMembers) == false)
                    return false;

                if (currentMembers.Length == 0)
                    return false; //vlan needs to be created first

                result = SetQBridgeVlanMembers(currentMembers, tag, true);
                if (result == false)
                    return false;

                result = sendSNMP("1.3.6.1.2.1.17.7.1.4.3.1.4." + tag, members, datatypes.str); //dot1qVlanStaticUntaggedPorts

            }
            return result;
        }
        /// 
        /// QBridge Byte Stream generation, used for modifing port membership in Vlan
        /// 
        /// portnumber to add/remove from vlan
        /// 
        private Byte[] GeneratePortByteStream(int newMemberPort, Byte[] currentMembers, bool addPort)
        {
            //Determine the port number we are working with for the given slot
            //Mod by 1000 to remove slot number; set remainder to portnumber
            //portnumber %= 1000;
            Byte[] holdingByte = new Byte[currentMembers.Length];
            Array.Copy(currentMembers, holdingByte, currentMembers.Length);


            //Determin which byte we are working with
            int byteposition = (newMemberPort - 1) / 8;

            //Mod to find the bit we are working with 
            int maskindex = (newMemberPort - 1) % 8;

            //Set the value in our Holding array for the corresponding bit
            if (addPort)
                holdingByte[byteposition] |= PORTMASKARRAY[maskindex];
            else
            {
                if ((holdingByte[byteposition] & PORTMASKARRAY[maskindex]) == 0) //check to see if the newMemberport is a member of the currentMembers
                    return holdingByte;


                if (currentMembers[byteposition] == 0)  //dont xor a null value.  If the port is not already a member dont try to remove it.
                    return holdingByte;
                // holdingByte[byteposition] = Convert.ToByte(currentMembers[byteposition] & INVERTED_PORTMASKARRAY[maskindex]);
                holdingByte[byteposition] = Convert.ToByte(currentMembers[byteposition] ^ PORTMASKARRAY[maskindex]);
            }
            return holdingByte;
        }

        private string ConvertToHex(object bytearray)
        {

            byte[] ba = (byte[])bytearray;

            StringBuilder hex = new StringBuilder(ba.Length * 2);
            foreach (byte b in ba)
                hex.AppendFormat("{0:x2}", b);
            return hex.ToString();
        }
        #endregion

Post 4: Green Monster Network plugin examples


Part of the Green Monster Project is to abstract the hardware for Vlan Management.  To do this I created a “plugin” for each network switch that we use at the EEC.  When action needs to be taken against a network switch.  Green Monster first queries the database (See post 2) to determine the DLL that it should load into the plugin manager.  Once it is loaded I can execute the exposed methods. 

This plugin model is the same process used for Power management and KVM management.  The interaction with the network switch is abstracted from the main code.  It is up to the plugin to know the best ways to execute a given command.  This allows for the plugin to use any available execution methods.  This can include SNMP, Netconf, scripted telnet, SSH, or other web services.  As long as the plugin has the ability to communicate via the protocol desired it should work. 

The majority of the code we use for device interaction is SNMP.  So I thought I would post how we use SNMP to do Vlan management.  Some of this was found by doing packet sniffs and working with the Manufactures engineers.

For SNMP in CSharp I use the nsoftware.com SSNMP library.  It is very fast and we had luck with it working well.  The code to make the “raw” snmp call is not included because that requires a license for the nsoftware library.  If you have that license and are interested please let me know and I can share some snip-its.

 
Extreme Networks (XOS based)
Brocade Networks (Turborion, MLX)

 

10 Years from last post

 Well world!   After the last almost 10 years I have been up to a few things in life and work.  Most recently I was working at Microsoft on ...