Monday, April 18, 2011

Replicating UNIX "tail -f" in PowerShell

Building upon my previous blog post, Unix Tail-like Functionality in PowerShell Revisited, I have completed the next step in providing tail functionality in my PowerShell scripts; the ability to emulate the "-f" argument.

From the TAIL(1) man page:
The -f option causes tail to not stop when end of file is reached, but rather to wait for additional data to be appended to the input.
My approach to replicating this functionality in PowerShell is to once again take advantage of System.IO.FileStream Class and read from the end of file as I did in my previous tail emulations. Originally, I thought this was going to be a more difficult task to accomplish but since I learned so much from my prior attempts, it turned out to be fairly simple and a lot less code to implement. My solution focuses on the fact that the file size grows as more data is appended to the text file. If I know how large the file was in a prior reading than it is currently, I know how many bytes have been added to the file and from there, I know the number of bytes I need to read from the end of the file and return to the console. All I need is a looping routine to constantly check the file for changes. In my code sample below, I emulate "tail -f" to the console by first reading the last 10 lines of a hypothetical BlackBerry Enterprise Server Management Agent log file (on an active server this file grows constantly) on a remote server as Unix "tail -f" would then start to monitor the log file for changes by comparing the file size waiting 100 milliseconds between each comparison. This process will continue until you send a break or the file is deleted.

With this bit of starter code, you should be able to implement some unique tools to monitor and react to data in log files in real-time.

UPDATE: I have made some improvements to this code here.
Function Read-EndOfFileByByteChunk($fileName,$totalNumberOfLines,$byteChunk) {
 if($totalNumberOfLines -lt 1) { $totalNumberOfLines = 1 }
 if($byteChunk -le 0) { $byteChunk = 10240 }
 $linesOfText = New-Object System.Collections.ArrayList
 if([System.IO.File]::Exists($fileName)) {
  $fileStream = New-Object System.IO.FileStream($fileName,[System.IO.FileMode]::Open,[System.IO.FileAccess]::Read,[System.IO.FileShare]::ReadWrite)
  $asciiEncoding = New-Object System.Text.ASCIIEncoding
  $fileSize = $fileStream.Length
  $byteOffset = $byteChunk
  [byte[]] $bytesRead = New-Object byte[] $byteChunk
  $totalBytesProcessed = 0
  $lastReadAttempt = $false
  do {
   if($byteOffset -ge $fileSize) {
    $byteChunk = $fileSize - $totalBytesProcessed
    [byte[]] $bytesRead = New-Object byte[] $byteChunk
    $byteOffset = $fileSize
    $lastReadAttempt = $true
   }
   $fileStream.Seek((-$byteOffset), [System.IO.SeekOrigin]::End) | Out-Null
   $fileStream.Read($bytesRead, 0, $byteChunk) | Out-Null
   $chunkOfText = New-Object System.Collections.ArrayList
   $chunkOfText.AddRange(([System.Text.RegularExpressions.Regex]::Split($asciiEncoding.GetString($bytesRead),"\r\n")))
   $firstLineLength = ($chunkOfText[0].Length)
   $byteOffset = ($byteOffset + $byteChunk) - ($firstLineLength)
   if($lastReadAttempt -eq $false -and $chunkOfText.count -lt $totalNumberOfLines) {
    $chunkOfText.RemoveAt(0)
   }
   $totalBytesProcessed += ($byteChunk - $firstLineLength)
   $linesOfText.InsertRange(0, $chunkOfText)
  } while($totalNumberOfLines -ge $linesOfText.count -and $lastReadAttempt -eq $false -and $totalBytesProcessed -lt $fileSize)
  $fileStream.Close()
  if($linesOfText.count -gt 1) {
   $linesOfText.RemoveAt($linesOfText.count-1)
  }
  $deltaLines = ($linesOfText.count - $totalNumberOfLines)
  if($deltaLines -gt 0) {
   $linesOfText.RemoveRange(0, $deltaLines)
  }
 } else {
  $linesOfText.Add("[ERROR] $fileName not found") | Out-Null
 }
 return $linesOfText
}
#--------------------------------------------------------------------------------------------------#
Function Read-FileUpdates($fileName,$startSize) {
 $asciiEncoding = New-Object System.Text.ASCIIEncoding
 while([System.IO.File]::Exists($fileName)) {
  $fileStream = New-Object System.IO.FileStream($fileName,[System.IO.FileMode]::Open,[System.IO.FileAccess]::Read,[System.IO.FileShare]::ReadWrite)
  $currentFileSize = $fileStream.Length
  if($currentFileSize -gt $startSize) {
   $byteChunk = $currentFileSize - $startSize
   [byte[]] $bytesRead = New-Object byte[] $byteChunk
   $fileStream.Seek((-$byteChunk), [System.IO.SeekOrigin]::End) | Out-Null
   $fileStream.Read($bytesRead, 0, $byteChunk) | Out-Null
   Write-Host ($asciiEncoding.GetString($bytesRead)) -noNewLine
   $startSize = $currentFileSize
  }
  $fileStream.Close()
  Start-Sleep -milliseconds 100
 }
}
#--------------------------------------------------------------------------------------------------#
Set-Variable -name inputFile -option Constant -value "\\japan-bes.ad.mydomain.local\E$\Program Files\Research In Motion\BlackBerry Enterprise Server\Logs\20110418\JAPAN-BES_MAGT_02_20110418_0001.txt"
#--------------------------------------------------------------------------------------------------#
if([System.IO.File]::Exists($inputFile)) {
 Write-Host (@(Read-EndOfFileByByteChunk $inputFile 10 1280) | Out-String) -noNewLine
 $fileStream = New-Object System.IO.FileStream($inputFile,[System.IO.FileMode]::Open,[System.IO.FileAccess]::Read,[System.IO.FileShare]::ReadWrite)
 $fileSize = $fileStream.Length
 $fileStream.Close()
 Read-FileUpdates $inputFile $fileSize
} else {
 Write-Host "Could not find $inputFile..." -foregroundColor Red
}

Monday, April 11, 2011

Audit Group Policy Objects

I needed to review a group of Group Policy Objects and determine if an Active Directory group is listed in the permissions for the GPO. If it was not, I needed to be aware of it. In the code sample, I take advantage of the Group Policy module to loop through all the GPOs in a domain that have "Workstation" in the display name, identify the ones that do not include the "ASIA\DesktopTeam" in the permission list and output them to a comma separated values text file.
#--------------------------------------------------------------------------------------------------#
Set-Variable -name accountDomain -option Constant -value "ASIA"
Set-Variable -name accountsAMAccountName -option Constant -value "DesktopTeam"
Set-Variable -name workingDomain -option Constant -value "asia.ad.mycompany.local"
Set-Variable -name matchFilter -option Constant -value "Workstation"
Set-Variable -name outputFile -option Constant -value  ($workingDomain + "-" + $matchFilter + ".csv")
#--------------------------------------------------------------------------------------------------#
if((Get-Host).Version.Major -ne 2) {
 Write-Host "You must have PowerShell V2 installed to use this script. Exiting!" -foregroundColor Red
 exit
}
if(!(Get-Module -ListAvailable | Where-Object { $_.Name -eq "GroupPolicy" }).Name) {
 Write-host "Unable to locate the GroupPolicy module. Exiting!" -foregroundColor Red
 exit
}
Import-Module -Name GroupPolicy

$badGpos = @()
$gpoList = Get-GPO -All -Domain $workingDomain | Where-Object { $_.DisplayName -match $matchFilter }
foreach($gpo in $gpoList) {
 $gpoPermissions = Get-GPPermissions -guid $gpo.Id -Domain $workingDomain -All
 $found = $false
 foreach($gpoPermission in $gpoPermissions) {
  if($gpoPermission.Trustee.Domain -eq $domain -and $gpoPermission.Trustee.Name -eq $sAMAccountName) {
   $found = $true
  }
 }
 if($found -eq $false) {
  $badGpos += $gpo
 }
}

$badGpos | Export-Csv -path $outputFile -noTypeInformation

Remove-Module -Name GroupPolicy