Powershell Log File Zipper…….

Another annoying repetitive task automated ! Ahhhhhhh :o)

The log files for our IIS servers build up over a couple of months and consume disk space to the point of becoming an issue. I had been manually logging on and creating a .zip file for each month and then dragging the individual files into the zip file.

Tedious and repetitive, just the sort of thing scripting was made for. Typicaly my scripts for Windows are in vbscript, but this time I decided it was time to look at powershell.

The logic was simple enough. First create a zip file for the previous month. The logfile names are in the format ‘u_ex.log’ where mm is the numerical month. I needed to get the current month, subtract one (compressing previous months logs) and then move any file with a mm figure that matched into the .zip file.

I found this article for creating the actual zip file itself. Then is was just a case of setting the paths and looping through each file in the directory (get-childitem makes this a breeze).

Debugging the script was a bit of a pain in the arse using the built in Windows tools, so I downloaded and installed PowerShell GUI from here. It’s still a work in progress in some respects, the built in intellisense isn’t 100% there just yet, but it does allow you to step through code and what the vaule of your variables and figure out what is happening.

Once I had the script working of the command line, it was just a case of scheduling the job to run on the 1st day of each month recurring. However, this proved to be almost as big a task as the script itself.

For some reason, running powershell.exe and passing the script to it as an argument, ever using the full -command “$ syntax failed to execute the script. In the end I had to create a batch/.cmd file and place the powershell command in there and schedule the batch file to run instead.

My only gripe is that the command to move the file into the zip file is asynchronus so I had to include a sleep/wait period to give the file time to be compressed before being moved into the .zip file. That being said, there will never be more the x31 files, and allowing x2 mins for each file to compress (extremely generous) means the script should never take more than x1 hour to complete, so as long as I run it during a quiet period it should not impact anything else.

I eagerly await the 1st of July to see if the scheduled job kicks in automatically on the live evironment……:o/

#declare functions here
function new-zipfile {
	param ($zipfile)
	if (! $zipfile.endswith('.zip')) {$zipfile += '.zip'}
	set-content $zipfile ("PK" + [char]5 + [char]6 + ("$([char]0)" * 18))
	(dir $zipfile).IsReadOnly = $false
}

#define variables here
#some strings and numbers we will need
$thismonthint = get-date -f "MM"
$prevmonthint = (get-date).addmonths(-1).tostring("MM")

$thismonthstr = get-date -f "MMM"
$prevmonthstr = (get-date).addmonths(-1).tostring("MMM")

$thisyearlongint = get-date -f "yyyy"
$thisyearshortint = get-date -f "yy"

$thislogdir = 'C:\weblogs\'
$thiszipfile = $thislogdir + $prevmonthstr + $thisyearlongint + '.zip'
$zipexists = test-path $thiszipfile

#start program here
#first pass, check for .zip files of previous months. if exists exit. if not exist, create empty .zip file
if (! $zipexists)
{
	echo 'zip file does not exist, creating zip file'
	new-zipfile $thiszipfile
}
else
{
	return
}

# move all log files where the month number matches the month number of the .zip file
# Jan = 01, Feb = 02, Mar =03 etc. etc.

foreach ($file in Get-ChildItem $thislogdir)
{
	# exclude the .zip files already in the directory (just in case we get a random month match in their filename
	if (! $file.name.endswith(".zip"))
	{
		# if the
		if ($file.Name.substring(6,2) -match $prevmonthint)
		{
			$zipfile = (New-Object -ComObject shell.application).NameSpace($thiszipfile)
			$zipfile.MoveHere($file.fullname)
			Start-Sleep -Seconds 120
		}
	}
}

squish !!

Tags: ,

5 Responses to “Powershell Log File Zipper…….”

  1. mel says:

    thank you!!!

  2. Casper says:

    Just what I was looking for!! I incorporated this into a cleanup script that cleans files on multiple servers. By placing it into an if-section that checks if the Microsoft Exchange Information Store Service is running it only zips to iis-logs on that server.

  3. scottb says:

    Glad it helped someone :o)

    Must confess I never managed to get it run run properly via the Windows task scheduler though ?!!?? I just used to fire it off on the 1st of each month and let it run :o/

  4. Frank Gallagher says:

    We modified the script sleep setting based on the information we found here at the URL below. It checks to make sure the the file is present in the zip before moving on to the next file. This change cut the runtime of the script in half for us.

    http://superuser.com/questions/290461/powershell-zip-file-synchronously

    In a nutshell, the idea behind this is testing the output zip archive to see if the file has been added. If not, sleep for a second and try again. If the filename does not occur in the archive, then we assume the zipping is still occurring.

    This seems to work well. Maybe to be even more “sure” we can add one more second for cleanup to occur after the file has been added.”

    Our version of the script:

    #declare functions here
    function new-zipfile {
    param ($zipfile)
    if (! $zipfile.endswith(‘.zip’)) {$zipfile += ‘.zip’}
    set-content $zipfile (“PK” + [char]5 + [char]6 + (“$([char]0)” * 18))
    (dir $zipfile).IsReadOnly = $false
    }

    #define variables here
    #some strings and numbers we will need
    $ShellApplication = New-Object -com Shell.Application

    $thismonthint = get-date -f “MM”
    $prevmonthint = (get-date).addmonths(-1).tostring(“MM”)

    $thisyearlongint = get-date -f “yyyy”

    $thislogdir = ‘E:\weblogs\W3SVC2\’
    $thiszipfile = $thislogdir + $thisyearlongint + $prevmonthint + ‘.zip’
    $zipexists = test-path $thiszipfile

    #start program here
    #first pass, check for .zip files of previous months. if exists exit. if not exist, create empty .zip file
    if (! $zipexists)
    {
    echo ‘zip file does not exist, creating zip file’
    new-zipfile $thiszipfile
    }
    else
    {
    return
    }

    # move all log files where the month number matches the month number of the .zip file
    # Jan = 01, Feb = 02, Mar = 03 etc. etc.
    foreach ($file in Get-ChildItem $thislogdir)
    {

    # exclude the .zip files already in the directory (just in case we get a random month match in their filename)
    if (! $file.name.endswith(“.zip”))
    {

    # if the
    if ($file.Name.substring(6,2) -match $prevmonthint)
    {
    $zipfile = $ShellApplication.NameSpace($thiszipfile)
    $zipfile.MoveHere($file.fullname)
    while($zipfile.Items().Item($file.name) -Eq $null)
    {
    start-sleep -seconds 1
    }
    }
    }
    }

  5. scottb says:

    Cool !! I left the company shortly after I put that script in, so I never got back round to revising it. Will have a play when I get some free time. Thanks ;op

Leave a Reply