Saturday 25 September 2010

PBL Delete based on date

I wrote a python script to delete the files from PBL(Public Build Library) .In PBL large number of files reside, as the nightly builds of every release gets uploaded here.  To prevent space constraint we discuss with Testing team and  delete the older builds which they no longer use like builds older than two days. But the deletion process needs to be done manually ,check for the files which are older than 2 days and delete in using cubit API client. So i decided to write a simple script which gets the user inputs like filename, retain files older than and command line arguments for  cubit API client. Below is my script and the usage pattern,

#!/usr/bin/env python

from os.path import exists, join
from string import split
from os import pathsep
import urllib2, string
import datetime
import re
import subprocess
import os, sys
from optparse import OptionParser

parser = OptionParser()
parser.add_option("-u", "--user", dest="user", help="Enter the username of the PBl user")
parser.add_option("-k", "--key", dest="key", help="Enter the API key of the PBL user")
parser.add_option("-p", "--project", dest="project", help="Enter the project name of the file to be deleted")
parser.add_option("-l", "--api_url", dest="api_url", help="Enter the API url")
parser.add_option("-a", "--address", dest="address", help="Enter the exact site address where the file resides eg:https://mgr.cubit.sp.collab.net/pbl/releng/pub/test/")
parser.add_option("-f", "--filename", dest="filename", help="Enter the name of the file to be deleted")
parser.add_option("-t", "--project_type", dest="project_type", help="Enter the project type")
parser.add_option("--temp_file", dest="temp_file", help="Enter the name of the temporary file to write the webpage content for parsing")
parser.add_option("-d", "--delete_files_older_than", type ="int", dest="delete_files_older_than", help="Enter the number of days i.e the files older than this number of days will be deleted")
parser.add_option("-r", "--remote_path", dest="remote_path", help="Enter the remote path of the file eg:/test")

(options, args) = parser.parse_args()

if not options.user or not options.key or not options.project or not options.api_url or not options.address or not options.filename or not options.project_type or not options.temp_file or not options.delete_files_older_than or not options.remote_path:
sys.exit(parser.print_help())

if os.name == "nt":
pbl_exec = "pbl.exe"
else:
pbl_exec = "pbl.py"

search_path = os.getenv('PATH')

file_found = 0
paths = string.split(search_path, pathsep)
for path in paths:
if exists(join(path, pbl_exec)):
file_found = 1
break

if file_found == 1:
pass
else:
sys.exit("Unable to find PBL executable in the PATH,please install Cubit API Client or add it to the path")

current_date = datetime.datetime.now().date().strftime("%Y-%b-%d")
current_date = datetime.datetime.strptime(current_date, '%Y-%b-%d')

address = options.address + "/.cubit_pbl_index.txt"
website = urllib2.urlopen(address)
website_html = website.read()

f = open(options.temp_file, 'w')
f.write(website_html)
f.close()

a = open(options.temp_file, 'r')
for line in a:
if re.search(options.filename, line):
file_date = line.split(',')[1].split()[0]
file_date = datetime.datetime.strptime(file_date, '%Y-%b-%d')
diff = current_date - file_date
diff = diff.days

if diff >= options.delete_files_older_than:
file_to_delete = line.split(',')[0]
cmd = 'pbl delete -l %s -u %s -k %s -t %s -p %s -r %s/%s' %(options.api_url, options.user, options.key, options.project_type, options.project, options.remote_path, options.file_to_delete)
cmd = cmd.strip()
subprocess.call(cmd)

a.close()
if os.path.isfile(options.temp_file):
os.unlink(options.temp_file)

--------------------------------------------------------------------------------------------

C:\>python pbl_delete_win.py

User:madhu

Key:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Project:test

Project-type:pub

api_url:https://

Address:https://

filename:test

retain_files_older_than:1 Remote_path:test

pbl: Deleting test/test-1.txt

pbl: OK

pbl: Deleting test/test-2.txt

pbl: OK

pbl: Deleting test/test-3.txt

pbl: OK

pbl: Deleting test/test-4.txt

pbl: OK

Still this script needs to refined  ,because the current implementation will work properly only if both client and server are there same time zone else the wrong files might get deleted and the wrong files might get deleted if the specified file name exist as a part in other file name .If u see the above output though i have passed  "test"  as the filename "test-1.txt" also got deleted ,this needs to be validated .

No comments:

Post a Comment