Tuesday, May 2, 2017

Python code snippet : Merge multiple CSVs (Column Wise) into a large CSV file using ID

Following is the python code snippet to merge multiple CSVs into a large CSV (column wise) using CSV's unique ID field. All input CSV's must have same length.

 import csv
 import itertools
 
 outCombinedStatTxtName ="FormattedTxtResults.csv"  
 tempCsvFiles = glob.glob(str(TempFolderPath)+'\*.csv')  
 openedFileList = [open(fn, 'rb') for fn in tempCsvFiles]  
 readers = [csv.reader(fn) for fn in openedFileList]  
   
 result = open(outCombinedStatTxtName, 'wb')  
 writer = csv.writer(result,delimiter=',')  
   
 multipleIndices = False  
 for row_chunks in itertools.izip(*readers):  
   tempFormattedRow = list(itertools.chain.from_iterable(row_chunks))  
   fidIndices = [i for i, x in enumerate(tempFormattedRow) if x == "ID"]  
   if(len(fidIndices) > 1):  
     fidIndices.pop(0)  
     redundantIndices = tuple(fidIndices)  
     multipleIndices = True  
       
   if(multipleIndices):  
     tempFormattedRow = [ tempFormattedRow[i] for i in xrange(len(tempFormattedRow)) if i not in set(redundantIndices) ]      
     writer.writerow(tempFormattedRow)  
   else:  
     writer.writerow(tempFormattedRow)  
 result.close()     
 [fn.close() for fn in openedFileList]  


Python

0 comments :

Post a Comment

 

© 2011 GIS and Remote Sensing Tools, Tips and more .. ToS | Privacy Policy | Sitemap

About Me