Process 100 files evenly across 10 nodes
I was recently asked to write a script assuming there are 100 files and they're all equal-sized, how to process them evenly across 10 nodes. I went in completely wrong direction suggesting regex where my regex-fu is really not that strong. Thinking it over on a different occasion, I came up with a simple script that will do just that. Of course the simplest solution is the best. As usual, your comments are welcome. __author__ = ' artem ' nodes = {} files = [] for filenum in range ( 100 ): filename = ' file_00 ' + str (filenum) files.append(filename) for node in range ( 10 ): subset = files[ 0 : 10 ] nodes[node] = subset files[ 0 : 10 ] = [] for node in nodes: print (node, nodes[node]) and the output: 0 ['file_000', 'file_001', 'file_002', 'file_003', 'file_004', 'file_005', 'file_006', 'file_007', 'file_008', 'file_00