Importing text files with space separators into a csv in python

Question

Having trouble importing the following space separated data file into python and splitting them into a dataframe that I can work with. raw data file looks like this:

3300 0.272 0.302  69 153 21  4 31 104  22  80  4  3 1 0 0 0 "Andre Dawson     "
2600 0.269 0.335  58 111 17  2 18  66  39  69  0  3 1 1 0 0 "Steve Buchele    "

import pandas as pd
data = pd.read_csv('../data/ABRMdata', header=None)
split_text = []
for line in data:
    split_text.append(line)

return split_text

and I only get [0] returned But I want the data returned in a list of lists:

[3300,0.272,0.302,69,153,21,4,31,104,22,80,4,3,1, 0,0,0,"Andre Dawson  "]
[2600,0.269,0.335,58,111,17,2,18,66,39,69, 0,3,1,1,0,0,"Steve Buchele    "]

Any ideas?

Answer 1

It appears you may in fact have a file that is not space separated, but one with fixed with fields. If this is the case, check out pandas.read_fwf. http://pandas.pydata.org/pandas-docs/version/0.17.0/generated/pandas.read_fwf.html

Answer 2

As David mentioned pandas read_fwf could be used to create dataframe can be converted to python dictionary using to_dict() and lot other data structures.

In [30]: df = pd.read_fwf("filefor",header=None)

In [31]: df
Out[31]: 
     0      1      2   3    4   5   6   7    8   9   10  11  12  13  14  15  \
0  3300  0.272  0.302  69  153  21   4  31  104  22  80   4   3   1   0   0   
1  2600  0.269  0.335  58  111  17   2  18   66  39  69   0   3   1   1   0   

   16      17       18 19  
0   0  "Andre   Dawson  "  
1   0  "Steve  Buchele  "  

In [32]: df.to_dict()
Out[32]: 
{0: {0: 3300, 1: 2600},
 1: {0: 0.27200000000000002, 1: 0.26899999999999996},
 2: {0: 0.30199999999999999, 1: 0.33500000000000002},
 3: {0: 69, 1: 58},
 4: {0: 153, 1: 111},
 5: {0: 21, 1: 17},
 6: {0: 4, 1: 2},
 7: {0: 31, 1: 18},
 8: {0: 104, 1: 66},
 9: {0: 22, 1: 39},
 10: {0: 80, 1: 69},
 11: {0: 4, 1: 0},
 12: {0: 3, 1: 3},
 13: {0: 1, 1: 1},
 14: {0: 0, 1: 1},
 15: {0: 0, 1: 0},
 16: {0: 0, 1: 0},
 17: {0: '"Andre', 1: '"Steve'},
 18: {0: 'Dawson', 1: 'Buchele'},
 19: {0: '"', 1: '"'}}

Yes it infered space inbetween last field as delimiter to avoid widths=[1,5....] could be used

Other ds for usage

df.to_clipboard  df.to_hdf        df.to_period     df.to_string
df.to_csv        df.to_html       df.to_pickle     df.to_timestamp
df.to_dense      df.to_json       df.to_records    df.to_wide
df.to_dict       df.to_latex      df.to_sparse     
df.to_excel      df.to_msgpack    df.to_sql        
df.to_gbq        df.to_panel      df.to_stata 

Answer 3

Do you need to use pandas?

This code would get you started outside of pandas. (its not correct to your specification)

import csv

with open('/Users/toasteez/desktop/file.txt', 'r') as csvfile:
    w = csv.reader(csvfile)
    for line in w:
        newline = str.replace(line[0],' ',',')
        print(newline)