Transform an Uploaded CSV File Upload with Flask

up vote
2
down vote

favorite

I have a simple Flask app hosted on Python Anywhere. The app provides an interface where a user can upload a CSV file, and the app inserts a new column into the uploaded CSV file by overwriting the first column (row[0]) and putting the original row[0] at the end of the file and downloads the transformed file for the user. So, the CSV input looks like this (first row is a header row that should be unmodified save for the addition of a 4th column name):

Pet1,Pet2,Pet3

Cats,Dogs,Mice

And the output should look like:

Pet1,Pet2,Pet3,Pet4

Birds,Dogs,Mice,Cats

Here is my code:

from flask import Flask, make_response, request

import operator

import io

import csv



app = Flask(__name__)



def transform(text_file_contents):

  reader = csv.reader(text_file_contents)

  all = 

  row = next(reader)

  row.append('Pet4')

  all.append(row)

  for row in reader:

    pet4 = row[0]

    row[0] = "Birds"

    row.append(pet4)

    all.append(row)

  sortedlist = sorted(all, key=operator.itemgetter(0))

  return sortedlist





@app.route('/')

def form():

  return """

    <html>

        <body>

            <h1>Upload a CSV File</h1>



            <form action="/transform" method="post" enctype="multipart/form-data">

                <input type="file" name="data_file" />

                <input type="submit" />

            </form>

        </body>

    </html>

"""



@app.route('/transform', methods=["POST"])

def transform_view():

  request_file = request.files['data_file']

  if not request_file:

    return "No file"



file_contents = request_file.stream.read().decode("utf-8")



result = transform(file_contents)



response = make_response(result)

response.headers["Content-Disposition"] = "attachment; filename=result.csv"

return response

This is the error I am getting:

Traceback (most recent call last):

File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app

response = self.full_dispatch_request()

File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request

rv = self.handle_user_exception(e)

File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception

reraise(exc_type, exc_value, tb)

File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise

raise value

File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request

rv = self.dispatch_request()

File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request

return self.view_functions[rule.endpoint](**req.view_args)

File "/home/user/mysite/callnumbers.py", line 62, in transform_view

result = transform(file_contents)

File "/home/user/mysite/callnumbers.py", line 18, in transform

row[0] = row[1]

IndexError: list index out of range

I can tell that my script isn't reading the uploaded input file correctly (hence not being able to find the data in the columns I am trying to reference and the IndexError) but I haven't been able to find any examples of Flask using the Python CSV library to parse an uploaded file and manipulate columns in the way I need to. How can I resolve this? Any pointers in the right direction are much appreciated!

edited Nov 14 at 23:01

Joel

1,5866719

asked Nov 14 at 22:30

lpm

156

1

According to this question, the solution is to do request_file.read() instead of request_file.stream.read(). I've never done it that way myself but it seemed to work for that question's OP.
– noslenkwah
Nov 14 at 22:44

add a comment |

up vote
2
down vote

favorite

Pet1,Pet2,Pet3

Cats,Dogs,Mice

And the output should look like:

Pet1,Pet2,Pet3,Pet4

Birds,Dogs,Mice,Cats

Here is my code:

from flask import Flask, make_response, request

import operator

import io

import csv



app = Flask(__name__)



def transform(text_file_contents):

  reader = csv.reader(text_file_contents)

  all = 

  row = next(reader)

  row.append('Pet4')

  all.append(row)

  for row in reader:

    pet4 = row[0]

    row[0] = "Birds"

    row.append(pet4)

    all.append(row)

  sortedlist = sorted(all, key=operator.itemgetter(0))

  return sortedlist





@app.route('/')

def form():

  return """

    <html>

        <body>

            <h1>Upload a CSV File</h1>



            <form action="/transform" method="post" enctype="multipart/form-data">

                <input type="file" name="data_file" />

                <input type="submit" />

            </form>

        </body>

    </html>

"""



@app.route('/transform', methods=["POST"])

def transform_view():

  request_file = request.files['data_file']

  if not request_file:

    return "No file"



file_contents = request_file.stream.read().decode("utf-8")



result = transform(file_contents)



response = make_response(result)

response.headers["Content-Disposition"] = "attachment; filename=result.csv"

return response

This is the error I am getting:

Traceback (most recent call last):

File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app

response = self.full_dispatch_request()

File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request

rv = self.handle_user_exception(e)

File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception

reraise(exc_type, exc_value, tb)

File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise

raise value

File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request

rv = self.dispatch_request()

File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request

return self.view_functions[rule.endpoint](**req.view_args)

File "/home/user/mysite/callnumbers.py", line 62, in transform_view

result = transform(file_contents)

File "/home/user/mysite/callnumbers.py", line 18, in transform

row[0] = row[1]

IndexError: list index out of range

edited Nov 14 at 23:01

Joel

1,5866719

asked Nov 14 at 22:30

lpm

156

1

According to this question, the solution is to do request_file.read() instead of request_file.stream.read(). I've never done it that way myself but it seemed to work for that question's OP.
– noslenkwah
Nov 14 at 22:44

add a comment |

up vote
2
down vote

favorite

Pet1,Pet2,Pet3

Cats,Dogs,Mice

And the output should look like:

Pet1,Pet2,Pet3,Pet4

Birds,Dogs,Mice,Cats

Here is my code:

from flask import Flask, make_response, request

import operator

import io

import csv



app = Flask(__name__)



def transform(text_file_contents):

  reader = csv.reader(text_file_contents)

  all = 

  row = next(reader)

  row.append('Pet4')

  all.append(row)

  for row in reader:

    pet4 = row[0]

    row[0] = "Birds"

    row.append(pet4)

    all.append(row)

  sortedlist = sorted(all, key=operator.itemgetter(0))

  return sortedlist





@app.route('/')

def form():

  return """

    <html>

        <body>

            <h1>Upload a CSV File</h1>



            <form action="/transform" method="post" enctype="multipart/form-data">

                <input type="file" name="data_file" />

                <input type="submit" />

            </form>

        </body>

    </html>

"""



@app.route('/transform', methods=["POST"])

def transform_view():

  request_file = request.files['data_file']

  if not request_file:

    return "No file"



file_contents = request_file.stream.read().decode("utf-8")



result = transform(file_contents)



response = make_response(result)

response.headers["Content-Disposition"] = "attachment; filename=result.csv"

return response

This is the error I am getting:

Traceback (most recent call last):

File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app

response = self.full_dispatch_request()

File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request

rv = self.handle_user_exception(e)

File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception

reraise(exc_type, exc_value, tb)

File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise

raise value

File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request

rv = self.dispatch_request()

File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request

return self.view_functions[rule.endpoint](**req.view_args)

File "/home/user/mysite/callnumbers.py", line 62, in transform_view

result = transform(file_contents)

File "/home/user/mysite/callnumbers.py", line 18, in transform

row[0] = row[1]

IndexError: list index out of range

edited Nov 14 at 23:01

Joel

1,5866719

asked Nov 14 at 22:30

lpm

156

Pet1,Pet2,Pet3

Cats,Dogs,Mice

And the output should look like:

Pet1,Pet2,Pet3,Pet4

Birds,Dogs,Mice,Cats

Here is my code:

from flask import Flask, make_response, request

import operator

import io

import csv



app = Flask(__name__)



def transform(text_file_contents):

  reader = csv.reader(text_file_contents)

  all = 

  row = next(reader)

  row.append('Pet4')

  all.append(row)

  for row in reader:

    pet4 = row[0]

    row[0] = "Birds"

    row.append(pet4)

    all.append(row)

  sortedlist = sorted(all, key=operator.itemgetter(0))

  return sortedlist





@app.route('/')

def form():

  return """

    <html>

        <body>

            <h1>Upload a CSV File</h1>



            <form action="/transform" method="post" enctype="multipart/form-data">

                <input type="file" name="data_file" />

                <input type="submit" />

            </form>

        </body>

    </html>

"""



@app.route('/transform', methods=["POST"])

def transform_view():

  request_file = request.files['data_file']

  if not request_file:

    return "No file"



file_contents = request_file.stream.read().decode("utf-8")



result = transform(file_contents)



response = make_response(result)

response.headers["Content-Disposition"] = "attachment; filename=result.csv"

return response

This is the error I am getting:

Traceback (most recent call last):

File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app

response = self.full_dispatch_request()

File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request

rv = self.handle_user_exception(e)

File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception

reraise(exc_type, exc_value, tb)

File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise

raise value

File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request

rv = self.dispatch_request()

File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request

return self.view_functions[rule.endpoint](**req.view_args)

File "/home/user/mysite/callnumbers.py", line 62, in transform_view

result = transform(file_contents)

File "/home/user/mysite/callnumbers.py", line 18, in transform

row[0] = row[1]

IndexError: list index out of range

python csv flask pythonanywhere

edited Nov 14 at 23:01

Joel

1,5866719

asked Nov 14 at 22:30

lpm

156

edited Nov 14 at 23:01

Joel

1,5866719

asked Nov 14 at 22:30

lpm

156

edited Nov 14 at 23:01

Joel

1,5866719

edited Nov 14 at 23:01

Joel

1,5866719

edited Nov 14 at 23:01

Joel

1,5866719

asked Nov 14 at 22:30

lpm

156

asked Nov 14 at 22:30

lpm

156

asked Nov 14 at 22:30

lpm

156

1

According to this question, the solution is to do request_file.read() instead of request_file.stream.read(). I've never done it that way myself but it seemed to work for that question's OP.
– noslenkwah
Nov 14 at 22:44

add a comment |

1

According to this question, the solution is to do request_file.read() instead of request_file.stream.read(). I've never done it that way myself but it seemed to work for that question's OP.
– noslenkwah
Nov 14 at 22:44

According to this question, the solution is to do request_file.read() instead of request_file.stream.read(). I've never done it that way myself but it seemed to work for that question's OP.
– noslenkwah
Nov 14 at 22:44

add a comment |

1 Answer
1

active

oldest

votes

up vote
0
down vote

The csv.reader() function takes a file object instead of text. One way to do it is to save the file then reopen it in the transform function.

from flask import Flask, make_response, request

import operator

import io

import csv    



app = Flask(__name__)





def transform():

  with open('myfile.csv', 'rb') as f:

    reader = csv.reader(f)  

    all = 

    row = next(reader)

    row.append('Pet4')

    all.append(row)

    for row in reader:

      pet4 = row[0]

      row[0] = "Birds"

      row.append(pet4)

      all.append(row)

    sortedlist = sorted(all, key=operator.itemgetter(0))  

  return sortedlist



@app.route('/')

def form():

  return """

    <html>

        <body>

            <h1>Upload a CSV File</h1>



            <form action="/transform" method="post" enctype="multipart/form-data">

                <input type="file" name="data_file" />

                <input type="submit" />

            </form>

        </body>

    </html>

"""



@app.route('/transform', methods=["POST"])

def transform_view():

  request_file = request.files['data_file']

  request_file.save("myfile.csv")

  if not request_file:

    return "No file"

  #file_contents = io.StringIO(request_file.stream.read().decode("UTF8"), newline=None)

  #file_contents = request_file.stream.read().decode("utf-8")



  result = transform()

  print result



  response = make_response(result)

  response.headers["Content-Disposition"] = "attachment; filename=result.csv"

  return response

if __name__ == '__main__':

    app.run()

edited Nov 15 at 17:03

answered Nov 15 at 15:19

Mike C.

156113

add a comment |

Your Answer

StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});

}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53309707%2ftransform-an-uploaded-csv-file-upload-with-flask%23new-answer', 'question_page');
}
);

Post as a guest

Name

Required, but never shown

1 Answer
1

active

oldest

votes

1 Answer
1

active

oldest

votes

up vote
0
down vote

The csv.reader() function takes a file object instead of text. One way to do it is to save the file then reopen it in the transform function.

from flask import Flask, make_response, request

import operator

import io

import csv    



app = Flask(__name__)





def transform():

  with open('myfile.csv', 'rb') as f:

    reader = csv.reader(f)  

    all = 

    row = next(reader)

    row.append('Pet4')

    all.append(row)

    for row in reader:

      pet4 = row[0]

      row[0] = "Birds"

      row.append(pet4)

      all.append(row)

    sortedlist = sorted(all, key=operator.itemgetter(0))  

  return sortedlist



@app.route('/')

def form():

  return """

    <html>

        <body>

            <h1>Upload a CSV File</h1>



            <form action="/transform" method="post" enctype="multipart/form-data">

                <input type="file" name="data_file" />

                <input type="submit" />

            </form>

        </body>

    </html>

"""



@app.route('/transform', methods=["POST"])

def transform_view():

  request_file = request.files['data_file']

  request_file.save("myfile.csv")

  if not request_file:

    return "No file"

  #file_contents = io.StringIO(request_file.stream.read().decode("UTF8"), newline=None)

  #file_contents = request_file.stream.read().decode("utf-8")



  result = transform()

  print result



  response = make_response(result)

  response.headers["Content-Disposition"] = "attachment; filename=result.csv"

  return response

if __name__ == '__main__':

    app.run()

edited Nov 15 at 17:03

answered Nov 15 at 15:19

Mike C.

156113

add a comment |

up vote
0
down vote

The csv.reader() function takes a file object instead of text. One way to do it is to save the file then reopen it in the transform function.

from flask import Flask, make_response, request

import operator

import io

import csv    



app = Flask(__name__)





def transform():

  with open('myfile.csv', 'rb') as f:

    reader = csv.reader(f)  

    all = 

    row = next(reader)

    row.append('Pet4')

    all.append(row)

    for row in reader:

      pet4 = row[0]

      row[0] = "Birds"

      row.append(pet4)

      all.append(row)

    sortedlist = sorted(all, key=operator.itemgetter(0))  

  return sortedlist



@app.route('/')

def form():

  return """

    <html>

        <body>

            <h1>Upload a CSV File</h1>



            <form action="/transform" method="post" enctype="multipart/form-data">

                <input type="file" name="data_file" />

                <input type="submit" />

            </form>

        </body>

    </html>

"""



@app.route('/transform', methods=["POST"])

def transform_view():

  request_file = request.files['data_file']

  request_file.save("myfile.csv")

  if not request_file:

    return "No file"

  #file_contents = io.StringIO(request_file.stream.read().decode("UTF8"), newline=None)

  #file_contents = request_file.stream.read().decode("utf-8")



  result = transform()

  print result



  response = make_response(result)

  response.headers["Content-Disposition"] = "attachment; filename=result.csv"

  return response

if __name__ == '__main__':

    app.run()

edited Nov 15 at 17:03

answered Nov 15 at 15:19

Mike C.

156113

add a comment |

up vote
0
down vote

The csv.reader() function takes a file object instead of text. One way to do it is to save the file then reopen it in the transform function.

from flask import Flask, make_response, request

import operator

import io

import csv    



app = Flask(__name__)





def transform():

  with open('myfile.csv', 'rb') as f:

    reader = csv.reader(f)  

    all = 

    row = next(reader)

    row.append('Pet4')

    all.append(row)

    for row in reader:

      pet4 = row[0]

      row[0] = "Birds"

      row.append(pet4)

      all.append(row)

    sortedlist = sorted(all, key=operator.itemgetter(0))  

  return sortedlist



@app.route('/')

def form():

  return """

    <html>

        <body>

            <h1>Upload a CSV File</h1>



            <form action="/transform" method="post" enctype="multipart/form-data">

                <input type="file" name="data_file" />

                <input type="submit" />

            </form>

        </body>

    </html>

"""



@app.route('/transform', methods=["POST"])

def transform_view():

  request_file = request.files['data_file']

  request_file.save("myfile.csv")

  if not request_file:

    return "No file"

  #file_contents = io.StringIO(request_file.stream.read().decode("UTF8"), newline=None)

  #file_contents = request_file.stream.read().decode("utf-8")



  result = transform()

  print result



  response = make_response(result)

  response.headers["Content-Disposition"] = "attachment; filename=result.csv"

  return response

if __name__ == '__main__':

    app.run()

edited Nov 15 at 17:03

answered Nov 15 at 15:19

Mike C.

156113

The csv.reader() function takes a file object instead of text. One way to do it is to save the file then reopen it in the transform function.

from flask import Flask, make_response, request

import operator

import io

import csv    



app = Flask(__name__)





def transform():

  with open('myfile.csv', 'rb') as f:

    reader = csv.reader(f)  

    all = 

    row = next(reader)

    row.append('Pet4')

    all.append(row)

    for row in reader:

      pet4 = row[0]

      row[0] = "Birds"

      row.append(pet4)

      all.append(row)

    sortedlist = sorted(all, key=operator.itemgetter(0))  

  return sortedlist



@app.route('/')

def form():

  return """

    <html>

        <body>

            <h1>Upload a CSV File</h1>



            <form action="/transform" method="post" enctype="multipart/form-data">

                <input type="file" name="data_file" />

                <input type="submit" />

            </form>

        </body>

    </html>

"""



@app.route('/transform', methods=["POST"])

def transform_view():

  request_file = request.files['data_file']

  request_file.save("myfile.csv")

  if not request_file:

    return "No file"

  #file_contents = io.StringIO(request_file.stream.read().decode("UTF8"), newline=None)

  #file_contents = request_file.stream.read().decode("utf-8")



  result = transform()

  print result



  response = make_response(result)

  response.headers["Content-Disposition"] = "attachment; filename=result.csv"

  return response

if __name__ == '__main__':

    app.run()

edited Nov 15 at 17:03

answered Nov 15 at 15:19

Mike C.

156113

edited Nov 15 at 17:03

answered Nov 15 at 15:19

Mike C.

156113

answered Nov 15 at 15:19

Mike C.

156113

answered Nov 15 at 15:19

Mike C.

156113

add a comment |

draft saved

draft discarded

Thanks for contributing an answer to Stack Overflow!

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

Some of your past answers have not been well-received, and you're in danger of being blocked from answering.

Please pay close attention to the following guidance:

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Name

Required, but never shown

Name

Required, but never shown

This page is only for reference, If you need detailed information, please check here

搜尋此網誌

Cfrgtkky