How to use a CSV file upload to change an OptionFields' input dynamically

I have completed the computer science tutorial. I saw the extra tasks where it mentions to make the app more dynamic by asking a user to upload a file to analyze and decided to try it out.

Since the variable file is a FileField and FileField returns a type file I assumed the best approach would be to add a parameter in the extract_data() function where the file is passed. That did not work as it gives an error that says: “‘FileField’ object has no attribute ‘file’” (note the files are csv)

Here is the code:

def extract_data(file):
data = pd.read_csv(file, sep=‘;’)
return data

class Parametrization(ViktorParametrization):
#uploading file
file = FileField(‘Upload a CSV’, file_types=[‘.csv’])

#optionfields
main_column = OptionField('Choose main property', options=extract_data(file.file).columns.values.tolist())
count_column = OptionField('Choose property to analyse', options=extract_data(file.file).columns.values.tolist())
1 Like

Hi Ibrahim,

Welcome to the community! And thank you for your question. If I understand you correctly, you would like to have a dynamic list of options based on the uploaded csv file. To achieve this, some things need to be understood first:

  1. Callback functions

One of the ways to create a dynamic option field is by defining a function that produces a list of options based on certain input and logic. These functions need to be defined in a certain format before they can be added to the options argument of the OptionField object. The format is as follows:

def name_of_callback_function(params, **kwargs):
    ...
    return [...]

The input that can be passed on in the key word arguments are:

  • params: the parameters defined in the parametrization
  • entity_id
  • entity_name

For your example, only the params will be necessary.

  1. The FileField

The FileField is used to if a developer wants to upload a file of sorts. To be able to retrieve the uploaded file, this can be retrieved from the name of the variable on which the FileField is defined. In your case, the name file. The object that is obtained, however, is not a File object, but rather a FileResource object. A FileResource object holds both the properties file (which returns a File object) and filename.

  1. Combining both

Knowing about this, to achieve your case, let us first rewrite your callback function:

import StringIO
import pandas as pd
from viktor import File
from viktor.api_v1 import FileResource

def extract_data(params, **kwargs):
    my_csv_file_resource: FileResource = params.file
    if not my_csv_file_resource:
        # in the case of no file uploaded, return an empty list of options
        return []
    my_csv_file: File = my_csv_file_resource.file
    # convert `File` to `StringIO`
    csv_stringio = StringIO(my_csv_file.getvalue())
    # convert `StringIO` to `DataFrame`
    df = pd.read_csv(csv_stringio, sep=‘;’)
    options = df.columns.values.tolist()
    return options

(For more information on converting a string to a pandas DataFrame object, refer to this link.)

And then, the parametrization:

from viktor.parametrization import ViktorParametrization, FileField, OptionField

class Parametrization(ViktorParametrization):
    #uploading file
    file = FileField(‘Upload a CSV’, file_types=[‘.csv’])

    #optionfields
    main_column = OptionField('Choose main property', options=extract_data)
    count_column = OptionField('Choose property to analyse', options=extract_data)

For more information, refer to the following link, which also describes how callback functions can be used to make an OptionField dynamic.

I hope this answers your question.

2 Likes

I’ve also changed the title of this topic to better suit the discussion.

This seems to work for options, however when i want to do something similar for a table or a dynamicarray i get an error that the function is not iterable:
cannot unpack non-iterable function object

Any thought on how to deal with that?

Hi @nielszee ,

Welcome to the community! And thanks for posting your question here.

I am trying to understand in what way are you trying to use a callback function for a table or dynamic array. Could you elaborate on your issue, maybe with a sample snippet to explain your case?

Thanks Marcel! So, i want to upload a file in which I stored some data. From this data i want to create a dynamic array.

class Parametrization(ViktorParametrization):
    step_1 = Step('Phase processing')
    step_1.file = FileField('Upload a file')
    step_1.array = DynamicArray("Overzicht", default = extract_data)
def extract_data(params, **kwargs):
    my_json_file_resource: FileResource = params.step_1.file
    if not my_json_file_resource:
        PHASE_DATA = PHASE_DATA_DUMMY
    else:
        my_json_file: File = my_json_file_resource.file
        with open(my_json_file) as f:
            data = json.load(f)
            PHASE_DATA = [data]
    Phases = []
    for phase in PHASE_DATA[0]:
        Phases.append({
            "phase_name": phase,
            "phase_type": "GGT",
            "phase_color": random_color()
        })
    return Phases

Let me know if this helps!

Ah okey, I see. This is a bit different from making options dynamic, as this requires fields to actually be filled. To fill any type of field, including a DynamicArray, one needs to use the SetParamsButton:

In your case, make sure to add your function extract_data in the Controller, and add the method name to SetParamsButton argument method:

class Parametrization(ViktorParametrization):
    step_1 = Step('Phase processing')
    step_1.file = FileField('Upload a file')
    step_1.set_params = SetParamsButton("Fill dynamic array", method="extract_data")
    step_1.array = DynamicArray("Overzicht", default = extract_data)

class MyController(ViktorController):
    ...

    def extract_data(params, **kwargs):
    my_json_file_resource: FileResource = params.step_1.file
    if not my_json_file_resource:
        PHASE_DATA = PHASE_DATA_DUMMY
    else:
        my_json_file: File = my_json_file_resource.file
        with open(my_json_file) as f:
            data = json.load(f)
            PHASE_DATA = [data]
    Phases = []
    for phase in PHASE_DATA[0]:
        Phases.append({
            "phase_name": phase,
            "phase_type": "GGT",
            "phase_color": random_color()
        })
    params.step_1.array = Phases
    return SetParamsResult(params)

Okay, that makes sense. Thanks for the quick repsonse! I tried to change the code. However now in the parametrization i get an error that “extract_data” is not defined?

If that is the case, try the following:

  • refresh the page
  • check that the method extract_data is defined in the Controller.

This is the current code:


class Parametrization(ViktorParametrization):
    step_1 = Step('Phase processing')
    step_1.file = FileField('Upload a file')
    step_1.set_params = SetParamsButton("Fill dynamic array", method="extract_data")
    step_1.array = DynamicArray("Overzicht", default = extract_data)
    step_1.array.phase_name = TextField("Phase name")
    step_1.array.phase_type = OptionField("Phase type", options=DEFAULT_PHASE_TYPES)
    step_1.array.phase_color = TextField('Pick a color (r,g,b)')

class Controller(ViktorController):
    label = 'My Entity Type'
    parametrization = Parametrization
    
    def extract_data(params, **kwargs):
        my_json_file_resource: FileResource = params.step_1.file
        if not my_json_file_resource:
            PHASE_DATA = PHASE_DATA_DUMMY
        else:
            my_json_file: File = my_json_file_resource.file
            with open(my_json_file) as f:
                data = json.load(f)
                PHASE_DATA = [data]
        Phases = []
        for phase in PHASE_DATA[0]:
            Phases.append({
                "phase_name": phase,
                "phase_type": "GGT",
                "phase_color": random_color()
            })
        params.step_1.array = Phases
        return params
    
    @PlotlyView('Line chart', duration_guess=1)
    def generate_plotly_view(self, params, **kwargs):
        # Make the line chart
        fig = go.Figure()
        phases = params.step_1.array
        for i, phase in enumerate(params.step_2.phase_selection):
            for plate in params.step_2.plate_selection:
                phase_name = str(phase).strip()
                plate_name = str(plate).strip()
                # fig.add_trace(go.Scatter(x=PHASE_DATA[0][phase_name][plate_name]["X"], y=PHASE_DATA[0][phase_name][plate_name]["Y"], mode='lines',name=phase,line=dict(color=viktor_color_to_rgb(phases[i]["phase_color"]), width=1.5,dash="dash")
                # ))
        
        return PlotlyResult(fig.to_json())

Oh my apologies, I forgot to add that the parameters need to be returned through a SetParamsResult. I’ve updated the code in my previous post

Unfortunately, same thing :slight_smile:

Looking at your code, I see that you still have the default=extract_data in your DynamicArray. Remove that, and try again.

Ah, now it actually loads. I now get an error that it gets multiple values for argument params. How does it know on calling the method which element in the parametrization to fill?

Your parametrization is a dictionary with values that represent your parametrization input. You could therefore either reconstruct the parametrization input as a dictionary, as shown in the documentation, or you could overwrite the parameters that you want to fill/change like I have done with params.step_1.array = Phases.

Looking at what might cause the error that you are experiencing, I think it has to do that your method does not have a self within it. You could do one of the following:

def extract_data(self, params, **kwargs):
   ...

or

@staticmethod
def extract_data(params, **kwargs):
1 Like

That worked. Awesome. Thanks for your patience.

1 Like