Memory error with downloadbutton

🗪 Support (Q&A)
1

Hi VIKTOR team,

I’d like to ask a question. Previously, my VIKTOR app worked fine. Now I updated the viktor-cli and a new geolib wheel. In my development environment the tools runs fine, but now I have published it and and running the tool gives me the following error:

image

Could you please help me? Thanks in advance

2

@yida.tao On a published environment the maximum memory limit is 500MB . Locally you can exceed this limit. In the docs at Command-line interface (CLI) | VIKTOR Documentation you can find the following command:

viktor-cli start --max-memory 500

This will ensure the same memory limit as in a published application. Hope you can debug the application in this fashion!

3

Hi Sylvain,

Thanks for your suggestion. I ran the tool using your proposed command. It still works in the development mode.

However, the published tool gives an error. It surprises me because the downloaded file is about 7 MB. So I’d say it is nowhere near the memory limit.

Could you please help me further?

4

Hi @yida.tao ,

We are aware that sometimes there is a mismatch on local memory consumption and memory consumption in production. This is high on our priority list.

For now the most straightforward solution would be to decrease memory consumption in your application. Can you perhaps share how you generate the download? Please be sure to remove any sensitive information before posting!

5

Hi @Sylvain,

Good to know this memory issue is something you are working hard on.

Unfortunately, I don’t know how I can decrease the memory consumption, nor can I easily share how the download is generated, because we make use of internal wheels. Could we please have a call?

6

Hi Viktor-team,

Unfortunately, this issue is still open. I wondered if you have any updates. Also, I’d like to ask if changing the heap size settings in Pycharm be similar to setting a memory limit on the viktor app while developing?

7

Hi @yida.tao

Unfortunately, we are still in the phase of investigating a way to accurately and reliably ‘predict’ an out-of-memory error in production within the development environment. We will make sure to communicate this on the Community Forum when we deploy a solution for this.

Since a VIKTOR app is running on your system using Python directly (or within a Docker container in case you are using “docker” isolation mode), and not in PyCharm, setting the heap size in PyCharm won’t influence this.

8

Hi Viktor team,

It has been a while. I am still awaiting news on this front, so my first Viktor app can be published and run without errors. It is for automatically generating appendices with input/output for stix files. My colleagues are also eagerly awaiting the online launch of the tool. It would be very convenient for any D-Stability project. Meanwhile, I had three questions:

  1. Do you have any updates on how to predict an out-of-memory error in production?
  2. Do you have any tips on pinpointing which part of the code is causing the large memory consumption?
  3. Is there any way to program around the memory error? So, for example, with try/except: try generating an elaborate appendix, except there is a memory error, generate a small appendix with the basics.

Thank you!

10

Hi Yida, I send you an email. Hopefully we can resolve this as soon as possible!

11

The app behaves quite well now. We could not pinpoint which of the following points is the bottleneck, but here is a list of causes of the memory error:

Thanks again for the help :smiley:

1 Like
12

I would like to chime in on the memory error issue, as it is something I had to deal with in my apps. I was able to fix the issue myself by rewriting some code to use memory more efficiently, but it was a frustrating experience to troubleshoot. The only way to properly test whether the app will run into this issue, is to publish the app and run some tests on the published version of the app. As my app depends on a lot of different parameters including multiple uploadfields, you can imagine how cumbersome this debugging process is.

The suggested CLI-command to limit max-memory does not accurately reflect the published version.
A way to properly debug this issue before publishing would be incredibly helpful!

2 Likes
13

Thank you Lloyd for adding your experience to the discussion. I am curious whether you have tried using memory profilers, such as memory-profiler?

I should add that I have not used this package, but I thought I’d put it out here to add to the conversation.

2 Likes