Author Topic: Large Excel export causes browser crash  (Read 940 times)

wd_perf

  • Pro Ultimate
  • Newbie
  • *
  • Posts: 26
    • View Profile
Large Excel export causes browser crash
« on: September 28, 2022, 04:22:46 am »
Previous topic: https://paramquery.com/forum/index.php?topic=3908.0

I believe the crash with large data sets may be caused because of the JSZip.generate function used. According to https://stuk.github.io/jszip/documentation/faq.html the generateAsync function would do better at avoiding browser crashes.

Would you be able to update the JSZip version used to take advantage of generateAsync?

wd_perf

  • Pro Ultimate
  • Newbie
  • *
  • Posts: 26
    • View Profile
Re: Large Excel export causes browser crash
« Reply #1 on: October 19, 2022, 06:06:55 am »
Upon further review, it looks like the crash happens before the zip, in getBody().

paramvir

  • Administrator
  • Hero Member
  • *****
  • Posts: 6318
    • View Profile
Re: Large Excel export causes browser crash
« Reply #2 on: October 19, 2022, 11:22:28 am »
Can you please attach sample Excel file which causes browser crash so that I can check it at my end.

wd_perf

  • Pro Ultimate
  • Newbie
  • *
  • Posts: 26
    • View Profile
Re: Large Excel export causes browser crash
« Reply #3 on: October 25, 2022, 01:49:07 am »
Hello, yes.
https://www.stats.govt.nz/assets/Uploads/International-trade/International-trade-June-2022-quarter/Download-data/international-trade-june-2022-quarter-csv.zip
In that zip file, the output_csv_full.csv can be imported but when exported in Excel format it crashes the browser tab.
Thank you.

wd_perf

  • Pro Ultimate
  • Newbie
  • *
  • Posts: 26
    • View Profile
Re: Large Excel export causes browser crash
« Reply #4 on: October 26, 2022, 10:00:45 am »
Hello again. I was able to get it working by removing all .join("") calls and modifying getSheet() to return buffered content as shown here: https://stackoverflow.com/a/46888173

wd_perf

  • Pro Ultimate
  • Newbie
  • *
  • Posts: 26
    • View Profile
Re: Large Excel export causes browser crash
« Reply #5 on: October 29, 2022, 05:26:18 am »
Actually, that was breaking on very very large exports. I had to rewrite things to take advantage of JSZip3's support for Node streams in the file creation function.