Capture data loading from fread into a shiny withProgress bar











up vote
1
down vote

favorite
3












I am loading in a large data file using fread which is from: library(data.table). fread by default shows the status of how long it is taking to read in the file but it only prints to the console.



I would like to capture the loading time of data and display it in real time in the shiny application so the user understands why there is a delay and roughly how long the data read is taking. I would like to use withProgress since this produces a loading bar and you can display text defining what the program is doing.



Typically, withProgress is incremented by a numeric scale using incProgress, however, I am looking for a way to do something like this:



withProgress(message = 'Loading Data', {
d <- fread("file_name.csv") # Pick a random (large) file to load
})


The problem with the above code is that the load bar isn't being incremented (I don't think that withProgress detects the loading by fread automatically). The load bar shows up and only increments a small amount but it is not correctly capturing the loading time from fread.



Update

I was able to find this similar question, however, the only answer given to this question uses the loading bar to indicate the progress of loading many files (Loaded 2 of n files). My problem is trying to show the progression of reading in a single large file.

I know fread knows how many rows a file contains after the initial scan because in the console it does show the progress. I just need to know how to tap into that to use it in the progress bar.



Here is an example of how fread displays the read in progress:
fread progress



fread Documentation

If you look up the documentation for fread this is what it says about the built in showProgess option:



TRUE displays progress on the console using r. 
It is produced in freads C code where the very nice (but R level)
txtProgressBar and tkProgressBar are not easily available.
showProgress = getOption("datatable.showProgress") # default: TRUE









share|improve this question




























    up vote
    1
    down vote

    favorite
    3












    I am loading in a large data file using fread which is from: library(data.table). fread by default shows the status of how long it is taking to read in the file but it only prints to the console.



    I would like to capture the loading time of data and display it in real time in the shiny application so the user understands why there is a delay and roughly how long the data read is taking. I would like to use withProgress since this produces a loading bar and you can display text defining what the program is doing.



    Typically, withProgress is incremented by a numeric scale using incProgress, however, I am looking for a way to do something like this:



    withProgress(message = 'Loading Data', {
    d <- fread("file_name.csv") # Pick a random (large) file to load
    })


    The problem with the above code is that the load bar isn't being incremented (I don't think that withProgress detects the loading by fread automatically). The load bar shows up and only increments a small amount but it is not correctly capturing the loading time from fread.



    Update

    I was able to find this similar question, however, the only answer given to this question uses the loading bar to indicate the progress of loading many files (Loaded 2 of n files). My problem is trying to show the progression of reading in a single large file.

    I know fread knows how many rows a file contains after the initial scan because in the console it does show the progress. I just need to know how to tap into that to use it in the progress bar.



    Here is an example of how fread displays the read in progress:
    fread progress



    fread Documentation

    If you look up the documentation for fread this is what it says about the built in showProgess option:



    TRUE displays progress on the console using r. 
    It is produced in freads C code where the very nice (but R level)
    txtProgressBar and tkProgressBar are not easily available.
    showProgress = getOption("datatable.showProgress") # default: TRUE









    share|improve this question


























      up vote
      1
      down vote

      favorite
      3









      up vote
      1
      down vote

      favorite
      3






      3





      I am loading in a large data file using fread which is from: library(data.table). fread by default shows the status of how long it is taking to read in the file but it only prints to the console.



      I would like to capture the loading time of data and display it in real time in the shiny application so the user understands why there is a delay and roughly how long the data read is taking. I would like to use withProgress since this produces a loading bar and you can display text defining what the program is doing.



      Typically, withProgress is incremented by a numeric scale using incProgress, however, I am looking for a way to do something like this:



      withProgress(message = 'Loading Data', {
      d <- fread("file_name.csv") # Pick a random (large) file to load
      })


      The problem with the above code is that the load bar isn't being incremented (I don't think that withProgress detects the loading by fread automatically). The load bar shows up and only increments a small amount but it is not correctly capturing the loading time from fread.



      Update

      I was able to find this similar question, however, the only answer given to this question uses the loading bar to indicate the progress of loading many files (Loaded 2 of n files). My problem is trying to show the progression of reading in a single large file.

      I know fread knows how many rows a file contains after the initial scan because in the console it does show the progress. I just need to know how to tap into that to use it in the progress bar.



      Here is an example of how fread displays the read in progress:
      fread progress



      fread Documentation

      If you look up the documentation for fread this is what it says about the built in showProgess option:



      TRUE displays progress on the console using r. 
      It is produced in freads C code where the very nice (but R level)
      txtProgressBar and tkProgressBar are not easily available.
      showProgress = getOption("datatable.showProgress") # default: TRUE









      share|improve this question















      I am loading in a large data file using fread which is from: library(data.table). fread by default shows the status of how long it is taking to read in the file but it only prints to the console.



      I would like to capture the loading time of data and display it in real time in the shiny application so the user understands why there is a delay and roughly how long the data read is taking. I would like to use withProgress since this produces a loading bar and you can display text defining what the program is doing.



      Typically, withProgress is incremented by a numeric scale using incProgress, however, I am looking for a way to do something like this:



      withProgress(message = 'Loading Data', {
      d <- fread("file_name.csv") # Pick a random (large) file to load
      })


      The problem with the above code is that the load bar isn't being incremented (I don't think that withProgress detects the loading by fread automatically). The load bar shows up and only increments a small amount but it is not correctly capturing the loading time from fread.



      Update

      I was able to find this similar question, however, the only answer given to this question uses the loading bar to indicate the progress of loading many files (Loaded 2 of n files). My problem is trying to show the progression of reading in a single large file.

      I know fread knows how many rows a file contains after the initial scan because in the console it does show the progress. I just need to know how to tap into that to use it in the progress bar.



      Here is an example of how fread displays the read in progress:
      fread progress



      fread Documentation

      If you look up the documentation for fread this is what it says about the built in showProgess option:



      TRUE displays progress on the console using r. 
      It is produced in freads C code where the very nice (but R level)
      txtProgressBar and tkProgressBar are not easily available.
      showProgress = getOption("datatable.showProgress") # default: TRUE






      r shiny progress-bar fread






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Nov 15 at 15:36

























      asked Nov 13 at 19:18









      Bear

      33814




      33814





























          active

          oldest

          votes











          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53288073%2fcapture-data-loading-from-fread-into-a-shiny-withprogress-bar%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown






























          active

          oldest

          votes













          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes
















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.





          Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


          Please pay close attention to the following guidance:


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53288073%2fcapture-data-loading-from-fread-into-a-shiny-withprogress-bar%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          mysqli_query(): Empty query in /home/lucindabrummitt/public_html/blog/wp-includes/wp-db.php on line 1924

          How to change which sound is reproduced for terminal bell?

          Can I use Tabulator js library in my java Spring + Thymeleaf project?