Results 1 to 4 of 4

Thread: Multiple File Acquisition

  1. #1

    Default Multiple File Acquisition

    I have 2000 files that need importing into lavastorm files are identical aside from the name as they are time stamped - is there a node or a way to bring all files in at the same time as they need to be joined together and then analyzed

  2. #2
    Lavastorm Employee
    Join Date
    Sep 2014
    Location
    London
    Posts
    8

    Default

    Hi Laura,

    Thanks for the question. What type of files are they?

    There are two parts to this. The first part is to acquire the files and the second is to read the records from the files.

    First part

    To bring in multiple files of the same type, you can use the Directory List node. You will need to enter the folder location of where the files are saved and also a file pattern that Lavastorm will use to acquire the files.

    For example, if the files you are acquiring are called:

    - Lavastorm-030717.csv
    - Lavastorm-040717.csv
    - Lavastorm-050717.csv

    You could use the naming pattern: Lavastorm-*.csv. The asterisk is the wildcard sign in Lavastorm. This will read in the multiple files and you should be able to see the file names in the output.

    Second part

    The second part is to read in the records from the files. To do this you'll want to attach the appropriate node to read in the type of files i.e. if you are trying to acquire CSV file, use a CSV File node. Connect the node to the Directory List node and in the FilenameExpr parameter, enter the input field that contains the file names. In this case, the input field name is called FileName. Run the node.

    You should see all records from all files. If you have any issues/questions, please let us know.

  3. #3

    Default

    Thank you - I've discovered another issue with the 2000 files, they have been saved as CSV but when the files are imported they are actually ; delimited. Is there a way to pick this up in a node?

  4. #4
    Lavastorm Employee stonysmith's Avatar
    Join Date
    Nov 2006
    Location
    Grapevine Tx
    Posts
    781

    Default

    Use the Delimited File node instead of the CSV node

    Field Delimiter = ;
    Record Delimiter = \n

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •