Hi, just trying to understand the behaviour we're seeing on the sort node.
This is desktop LS v6, run on the same PC with no other work being run at the same time.
Dataset is 34m rows with 7 columns, sorting on 2 columns - a 10 character string and a datetime.

A week ago when I ran this node, after an hour and a half it still wasn't completed. It didn't error, just kept running, forcing me to sort the data external to Lavastorm and re-import.

Yesterday it completed in 22 mins so I thought it was just an error last time.

However today it's been running for over an hour 10 mins again and still running.

The source data is exactly the same in each case, and again the PC is only being used for this one purpose with nothing else running as verified in Task Manager.

What could be driving this wildly different performance, and what optimisation steps could I do ? The 10 character string is actually all numbers, so not sure if changing it to int() or something would make any difference.

Thanks,
Dave