Question

What is good value of test split for 1 million rows?

Answer 1

 

If you are using SAS 9.4,

you may want to use Proc DS2 to process your ~28 Million Rows data set in a Multi-threaded way.

Here is how you can do it

proc ds2;

/* Declare a thread that would split the long delimited string into 700 variables */ thread splitter/overwrite=yes;

dcl double y count;

dcl bigint thisThread;

dcl vararray varchar(10) var[1:700] var1-var700;

drop count;

method run();

set test;

do i = LBOUND(var) to HBOUND(var); var[i]=scan(row,i,'|','m');

end;

thisThread=_threadid_;

count+1;

end;

method term(); put '**Thread' _threadid_ 'processed' count 'rows:';

end; endthread; run; /*

Use 6 threads to process the large ~1 Million rows */ data split/overwrite=yes; dcl thread splitter st; method run();

set from st threads=6;

end;

enddata;

run;

quit;