AnsweredAssumed Answered

Very slow writing of many ARB sequence files to the N1572B

Question asked by jakelishman on Aug 13, 2015
Hello,

For some of the experiments we're doing, we need to be able to write around 16,000 sequence files to our N1572B as quickly as possible.  Each sequence file is reasonably small (usually no more than 6 waveform segments), but there are a large number of them.  What is the best way to write these?

Our current method is to send the SCPI command ":RAD:ARB:SEQ ,,,ALL\[,<...>]" for each sequence.  This, however, seems to take an exorbitant amount of time - for some reason the time to write "n" sequences seems to be almost exponential in n.  Even for our short sequences, this takes more than an hour to write over ethernet (I've never seen it finish - we've left it running while we went to lunch, and it's never finished in that time).  There's a minimal working example of this behaviour in the file "sequence_command.py" (run with "python2 sequence_command.py ").

I have now plugged in an external USB drive, which has helped significantly with general sequence write speeds, but not brought large numbers of sequences down to a reasonable time.  It still takes an indeterminate amount of time to write 16k sequence files.

I have also tried sending a large number of files over FTP, but this has proved to be the slowest method I've tried so far.  I used FileZilla to send the files all at once, but this was taking around 0.5-1.0s a file right from the beginning, and then it slowed down as time went on.

My most recent attempt has been to send the files using the ":MMEM:DATA" command, and creating a datablock.  My example of this is "datablock.py", and an example sequence file (this particular one has only one segment in it, to be as simple as possible) is "one_segment.seq".  The test can be run with "python2 datablock.py ". This has been by far the most successful, bringing the writing of 1024 very simple sequence files down from ~240s to ~30s, but after about 30 minutes this times out on a single "visa_instrument.write" call when trying to write 16k sequences.  By looking at what was sucessfully written, I see that approximately 4000 sequences were written before the timeout (2000ms) occurred.

I've also found that sometimes the time taken for either SCPI-based method is significantly longer.  I think this is something to do with the number of files which exist on the storage medium at the time, but it's very hard for me to pin down, because it's not 100% consistent.  This can sometimes cause either method to quadruple or worse in time taken.

I've attached two graphs (datablock_out_2048.pdf and command_out_2048.pdf), each of an attempt to write 2048 sequences to the machine over ethernet using either the ":RAD:ARB:SEQ" command or the ":MMEM:DATA" command.  These are each only from one run, and due to the effect mentioned in the previous paragraph, I'm not sure how representative either is of this run-time.  Both tests were run with no files on the internal storage when I began.  I have also included graphs of one run, each taken immediately after the previous run had completed, so each had the files pre-existing on the USB stick.  These are files datablock_out_2048_2.pdf and command_out_2048_2.pdf.  Strangely, at 2048 sequences, the ":RAD:ARB:SEQ" command performed better at these tests, though again this could be due to the slowdown effect.

Total times taken were:
- datablock_out_2048.pdf - *5m55*
- datablock_out_2048_2.pdf - *6m10*
- command_out_2048.pdf - *4m16*
- command_out_2048_2.pdf - *2m02*

The datablock ones were uncharacteristically slow here, and the sequence command ones were faster than usual, but this may be bias since I don't have too much data.

Writing of waveform segments is fine and quick, but sequences are very slow.  Sadly, we would quickly run out of memory if we were forced to write everything as segment files, so this is not an option.

Is there anything I could be doing to speed up the writing of these sequences?  Also, what causes the strange increases in sequence writing time as index increases, and why doesn't it seem to depend on the number of files on the disk (as one might expect from FAT32) - since starting the experiment with a drive full of sequences begins again from quick writing?

Thanks in advance!

Outcomes