I've recently introduced an issue in my workflow when using Common Lisp's UIOP standard facility for launching asynchronous system calls. It appears to be quite a critical one since I'm planning to use that functionality a lot with my side-effect manager.

The following code will fail after a couple hundred (maybe thousand) of iterations, complaining about "too many open files":

  (loop
 :repeat 1000
 :do  (let ((proc (uiop:launch-program
                   "cat ~/.bashrc"
                   :output :stream :error-output :stream)))

        (print (uiop:process-info-pid proc)) ; show PID

        (uiop:slurp-stream-lines        ; collect output lines
         (uiop:process-info-output proc))

        (unless (= 0 (uiop:wait-process proc)) ; wait till end
          (print "Fail!"))))
 

Experimentally I found out that uiop:close-streams should be used to mitigate the bug. Which is in fact written in the documentation string, saying:

Needs to be run whenever streams were requested by passing :stream to :input, :output, or :error-output.

Which takes place since I do request those outputs as streams.

But how do I know about it in advance? Well, turns out that the corresponding chapter in the ASDF manual opens with description of close-streams, because alphabet. So it's one of the "RTFM" cases.

So this works, as many-many times I want:

 (loop
 :repeat 100000
 :do  (let ((proc (uiop:launch-program
                   "cat .bashrc"
                   :output :stream :error-output :stream)))

        (print (uiop:process-info-pid proc)) ; show PID

        (uiop:slurp-stream-lines        ; collect output lines
         (uiop:process-info-output proc))

        (unless (= 0 (uiop:wait-process proc)) ; wait till end
          (print "Fail!"))

        (uiop:close-streams proc))) ; <- DAT FIXES

And given that I defined a macro for this most common pattern of program launching, just not to forget to close-streams again. A raw anaphora will serve till I produce something more clever:

 (defmacro aproc (command &body body)
  "Anaphoric process launcher. Refer to process instance as PROC in &BODY."
  `(let ((proc (uiop:launch-program
                ,command :output :stream :error-output :stream)))
     (values ,@body (uiop:close-streams proc))))
Unless otherwise credited all material Creative Commons License by Vladimir Dikan.
Powered by Coλeslaw. Design elements' artwork created with MagicaVoxel.