About me: My name is Solène Rapenne. I like learning and sharing my knowledge related to IT stuff. Hobbies: '(BSD OpenBSD h+ Lisp cmdline gaming internet-stuff Crossbow). I love percent and lambda characters. OpenBSD developer solene@.

Contact me: solene on Freenode, solene+www at dataswamp dot org or solene@bsd.network (mastodon). If for some reason you want to give me some money, I accept paypal at the address donate@perso.pw.

Common LISP awk macro for easy text file operations

Written by Solène, on 04 February 2020.
Tags: #awk #lisp

Comments on Mastodon

I like Common LISP and I also like awk. Dealing with text files in Common LISP is often painful. So I wrote a small awk like common lisp macro, which helps a lot dealing with text files.

Here is the implementation, I used the uiop package for split-string function, it comes with sbcl. But it's possible to write your own split-string or reused the infamous split-str function shared on the Internet.

(defmacro awk(file separator &body code)
  "allow running code for each line of a text file,
   giving access to NF and NR variables, and also to
   fields list containing fields, and line containing $0"
    `(progn
       (let ((stream (open ,file :if-does-not-exist nil)))
         (when stream
           (loop for line = (read-line stream nil)
              counting t into NR
              while line do
                (let* ((fields (uiop:split-string line :separator ,separator))
                       (NF (length fields)))
                  ,@code))))))

It's interesting that the "do" in the loop could be replaced with a "collect", allowing to reuse awk output as a list into another function, a quick example I have in mind is this:

;; equivalent of awk '{ print NF }' file | sort | uniq
;; for counting how many differents fields long line we have
(uniq (sort (awk "file" " " NF)))

Now, here are a few examples of usage of this macro, I've written the original awk command in the comments in comparison:

;; numbering lines of a text file with NR
;; awk '{ print NR": "$0 }' file.txt
;;
(awk "file.txt" " "
     (format t "~a: ~a~%" NR line))

;; display NF-1 field (yes it's -2 in the example because -1 is last field in the list)
;; awk -F ';' '{ print NF-1 }' file.csv
;;
(awk "file.csv" ";"
     (print (nth (- NF 2) fields)))

;; filtering lines (like grep)
;; awk '/unbound/ { print }' /var/log/messages
;;
(awk "/var/log/messages" " "
     (when (search "unbound" line)
       (print line)))

;; printing 4nth field
;; awk -F ';' '{ print $4 }' data.csv
;;
(awk "data.csv" ";"
     (print (nth 4 fields)))