{"id":137,"date":"2017-11-07T21:53:19","date_gmt":"2017-11-08T05:53:19","guid":{"rendered":"http:\/\/flenniken.net\/blog\/?p=137"},"modified":"2024-10-19T15:31:36","modified_gmt":"2024-10-19T22:31:36","slug":"nim-macros","status":"publish","type":"post","link":"https:\/\/flenniken.net\/blog\/nim-macros\/","title":{"rendered":"Nim Macros"},"content":{"rendered":"<p><strong>The Nim programming language<\/strong> provides powerful meta-programming capabilities with nim macros. Macros run at compile time and they operate on nim\u2019s abstract syntax tree (AST) directly. You write macros using the regular nim language.<\/p>\n<p>This post tells how to write nim macros with lots of simple examples.<\/p>\n<p>You can write a macro using a string of code or by creating AST structures. I call these two styles of macros:<\/p>\n<ul>\n<li>text style macro<\/li>\n<li>AST style macro<\/li>\n<\/ul>\n<p>You can also categorize nim macros by how you invoke them.<\/p>\n<h2 id=\"expression-macro\">1. Expression macro<\/h2>\n<p>You invoke an expression macro like a procedure call and it will generate new code at that point in the program. The macro generates AST from scratch and it is inserted at the point it is called.<\/p>\n<h2 id=\"pragma-macro\">2. Pragma macro<\/h2>\n<p>You invoke a pragma macro when a procedure is compiled by tagging the procedure with a pragma and naming the macro as the pragma. The procedure AST is passed to the macro for modification.<\/p>\n<h2 id=\"block-macro\">3. Block macro<\/h2>\n<p>You invoke a block macro when a block of code is compiled by naming the macro as the block. The block AST is passed to the block macro for modification.<\/p>\n<p>You define each type of macro the same except the pragma and block macros have a hidden last parameter for the AST. In all cases macros return an AST.<\/p>\n<h1 id=\"simple-example\">Simple Example<\/h1>\n<p>Here is a simple nim program stored in the file t.nim. We will use it to investigate nim macros.<\/p>\n<pre class=\"nim\"><code>proc hello() =\r\n  echo \"hi\"\r\n\r\nhello()\r\n<\/code><\/pre>\n<p>The program defines the hello procedure then calls it.<\/p>\n<p>Here is the output when compiling and running the program. All the Hint lines have been removed from the output for simplicity.<\/p>\n<pre><code>nim c -r t\r\nhi<\/code><\/pre>\n<h1 id=\"text-style-expression-macro\">Text Style Expression Macro<\/h1>\n<p>Let\u2019s write a text style expression macro to generate the hello proc above.<\/p>\n<pre class=\"nim\"><code>import macros\r\n\r\nmacro gen_hello(): typed =\r\n  let source = \"\"\"\r\nproc hello() =\r\n  echo \"hi\"\r\n\"\"\"\r\n  result = parseStmt(source)\r\n\r\ngen_hello()\r\nhello()<\/code><\/pre>\n<p>Here is the output when compiling and running:<\/p>\n<pre><code>\r\nnim c -r t\r\nhi\r\n<\/code><\/pre>\n<p>The macro is defined like a procedure except you use \u201cmacro\u201d instead of \u201cproc\u201d. The result is the AST you want to insert at the point the macro is called. The parseStmt converts the string to AST. The \u201chello()\u201d call calls the hello procedure generated by the macro.<\/p>\n<h1 id=\"ast-style-expression-macro\">AST Style Expression Macro<\/h1>\n<p>Now lets write the same expression macro in AST style.<\/p>\n<p>Before we do that we need to know what the AST looks like. You could consult the macro module docs. But it is easier run a couple of macros in the macros module to dump out the code so you can see the AST.<\/p>\n<p>For example here is code to dump out our simple hello program using the dumpTree macro.<\/p>\n<pre class=\"nim\"><code>import macros\r\ndumpTree:\r\n  proc hello() =\r\n    echo \"hi\"<\/code><\/pre>\n<p>When running it you get a list of AST nodes indented to show the hierarchy. At the root is a StmtList (statement list) and it contains one node called ProcDef for the definition of the procedure named hello.<\/p>\n<pre><code>StmtList\r\n  ProcDef\r\n    Ident !\"hello\"\r\n    Empty\r\n    Empty\r\n    FormalParams\r\n      Empty\r\n    Empty\r\n    Empty\r\n    StmtList\r\n      Command\r\n        Ident !\"echo\"\r\n        StrLit hi<\/code><\/pre>\n<p>You can also use the dumpAstGen macro. It will generate the textual code needed to build the AST.<\/p>\n<pre class=\"nim\"><code>import macros\r\ndumpAstGen:\r\n  proc hello() =\r\n    echo \"hi\"<\/code><\/pre>\n<p>When running it you get:<\/p>\n<pre class=\"nim\"><code>nnkStmtList.newTree(\r\n  nnkProcDef.newTree(\r\n    newIdentNode(!\"hello\"),\r\n    newEmptyNode(),\r\n    newEmptyNode(),\r\n    nnkFormalParams.newTree(\r\n      newEmptyNode()\r\n    ),\r\n    newEmptyNode(),\r\n    newEmptyNode(),\r\n    nnkStmtList.newTree(\r\n      nnkCommand.newTree(\r\n        newIdentNode(!\"echo\"),\r\n        newLit(\"hi\")\r\n      )\r\n    )\r\n  )\r\n)<\/code><\/pre>\n<p>Now that we know the required AST we can write our AST style expression macro. We take the dumpAstGen output and assign it to result.<\/p>\n<pre class=\"nim\"><code>import macros\r\nmacro gen_hello(): typed =\r\n  result = nnkStmtList.newTree(\r\n    nnkProcDef.newTree(\r\n      newIdentNode(!\"hello\"),\r\n      newEmptyNode(),\r\n      newEmptyNode(),\r\n      nnkFormalParams.newTree(\r\n        newEmptyNode()\r\n      ),\r\n      newEmptyNode(),\r\n      newEmptyNode(),\r\n      nnkStmtList.newTree(\r\n        nnkCommand.newTree(\r\n          newIdentNode(!\"echo\"),\r\n          newLit(\"hi\")\r\n        )\r\n      )\r\n    )\r\n  )\r\ngen_hello()\r\nhello()<\/code><\/pre>\n<p>Here is the output when compiling and running:<\/p>\n<pre><code>nim c -r t\r\nhi<\/code><\/pre>\n<h1 id=\"pragma-macro-1\">Pragma Macro<\/h1>\n<p>A pragma macro has the same name as a nim pragma. A pragma is specified with curly bracks like: {.pragma echoName.}. You add pragmas to procedures.<\/p>\n<p>Let\u2019s write a pragma macro to display the procedure\u2019s name when the procedure is called. For our hello procedure the macro would transform it to:<\/p>\n<pre class=\"nim\"><code>proc hello():\r\n  echo \"hello\"\r\n  echo \"hi\"<\/code><\/pre>\n<p>Looking back at the output from dumpAstGen we see the AST structure we need to generate a command that echos \u201chello\u201d.<\/p>\n<pre class=\"nim\"><code>    nnkStmtList.newTree(\r\n      nnkCommand.newTree(\r\n        newIdentNode(!\"echo\"),\r\n        newLit(\"hello\")\r\n      ),<\/code><\/pre>\n<p>But we do not want to show \u201chello\u201d for all procedures but instead show the procedure\u2019s name. The name comes from the top of the tree in the IdentNode.<\/p>\n<pre class=\"nim\"><code>nnkStmtList.newTree(\r\n  nnkProcDef.newTree(\r\n    newIdentNode(!\"hello\"),<\/code><\/pre>\n<p>Here is our starting attempt at writing the pragma macro. The pragma and macro are called echoName. The macro is passed the AST of the procedure, in this case the procedure is main. The main procedure is annotated with the pragma. Notice it goes at the end of the procedure definition. The line \u201clet msg = name(x)\u201d gets the procedure name and the next line displays it.<\/p>\n<pre class=\"nim\"><code>import macros\r\n\r\nmacro echoName(x: untyped): untyped =\r\n  let msg = name(x)\r\n  echo msg\r\n\r\nproc main (p: int): string {.echoName.} =\r\n  result = \"test\"<\/code><\/pre>\n<p>Here is the output when compiling and running. During the macro processing step it outputs \u201cmain\u201d. You can debug your macro with echo statements.<\/p>\n<pre><code>nim c -r t\r\nHint: used config file '\/usr\/local\/Cellar\/nim\/0.17.2\/nim\/config\/nim.cfg' [Conf]\r\nHint: system [Processing]\r\nHint: t [Processing]\r\nHint: macros [Processing]\r\nmain\r\nHint:  [Link]<\/code><\/pre>\n<p>The \u201cname\u201d procedure is defined in the macro module. It returns the name of the procedure given a procedure AST node.<\/p>\n<p>The meta-type \u201cuntyped\u201d matches anything. It is lazy evaluated so you can pass undefined symbols to it.<\/p>\n<p>There are two other meta-types, typed and typedesc. They are not lazy evaluated.<\/p>\n<p>Here is a working pragma macro that echoes the procedure name when it is called. The \u201clet name = $name(x)\u201d line gets the name of the procedure as a string. The next line creates a new node that echoes the name. The insert adds the node to the body of the procedure as the first statement. You can use treeRepr for debugging.<\/p>\n<p>Our pragma macro is invoked at compile time for each proc tagged with the {.echoName.} pragma.<\/p>\n<pre class=\"nim\"><code>import macros\r\n\r\nmacro echoName(x: untyped): untyped =\r\n  let name = $name(x)\r\n  let node = nnkCommand.newTree(newIdentNode(!\"echo\"), newLit(name))\r\n  insert(body(x), 0, node)\r\n  # echo \"treeRepr = \", treeRepr(x)\r\n  result = x\r\n\r\nproc add(p: int): int {.echoName.} =\r\n  result = p + 1\r\n\r\nproc process(p: int) {.echoName.} =\r\n  echo \"ans for \", p, \" is \", add(p)\r\n\r\nprocess(5)\r\nprocess(8)<\/code><\/pre>\n<p>Here is the output when compiling and running:<\/p>\n<pre><code>nim c -r t\r\n\r\nprocess\r\nadd\r\nans for 5 is 6\r\nprocess\r\nadd\r\nans for 8 is 9<\/code><\/pre>\n<p>Now we enhance the macro to show how you pass parameters to pragmas. In this example we pass a custom message string. When invoking the pragma you add the parameter after a colon as shown below.<\/p>\n<p>By default all arguments are AST expressions. The msg string is passed to the macro as a StrLit node, which happens to be what the newIdentNode procedure requires.<\/p>\n<p>The ! operator in the macro module creates an identifier node from a string.<\/p>\n<pre class=\"nim\"><code>import macros\r\n\r\nmacro echoName(msg: untyped, x: untyped): untyped =\r\n  let node = nnkCommand.newTree(newIdentNode(!\"echo\"), msg)\r\n  insert(body(x), 0, node)\r\n  result = x\r\n\r\nproc add(p: int): int {.echoName: \"calling add proc\".} =\r\n  result = p + 1\r\n\r\nproc process(p: int) {.echoName: \"calling process\".} =\r\n  echo \"ans for \", p, \" is \", add(p)\r\n\r\nprocess(5)\r\nprocess(8)<\/code><\/pre>\n<p>Here is the output when compiling and running:<\/p>\n<pre><code>calling process\r\ncalling add proc\r\nans for 5 is 6\r\ncalling process\r\ncalling add proc\r\nans for 8 is 9<\/code><\/pre>\n<h1 id=\"pass-normal-parameters\">Pass Normal Parameters<\/h1>\n<p>You can pass normal types to macros with the \u201cstatic\u201d syntax. Here is an example of passing an int. The macro echoes the name concatenated with the number.<\/p>\n<pre class=\"nim\"><code>import macros\r\n\r\nmacro echoName(value: static[int], x: untyped): untyped =\r\n  let node = nnkCommand.newTree(newIdentNode(!\"echo\"), newLit($name(x) &amp; $value))\r\n  insert(body(x), 0, node)\r\n  result = x\r\n\r\nproc add(p: int): int {.echoName: 42} =\r\n  result = p + 1\r\n\r\nproc process(p: int) {.echoName: 43} =\r\n  echo \"ans for \", p, \" is \", add(p)\r\n\r\nprocess(5)\r\nprocess(8)<\/code><\/pre>\n<p>output:<\/p>\n<pre><code>process43\r\nadd42\r\nans for 5 is 6\r\nprocess43\r\nadd42\r\nans for 8 is 9<\/code><\/pre>\n<h1 id=\"multiple-macro-parameters\">Multiple Macro Parameters<\/h1>\n<p>You can pass one parameter to a pragma macro. If you want to pass more values, you can use a tuple. Here is an example of passing a number and a string to the macro.<\/p>\n<pre class=\"nim\"><code>import macros\r\n\r\ntype\r\n  Parameters = tuple[value: int, ending: string]\r\n\r\nmacro echoName(p: static[Parameters], x: untyped): untyped =\r\n  # echo \"x = \", treeRepr(x)\r\n  let node = nnkCommand.newTree(newIdentNode(!\"echo\"),\r\n               newLit($name(x) &amp; $p.value &amp; p.ending))\r\n  insert(body(x), 0, node)\r\n  result = x\r\n\r\nproc add(p: int): int {.echoName: (42, \"p1\").} =\r\n  result = p + 1\r\n\r\nproc process(p: int) {.echoName: (43, \"p2\").} =\r\n  echo \"ans for \", p, \" is \", add(p)\r\n\r\nprocess(5)\r\nprocess(8)<\/code><\/pre>\n<p>Here is the output when compiling and running:<\/p>\n<pre><code>process43p2\r\nadd42p1\r\nans for 5 is 6\r\nprocess43p2\r\nadd42p1\r\nans for 8 is 9<\/code><\/pre>\n<h1 id=\"block-macro-1\">Block Macro<\/h1>\n<p>If you name a block with the name of a macro, the macro is invoked when the block is compiled. The AST of the block is passed to the macro as the last parameter.<\/p>\n<p>Here is an example block macro that prints out the AST past to it.<\/p>\n<pre class=\"nim\"><code>import macros\r\n\r\nmacro echoName(x: untyped): untyped =\r\n  echo \"x = \", treeRepr(x)\r\n  result = x\r\n\r\nechoName:\r\n  proc add(p: int): int =\r\n    result = p + 1\r\n\r\n  proc process(p: int) =\r\n    echo \"ans for \", p, \" is \", add(p)\r\n\r\nprocess(5)\r\nprocess(8)<\/code><\/pre>\n<p>Here is the results when compiling and running.<\/p>\n<pre><code>x = StmtList\r\n  ProcDef\r\n    Ident !\"add\"\r\n    Empty\r\n    Empty\r\n    FormalParams\r\n      Ident !\"int\"\r\n      IdentDefs\r\n        Ident !\"p\"\r\n        Ident !\"int\"\r\n        Empty\r\n    Empty\r\n    Empty\r\n    StmtList\r\n      Asgn\r\n        Ident !\"result\"\r\n        Infix\r\n          Ident !\"+\"\r\n          Ident !\"p\"\r\n          IntLit 1\r\n  ProcDef\r\n    Ident !\"process\"\r\n    Empty\r\n    Empty\r\n    FormalParams\r\n      Empty\r\n      IdentDefs\r\n        Ident !\"p\"\r\n        Ident !\"int\"\r\n        Empty\r\n    Empty\r\n    Empty\r\n    StmtList\r\n      Command\r\n        Ident !\"echo\"\r\n        StrLit ans for\r\n        Ident !\"p\"\r\n        StrLit  is\r\n        Call\r\n          Ident !\"add\"\r\n          Ident !\"p\"<\/code><\/pre>\n<p>To write the macro so it prints out the name of the procedures when called, we need to find the procedure nodes in the AST and add the echo as before.<\/p>\n<p>You can use the children procedure to loop through the child nodes of the AST statements. You find the proc\u2019s by checking for the node type nnkProcDef. You add nnk prefix to the names output by treeRepr. Notice we added echo statements to the block that we need to skip over.<\/p>\n<pre class=\"nim\"><code>import macros\r\n\r\nmacro echoName(x: untyped): untyped =\r\n  for child in x.children():\r\n    if child.kind == nnkProcDef:\r\n      let node = nnkCommand.newTree(newIdentNode(!\"echo\"),\r\n                   newLit($name(child)))\r\n      insert(body(child), 0, node)\r\n  result = x\r\n\r\nechoName:\r\n  echo \"an echo statement\"\r\n  proc add(p: int): int =\r\n    result = p + 1\r\n  echo \"another echo statement\"\r\n  proc process(p: int) =\r\n    echo \"ans for \", p, \" is \", add(p)\r\n\r\nprocess(5)\r\nprocess(8)<\/code><\/pre>\n<p>Here is the output:<\/p>\n<pre><code>an echo statement\r\nanother echo statement\r\nprocess\r\nadd\r\nans for 5 is 6\r\nprocess\r\nadd\r\nans for 8 is 9<\/code><\/pre>\n<p>The following shows how to pass parameters to your block macros. In this example we pass the string \u201ccalled\u201d .<\/p>\n<pre class=\"nim\"><code>import macros\r\n\r\nmacro echoName(msg: static[string], x: untyped): untyped =\r\n  for child in x.children():\r\n    if child.kind == nnkProcDef:\r\n      let node = nnkCommand.newTree(newIdentNode(!\"echo\"),\r\n                   newLit($name(child) &amp; \"-\" &amp; $msg))\r\n      insert(body(child), 0, node)\r\n  result = x\r\n\r\nechoName(\"called\"):\r\n  echo \"an echo statement\"\r\n  proc add(p: int): int =\r\n    result = p + 1\r\n  echo \"another echo statement\"\r\n  proc process(p: int) =\r\n    echo \"ans for \", p, \" is \", add(p)\r\n\r\nprocess(5)\r\nprocess(8)<\/code><\/pre>\n<p>Here is the output:<\/p>\n<pre><code>an echo statement\r\nanother echo statement\r\nprocess-called\r\nadd-called\r\nans for 5 is 6\r\nprocess-called\r\nadd-called\r\nans for 8 is 9<\/code><\/pre>\n<h1 id=\"more-information\">More Information<\/h1>\n<p>See the macro module documentation for the complete AST syntax and other useful procedures and operators.<\/p>\n<p><a href=\"https:\/\/nim-lang.org\/docs\/macros.html\">https:\/\/nim-lang.org\/docs\/macros.html<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The Nim programming language provides powerful meta-programming capabilities with nim macros. Macros run at compile time and they operate on nim\u2019s abstract syntax tree (AST) directly. You write macros using the regular nim language. This post tells how to write &hellip; <a href=\"https:\/\/flenniken.net\/blog\/nim-macros\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[25],"tags":[],"class_list":["post-137","post","type-post","status-publish","format-standard","hentry","category-nim"],"_links":{"self":[{"href":"https:\/\/flenniken.net\/blog\/wp-json\/wp\/v2\/posts\/137","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/flenniken.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/flenniken.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/flenniken.net\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/flenniken.net\/blog\/wp-json\/wp\/v2\/comments?post=137"}],"version-history":[{"count":4,"href":"https:\/\/flenniken.net\/blog\/wp-json\/wp\/v2\/posts\/137\/revisions"}],"predecessor-version":[{"id":188,"href":"https:\/\/flenniken.net\/blog\/wp-json\/wp\/v2\/posts\/137\/revisions\/188"}],"wp:attachment":[{"href":"https:\/\/flenniken.net\/blog\/wp-json\/wp\/v2\/media?parent=137"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/flenniken.net\/blog\/wp-json\/wp\/v2\/categories?post=137"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/flenniken.net\/blog\/wp-json\/wp\/v2\/tags?post=137"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}