Elixir meta-programming -- generating code from specs

5.1   Reading the specification file at compile-time

I encoded that specification of the functions in a Yaml file (see: https://yaml.org/). In retrospect, that was a questionable decision. The reason is that the Erlang/Elixir Yaml parser needs to load a NIF (a native implemented function, one that is implemented in C) and that NIF does not seem to get loaded early enough to be available when Elixir/IEx is started and out-of-date files are compiled. Okie-dokie, trying to turn this into a learning experience -- I wrote a function to read and parse the Yaml file and then save the resulting data structure as a string in a file.

You can read the Yaml file with something like the following:

{:ok, [spec | _]} = :fast_yaml.decode_from_file(in_spec_file)

And, you can write it out with this:

spec_str = Kernel.inspect(spec)
File.write("data_outfile.exs", spec_str)

See function save_yaml_spec_to_text_file/2 in the sample code.

Then, at compile-time, you can read in that Elixir data structure and save it in a module attribute (@methods in this case) with something like the following:

{:ok, spec_txt} = File.read("Data/module03.exs")
{spec, _} = Code.eval_string(spec_txt)
@methods :proplists.get_value("instance_methods", spec)

5.2   Generating a table at compile-time

I generated a dispatch table as an Elixir keyword list. In this case, a dispatch table is not really needed, since the user is passing in the name of the requested function as an atom. But, it's worth doing for demonstration purposes.

The dispatch table is generated be the macro ModuleGeneratorMacros.gen_dispatch_table:

defmacro gen_dispatch_table() do
  methods = ModuleGeneratorHelpers.get_methods()
  ast = for item <- methods do
    quote bind_quoted: [method_name: String.to_atom(:proplists.get_value("method_name", item))] do
      {method_name, method_name}
    end
  end
  #IO.inspect(ast, label: "ast")
  ast
end

Notes:

• Notice the use of the Elixir list comprehension wrapped around quote/2. That produces a list of ASTs, which results in a list containing the items in a keyword list. In effect, we are injecting those ASTs into our Module. That's what happens when we call a macro.
• And, we need to inject the value of the method name (as an atom) twice. So, rather than evaluate String.to_atom(:proplists.get_value("method_name", item)) twice, we use the bind_quoted option to the quote macro.

5.3   Generating functions at compile time

Normally (if you can consider meta-programming normal at all) we'd define a macro (with defmacro) to produce and insert an AST into our module. But, in this case, what we want to define are functions defined with the def expression. However, def itself is defined as a macro. And, that means that it returns an AST all by itself. And, that's what we want to insert, an AST.

So, all we need to do is repeatedly, in a loop, create def expressions. Here is some code that does that:

ModuleGeneratorHelpers.get_methods()
|> Enum.each(fn (item) ->
  method_name = :proplists.get_value("method_name", item)
  method_name_atom = String.to_atom(method_name)
  method_body_str = :proplists.get_value("method_body", item)
  {:ok, method_body_ast} = Code.string_to_quoted(method_body_str)
  # IO.inspect(method_body_ast, label: "method_body_ast")
  def unquote(method_name_atom)(args, opts) do
    unquote(method_body_ast)
  end
end)

Notes:

• For each method or function, we capture the function name as an atom and the function body as a string.
• In order to insert the name and the body, we unquote them. If we did not do so, we'd be inserting the variable names ("method_name_atom" and "method_body_str") rather than their values.
• The call to IO.inspect/2, which is commented out, is for debugging purposes.