Creating an Erlang application using OTP application and supervisor behaviors

1   Goals and preliminaries

An earlier blog article explained how to implement a module based on the OTP gen_fsm behavior. In this article we will learn how to fit that into and create a larger unit that is able to both start-up test_fsm.erl and supervise that process as it runs. In doing so, we'll learn how to use the OTP application and supervisor behaviors and also how to create the directory structure they fit into.

Credits -- These examples were created with help from:

OTP behavior skeletons -- I used Vim and vimerl to create skeletons for the OTP behaviors in this example (application, supervisor, and gen_fsm). The Erlang mode for the Emacs text editor has a similar capability. Even if you use neither Vim nor Emacs, you can still down load vimerl, find the skeletons under vimerl/plugin/erlang_skels, and copy and edit whichever ones you need.

OTP and Erlang documentation -- You can find documentation on Erlang OTP behaviors here:

2   Implementation procedure

We'll perform the following steps.

2.1   Create the directory structure

Our gen_fsm example converts to lower and upper case. So, lets call our example application char_case_app.

Now, create the following directory and subdirectories:

char_case_app
|-- doc
|-- ebin
|-- include
|-- priv
`-- src

We're going to add a configuration file inside of ebin. And, we are going to put our gen_fsm, supervisor, and application modules inside of src.

2.2   Write the configuration files

Here is ebin/char_case.app:

%% vim:ft=erlang:

{application, char_case,
 [{description, "Character case converter using OTP gen_fsm, supervisor, and application behaviors"},
  {vsn, "0.1.0"},
  {modules, [
            char_case_app,
            char_case_sup,
            %char_case_server,
            char_case_fsm
            ]},
  {registered, [
                char_case_sup,
                char_case_fsm
               ]},
  {applications, [kernel, stdlib]},
  {mod, {char_case_app, []}}
 ]}.

2.3   Implement the application behavior

Here is char_case/src/char_case_app.erl:

-module(char_case_app).

-behaviour(application).

%% Application callbacks
-export([start/2,
         stop/1]).

%%%===================================================================
%%% Application callbacks
%%%===================================================================

%%--------------------------------------------------------------------
%% @private
%% @doc
%% This function is called whenever an application is started using
%% application:start/[1,2], and should start the processes of the
%% application. If the application is structured according to the OTP
%% design principles as a supervision tree, this means starting the
%% top supervisor of the tree.
%%
%% @spec start(StartType, StartArgs) -> {ok, Pid} |
%%                                      {ok, Pid, State} |
%%                                      {error, Reason}
%%      StartType = normal | {takeover, Node} | {failover, Node}
%%      StartArgs = term()
%% @end
%%--------------------------------------------------------------------
start(_StartType, _StartArgs) ->
    case char_case_sup:start_link() of
      {ok, Pid} ->
          {ok, Pid};
      Other ->
          {error, Other}
    end.

%%--------------------------------------------------------------------
%% @private
%% @doc
%% This function is called whenever an application has stopped. It
%% is intended to be the opposite of Module:start/2 and should do
%% any necessary cleaning up. The return value is ignored.
%%
%% @spec stop(State) -> void()
%% @end
%%--------------------------------------------------------------------
stop(_State) ->
    ok.

%%%===================================================================
%%% Internal functions
%%%===================================================================

2.4   Implement the supervisor behavior

Here is char_case/src/char_case_sup.erl:

-module(char_case_sup).

-behaviour(supervisor).

%% API functions
-export([start_link/0]).

%% Supervisor callbacks
-export([init/1]).

-define(SERVER, ?MODULE).
-define(CHILD(Id, Mod, Type, Args), {Id, {Mod, start_link, Args},
                                     permanent, 5000, Type, [Mod]}).

%%%===================================================================
%%% API functions
%%%===================================================================

%%--------------------------------------------------------------------
%% @doc
%% Starts the supervisor
%%
%% @spec start_link() -> {ok, Pid} | ignore | {error, Error}
%% @end
%%--------------------------------------------------------------------
start_link() ->
    supervisor:start_link({local, ?SERVER}, ?MODULE, []).

%%%===================================================================
%%% Supervisor callbacks
%%%===================================================================

%%--------------------------------------------------------------------
%% @private
%% @doc
%% Whenever a supervisor is started using supervisor:start_link/[2,3],
%% this function is called by the new process to find out about
%% restart strategy, maximum restart frequency and child
%% specifications.
%%
%% @spec init(Args) -> {ok, {SupFlags, [ChildSpec]}} |
%%                     ignore |
%%                     {error, Reason}
%% @end
%%--------------------------------------------------------------------
init([]) ->
%~     Server = {char_case_server, {char_case_server, start_link, []},
%~               permanent, 2000, worker, [char_case_server]},
    Server = {char_case_fsm, {char_case_fsm, start_link, []},
              permanent, 2000, worker, [char_case_fsm]},
    Children = [Server],
    RestartStrategy = {one_for_one, 0, 1},
    {ok, {RestartStrategy, Children}}.

%%%===================================================================
%%% Internal functions
%%%===================================================================

2.5   Implement the service itself

This is the module in which our work is done.

Here is char_case/src/char_case_fsm.erl:

-module(char_case_fsm).

-behaviour(gen_fsm).

%% API functions
-export([
        start_link/0,
        start_link/1,
        start_link/2,
        start_link/3,
        stop/0,
        help/0,
        set_from_pid/1,
        process/1
        ]).

%% gen_fsm callbacks
-export([init/1,
         lower/2,
         upper/2,
         handle_event/3,
         handle_sync_event/4,
         handle_info/3,
         terminate/3,
         code_change/4]).

%% testing
-export([test/1]).

% define the record to hold data items to be passed from one callback
% to the next, for example between calls to lower/2 and upper/2.
-record(state, {frompid,       % the process ID of the requestor
                outdata,       % the collected output data
                debug=false    % if true, print debugging info
                }).

-define(SERVER, ?MODULE).

%%%===================================================================
%%% API functions
%%%===================================================================

%%--------------------------------------------------------------------
%% @doc
%% Creates a gen_fsm process which calls Module:init/1 to
%% initialize. To ensure a synchronized start-up procedure, this
%% function does not return until Module:init/1 has returned.
%%
%% @spec start_link(FromPid) -> {ok, Pid} | ignore | {error, Error}
%% @end
%%--------------------------------------------------------------------
start_link() ->
    gen_fsm:start_link({local, ?SERVER}, ?SERVER, [nil, false], []).
start_link(FromPid) ->
    %gen_fsm:start_link({local, ?SERVER}, ?SERVER, [Data], []).
    gen_fsm:start_link({local, ?SERVER}, ?SERVER, [FromPid, false], []).
start_link(FromPid, Debug) ->
    case Debug of
        debug ->
            gen_fsm:start_link({local, ?SERVER}, ?SERVER, [FromPid, true], []);
        _ ->
            gen_fsm:start_link({local, ?SERVER}, ?SERVER, [FromPid, false], [])
    end.
start_link(Name, FromPid, Debug) ->
    gen_fsm:start_link({local, Name}, ?SERVER, [FromPid, Debug], []).

%%--------------------------------------------------------------------
%% @doc
%% Saves the requestor's PID in the state info so that the FSM
%% knows who to send the result to.
%% For use by code that cannot pass the PID in start_link/1.
%%
%% @spec set_from_pid(FromPid) -> ok
%% @end
%%--------------------------------------------------------------------
set_from_pid(FromPid) ->
    gen_fsm:send_event(?SERVER, {set_from_pid, FromPid}),
    ok.

%%--------------------------------------------------------------------
%% @doc
%% Process an input sequence and return a result string.
%% Sample call:
%%     test_gen_fsm:start_link(MyPid),
%%     Data = [
%%             {chr, "m"},
%%             {chr, "n"},
%%             {chr, "o"},
%%             {cmd, upper},
%%             {chr, "p"},
%%             {chr, "q"},
%%             {chr, "r"},
%%             {cmd, lower},
%%             {chr, "t"},
%%             {chr, "u"},
%%             {cmd, upper},
%%             {chr, "v"},
%%             {chr, "w"},
%%             'end'
%%            ],
%%     test_gen_fsm:process(Data),
%%     receive
%%         {ok, Result} ->
%%         ...
%%
%% @spec process(Data) -> ok
%% @end
%%--------------------------------------------------------------------
process(Data) ->
    process_items(Data).

process_items([]) ->
    ok;
process_items([Item | Rest]) ->
    gen_fsm:send_event(?SERVER, Item),
    process_items(Rest).

%%--------------------------------------------------------------------
%% @doc
%% Stop the FSM.
%%
%% @spec stop() -> {ok, String} | {error, Reason}
%% @end
%%--------------------------------------------------------------------
stop() ->
    %gen_fsm:stop(?SERVER, normal, infinity),
    gen_fsm:stop(?SERVER),
    ok.

%%--------------------------------------------------------------------
%% @doc
%% Display a help message.
%%
%% @spec help() -> ok
%% @end
%%--------------------------------------------------------------------
help() ->
    Msg = [
        "~nSynopsis:~n",
        "    Convert the case of a stream of characters depending on commands~n",
        "    embedded in that list.~n",
        "Usage:~n",
        "    char_case_fsm:process(DataList)~n",
        "    char_case_fsm:test(Code)        Code = one | two | three~n",
        "Example:~n",
        "    application:start(char_case)~n",
        "    char_case_fsm:test(three)~n",
        "    application:stop(char_case)~n",
        "~n"
          ],
    Msg1 = lists:concat(Msg),
    io:format(Msg1),
    ok.

%%%===================================================================
%%% gen_fsm callbacks
%%%===================================================================

%%--------------------------------------------------------------------
%% @private
%% @doc
%% Whenever a gen_fsm is started using gen_fsm:start/[3,4] or
%% gen_fsm:start_link/[3,4], this function is called by the new
%% process to initialize.
%%
%% @spec init(Args) -> {ok, StateName, State} |
%%                     {ok, StateName, State, Timeout} |
%%                     ignore |
%%                     {stop, StopReason}
%% @end
%%--------------------------------------------------------------------
init([FromPid, Debug]) ->
    State = #state{frompid=FromPid, outdata=[], debug=Debug},
    {ok, lower, State}.

%%--------------------------------------------------------------------
%% @private
%% @doc
%% There should be one instance of this function for each possible
%% state name. Whenever a gen_fsm receives an event sent using
%% gen_fsm:send_event/2, the instance of this function with the same
%% name as the current state name StateName is called to handle
%% the event. It is also called if a timeout occurs.
%%
%% @spec state_name(Event, State) ->
%%                   {next_state, NextStateName, NextState} |
%%                   {next_state, NextStateName, NextState, Timeout} |
%%                   {stop, Reason, NewState}
%% @end
%%--------------------------------------------------------------------
lower(Event, #state{frompid=FromPid, outdata=OutData, debug=Debug} = State) ->
    case Debug of
        true ->
            io:format("(lower) Event: ~p~n", [Event]);
        _ ->
            ok
    end,
    Item = Event,
    {StateName, State1} = case Item of
          'end' ->
              OutData2 = lists:reverse(OutData),
              FromPid ! {ok, OutData2},
              {'lower', State#state{outdata=[]}};
          {'set_from_pid', FromPid1} ->
              {lower, State#state{frompid=FromPid1}};
          {cmd, Cmd} ->
              case Cmd of
                  lower ->
                      {lower, State};
                  upper ->
                      {upper, State}
              end;
          {chr, Chr} ->
              OutData2 = [string:to_lower(Chr) | OutData],
              {lower, State#state{outdata=OutData2}}
    end,
    {next_state, StateName, State1}.

upper(Event, #state{frompid=FromPid, outdata=OutData, debug=Debug} = State) ->
    case Debug of
        true ->
            io:format("(upper) Event: ~p~n", [Event]);
        _ ->
            ok
    end,
    Item = Event,
    {StateName, OutData1} = case Item of
        'end' ->
            OutData2 = lists:reverse(OutData),
            FromPid ! {ok, OutData2},
            {'lower', []};
        {cmd, Cmd} ->
            case Cmd of
                lower ->
                    {lower, OutData};
                upper ->
                    {upper, OutData}
            end;
        {chr, Chr} ->
            OutData2 = [string:to_upper(Chr) | OutData],
            {upper, OutData2}
    end,
    {next_state, StateName, State#state{outdata=OutData1}}.

%%--------------------------------------------------------------------
%% @private
%% @doc
%% There should be one instance of this function for each possible
%% state name. Whenever a gen_fsm receives an event sent using
%% gen_fsm:sync_send_event/[2,3], the instance of this function with
%% the same name as the current state name StateName is called to
%% handle the event.
%%
%% @spec state_name(Event, From, State) ->
%%                   {next_state, NextStateName, NextState} |
%%                   {next_state, NextStateName, NextState, Timeout} |
%%                   {reply, Reply, NextStateName, NextState} |
%%                   {reply, Reply, NextStateName, NextState, Timeout} |
%%                   {stop, Reason, NewState} |
%%                   {stop, Reason, Reply, NewState}
%% @end
%%--------------------------------------------------------------------
%~ state_name(_Event, _From, State) ->
%~     Reply = ok,
%~     {reply, Reply, state_name, State}.

%%--------------------------------------------------------------------
%% @private
%% @doc
%% Whenever a gen_fsm receives an event sent using
%% gen_fsm:send_all_state_event/2, this function is called to handle
%% the event.
%%
%% @spec handle_event(Event, StateName, State) ->
%%                   {next_state, NextStateName, NextState} |
%%                   {next_state, NextStateName, NextState, Timeout} |
%%                   {stop, Reason, NewState}
%% @end
%%--------------------------------------------------------------------
handle_event(_Event, StateName, State) ->
    {next_state, StateName, State}.

%%--------------------------------------------------------------------
%% @private
%% @doc
%% Whenever a gen_fsm receives an event sent using
%% gen_fsm:sync_send_all_state_event/[2,3], this function is called
%% to handle the event.
%%
%% @spec handle_sync_event(Event, From, StateName, State) ->
%%                   {next_state, NextStateName, NextState} |
%%                   {next_state, NextStateName, NextState, Timeout} |
%%                   {reply, Reply, NextStateName, NextState} |
%%                   {reply, Reply, NextStateName, NextState, Timeout} |
%%                   {stop, Reason, NewState} |
%%                   {stop, Reason, Reply, NewState}
%% @end
%%--------------------------------------------------------------------
handle_sync_event(_Event, _From, StateName, State) ->
    Reply = ok,
    {reply, Reply, StateName, State}.

%%--------------------------------------------------------------------
%% @private
%% @doc
%% This function is called by a gen_fsm when it receives any
%% message other than a synchronous or asynchronous event
%% (or a system message).
%%
%% @spec handle_info(Info,StateName,State)->
%%                   {next_state, NextStateName, NextState} |
%%                   {next_state, NextStateName, NextState, Timeout} |
%%                   {stop, Reason, NewState}
%% @end
%%--------------------------------------------------------------------
handle_info(_Info, StateName, State) ->
    {next_state, StateName, State}.

%%--------------------------------------------------------------------
%% @private
%% @doc
%% This function is called by a gen_fsm when it is about to
%% terminate. It should be the opposite of Module:init/1 and do any
%% necessary cleaning up. When it returns, the gen_fsm terminates with
%% Reason. The return value is ignored.
%%
%% @spec terminate(Reason, StateName, State) -> void()
%% @end
%%--------------------------------------------------------------------
terminate(Reason, _StateName, _State) ->
    io:format("Terminated with Reason: ~p~n", [Reason]),
    ok.

%%--------------------------------------------------------------------
%% @private
%% @doc
%% Convert process state when code is changed
%%
%% @spec code_change(OldVsn, StateName, State, Extra) ->
%%                   {ok, StateName, NewState}
%% @end
%%--------------------------------------------------------------------
code_change(_OldVsn, StateName, State, _Extra) ->
    {ok, StateName, State}.

%%%===================================================================
%%% Internal functions
%%%===================================================================

test(one) ->
    Data = [
            {chr, "a"},
            {chr, "b"},
            {chr, "c"},
            {cmd, upper},
            {chr, "d"},
            {chr, "e"},
            {chr, "f"},
            {cmd, lower},
            {chr, "g"},
            {chr, "h"},
            {chr, "i"},
            {cmd, upper},
            {chr, "j"},
            'end'
           ],
    % test without debug
    ?MODULE:start_link(self(), nodebug),
    ?MODULE:process(Data),
    receive
        {ok, Output1} ->
            io:format("Output: ~p~n", [Output1]),
            ok
    end,
    ?MODULE:stop(),
    % test with debug
    io:format("-------------------------------------------~n"),
    ?MODULE:start_link(self(), debug),
    ?MODULE:process(Data),
    receive
        {ok, Output2} ->
            io:format("Output: ~p~n", [Output2]),
            ok
    end,
    ?MODULE:stop(),
    ok;
test(two) ->
    Data1 = [
            {chr, "a"},
            {chr, "b"},
            {chr, "c"},
            {cmd, upper},
            {chr, "d"},
            {chr, "e"},
            {chr, "f"},
            {cmd, lower},
            {chr, "g"},
            {chr, "h"},
            {chr, "i"},
            {cmd, upper},
            {chr, "j"},
            'end'
           ],
    Data2 = [
            {chr, "m"},
            {chr, "n"},
            {chr, "o"},
            {cmd, upper},
            {chr, "p"},
            {chr, "q"},
            {chr, "r"},
            {cmd, lower},
            {chr, "t"},
            {chr, "u"},
            {cmd, upper},
            {chr, "v"},
            {chr, "w"},
            'end'
           ],
    % test without debug
    ?MODULE:start_link(self(), debug),
    ?MODULE:set_from_pid(self()),
    ?MODULE:process(Data1),
    receive
        {ok, Output1} ->
            io:format("Output: ~p~n", [Output1]),
            ok
    end,
    ?MODULE:stop(),
    % test with debug
    io:format("-------------------------------------------~n"),
    ?MODULE:start_link(self(), nodebug),
    ?MODULE:process(Data1),
    receive
        {ok, Output2} ->
            io:format("Output: ~p~n", [Output2]),
            ok
    end,
    ?MODULE:process(Data2),
    receive
        {ok, Output3} ->
            io:format("Output: ~p~n", [Output3]),
            ok
    end,
    ?MODULE:stop(),
    ok;
test(three) ->
    Data1 = [
            {chr, "a"},
            {chr, "b"},
            {chr, "c"},
            {cmd, upper},
            {chr, "d"},
            {chr, "e"},
            {chr, "f"},
            {cmd, lower},
            {chr, "g"},
            {chr, "h"},
            {chr, "i"},
            {cmd, upper},
            {chr, "j"},
            'end'
           ],
    Data2 = [
            {chr, "m"},
            {chr, "n"},
            {chr, "o"},
            {cmd, upper},
            {chr, "p"},
            {chr, "q"},
            {chr, "r"},
            {cmd, lower},
            {chr, "t"},
            {chr, "u"},
            {cmd, upper},
            {chr, "v"},
            {chr, "w"},
            'end'
           ],
    % test without debug
    %?MODULE:start_link(self(), debug),
    ?MODULE:set_from_pid(self()),
    ?MODULE:process(Data1),
    receive
        {ok, Output1} ->
            io:format("Output: ~p~n", [Output1]),
            ok
    end,
    %?MODULE:stop(),
    % test with debug
    io:format("-------------------------------------------~n"),
    %?MODULE:start_link(self(), nodebug),
    ?MODULE:process(Data1),
    receive
        {ok, Output2} ->
            io:format("Output: ~p~n", [Output2]),
            ok
    end,
    ?MODULE:process(Data2),
    receive
        {ok, Output3} ->
            io:format("Output: ~p~n", [Output3]),
            ok
    end,
    %?MODULE:stop(),
    ok.

Notes:

  • There is a test function (with multiple clauses) that can be used to test and run our code.

3   Building and compiling

Use this:

$ cd path/to/char_case
$ erlc -o ebin src/*.erl

4   Using our application

Now that we've implemented our application (called a "release" in Erlang terms, I believe), here are several ways that we can use it:

4.1   In the Erlang erl shell on the local machine

We can run it locally as follows:

$ cd path/to/char_case
$ erl -pa ebin -name session1@crow.local -setcookie xxx
(session1@crow.local)1> application:start(char_case).
ok
(session1@crow.local)2> char_case_fsm:test(three).
Output: ["a","b","c","D","E","F","g","h","i","J"]
-------------------------------------------
Output: ["a","b","c","D","E","F","g","h","i","J"]
Output: ["m","n","o","P","Q","R","t","u","V","W"]
ok
(session1@crow.local)3> application:stop(char_case).

=INFO REPORT==== 21-Jun-2016::11:05:43 ===
    application: char_case
    exited: stopped
    type: temporary
ok

Notes:

  • Notice how we start up the Erlang shell. The -pa ebin gives us access to the code in the ebin subdirectory. The -name and -setcookie options will be used later when we access our application across the network.
  • The line with application:start starts up and initializes the application and the supervisor behaviors.
  • Line 2 char_case_fsm:test(three) runs test 3.
  • Line 3 containing application:stop shuts down the application and supervisor.

4.2   From a remote machine on the local network (LAN)

Here is a module that we can use to run a test from a remote machine:

-module(test_remote01).

-export([main/0]).

main() ->
    rpc:call('session1@crow.local', application, start, [char_case]),
    A1 = rpc:call('session1@crow.local', char_case_fsm, test, [three]),
    A2 = rpc:call('session1@crow.local', char_case_fsm, test, [three]),
    rpc:call('session1@crow.local', application, stop, [char_case]),
    io:format("===============================================================~n"),
    rpc:call('session1@crow.local', test01, start, [30]),
    A3 = rpc:call('session1@crow.local', test01, rpc, [add, 11, 6]),
    A4 = rpc:call('session1@crow.local', test01, rpc, [add, 11, 7]),
    A5 = rpc:call('session1@crow.local', test01, rpc, [mult, 11, 8]),
    rpc:call('session1@crow.local', test01, stop, []),
    Replies = [A1, A2, A3, A4, A5],
    io:format("Replies: ~p~n", [Replies]),
    {ok, Replies}.

Notes:

  • We use the Erlang rpc module to make remote procedure calls to the node and session that can run our application.

And, here is how we can run that:

$ erl -name session1@bluejay.local -setcookie xxx
Erlang/OTP 17 [erts-6.2] [source] [smp:4:4] [async-threads:10] [kernel-poll:false]

Eshell V6.2  (abort with ^G)
(session1@bluejay.local)1> c(test_remote01).
{ok,test_remote01}
(session1@bluejay.local)2> test_remote01:main().
Output: ["a","b","c","D","E","F","g","h","i","J"]
-------------------------------------------
Output: ["a","b","c","D","E","F","g","h","i","J"]
Output: ["m","n","o","P","Q","R","t","u","V","W"]
Output: ["a","b","c","D","E","F","g","h","i","J"]
-------------------------------------------
Output: ["a","b","c","D","E","F","g","h","i","J"]
Output: ["m","n","o","P","Q","R","t","u","V","W"]
===============================================================
Response: {31,17}
Response: {32,18}
Response: {33,88}
Replies: [ok,ok,{ok,{31,17}},{ok,{32,18}},{ok,{33,88}}]
{ok,[ok,ok,{ok,{31,17}},{ok,{32,18}},{ok,{33,88}}]}

Notes:

  • We start the erl shell with the same node/session name and the same -setcookie as the machine we are going to communicate with.
  • Line 1 compiles the test module, to make sure that we are using the latest code.
  • Line 2 runs our test, which makes the remote rpc calls to the node and session that can run our application.

And, finally, here is an escript script that we can use as a convenience to run the above module (test_remote01.erl):

#!/usr/bin/env escript
%%! -name session1@bluejay.local -setcookie aaa

main(_) ->
    Node = node(),
    Cookie = erlang:get_cookie(),
    io:format("Node: ~p  Cookie: ~p~n", [Node, Cookie]),
    test_remote01:main(),
    good.

Notes:

  • Notice how we specify the node/session name and the session cookie in the 2nd line of the script. The "%%!" is required. See the escript documentation for more on this: http://erlang.org/doc/man/escript.html
  • Then, after getting and printing out the node/session name and cookie (for informational purposes), we call the main function in the test_remote01.erl module.

links