1   Introduction

In a previous post (http://davekuhlman.org/cowboy-rest-add-get-update-list.html and http://davekuhlman.org/cowboy-rest-add-get-update-list.html#problems-enhancements-quibbles) we mentioned a deficiency in the example application, namely that the request handler opened and closed the DETS database for each request handled. The example discussed in this current post addresses that issue.

In that previous post we looked at a REST application built on top of the Erlang Cowboy Web server. It was a CRUD application in that it supported simple create, read, update, and delete operations. The example REST application described in the current post supports the same operations and likewise stores its records in a DETS database, but implements a separate supervised OTP process that "owns" the database resource, performs all needed operations on that resources, and keeps the DETS database open across responses to multiple requests.

You can find the source for this example in the cowboy_rest_crud_otp_process/ subdirectory of the repository at https://github.com/dkuhlman/cowboy_rest_examples.git.

2   Explanation

Here is a summary of the required changes:

  1. Add src/db_sup.erl -- This is the supervisor for the database server process.
  2. Add src/db_server.erl -- This implements the actual database server. It is built on top of the gen_server behavior.
  3. Add code to src/rest_update_app.erl that starts the supervisor which in turn starts the server
  4. Modify the code in src/db_update_handler.erl so that it makes calls to db_server rather than performing database operations itself.

3   Details

Note that the files of interest in this modified example are listed in full in section The code.

Both the DB supervisor and the DB server were built on top of the skeletons generated by Vim using the vim-erlang-skeletons plugin. See: https://github.com/dkuhlman/vim-erlang-skeletons.git and https://github.com/vim-erlang. Similar OTP skeletons are available through Emacs.

3.1   The DB supervisor

src/db_sup.erl:

init([]) ->
    RestartStrategy = one_for_one,
    MaxRestarts = 1000,
    MaxSecondsBetweenRestarts = 3600,
    SupFlags = {RestartStrategy, MaxRestarts, MaxSecondsBetweenRestarts},
    Restart = permanent,
    Shutdown = 2000,
    Type = worker,
    AChild = {db_server, {db_server, start_link, []},
                      Restart, Shutdown, Type, [db_server]},
    {ok, {SupFlags, [AChild]}}.

Notes:

  • I replaced the skeleton's name for the child with "db_server".

3.2   The application

We add the following code to start/2 in rest_update_app.erl:

db_sup:start_link(),

Notes:

  • The above line calls db_sup:start_link/0, which calls supervisor/start_link/3, which results in a call to the callback db_sup:init/1.
  • It's the code in db_sup:init/1 causes our supervisor (src_db_sup.erl) to start up the database server process (implemented in src/db_server.erl).

3.3   The DB server

The DB server is implemented on top of the gen_server OTP behavior.

We implement an API to be used by our REST handler (src/db_update_handler.erl). The API includes an exported function for each of the database operations that we support: add_rec/1, get_rec/1, get_record_list/0, update_rec/1, and delete_rec/1.

Each of these functions in our externally visible API simply forwards the request through gen_server to the handle_call/3 callback function. Here is an example:

get_rec(RecordId) ->
    gen_server:call(?SERVER, {get_rec, RecordId}).

Notes:

  • The second argument to the call to gen_server:call/2 is passed as the first argument to the callback handle_call/3.

Here is the clause of handle_call/3 that handles the above call:

handle_call({get_rec, Key}, _From, State) ->
    Records = dets:lookup(records_db, Key),
    Reply = case Records of
        [{_RecordId, Data}] ->
            {ok, Data};
        [] ->
            {error, not_found};
        _ ->
            {error, too_many_records}
    end,
    {reply, Reply, State};

Notes:

  • The clause contains the reworked code from the previous post and example that retrieved a record from the DETS database.
  • The result of our database lookup is returned as the Reply in the returned term {reply, Reply, State}. The value of Reply is returned in turn by our API function get_rec/1.

4   The code

4.1   The DB supervisor code

src/db_sup.erl:

%%%-------------------------------------------------------------------
%%% @author Dave Kuhlman
%%% @copyright (C) 2016, Dave Kuhlman
%%% @doc
%%%
%%% @end
%%% Created : 2016-12-19 15:35:26.624587
%%%-------------------------------------------------------------------
-module(db_sup).

-behaviour(supervisor).

%% API
-export([start_link/0]).

%% Supervisor callbacks
-export([init/1]).

-define(SERVER, ?MODULE).

%%%===================================================================
%%% API functions
%%%===================================================================

%%--------------------------------------------------------------------
%% @doc
%% Starts the supervisor
%%
%% @spec start_link() -> {ok, Pid} | ignore | {error, Error}
%% @end
%%--------------------------------------------------------------------
start_link() ->
    supervisor:start_link({local, ?SERVER}, ?MODULE, []).

%%%===================================================================
%%% Supervisor callbacks
%%%===================================================================

%%--------------------------------------------------------------------
%% @private
%% @doc
%% Whenever a supervisor is started using supervisor:start_link/[2,3],
%% this function is called by the new process to find out about
%% restart strategy, maximum restart frequency and child
%% specifications.
%%
%% @spec init(Args) -> {ok, {SupFlags, [ChildSpec]}} |
%%                     ignore |
%%                     {error, Reason}
%% @end
%%--------------------------------------------------------------------
init([]) ->
    RestartStrategy = one_for_one,
    MaxRestarts = 1000,
    MaxSecondsBetweenRestarts = 3600,
    SupFlags = {RestartStrategy, MaxRestarts, MaxSecondsBetweenRestarts},
    Restart = permanent,
    Shutdown = 2000,
    Type = worker,
    AChild = {db_server, {db_server, start_link, []},
                      Restart, Shutdown, Type, [db_server]},
    {ok, {SupFlags, [AChild]}}.

%%%===================================================================
%%% Internal functions
%%%===================================================================

Notes:

  • The work is done in init/1. It starts our child process, which is implemented in src/db_server.erl. And because we use the one_for_one restart strategy, this supervisor will restart its child (src/db_server.erl) if it fails.
  • For more information on OTP supervisors and the one_for_one restart strategy, see: http://erlang.org/doc/man/supervisor.html.

4.2   The DB server code

src/db_server.erl:

%%%-------------------------------------------------------------------
%%% @author Dave Kuhlman
%%% @copyright (C) 2016, Dave Kuhlman
%%% @doc
%%%
%%% @end
%%% Created : 2016-12-19 14:06:45.650615
%%%-------------------------------------------------------------------
-module(db_server).

-behaviour(gen_server).

%% API
-export([start_link/0]).

%% gen_server callbacks
-export([init/1,
         handle_call/3,
         handle_cast/2,
         handle_info/2,
         terminate/2,
         code_change/3]).

%% db_server API
-export([
         get_rec/1,
         get_all_recs/0,
         delete_rec/1,
         create_rec/1,
         update_rec/2
        ]).

-define(SERVER, ?MODULE).

-record(state, {}).

%%%===================================================================
%%% API
%%%===================================================================

%%--------------------------------------------------------------------
%% @doc
%% Starts the server
%%
%% @spec start_link() -> {ok, Pid} | ignore | {error, Error}
%% @end
%%--------------------------------------------------------------------
start_link() ->
    gen_server:start_link({local, ?SERVER}, ?MODULE, [], []).

%%--------------------------------------------------------------------

get_rec(RecordId) ->
    io:fwrite("(get_rec) RecordId: ~p~n", [RecordId]),
    gen_server:call(?SERVER, {get_rec, RecordId}).

get_all_recs() ->
    io:fwrite("(get_all_recs)~n"),
    gen_server:call(?SERVER, {get_all_recs}).

delete_rec(RecordId) ->
    io:fwrite("(delete_rec) RecordId: ~p~n", [RecordId]),
    gen_server:call(?SERVER, {delete_rec, RecordId}).

create_rec(Content) ->
    io:fwrite("(create_rec) ~n", []),
    gen_server:call(?SERVER, {create_rec, Content}).

update_rec(RecordId, Content) ->
    io:fwrite("(update_rec) RecordId: ~p  Content: ~p~n", [RecordId, Content]),
    gen_server:call(?SERVER, {update_rec, RecordId, Content}).

%%%===================================================================
%%% gen_server callbacks
%%%===================================================================

%%--------------------------------------------------------------------
%% @private
%% @doc
%% Initializes the server
%%
%% @spec init(Args) -> {ok, State} |
%%                     {ok, State, Timeout} |
%%                     ignore |
%%                     {stop, Reason}
%% @end
%%--------------------------------------------------------------------
init([]) ->
    {ok, Recordfilename} = application:get_env(rest_update, records_file_name),
    dets:open_file(records_db, [{file, Recordfilename}, {type, set}]),
    {ok, Statefilename} = application:get_env(rest_update, state_file_name),
    dets:open_file(state_db, [{file, Statefilename}, {type, set}]),
    {ok, #state{}}.

%%--------------------------------------------------------------------
%% @private
%% @doc
%% Handling call messages
%%
%% @spec handle_call(Request, From, State) ->
%%                                   {reply, Reply, State} |
%%                                   {reply, Reply, State, Timeout} |
%%                                   {noreply, State} |
%%                                   {noreply, State, Timeout} |
%%                                   {stop, Reason, Reply, State} |
%%                                   {stop, Reason, State}
%% @end
%%--------------------------------------------------------------------

handle_call({get_rec, Key}, _From, State) ->
    Records = dets:lookup(records_db, Key),
    io:fwrite("(call get_rec) Records: ~p~n", [Records]),
    Reply = case Records of
        [{_RecordId, Data}] ->
            {ok, Data};
        [] ->
            {error, not_found};
        _ ->
            {error, too_many_records}
    end,
    {reply, Reply, State};

handle_call({get_all_recs}, _From, State) ->
    F = fun (Item, Acc) -> Acc1 = [Item | Acc], Acc1 end,
    Items = dets:foldl(F, [], records_db),
    Reply = {ok, Items},
    {reply, Reply, State};

handle_call({delete_rec, RecordId}, _From, State) ->
    Reply = case dets:lookup(records_db, RecordId) of
        [] ->
            {error, not_found};
        _ ->
            dets:delete(records_db, RecordId)
    end,
    {reply, Reply, State};

handle_call({create_rec, Content}, _From, State) ->
    RecordId = generate_id(),
    ok = dets:insert(records_db, {RecordId, Content}),
    ok = dets:sync(records_db),
    Reply = {ok, RecordId},
    {reply, Reply, State};

handle_call({update_rec, RecordId, NewContent}, _From, State) ->
    DBResponse = dets:lookup(records_db, RecordId),
    Reply = case DBResponse of
        [_] ->
            ok = dets:insert(records_db, {RecordId, NewContent}),
            ok = dets:sync(records_db),
            Response = io_lib:format("/get/~s", [RecordId]),
            Response1 = list_to_binary(Response),
            {ok, Response1};
        [] ->
            {error, not_found}
    end,
    {reply, Reply, State}.

%%--------------------------------------------------------------------
%% @private
%% @doc
%% Handling cast messages
%%
%% @spec handle_cast(Msg, State) -> {noreply, State} |
%%                                  {noreply, State, Timeout} |
%%                                  {stop, Reason, State}
%% @end
%%--------------------------------------------------------------------
handle_cast(_Msg, State) ->
        {noreply, State}.

%%--------------------------------------------------------------------
%% @private
%% @doc
%% Handling all non call/cast messages
%%
%% @spec handle_info(Info, State) -> {noreply, State} |
%%                                   {noreply, State, Timeout} |
%%                                   {stop, Reason, State}
%% @end
%%--------------------------------------------------------------------
handle_info(_Info, State) ->
        {noreply, State}.

%%--------------------------------------------------------------------
%% @private
%% @doc
%% This function is called by a gen_server when it is about to
%% terminate. It should be the opposite of Module:init/1 and do any
%% necessary cleaning up. When it returns, the gen_server terminates
%% with Reason. The return value is ignored.
%%
%% @spec terminate(Reason, State) -> void()
%% @end
%%--------------------------------------------------------------------
terminate(_Reason, _State) ->
    dets:close(state_db),
    dets:close(records_db),
    ok.

%%--------------------------------------------------------------------
%% @private
%% @doc
%% Convert process state when code is changed
%%
%% @spec code_change(OldVsn, State, Extra) -> {ok, NewState}
%% @end
%%--------------------------------------------------------------------
code_change(_OldVsn, State, _Extra) ->
        {ok, State}.

%%%===================================================================
%%% Internal functions
%%%===================================================================

generate_id() ->
    Records = dets:lookup(state_db, current_id),
    Response = case Records of
        [{current_id, CurrentId}] ->
            NextId = CurrentId + 1,
            dets:insert(state_db, {current_id, NextId}),
            Id = lists:flatten(io_lib:format("id_~4..0B", [CurrentId])),
            Id;
        [] ->
            error
    end,
    Response.

5   Additional comments

Suggestions for future work:

  • The skeletons generated by the Vim vim-erlang-skeletons plugin use the old style type specifications. I really should update those to the style of type specifications described here: http://erlang.org/doc/reference_manual/typespec.html.
  • Creating a modified version of this example application that uses Mnesia instead of DETS might be useful.

- Dave Kuhlman