View source for Processing HTML Forms Using the POST Method

== Overview ==

With HTML forms you can use the GET method to process user input. This is quite simple to use, particularly because Nut/OS provides the three API functions <tt>NutHttpGetParameterCount</tt>, <tt>NutHttpGetParameterName</tt> and <tt>NutHttpGetParameterValue</tt> to extract the parameters and its values from the request string.

However, the total size of the request string is limited with most browsers and HTTP servers. In Nut/OS the maximum size is specified by HTTP_MAX_REQUEST_SIZE, which is 256 bytes only, by default. As an alternative, the POST method may be used instead.

I will present a simple web-based chat application to demonstrate how to use the POST method in Nut/OS.

== The Chat Server ==

This application allows any number of users to chat online, using a webbrowser to connect to a chat server running on an Ethernut board.

=== Header Files ===

System configurations are stored in

<source lang="c">
#include <cfg/os.h>
</source>

and can be used to set up the right thread stack size, based on the CPU architecture and the compile mode. When compiling with debug information, more stack is required and <tt>NUT_THREAD_STACK_MULT</tt> and <tt>NUT_THREAD_STACK_ADD</tt> will have higher values.

<source lang="c">
/*! \brief Server thread stack size. */
#ifndef HTTPD_SERVICE_STACK
#if defined(__AVR__)
#define HTTPD_SERVICE_STACK ((580 * NUT_THREAD_STACK_MULT) + NUT_THREAD_STACK_ADD)
#elif defined(__arm__)
#define HTTPD_SERVICE_STACK ((1024 * NUT_THREAD_STACK_MULT) + NUT_THREAD_STACK_ADD)
#else
#define HTTPD_SERVICE_STACK  ((2048 * NUT_THREAD_STACK_MULT) + NUT_THREAD_STACK_ADD)
#endif
#endif
</source>

Hardware-specific dependencies are handled in

<source lang="c">
#include <dev/board.h>
</source>

This will internally include several other device-specific header files, but not necessarily the header for the file system driver. Our application will use the simple UROM filesystem.

<source lang="c">
#include <dev/urom.h>
</source>

The application calls a number of basic API function, defined in the following header files:

<source lang="c">
#include <sys/version.h>
#include <sys/thread.h>
#include <sys/timer.h>
</source>

All required definitions for the network stuff are provided by these header files:

<source lang="c">
#include <sys/confnet.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <pro/httpd.h>
#include <pro/dhcp.h>
</source>

Last not least, the application uses a number of standard C library functions.

<source lang="c">
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include <io.h>
</source>

=== Hard-Coded Parameters ===

To keep the application simple, only the number of buffered chat lines has been made configurable on top of the source code.

<source lang="c">
/*! \brief Max. number of chat lines to display. */
#define MAX_CHAT_LINES  5
</source>

This parameter fixes the size of a global array, which is used to buffer the latest chat message.

<source lang="c">
/*! \brief Array of chat lines. */
static char *chat_lines[MAX_CHAT_LINES];

/*! \brief Current length of all chat lines, including list tags. */
static int chat_len;
</source>

The second variable <tt>chat_len</tt> keeps track of the total number of bytes contained in all lines.

=== main ===

When the system is started, Nut/OS will do some internal initialization and then call function <tt>main</tt> in the application code. Our chat application is implemented by an HTTP server and the main routine looks quite similar to the main routine of the HTTP daemon sample included in the Nut/OS distribution.

{| class="wikitable" width="100%"
|<source lang="c" line start="321">
/*!
 * \brief Main application routine.
 */
int main(void)
{
    uint32_t baud = 115200;
    int i;

    /* Initialize the stdout device. */
    NutRegisterDevice(&DEV_DEBUG, 0, 0);
    freopen(DEV_DEBUG_NAME, "w", stdout);
    _ioctl(_fileno(stdout), UART_SETSPEED, &baud);
    printf("\n\nHTTP POST Sample - Nut/OS %s\n", NutVersionString());

    /* Configure the Ethernet interface. */
    if (NutRegisterDevice(&DEV_ETHER, 0, 0)) {
        puts("Registering device failed");
    }
    printf("Configure %s...", DEV_ETHER_NAME);
    if (NutDhcpIfConfig(DEV_ETHER_NAME, 0, 60000)) {
        puts("failed");
    } else {
        puts("OK");
    }
    printf("%s ready\n", inet_ntoa(confnet.cdn_ip_addr));

    /* Register the file system device. */
    NutRegisterDevice(&devUrom, 0, 0);

    /* Register the CGI functions. */
    NutRegisterCgi("form.cgi", SendForm);
    NutRegisterCgi("post.cgi", ProcessForm);

    /* Start four server threads, running concurrently. */
    for (i = 1; i <= 4; i++) {
        NutThreadCreate("http", Service, NULL, HTTPD_SERVICE_STACK);
    }
    /* Processes clients in an endless loop. */
    for (;;) {
        ProcessNextClient();
    }
    return 0;
}
</source>
|}

=== HTTP Server Threads ===

As you may know, Nut/OS doesn't provide a backlog for incoming connections. If a TCP connection is established, incoming request for additional connections to the same port are rejected. The typical solution is to start more than one thread to service a specific TCP port. This is done in the main function on lines 355-357, where 4 additional threads are started.

The thread function itself does nothing else but calling <tt>ProcessNextClient</tt> in an endless loop.

{| class="wikitable" width="100%"
|<source lang="c" line start="309">
/*! \fn Service(void *arg)
 * \brief HTTP service thread.
 *
 * Processes clients in an endless loop.
 */
THREAD(Service, arg)
{
    for (;;) {
        ProcessNextClient();
    }
}
</source>
|}

The same loop is used at the end of the main function (lines 359-361), so in total 5 threads are running to server up to 5 concurrent client requests. Each thread calls <tt>ProcessNextClient</tt> over and over again.

{| class="wikitable" width="100%"
|<source lang="c" line start="282">
/*!
 * \brief Process next HTTP client.
 *
 * This function waits for a client connect, processes the HTTP request 
 * and disconnects.
 */
static void ProcessNextClient(void)
{
    TCPSOCKET *sock;
    FILE *stream;

    /* Create a socket. */
    sock = NutTcpCreateSocket();
    /* Listen on port 80. This call will block until we get a 
       connection from a client. */
    NutTcpAccept(sock, 80);
    /* Associate a stream with the socket so we can use standard 
       I/O calls. */
    stream = _fdopen((int) ((uptr_t) sock), "r+b");
    /* Process HTTP request. */
    NutHttpProcessRequest(stream);
    /* Destroy the stream device. */
    fclose(stream);
    /* Close our socket. */
    NutTcpCloseSocket(sock);
}
</source>
|}

=== CGIs ===

In function <tt>ProcessNextClient</tt> the Nut/OS API function <tt>NutHttpProcessRequest</tt> is called to process the HTTP request. No further effort is required to handle static content like HTML files or images.

Dynamic content may be created by CGI routines. Two such CGI routines are registered in lines 351 and 352 of the main function, <tt>SendForm</tt> and <tt>ProcessForm</tt>, registered for the URLs <tt>/cgi-bin/form.cgi</tt> and <tt>/cgi-bin/post.cgi</tt>, resp. If a client requests one of these, then <tt>NutHttpProcessRequest</tt> will call its registered routine.

The first CGI routine <tt>SendForm</tt> will send an HTML page containing a simple form.

{| class="wikitable" width="100%"
|<source lang="c" line start="160">
/*!
 * \brief Send HTML page with a chat form to the client.
 *
 * This routine must have been registered by NutRegisterCgi() and is
 * automatically called by NutHttpProcessRequest() when the client
 * request the URL '/cgi-bin/form.cgi'.
 *
 * \param stream Stream of the HTTP connection.
 * \param req    HTTP request information.
 *
 * \return Always 0.
 */
int SendForm(FILE * stream, REQUEST * req)
{
    static CONST char *html_top =
        "<html>\r\n"
        "<head>\r\n"
        "  <title>Form Post Test</title>\r\n"
        "</head>\r\n"
        "<body>\r\n"
        "  <form action=\"/cgi-bin/post.cgi\" method=\"post\">\r\n"
        "    <textarea name=\"text\"></textarea><br />\r\n"
        "    <input type=\"submit\" value=\"Update\">\r\n" "  </form>\r\n" "  <ul>\r\n";
    static CONST char *html_bot = "  </ul>\r\n" "</body>\r\n" "</html>\r\n";
    int form_len = sizeof(html_top) + sizeof(html_bot) - 2;
    int i;

    /* Send the HTTP header. */
    NutHttpSendHeaderTop(stream, req, 200, "Ok");
    NutHttpSendHeaderBottom(stream, req, "text/html", chat_len + form_len);
    /* Send the HTML code. */
    fputs(html_top, stream);
    for (i = 0; i < MAX_CHAT_LINES; i++) {
        if (chat_lines[i]) {
            /* Enclose chat lines by list tags. */
            fputs("<li>", stream);
            fputs(chat_lines[i], stream);
            fputs("</li>\r\n", stream);
        }
    }
    fputs(html_bot, stream);
    fflush(stream);

    return 0;
}
</source>
|}

The second CGI routine <tt>ProcessForm</tt> will send will process the submit of the form.

{| class="wikitable" width="100%"
|<source lang="c" line start="206">
/*!
 * \brief Process a POST of the chat form.
 *
 * This routine must have been registered by NutRegisterCgi() and is
 * automatically called by NutHttpProcessRequest() when the client
 * request the URL '/cgi-bin/post.cgi'.
 *
 * \param stream Stream of the HTTP connection.
 * \param req    HTTP request information.
 *
 * \return Always 0.
 */
int ProcessForm(FILE * stream, REQUEST * req)
{
    static const char redir_body[] = "<html><body><a href=\"/\">Continue</a></body></html>\r\n";
    char *cp;

    if (req->req_method == METHOD_POST) {
        int rc;
        int i;
        int sz = req->req_length;
        char *buf = malloc(sz + 1);

        /* Read the complete contents. */
        while (sz > 0) {
            /* Get the next parameter. */
            rc = GetEncodedString(stream, buf, sz, '=');
            if (rc <= 0) {
                /* Failed to read encoded string. */
                break;
            }
            sz -= rc;
            if (strcmp(buf, "text") == 0 && sz) {
                /* Get the value of the parameter 'text'. */
                rc = GetEncodedString(stream, buf, sz, '&');
                if (rc <= 0) {
                    /* Failed to read encoded string. */
                    break;
                }
                sz -= rc;

                /* Remove HTML tags. */
                for (cp = buf; *cp; cp++) {
                    if (*cp == '<' || *cp == '>') {
                        *cp = ' ';
                    }
                }

                /* Append the new chat line to the chat line array. Note,
                   that we add 11 to the size of a chat line to reflect
                   the size of the surrounding list tags. These tags are
                   not stored in the line buffer. */
                if (chat_lines[0]) {
                    chat_len -= strlen(chat_lines[0]) + 11;
                    free(chat_lines[0]);
                }
                for (i = 0; i < MAX_CHAT_LINES - 1; i++) {
                    chat_lines[i] = chat_lines[i + 1];
                }
                chat_lines[i] = strdup(buf);
                chat_len += strlen(buf) + 11;
            }
        }
        free(buf);
    }

    /* Redirect the client to our home page. */
    NutHttpSendHeaderTop(stream, req, 303, "Redir");
    fputs("Location: /\r\n", stream);
    NutHttpSendHeaderBottom(stream, req, "text/html", sizeof(redir_body) - 1);
    fputs(redir_body, stream);
    fflush(stream);

    return 0;
}
</source>
|}

To access the form, the user must request the URL http://mynutip/cgi-bin/form.cgi, which is a bit complicated. Just entering http://mynutip will result in error 404, because in this case the API will look for index.html, index.shtml or similar.

A nice solution is to create index.html containing a redirection to our CGI.

<source lang="html4strict">
<html>
<head>
  <meta http-equiv="refresh" content="0; URL=/cgi-bin/form.cgi">
</head>
</html>
</source>

This file will be placed in subdirectory www, which will be converted by crurom into urom.c.

=== Another way of processing POSTed form data ===

Current Nut/OS directly support the POST method when processing a form submit. Instead of manually parsing the form data like in the example above you can just use the function "NutHttpProcessPostQuery(stream, req);". The same CGI would look like this:

{| class="wikitable" width="100%"
|<source lang="c" line start="206">
/*!
 * \brief Process a POST of the chat form in the same way as using a GET request
 *
 * This routine must have been registered by NutRegisterCgi() and is
 * automatically called by NutHttpProcessRequest() when the client
 * request the URL '/cgi-bin/post_new_style.cgi'.
 *
 * \param stream Stream of the HTTP connection.
 * \param req    HTTP request information.
 *
 * \return Always 0.
 */
int ProcessFormNewStyle(FILE * stream, REQUEST * req)
{
    static const char redir_body[] = "<html><body><a href=\"/\">Continue</a></body></html>\r\n";
    char *cp;

    if (req->req_method == METHOD_POST) {
        char *name;
        char *value;
        int idx;
        int count;

        NutHttpProcessPostQuery(stream, req);
        count = NutHttpGetParameterCount(req);

        for (idx = 0; idx < count; idx++) {
            name  = NutHttpGetParameterName(req, idx);
            value = NutHttpGetParameterValue(req, idx);

            if (strcmp(name, "text") == 0) {
                /* Remove HTML tags. */
                for (cp = value; *cp; cp++) {
                    if (*cp == '<' || *cp == '>') {
                        *cp = ' ';
                    }
                }

                /* Append the new chat line to the chat line array. Note,
                   that we add 11 to the size of a chat line to reflect
                   the size of the surrounding list tags. These tags are
                   not stored in the line buffer. */
                if (chat_lines[0]) {
                    chat_len -= strlen(chat_lines[0]) + 11;
                    free(chat_lines[0]);
                }
                for (i = 0; i < MAX_CHAT_LINES - 1; i++) {
                    chat_lines[i] = chat_lines[i + 1];
                }
                chat_lines[i] = strdup(value);
                chat_len += strlen(value) + 11;
            }
        }
    }

    /* Redirect the client to our home page. */
    NutHttpSendHeaderTop(stream, req, 303, "Redir");
    fputs("Location: /\r\n", stream);
    NutHttpSendHeaderBottom(stream, req, "text/html", sizeof(redir_body) - 1);
    fputs(redir_body, stream);
    fflush(stream);

    return 0;
}

</source>
|}

=== POST Method Processing in Application Code ===

Early versions of the Nut/OS API did not directly support the POST method when processing a form submit. This has to be done by the application. This is _not_ needed if you implement your CGIs using the NutHttpProcessPostQuery(stream, req); method as shown in the above sample.

{| class="wikitable" width="100%"
|<source lang="c" line start="80">

/*!
 * \brief Read hex-encoded character from stream.
 *
 * \param stream The stream to read from.
 *
 * \return Decoded character or EOF on errors.
 */
static int GetEncodedHexChar(FILE * stream)
{
    int i;
    int c = 0;
    int ch;

    for (i = 0; i < 2; i++) {
        c <<= 4;
        ch = fgetc(stream);
        if (ch >= '0' && ch <= '9') {
            c |= ch - '0';
        } else if (ch >= 'A' && ch <= 'F') {
            c |= ch - ('A' - 10);
        } else if (ch >= 'a' && ch <= 'f') {
            c |= ch - ('a' - 10);
        } else {
            /* Bad hex code. */
            return EOF;
        }
    }
    return c;
}

/*!
 * \brief Read encoded string from stream.
 *
 * This function reads a specified number of characters from a stream,
 * but not beyond a given end-of-string character.
 *
 * \param stream The stream to read from.
 * \param str    Pointer to a buffer that receives the decoded string.
 *               The string will be terminated with ASCII 0.
 * \param siz    Maximum number of characters to read.
 * \param eos    End of string character.
 *
 * \return Number of characters read or -1 on errors.
 */
static int GetEncodedString(FILE * stream, char *str, int siz, int eos)
{
    int ch;
    int rc;

    for (rc = 0; rc < siz; rc++) {
        ch = fgetc(stream);
        if (ch == EOF) {
            return -1;
        }
        if (ch == eos) {
            rc++;
            break;
        }
        if (ch == '+') {
            ch = ' ';
        } else if (ch == '%') {
            if (rc > siz - 3) {
                return -1;
            }
            ch = GetEncodedHexChar(stream);
            if (ch == EOF) {
                return -1;
            }
            rc += 2;
        }
        if (str) {
            *str++ = (char) ch;
        }
    }
    if (str) {
        *str = '\0';
    }
    return rc;
}
</source>
|}