GNU Artanis web-framework Manual ¶

1.1 Conventions used in this manual ¶

In this manual the following syntax is used to demonstrate the use of the API:

(api-name arg1 arg2 #:key0 val0 ... [optional-arg1 <- default-value1] ...)

If you are new to Scheme, it is recommended that you read the Basic in Scheme chapter first.

1.2 No warranty ¶

We distribute software in the hope that it will be useful, but without any warranty. No author or distributor of this software accepts responsibility to anyone for the consequences of using it or for whether it serves any particular purpose or works at all, unless they say so in writing. This is exactly the same warranty that proprietary software companies offer: none.

3.1 For Guix users ¶

GNU Guix is the advanced functional & transactional package manager, we recommend you to try it. You may read the GNU Guix installation .

guix install artanis

3.2 For Ubuntu/Debian users to install dependencies ¶

apt-get -qq -y install texinfo guile-3.0 guile-3.0-dev build-essential automake git autoconf libtool libmariadb-dev-compat libmariadb-dev libnss3 libnss3-dev

apt-get -qq -y install gettext redis redis-server libcurl4-openssl-dev

3.3 For hardcore users ¶

Install GNU Guile-3.0.5 or higher version:

Since GNU Artanis-0.5, GNU Guile-3+ is required.

Install pre-requisites (typically available in linux package distributions): a C compiler, [[https://www.gnu.org/software/autoconf/]autoconf], [[https://www.gnu.org/software/automake/]automake], [[https://sourceware.org/libffi/]libffi], [[https://curl.se/libcurl/]libcurl], [[https://www.gnu.org/software/libtool/]libtool and libltdl], [[https://www.gnu.org/software/libunistring/]libunistring], [[https://www.perl.org/]perl], [[https://firefox-source-docs.mozilla.org/security/nss/index.html]libnss (Network Security Services)], [[https://tiswww.case.edu/php/chet/readline/rltop.html]readline] and [[http://zlib.net/]zlib].

wget -c ftp://ftp.gnu.org/gnu/guile/guile-3.0.10.tar.xz
tar xvf guile-3.0.10.tar.xz
cd guile-3.0.10 && ./configure && make #(NOTE: this may take a very long time)
sudo make install

I would NOT recommend trying to compile/install Guile from Git repo, since it’ll take a long time.

Install dependencies:

• guile-dbi-2.1.8 (The tarball included guile-dbd) [Optional]

wget -c https://github.com/opencog/guile-dbi/archive/guile-dbi-2.1.8.tar.gz

tar xvzf guile-dbi-2.1.8.tar.gz
cd guile-dbi-guile-dbi-2.1.7/guile-dbi && ./autogen.sh --no-configure && ./configure && make
sudo make install
cd ../..

• guile-dbd [Optional]. The dbd plugins connect to an actual database server.

cd guile-dbi-guile-dbi-2.1.8/guile-dbd-mysql && ./autogen.sh --no-configure && ./configure && make
sudo make install

MySQL is used for the examples in this manual. You may find dbd plugins for other databases here. The installation process is identical.

• guile-curl for the client module.

git clone https://github.com/spk121/guile-curl.git
cd guile-curl && git checkout v0.9
./bootstrap && ./configure --prefix=/usr && make -j
make install
ln -s /usr/lib/guile/3.0/extensions/libguile-curl.* /usr/lib/
ldconfig

NOTE: You may need to make necessary soft links according to above script.

• guile-redis:

git clone https://github.com/aconchillo/guile-redis.git
cd guile-redis
git checkout -b 2.2.0
autoreconf -vif
./configure --prefix=/usr
make
make install

• guile-json:

git clone https://github.com/aconchillo/guile-json.git
cd guile-json
git checkout -b 4.7.3
autoreconf -iv
./configure --prefix=/usr
make -j5
sudo make install
sudo ldconfig

• libnss (Network Security Services):

Please follow the [[https://firefox-source-docs.mozilla.org/security/nss/build.html] official guide]. It’s better to install it with your distro’s pkg manager.

Install the latest GNU Artanis:

,#+begin_src null wget -c http://ftp.gnu.org/gnu/artanis/artanis-latest.tar.bz2 tar xvjf artanis-latest.tar.bz2 cd artanis-latest && ./autogen.sh –no-configure && ./configure && make sudo make install

3.4 For contributors ¶

First of all, thank you for contributing! You may clone the main git repository, or the mirror on GitLab:

git clone git://git.savannah.gnu.org/artanis.git

# mirror on GitLab
git clone https://gitlab.com/hardenedlinux/artanis.git

4.1 Config APIs ¶

To change the default configurations:

(conf-set! key value)
;;e.g
(conf-set! 'debug-mode #t)

To get the current configuration:

(get-conf key)
;;e.g
(get-conf '(server charset))

To get the current hostname in GNU Artanis environment.

(current-myhost)

5.1 Use Guile REPL and verify GNU Artanis installation ¶

If you are already familiar with Guile, you may skip this section.

Type ‘guile’ in your console to enter the Guile REPL. You should see the following text displayed on your screen:

GNU Guile 3.0.9
Copyright (C) 1995-2023 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)>

Welcome to Guile world! We are now going to play with GNU Artanis. Before we start, we need to check that GNU Artanis is installed correctly:

(Just type them, you don’t have to understand them at present)

,use (artanis artanis)
artanis-version

The expected output should be similar to this:

$1 = "GNU Artanis-x.x.x"

5.2 Simple HTTP server ¶

Run this code in your console:

guile -c "(use-modules (artanis artanis))(init-server)(run)"
## You'll see this screen:
Anytime you want to quit just try Ctrl+C, thanks!
http://127.0.0.1:3000

Assuming there’s a file named "index.html" in the current path. Now you may try http://localhost:3000/index.html in your browser. It’s just simply fetching static file by the URL: http://localhost:3000/path/filename

5.3 Try simple URL remapping ¶

Type these code in Guile REPL:

(use-modules (artanis artanis))
(init-server)
(get "/hello" (lambda () "hello world"))
(run #:port 8080)

Now you can visit http://localhost:8080/hello with your browser, and (hopefully) see the result.

/If you encounter "[EXCEPTION] favicon.ico is abnormal request" , please just ignore that warning.

Let me explain the code:

• line 1: Load GNU Artanis module, (artanis artanis) is the name.

• line 2: The first argument get is GNU Artanis’ API correspondence to the GET method of the HTTP protocol. The second argument "/hello" is the URL rule to register showing in the address line of e.g. firefox. The third argument is the handler which will be triggered if the registered URL rule is hit.
• line 3: Run the GNU Artanis web server, and listen on socket port 8080.

You may type Ctrl+C to quit and stop the server, see also the message printed on the screen accordingly.

5.4 More complex URL remapping ¶

Try this code:

(use-modules (artanis artanis))
(init-server)
(get "/hello/:who"
  (lambda (rc)
    (format #f "<p>hello ~a</p> " (params rc "who"))))
(run #:port 8080)

Now you can try http://localhost:8080/hello/artanis in your browser.

There are two differences compared to the simpler example:

• 1. The special rule, "/hello/:who", :who means you can use params to refer to the value of the par of the URL with the key "who". Like: (params rc "who").
• 2. You may have noticed that the handler is being defined as an anonymous function with lambda has one argument rc. It means route context which preserves all the related context information. Many GNU Artanis APIs need it, e.g. params.

And format is a Scheme lib function. It is similar to sprintf in the C language, which outputs text with a formatted pattern. The second argument #f (means FALSE) indicates that the formatted output should be returned in a string rather than printed out.

5.5 Regex in URL remapping ¶

You can use regular expressions as a URL rule argument.

(use-modules (artanis artanis))
(init-server)
(get "/.+\\.(png|gif|jpeg)" static-page-emitter)
(run #:port 8080)

static-page-emitter is a GNU Artanis API that sends a static file (images, data files) to the client.

5.6 Database operating ¶

GNU Artanis supports mysql/postgresql/sqlite3. We use mysql as an example here.

Please ensure that your DB service was started before you run this code.

If you encounter any problems, it’s very likely it’s with your DB config.

You can use a DB (such as mysql) with GUI tools such as "adminer", independently of the running web-server, e.g. artanis-based.

(use-modules (artanis artanis))
(init-server)
(define conn (connect-db 'mysql #:db-username "your_db_username"
                         #:db-name "your_db_name" #:db-passwd "your_passwd"))
(define mtable (map-table-from-DB conn))
((mtable 'create 'Persons '((name varchar 10)
                            (age integer)
                            (email varchar 20)))
 'valid?)
;; ==> #t
(mtable 'set 'Persons #:name "nala" #:age 20 #:email "nala@artanis.dev")
(mtable 'get 'Persons #:columns '(name email))
;; ==> ((("name" . "nala") ("email" . "nala@artanis.com")))

• map-table-from-DB is the GNU Artanis API handling tables in a DB. Here, we define this mapping as the var mtable.

• We can use mtable to handle tables, you can get the values from a table with the ’get command.

• mtable is a function which accepts the first argument as a command, say ’create, which is a command to create a new table. ’set is used to insert/update values in the table. And the ’get command to fetch the values of specific columns.

• The second argument of mtable is the name of the table as you can guess. Please note that it is case sensitive, while the column name isn’t.

• The ’create command returns a function too, which also accepts an argument as a command. Here, we use the ’valid? command to check if the table has been created successfully.

This was just a simple introduction. You may read the DB section in this manual for details.

Of course, you can use DB in your web application.

(get "/dbtest" #:conn #t ; apply for a DB connection from pool
     (lambda (rc)
       (let ((mtable (map-table-from-DB (:conn rc))))
         (object->string
          (mtable 'get 'Persons #:columns '(name email))))))

(run #:use-db? #t #:dbd 'mysql #:db-username "your_db_username"
     #:db-name "your_db_name" #:db-passwd "your_passwd" #:port 8080)

Now, try loading http://localhost:8080/dbtest in your browser.

Here is quick explanation:

• The keyword-value pair #:conn #t means applying for a DB connection from connection-pool. Then you can use (:conn rc) to get the allocated connection for DB operations.
• Finally, the handler needs to return a string as the HTTP response body, so in this example,

we have to use the Guile API object->string to convert the query result into a string.

Exercise: Return a beautiful table in HTML rather than using object->string.

6.1 For newbies ¶

If you’re not familiar with Scheme and Guile in particular, here is a simple tutorial for you.

If you already know the basics of the Scheme language, please feel free to skip this section.

I would recommend newbies to type/paste the code in Guile REPL following the guide in tutorial: Learn Scheme in 15 minutes

And here’s a nice section in the Guile manual for basics in Scheme: Hello Scheme

Please don’t spend too much time on these tutorials, the purpose is to let newbies get a little familiar with the grammar of Scheme.

6.2 For Pythonistas ¶

These are good pythonic articles for Pythoners:

1. Guile basics from the perspective of a Pythonista
2. Going from Python to Guile Scheme

Still, please don’t spend too much time on them, the purpose is to let newbies get a little familiar with the grammar of Scheme.

6.3 For Rubyist ¶

Here’s a great article for Rubyist to learn Scheme:

1. Scheme for ruby programmers

6.4 For deep learners ¶

These two books are very good for learning Scheme seriously:

1. The Scheme Programming Language
2. Structure and Interpretation of Computer Programs(SICP)

Please don’t bother reading them if you simply want to use GNU Artanis to build your web application/site in few minutes.

And if you really want to try to work these books seriously, please ignore GNU Artanis before you are done with them.

But once you’re done reading them carefully, you may want to write a new GNU Artanis all by yourself!

Hold your horses. ;-)

7.1 How to run a site with GNU Artanis ¶

This is the simplest way to run a site:

#!/bin/env guile
!#
(use-modules (artanis artanis))
(init-server)
(get "/hello" (lambda () "hello world"))
(run)

7.2 Initialization ¶

It’s better to use (init-server) to init GNU Artanis.

(init-server #:statics '(png jpg jpeg ico html js css)
             #:cache-statics? #f #:exclude '())

#:statics specifies the static files with the file name extensions. GNU Artanis is based on URL remapping, so the requested URL will have to end in the requested file name, matching the string defined, and returning the file without any extra defintions per file type. By default, it covers the most common static file types.

#:cache-statics? controls whether the static files should be cached.

#:exclude specifies the types should be excluded. This is useful when you want to generate files dynamically. Even JavaScript/CSS could be generated dynamically, so it depends your design.

7.3 Registering handlers for HTTP methods ¶

Please read URL handling.

7.4 Emit a Response ¶

(response-emit body #:status 200 #:headers '() #:mtime (current-time))

body is the response body, it can be a bytevector or literal string (in HTML).

#:status is the HTTP status, 200 in default, which means OK.

#:headers lets you specify custom HTTP headers. The headers must follow a certain format. Please read Response Headers for details.

#:mtime specifies the modified time in the response. GNU Artanis will generate it for you when not defined.

(emit-response-with-file filename [headers <- '()])

filename is the filename to be sent as a response.

[headers] is the custom HTTP headers.

7.5 Running the server ¶

(run #:host #f #:port #f #:debug #f #:use-db? #f
     #:dbd #f #:db-username #f #:db-passwd #f #:db-name #f)

keywords with the value #f, as default, will get the values from the config file.

But you can define them as well.

#:host the hostname.

#:port the socket port of the server.

#:debug set #t if you want to enable debug mode. Logs will be more verbose.

#:use-db? set #t if you want to use DB, and GNU Artanis will initialize DB connections.

#:dbd choose a dbd. These are the supported three: mysql, postgresql, and sqlite3.

#:db-username the username of your DB server.

#:db-passwd the DB password for the user above.

#:db-name the DB name to use.

7.6 Working with Nginx ¶

You may try GNU Artanis+Nginx with a reverse proxy.

Although GNU Artanis has good server core, I would recommend you use Nginx as the front server. In addition to the enhanced performance, it’ll also be less vulnerable to attacks.

These are some sample lines for /etc/nginx/nginx.conf:

location / {
proxy_pass http://127.0.0.1:1234;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}

To make it work, restart Nginx after editing the file:

sudo service nginx restart

And run GNU Artanis:

(run #:port 1234)

8.1 art create ¶

If you want to set up your site/app in an application directory, and take advantage of the MVC features, you have to use this command to create the application directory first.

art create proj_path

8.2 art draw ¶

This command will generate the specified components:

Usage:
  art draw <component> NAME [options]

component list:
  model
  controller
  migration
  view
  lib

Options:
  -h, [--help]     # Print this screen
  -d, [--dry]      # Dry run but do not make any changes
  -f, [--force]    # Overwrite files that already exist
  -s, [--skip]     # Skip files that already exist
                   # If -s and -f are both provided, -f will be enabled
  -q, [--quiet]    # Suppress status output

Example:
  art draw model myblog

Please see MVC to learn more about how to use these components.

8.3 art api ¶

This command is used to generate RESTful API skeleton.

Usage:
  art api [options]

Options:
  -h, [--help]     # Print this screen
  -d, [--dry]      # Dry run but do not make any changes
  -f, [--force]    # Overwrite files that already exist
  -s, [--skip]     # Skip files that already exist
                   # If -s and -f are both provided, -f will be enabled
  -q, [--quiet]    # Suppress status output
  -l, [--list]     # List all defined APIs
  -m, [--mode]     # Specify the WebAPI mode, the default is "restful"
  -v, [--ver]      # Specify the WebAPI version, omit it to auto detect
  -c, [--create]   # Create WebAPI

Example:
  art api -c
  art api -c -v v3

Please see RESTful API for more details.

8.4 art migrate ¶

Migrate is used for Database migration.

Usage:
  art migrate operator name [OPTIONS]

Operators:
  up
  down

OPTIONS:
  VERSION=version

Please see Migration for more detail.

8.5 art work ¶

This command is used to start the server when run in a project directory:

Usage:
  art work [options]

Options:
  -c, [--config=CONFIG]          # Specify config file
                                   Default: conf/artanis.conf
                                            if no, /etc/artanis/artanis.conf
  -h, [--host=HOST]              # Specify the network host
                                   Default: 0.0.0.0
  -d, [--usedb]                  # Whether to use Database
                                   Default: false
  -b, [--dbd=DBD]                # Specify DBD, mysql/postgresql/sqlit3
                                   Default: mysql
  -n, [--name=DATABASE_NAME]     # Database name
                                   Default: artanis
  -w, [--passwd=PASSWD]          # Database password
                                   Default: none
  -u, [--user=USER]              # Database user name
                                   Default: root
  -p, [--port=PORT]              # Specify listening port
                                   Default: 3000
  -g, [--debug]                  # Debug mode
                                   Default: disable
  -s, [--server=SERVER]          # Specify server core
                                   Default: Ragnarok (New server core since 0.2)
  --refresh                      # Clean caches, and force to re-compile all source code.
  --help                         # Show this screen

• For server core alternatives, please see server.config in config.
• For Database (DBD) alternatives, please see database.config in config.

NOTE: Please make sure use art work --refresh to recompile WebApp code each time you upgrade GNU Artanis. And if you want to clean the caches for debug, --refresh is also your friend.

9.1 Introduction to URL remapping ¶

URL remapping is used to modify a web URL’s appearance to provide short, pretty or fancy, search engine friendly URLs. It’s largely used in modern WAFs(web application framework) to provide RESTful web APIs.

9.2 URL handling ¶

According to RFC2616, the methods include GET, POST, etc. However, because of the naming conflict, GNU Artanis provides the methods by this list:

• get
• post
• put
• patch
• page-delete
• page-options

In GNU Artanis the HEAD method is handled by the server, so you can’t define specific handlers directly for it within GNU Artanis.

Usage:

(method rule handler)

And the handler could be one of two types, depending on your needs:

(lambda ()
  ...
  ret)

(lambda (rc)
  ...
  ret)

ret also has two types:

• 1. literal string as the returned response body
• 2. See Emit a Response

(get "/hello" (lambda () "hello world"))

For a POST method:

(post "/auth" (lambda (rc) ...))

9.3 OPTIONS method ¶

OPTIONS method is used to return the other methods supported by the server-side at the given URL. By default in GNU Artanis, for a specified URL, every method whose handler was registerted would be added to its own OPTIONS list.

If you want to deny certain methods for security consideration, you may need to set server.allowedmethods in config.

NOTE: If you’re not trying to use Cross-Origin Resource Sharing (CORS), then you may not need to care about it.

9.4 Get parameters from a URL ¶

(params rc name)
;; e.g
(get "/hello/:who" (lambda (rc) (params rc "who")))

9.5 Redirecting ¶

(redirect-to rc path #:status 301 #:type '(text/html) #:headers '())
;; e.g
(get "/aaa" (lambda (rc) (redirect-to rc "/bbb")))
(get "/bbb" (lambda () "ok bbb"))

The path could be 2 possible patterns:

• URI created by string->uri. For example, (redirect-to rc (string->uri "https://nalaginrut.com/feed/atom")).
• A relative path which will finally be converted to absolute URL append the host.addr. For example, (redirect-to rc "/login").

NOTE: redirect-to will always use absolute URL implicitly. Although the standard supports relative URL, there maybe some problems when you’re behind a proxy. So we use absolute URL to avoid these issues.

10.1 Route context APIs ¶

(rc-path <route-context>)

• Get the requested path, that is to say, the actual URI visited by the client.

;; e.g
(get "/hello/world" (lambda (rc) (rc-path rc)))
;; visit localhost:3000/hello/world or from any port you specified
;; the result is "/hello/world".
(get "/hello/:who" (lambda (rc) (rc-path rc)))
;; visit localhost:3000/hello/world or from any port you specified
;; the result is "/hello/world".

(rc-req <route-context>)

• Get the current HTTP request wrapped in record-type. About HTTP request please see HTTP Request. It stores HTTP request of Guile.

(rc-body <route-context>)

• Get the current request body:
  • For a regular HTTP request, the body should be a bytevector;
  • For a Websocket request, the body should be Websocket frame as a record-type.

(rc-method <route-context>)

• Get the current requested HTTP method.

(rc-conn <route-context>)

• Get the current DB connection if you’ve requested one, please checkout DB shortcut.

(rc-qt <route-context>)

• Get query table, which is a key-value list parsed from query string.

(rc-handler <route-context>)

• Get the current request handler. The tricky part is that you can only get this handler within this handler unless you can go no where to run rc-handler correctly.
  • It’s on your own risk to use this API. But now that we have powerful first class lambda, you may do some magic. Well, depends on you.

(rc-mtime <route-context>) ; getter
(rc-mtime! <route-context>) ; setter

• You may set it in the handler to return you customized modified time. For static pages, the mtime is set automatically. But sometimes people may want to set it in a dynamic generated page.

(rc-cookie <route-context>)

• The cookies parsed from request header.

(rc-set-cookie! <route-context>)

• Set response cookie from server side. If you want to return cookies to the client, please use it.

There’re other APIs in route-context, but they’re largely used for internals of Artanis, rarely useful for users. So we don’t list them here.

11.1 HTTP status code ¶

HTTP response status codes are the error numbers show the status of a specific HTTP request. The responses are grouped in five classes:

• Informational responses (100–199)
• Successful responses (200–299)
• Redirects (300–399)
• Client errors (400–499)
• Server errors (500–599)

11.2 The default behaviour of status handling ¶

On success, GNU Artanis returns the status code 200.

If there’s any issue prevents GNU Artanis to generate requested result successfully, then it’ll throw the relavent exception somewhere issue happened. And finally it will be caught by the server-core. In theory, the status code other than 200 will trigger a system page generating operation. By default, status pages are put in /etc/artanis/pages, but you may override it in your_app_folder/sys/pages. Each system page named with the status code, say, 404.html. So you may easily customize your preferred status page.

11.3 Dynamic status page generating ¶

Sometimes you may need more complex status page generating. For example, you want to put a random public service advertising when the visitors encountered a 404 (page missing). Then you can do it like this:

(http-status status-code thunk-handler)

• status-code must be an integer between 100~599.
• Thunk implies a function without any argument.

Let’s see an example:

;; You may put this code in any controller module.
(http-status 404
 (lambda ()
   (view-render "psd_404" (the-environment))))

In this example, "psd" stands for public-service-advertising, and "psd_404" tells the view-render to find "psd_404.html" in your_app_folder/sys/pages/psd_404.html. So you can render the HTML template to generate a status page. Of course, there’re a few ways to implement the similar idea. If you don’t want to render template on the backend, you may generate a JSON and let the frontend framework to generate it.

12.1 Controllers/Views ¶

Let’s make a controller named article:

art draw controller article show edit

show and edit are the name of methods for the controller named article.

This will generate both a controller and a view for article:

drawing    controller article
working    Controllers `article.scm'
create     app/controllers/article.scm
working    Views `article'
create     app/views/article/show.html.tpl
create     app/views/article/edit.html.tpl

These three files are generated:

app/controllers/article.scm
app/views/article/show.html.tpl
app/views/article/edit.html.tpl

Based on this, the controller article will have two methods mapped to the URL rule, show and edit. As part of the view component. An HTML template is generated for each method. For the show method for example, the view file show.html.tpl is created. For the controller component, you get a show method handler, as:

(article-define show
  (lambda (rc)
    "<h1>This is article#show</h1><p>Find me in app/views/article/show.html.tpl</p>"
    ;; TODO: add controller method `show'
    ;; uncomment this line if you want to render view from template
    ;; (view-render "show")
    ))

Of course, you’re free to use or not use these templates. If you want to use the view template, just uncomment the last line (view-render "show").

NOTE: The views template generated by MVC will defaultly announce FreeJS. The reason was well explained in The Javascript Trap. It’s optional, you may remove it with your free will, but I put it there in the hope that you can support free software with us.

For more detail about template in Views, please see Layouts and Rendering in GNU Artanis.

12.2 Models ¶

Models contains operations of database.

art create mytest
cd mytest

# Create model
art draw model user
# drawing    model user
# working    Models `user.scm'

# check result
ls app/models/
# user.scm

(import (artanis mvc model))
;; Model user definition of mytest
;; Please add your license header here.
;; This file is generated automatically by GNU Artanis.
(create-artanis-model user) ; DO NOT REMOVE THIS LINE!!!

+-------------------------------------------------------------+
| Field | Type        | Null | Key | Default | Extra          |
|-------+-------------+------+-----+---------+----------------|
| id    | auto        | NO   | PRI | NULL    | auto_increment |
| name  | char-field  | NO   |     | NULL    |                |
| email | char-field  | NO   |     | NULL    |                |
+-------------------------------------------------------------+

(create-artanis-model
 user  ; model name
 (:deps) ; deps list
 (id auto (#:primary-key))
 (name char-field (#:not-null #:maxlen 128))
 (email char-field (#:not-null #:maxlen 128))
 )

(create-artanis-model
 article  ; model name
 (:deps user) ; deps list
 (id auto (#:primary-key))
 (author_id tinyint (#:not-null))
 (title char-field (#:not-null #:maxlen 128))
 (timestamp bigint (#:unsigned #:not-null))
 )

12.3 Model mapping ¶

If you defined a model, for example, an article table:

(create-artanis-model
 article  ; model name
 (:deps user) ; deps list
 (id auto (#:primary-key))
 (author_id tinyint (#:not-null))
 (title char-field (#:not-null #:maxlen 128))
 (timestamp bigint (#:unsigned #:not-null))
 )

There will be a closure named $article in this module, and could be exported. The $ sign is auto added to the name of the table name. We call it model mapping so that you can control this table easily.

($article 'get #:columns (columns ...)
               #:functions '() ; put SQL functions here, e.g, '((count Article.name))
               #:ret 'all ; values: top | all | postive-number
               #:group-by column-name
               #:order-by "column-name ASC | DESC"
               #:condition "" ; (where #:name "nala")
               #:foreach '() ; #:foreach '(city ("sz" "bj" "sh"))
               #:dump #f ; set #t if you want to check the SQL string
               #:mode 'raw ; raw to return all rows; getter to return a getter function)

($article 'set #:key value ...)

;; for example
($article 'set #:title "new title" #:timestamp (current-time)
               #:condition (where #:id 1))

($article 'table-exists?)

($article 'exists? fields ...)

($article 'ci-exists? fields ...)

($article 'schema)

($article 'create fields-definitions ...)

($article 'remove)

($article 'mod definitions ...)

13.1 Query string from GET ¶

The query string would be encoded in the URL on the GET method.

http://example.com/over/there?name=ferret&color=purple

Please notice that URL-remapping supports regex. So you could register a URL rule like this:

(get "/there?"
  (lambda (rc)
    (get-from-qstr rc "name")))

Or it will throw 404 since URL-remapping failed to hit the rule with the query string.

13.2 Query string from POST ¶

The query string would be encoded in the HTTP body on the POST method.

There’s only a slight difference when you pass query string by POST instead of with GET: you don’t need a URL rule using a regex, so the "?" is unnecessary here.

(post "/there" #:from-post 'qstr
      (lambda (rc)
        (:from-post rc 'get "name")))

Please notice that #:from-post 'qstr is necessary when you’re trying to get data from POST. And you should use :from-post to get related data from query-string.

#:from-post <mode>

The mode includes:

• #t or 'qstr: handle query-string for you.
• 'json: returns a parsed json as hashtable.
• 'qstr-safe: similar to ’qstr, but try to eliminate evil HTML entities first.
• 'bv: returns the body as bytevector.
• 'store rest ...: It’s for Upload files.

The reason to design :from-post is for the efficient purpose. Artanis will not try to auto parse POST body as query-string for at least 2 reasons:

• 1. It may not be query-string, maybe json, or uploaded file
• 2. It may be long query-string, and could be deleyed to parse. This is

useful to avoid redundant parsing. However, each time you call :from-post, it will parse the query-string again, the correct way to fetch multiple values is:

(:from-post rc 'get-vals "key1" "key2" "key3")

;; For example:
;; let-values is imported from srfi-11
(let-values (((title sub-title old-passwd new-passwd)
              (:from-post rc 'get-vals "title" "sub-title"
                          "old-passwd" "new-passwd")))
  ..........
  )

BTW, you may get the parsed query-string as an assoc-list as well:

(let ((qstr (:from-post rc 'get)))
  (assoc-ref qstr "key-1"))

14.1 Templating ¶

Templating provides a way to mix programmatic code into HTML.

14.2 Templating for Pythoners ¶

If you’re familiar with Django, which implemented a DSL(Domain Specific Language) to express presentation rather than program logic. Templating in GNU Artanis follows a different philosophy.

Templating in GNU Artanis, is just writing Scheme code in the HTML document. Why? Because of the philosophy of FP(Functional Programming), everything is a function. So, (filesizeformat size) is easy enough to grasp for anyone with scheme experience. It’s just a simple function calling in prefix-notation. There’s no need to implement DSL like size|filesizeformat to increase the complexity of code. Let alone the syntax is very different from Python.

The syntax size | filesizeformat follows postfix-notation, used in stack-based languages, say Forth. Such a language used to delegate another programming paradigm named concatenative programming. It’s very different from the paradigm of Scheme (functional programming), and the paradigm of Python (imperative programming).

The philosophy of GNU Artanis templating is to bring it into correspondence with the paradigm of the language. And reduce unnecessary complexities. KISS.

14.3 Templating for Rubyists ¶

Templating in GNU Artanis looks very similar to Rails.

The Rails code:

<% if( @fullscreen == 1 ) %>
<%= "<div class='full'><p>...</p></div>" %>
<% end %>

And the same function in GNU Artanis code:

<% (if (= fullscreen 1) %>
       <% "<div class='full'><p>...</p></div>" %>
       <% ) %>

14.4 Templating APIs ¶

(tpl->response filename/sxml [environment <- (the-environment)] [escape? <- #f])

(tpl->html filename/sxm [environment <- (the-environment)] [escape? <- #f])

The difference is that tpl->html returns a string, but tpl->response returns an HTTP object response.

[environment] is the environment you want to use. We often ignore it. If you want to ref some vars defined outside your template string, you need to pass this variable.

[escape?] If you want to char-escape the HTML with the returned string, set it to #t.

There are two main ways of writing templates:

14.5 Embedded Templates ¶

Example: Write a tpl file named "my.tpl":

<html>
  <p> <%= "This is tpl test!" %> </p>
  <p> <% (format #t "And this is ~a" (getcwd)) %> </p>
  <p> <%= external-var %> </p>
</html>

The filename extension ".tpl" is mandatory when using the MVC. Since the MVC will find the template by detecting controller name automatically.

If you don’t use the MVC, and are rather writing all in one script file loading GNU Artanis modules. Then you don’t need to follow this rule.

NOTE: Don’t wrap code in double-quotes, for example:

<a href="<%= my-url %>">click me</a> <!-- Wrong! -->
<a href=<%= my-url %> >click me</a> <!-- Correct! -->

If you need to output a double-quoted string, please use object->string to convert in Scheme first.

<a href=<%= (object->string my-url) %> >click me</a> <!-- If my-url is not properly quoted -->

(get "/test"
  (lambda (rc)
    (let ((external-var 123))
      (tpl->response "my.tpl" (the-environment)))))
(run #:port 8080)

In this case, make sure to put my.tpl in the same path as your GNU Artanis code.

Since external-var is defined outside the file "my.tpl", and it’s bound in let with 123, you have to pass (the-environment). Or the template render will complain aobut not being able to find the variable external-var.

If you don’t need to refer to any external vars, just use (tpl->response "file.tpl").

To test, access http://localhost:3000/test in your browser.

<html>
  <head>
    <@icon favicon.ico %>
      <@js functions.js %>
        <@css blog.css %>
  </head>

  <@include sidebar.html %>

    <body>
      ...
    </body>
</html>

;; @include is the command name, not <@ include filename %>
<@include filename.html %>

/current_toplevel/pub/filename.html

<@css filename.css %>

<link rel="stylesheet" href="/css/filename.css">

<@js filename.js %>

<script type="text/javascript" src="/js/filename.js"> </script>

<@icon favicon.ico %>

<link rel="icon" href="/img/favicon.ico" type="image/x-icon">

14.6 SXML Templates ¶

SXML is an alternative syntax for writing XML data, using the form of S-expressions.

SXML is to Scheme as JSON is to ECMAScript(the so-called Javascript). Maybe this explains it clearer.

One benefit of SXML is that it takes advantage of the quasiquote in Scheme. Please search in the internet "scheme quasiquote" for more details.

This is an SXML expression:

(tpl->response '(html (body (p (@ (id "content")) "hello world"))))

The above would result the following HTML code:

<html><body><p id="content">hello world</p></body></html>

Sometimes you may need quasiquote to refer to a variable, for example:

(let ((content "hello world"))
  (tpl->response `(html (body (p (@ (id "content")) ,content)))))

Here, the "html" block is being quoted with the backtick, which in combination with a "," character before the variable name, makes the variable be referred to instead of just passing a string.

15.1 DB init hooks ¶

Sometimes you need to do some configurations before using DB, Artanis provide an API for that, you should put it to ENTRY file before run the server.

For example, assuming you’re using MySQL/MariaDB, and you need to configure it to UTF-8, you should add these lines to you ENTRY file. Or if you’re using minimal mode without application folder, just put it before running the server.

(run-when-DB-init!
 (lambda (conn)
   (DB-query conn "set names utf8;")))

NOTE: Don’t forget ‘!’ here, it implies the side-effects in Scheme!

15.2 DB connection pool ¶

GNU Artanis will create a connection pool when you run art work. Its size is decided by db.poolsize. The pool is increasing by default for development and test. It is strongly recommended you to set db.pool to fixed when you deploy. The increasing pool will cause the effects that is similar to the memory leak. If you set it to fixed, then the request handler will be safely scheduled when there’s no available DB connection, and it will be awake when it’s available.

15.3 Migration ¶

Migrations provide a way to do complicated modification of tables in a database by GNU Artanis. Here’s an example.

First, draw a migration:

# art draw migration person
drawing    migration person
working    Migration `20151107040209_person.scm'

You’ll see something similar like above.

Then you’d edit the file db/migration/20151107040209_person.scm:

(migrate-up
 (create-table
  'person
  '(id auto (#:primary-key))
  '(name char-field (#:not-null #:maxlen 10))
  '(age tiny-integer (#:not-null))
  '(email char-field (#:maxlen 20))))

(migrate-down
 (drop-table 'person))

Then you run the up command for migration:

art migrate up person

Then migrate-up function will be called, and this will create a table named person:

+-------+---------------------+------+-----+---------+----------------+
| Field | Type                | Null | Key | Default | Extra          |
+-------+---------------------+------+-----+---------+----------------+
| id    | bigint(20) unsigned | NO   | PRI | NULL    | auto_increment |
| name  | varchar(10)         | NO   |     | NULL    |                |
| age   | tinyint(4)          | NO   |     | NULL    |                |
| email | varchar(20)         | YES  |     | NULL    |                |
+-------+---------------------+------+-----+---------+----------------+

If you run the down command of migration, as:

art migrate down person

The table person will be dropped.

15.4 ORM problem ¶

ORM stands for Object Relational Mapping, which is a popular approach to handle relational DB nowadays, especially for Object-Oriented Programming.

Of course, Guile has it’s own Object System named GOOPS. Users can use OOP with it. And it’s possible to implement ORM in GNU Artanis as well.

However, FP fans realized that they don’t have to use OOP if they can use FP features reasonably.

Besides, there’re some critics about ORM:

• ORM Hate
• Vietnam of Computer Science
• Object-Relational Mapping is the Vietnam of Computer Science

Here are some known ways to solve ORM related problems:

• 1. Give up ORM.

• 2. Give up relational storage model. Don’t use a relational DB. Use another DB style, such as No-SQL. Well, this way is not cool when you must use a relational DB.

• 3. Manual mapping. Write SQL code directly. It’s fine sometimes. But the code increases when things get complicated. Refactoring and reusing would be worth to consider.

• 4. Limited ORM. Limiting the utility of ORM. And use ORM to solve part of your work rather than whole, depends on you. This may avoid some problems.

• 5. SQL related DSL. Design a new language. Microsoft’s LINQ is such a case.

• 6. Integration of relational concepts into frameworks. Well, harder than 5, but worth to try.

• 7. Stateless. This is the critical hit to counter complexity and unreliability.

Basically, GNU Artanis has no ORM yet, and maybe never will. GNU Artanis is trying to experiment new ways to solve the problems of ORM.

GNU Artanis provides three ways to complete this mission. All of them, are experimental at present.

• SSQL (1,3,5)
• FPRM (4,7)
• SQL Mapping (1,3,6)

15.5 SSQL ¶

The concept of SSQL is to write SQL in s-expression.

Usage:

(->sql sql-statement)
(where #:key val ... [literal string])
(having #:key val ... [literal string])
(/or conds ...)
(/and conds ...)

For example:

(->sql select * from 'Persons (where #:city "Tokyo"))
(->sql select '(age name) from 'Persons (where "age < 30"))

The SQL update command is quite different to SQL grammar. Example:

(->sql update 'table set (list (list phone_number "08080001234")) (where #:name "john"))

15.6 FPRM (experimental) ¶

FPRM stands for Functional Programming Relational Mapping. It’s a new word I invented. But it’s not new concept. FP here indicates stateless.

FPRM is still experimental and work-in-progress.

;; usage 1:
(connect-db dbd init-str)

;; usage 2:
(connect-db dbd #:db-name "artanis" #:db-username "root" #:db-passwd ""
            #:proto "tcp" #:host "localhost" #:port 3306)

(connect-db "mysql" "root:123:artanis:tcp:localhost:3306")

(define m (map-table-from-DB rc/conn))

(m 'create table-name defs #:if-exists? #f #:primary-keys '() #:engine #f)

'((name varchar 10) (age integer) (email varchar 20))

(m 'get table-name #:columns '(*) #:functions '() #:ret 'all
   #:group-by #f #:order-by #f)

#:functions '((count Persons.Lastname))

(m 'get 'Persons #:columns '(Lastname City) #:ret 'top)

(m 'set table-name . kargs)

(m 'set 'Persons #:name "nala" #:age 99 #:email "nala@artanis.com")

(m 'drop table-name)

;; case sensitive
(m 'exists? table-name . columns)
;; or for case-insensitive
(m 'ci-exists? table-name . columns)

(m 'exists? 'Persons 'city 'lastname)

(m 'schema table-name)

15.7 SQL Mapping (experimental) ¶

To be continued …

17.1 JSON ¶

GNU Artanis integrates the third-party module guile-json to parse json. You can use the #:mime method to handle JSON:

(get "/json" #:mime 'json
     (lambda (rc)
       (let ((j (scm->json-string '(("name" . "nala") ("age" . 15)))))
         (:mime rc j))))

For example:

(define my-json
  '((name . nala) (age . 15)
    (read_list
     ("book1" "The interpreter and structure of Artanis")
     ("book2" "The art of Artanis programming"))))
(scm->json-string my-json) ; scm->json will print a json string
;; ==> {"name" : "nala",
;;      "age" : 15,
;;      "read_list" : {"book2" : "The art of Artanis programming",
;;                     "book1" : "The interpreter and structure of Artanis"}}

scm->json will return a hash-table to represent a JSON object.

If you need to format JSON as a string to return to the client, please use scm->json-string.

17.2 CSV ¶

GNU Artanis integrates the third-party module guile-csv to parse csv. You can use the #:mime method to handle CSV:

(get "/csv" #:mime 'csv
     (lambda (rc)
       (:mime rc '(("a" "1") ("b" "2")))))

17.3 XML ¶

In Scheme, XML is handled with SXML. Another way would be to append the text to a common string.

(get "/xml" #:mime 'xml
     (lambda (rc)
       (:mime rc '(*TOP* (WEIGHT (@ (unit "pound"))
                                 (NET (@ (certified "certified")) "67")
                                 (GROSS "95"))))))

The rendered result to the client will be:

<WEIGHT unit="pound">
  <NET certified="certified">67</NET>
  <GROSS>95</GROSS>
</WEIGHT>

17.4 SXML ¶

You can use SXML to replace XML for exchanging data format. This way saves some bandwidth.

(get "/sxml" #:mime 'sxml
     (lambda (rc)
       (:mime rc '((a 1) (b 2)))))

18.1 Receive an upload from the client ¶

The typical configuration of an uploading WebAPI looks like this:

(post "/upload" #:from-post '(store #:path "upload" #:sync #f)
      (lambda (rc)
        (case (:from-post rc 'store)
          ((success) (response-emit "upload succeeded!"))
          ((none) (response-emit "No uploaded files!"))
          (else (response-emit "Impossible! please report bug!")))))

However, you may use the low-level API for more configurations as well:

(store-uploaded-files rc #:path (current-upload-path)
                      #:uid #f
                      #:gid #f
                      #:simple-ret? #t
                      #:mode #o664
                      #:path-mode #o775
                      #:sync #f)

rc is the route-context.

#:path is the specified path to put uploaded files.

#:uid is new UID for the uploaded files, #f uses the default UID.

#:gid specifies the GID.

#:simple-ret? specifies the mode of return:

• if #t, there’re only two possible return value, ’success for success, ’none for nothing has been done.
• if #f, and while it’s successful, it returns a list to show more details: (success size-list filename-list).

#:mode chmod files to mode.

#:path-mode chmod upload path to mode.

#:sync sync while storing files.

18.2 Send an upload to a Server ¶

Although GNU Artanis is often used in server-side, we provide this function for users to upload files from the client.

(upload-files-to uri pattern)

uri is standard HTTP URL:

scheme://[user:password@]domain:port/path?query_string#fragment_id

pattern should be: ((file filelist …) (data datalist …)), for example:

(upload-files-to "ftp://nala:123@myupload.com/"
                 '((data ("data1" "hello world"))
                   (file ("file1" "filename") ("file2" "filename2"))))

19.1 Session backend ¶

Artanis provide several backends for implementing sessions in the lower-level. Please take a look at the description of session.backend in your artanis.conf.

21.1 Init Authentication ¶

GNU Artanis provides flexible mechanism for authentication.

You can use #:auth mode to define a URL rule handler.

(get "/certain-rule" #:auth mode
     (lambda (rc) ...))

mode can be:

• SQL as string template. You can write your own custom SQL string to fetch & check your username and password.
• ('basic (lambda (rc user passwd) ...)) init Basic Authentication mode. user is the username, and passwd is the password.
• ('table table-name username-field passwd-field) init a common Authentication mode. The passwd will be encrypted by the default algorithm.
• ('table table-name username-field passwd-field crypto-proc) similar to the above item, but encrypt passwd with crypto-proc.
• (table-name crypto-proc), so the passwd field will be "passwd" and username will be "username" by default.You can encrypt the passwd with crypto-proc.

Available crypto-proc helper functions listed here:

(string->md5 <string>)
(string->sha-1 <string>)
(string->sha-224 <string>)
(string->sha-256 <string>)
(string->sha-384 <string>)
(string->sha-512 <string>)

NOTE: Please make sure that the username-field and passwd-field must be the same with the field name specifed in the submit form of you web page code.

For example, if there is a form on you page:

<input type="password" name="passwd">

Please notice that name of password input was specified to "passwd".

Then you should write authentication like this:

(post "/auth" #:auth '(table user "user" "passwd") #:session #t
      (lambda (rc) ...))

Please notice that the "passwd" here is the same with what you specified in the form.

21.2 Basic Authentication ¶

The HTTP Basic authentication (BA) implementation is the simplest technique for enforcing access control to web resources, as it doesn’t require cookies, session identifiers, or login pages. But rather uses static, standard HTTP headers, which means that no extra handshakes are necessary for the connection.

The BA mechanism provides no protection for the transmitted credentials. They are merely encoded with Base64, but not encrypted or hashed in any way. For that reason, Basic Authentication is typically used over HTTPS.

GNU Artanis doesn’t support HTTPS at present. There are plans to support it in the future.

Let’s see a simple example:

(define (my-checker rc user passwd)
  (and (string=? user "jack") (string=? passwd "123")))

(post "/bauth" #:auth `(basic ,my-checker)
      (lambda (rc)
        (if (:auth rc)
            "auth ok"
            (throw-auth-needed))))

Another simple way to provide authentication is to compare the passsword stored in a database table:

(post "/bauth" #:auth `(basic Person username passwd)
      (lambda (rc) ... ))

NOTE: Assuming username and passwd are columns of the Person table.

You have to define your own checker with the anonymous function (lambda (rc u p) ...). #t to show success, and #f to fail.

APIs:

• (:auth rc) will check if Basic Authentication succeeded, #f if not.
• (throw-auth-needed) is a useful helper function to ask for auth in client side.

21.3 Common Authentication ¶

There are multiple authentication methods that can be used by developers. Most of them are sort of tricky hacks. Here are the most common.

The most common, and relatively safe way to authenticate, is to use the POST method, and check the username and passwd from a table in the DB.

There are several ways to provide authentication.

The simplest case is for String Template:

#:auth "string-template"

If you save the account data in a database table, then you can use the table mode:

#:auth `(table ,table-name [,username-field] [,passwd-field] [,salt-field] [,hmac])

NOTE: The square-braced [args] above are optional.

The default values for the optional items are:

• username-field: username
• passwd-field: passwd
• salt-field: salt

TODO: remove the brackets for salt-field, as it says below that it’s not optional. And specify whether "optional" means that can be skipped or set to #f to use the default value.

GNU Artanis requires a salted password, it’s not optional.

So please prepare a field in a table for the salt string. It’s your responsibility to generate a salt string, please see Random String Generator. When authenticating, please specify the salt field name in the salt-field argument.

For hmac item, please see HMAC.

21.4 Login Authentication ¶

Usually, when doing a login, you will need both #:auth and #:session options for a long time session. The first step is to authenticate, if it’s successful, then spawn a new session for this request.

Here is a simple example:

(post "/auth"
 #:auth '(table user "user" "passwd")
 #:session #t
 (lambda (rc)
   (cond
    ((:session rc 'check) "auth ok (session)")
    ((:auth rc)
     (:session rc 'spawn)
     "auth ok")
    (else (redirect-to rc "/login?login_failed=true")))))

NOTE: The passwd will be encrypted by the default algorithm.

21.5 Authenticate checking ¶

If certain page requires authencation first, then how to check it properly?

In GNU Artanis, the session-id is the only token to check if the client has already been authenticated. By default, the session-id is named as SID in cookies. You may use #:with-auth to do all the works automatically for you.

(get "/dashboard"
  #:with-auth <options>
  (lambda (rc)
    (view-render "dashboard" (the-environment))))

For example, assuming you have a page /dashboard requires login, then you may set #:with-auth with certain option. We will explain this option later. Each time user visit /dashboard page, GNU Artanis will check if there’s valid session-id from client’s cookies, if yes, then run the handler to generate the response; if no, then jump to the related auth failed handler, depends on the option you specified.

Here’s available options:

• #t means the default failure activity: redirect to /login page.
• URL in string, specify the login page URL. For example, #:with-auth "/admin/login".
• 'status will return a 401 page with status code 401, which could be checked by cilent.
• Thunk can be used to generate customized response by users. A thunk is a function without parameter. For example, #:with-auth (lambda () (scm->json-string '((status . 401) (reason . "No auth")))). This is useful to customize your protocol in JSON for RESTful API.

NOTE: Different from other shortcuts, there’s no :with-auth apply for user customized operations inside the handler. If you use #:with-auth, then all related works are handled by GNU Artanis.

21.6 Customized Authentication ¶

;; (define (checker username passwd) ...)
#:auth `(post ,username ,passwd ,checker)

This mode will parse and return username and password by the specified field name in query-string. What matters here is that you can write your own checker for customization.

21.7 Websocket Authentication ¶

See Websocket authencation

21.8 HMAC ¶

HMAC is a hash-based message authentication code. It’s dangerous to store the passwd in a raw string. A safer way is to salt then hash with a strong cryptograpic hash function when storing the passwd.

The default salt is a random string got from the operating system. And the default cryptographic hash function is SHA256. You can set your own HMAC function, as in this example:

(define (my-hmac passwd salt)
  (string->sha-512 (format #f "~a-~a-~a" passwd salt (current-time))))

(post "/auth" #:auth `(table user "user" "passwd" "salt" ,my-hmac)
      ...... )

The default HMAC function is:

(define (default-hmac passwd salt)
  (string->sha-256 (string-append passwd salt)))

For more on hash functions, please refer to Cryptographic hash functions.

22.1 On web caching ¶

Web caching is very important nowadays. This section discusses proper web caching. It is not a full product guide document, but it may help to understand how to cache data in GNU Artanis.

(to be continued…)

22.2 Cache APIs ¶

You can use #:cache mode to define a URL rule handler.

(get "/certain-rule" #:cache mode
     (lambda (rc) ...))

NOTE: the default value of "maxage" (3600 seconds) is defined by cache.maxage in /etc/artanis/artanis.conf.

mode can be:

• #t to enable caching the page.
• #f to disable caching the page explicitly. It’s the default value.
• ('static [maxage <- 3600]) to be used for static files. The URL rule must be a real path to a static file.
• (filename [maxage <- 3600]) to cache a static file. This is useful when you don’t want to reveal the actual path of the static file, but use a fake URL for it.
• ('public filename [maxage <- 3600]) to allow proxies cache the content of specified static file. If HTTP authentication is required, responses are automatically set to "private".
• ('private filename [maxage <- 3600]) to not allow proxies cache the content of specified static file.

Let’s set a simple cache setting for dynamic content:

(get "/new" #:cache #t
     (lambda (rc)
       (:cache rc "hello world")))

If you want to cache a static file, and permit proxies cache the content:

(get "/hide" #:cache '(public "/some.html")
     (lambda (rc)
       (:cache rc)))

But, if your current URL rule is used for authentication (once you use #:auth), the cache will be changed to private even if you specify public.

(get "/pauth"
  #:auth `(basic ,(lambda (rc u p) (and (string=? u "nala")
                                        (string=? p "123"))))
  #:cache '(public "/some.html") ; will be changed to 'private' forcely.
  (lambda (rc) (:cache rc)))

23.1 What is shortcuts? ¶

shortcuts are a series of special functions. They’re used to simplify the complex operations, according to the configuration specified by the related keyword, set after a URL-rule.

It was named OHT which stands for Optional Handler Table, which indicates the basic principle to be implemented. But it was too hard to remember. So let’s just call it shortcut.

Anyway, you may find them in the module (artanis oht).

It’s good practice to use shortcuts as possible and avoid calling low-level APIs.

Each shortcut consists of 2 parts: config and apply.

config is to configure a certain service for the specific URL rule. This configuration will only be availble to this URL rule, and independent to other registered URL rules.

apply is used to call specific functions related to your configuration in the config step. The first argument of the apply method must be a route-context (route context).

23.2 Database connection ¶

This is a useful feature to use when you connect to a database. The shortcut provides a way to interact with the raw connection. The connection is fetched from the connection pool, which is created at GNU Artanis’s start up.

;; config
#:conn #t

;; apply
(:conn <route-context> [sql])

• The second argument is optional, if it’s missing, then :conn will return the raw connection after applying (:conn rc).
  • NOTE: If you haven’t set #:conn #t, and applied (:conn rc), then (rc-conn rc) will return #f. This is why you shouldn’t use low-level (rc-conn rc).
• If you pass a second argument, it should be a valid SQL query string. The returned value is described in DB connection pool.
  • You can create the SQL query string with SSQL.

23.3 Raw SQL ¶

This shortcut is useful for a simple one-shot query.

;; config
#:raw-sql sql

;; apply
(:raw-sql <route-context> mode)

Sql must be a valid SQL query string.

Mode is one of:

• ’all for getting all the results.
• ’top for getting the first result.
• A positive integer to indicate how many results should be returned.

23.4 String template ¶

This is a shortcut for string template. Sometimes it’s useful when you just need a quick way to use a string template. It doesn’t support multi templates, so if you do need to, please use the traditional String Template.

;; config
#:str "string template"

;; apply
(:str <route-context> key-values ...)

Please checkout string template to find out how to use the string-template and key-values.

23.5 SQL-Mapping shortcut (unfinished) ¶

This is related to SQL-Mapping, which is still experimental, maybe you should wait for the next version if you wish to use it.

;; config
#:sql-mapping config-patterns

;; apply
(:sql-mapping <route-context> command ...)

config-patterns can be any of:

• #t enable the simple sql-mapping.
• `(path ,path ,name) Fetch the sql-mapping with name in the specified path.
  • name must be an object of the symbol type.
  • path must be an object of the string type, and an existing path in your filesystem.
• `(add ,name ,sql-template) Fetch the sql-mapping with name rendered from sql-template.
  • name should be an object of the symbol type.
  • sql-template is described in more detail in SQL-Mapping.

23.6 Authentication shortcut ¶

See Authentication.

24.1 Websocket introduction ¶

Websockets are becoming more and more important for modern web development. GNU Artanis is trying to provide an industrial strength and efficient Websocket implementation. Websockets are important for GNU Artanis’s design. Please see Principles for more details.

24.2 Websocket basic usage ¶

In GNU Artanis, a Websocket handling is triggered by setting it on a specific URL. You should use #:websocket to configure the Websocket. Here’s the API definition:

#:websocket (proto protocol-name ['inexclusive])

’inexclusive is optional. By default, each Websocket connection only servs one client. However, sometimes we do need to multicast the messages to several clients. Enable ’inexclusive is the easiest way, but it may cause the similar issue to file-description-leaking. There’re two suggestions:

• Set and check security token to prevent malicious programs establish massive connections.
• If you want to do muticasting as in pub/sub, then use different token or named-pipe to subscribe. And maintain a subscribers table for multicasting.

Here’s a simple example for common cases:

(use-modules (artanis artanis))

(get "/echo" #:websocket '(proto echo)
     (lambda (rc)
       (:websocket rc 'payload)))

(run #:port 3000)

In this simple test, we choose the simplest echo protocol of the Websocket. This will return back the string sent from the client. Let’s also write a simple javascript function for the web frontend:

function WebSocketTest()
{
    if ("WebSocket" in window)
    {
        document.write("<p>WebSocket is supported by your Browser!</p>");

        // Let us open a web socket
        var ws = new WebSocket("ws://localhost:3000/echo");

        ws.onopen = function()
        {
            // Web Socket is connected, send data using send()
            ws.send("hello world");
            document.write("<p>Message is sent...</p>");
        };

        ws.onmessage = function (evt)
        {
            var received_msg = evt.data;
            document.write("<p>hello welcome...</p>");
        };

        ws.onclose = function()
        {
            // websocket is closed.
            document.write("<p>Connection is closed...</p>");
        };

        window.onbeforeunload = function(event) {
            socket.close();
        };
    }
    else
    {
        // Your browser doesn't support WebSockets
        document.write("<p>WebSocket NOT supported by your Browser!</p>");
    }
}

24.3 Websocket named-pipe ¶

Artanis provides the named-pipe based on Websocket, which is very useful to interact between the client and server. NOTE: The Websocket named-pipe is very useful to implement server-push messaging.

Here is the critical API:

(named-pipe-subscribe <route-context>)

Let’s try a simple example:

(get "/robot" #:websocket '(proto echo) named-pipe-subscribe)

Here we register a WebAPI named "/robot", and configure it a Websocket API by #:websocket '(proto echo). As you may see, we specify its protocol to a simple echo protocal, it is to say, it will redirect what it receives. Each time it receives a message, it will redirect the message to the specified named-pipe, and the subscriber of that named-pipe will get the message instantly. The name of the pipe is specified by the client like this:

var ws = new WebSocket("ws://localhost:3000/robot?artanis_named_pipe=robot-1");

The query-string "artanis_named_pipe" is a special key in Artanis, you MUST use this key to specify the pipe name. Here we specify the pipe name as "robot-1".

Now we setup the named-pipe, however, we still need to setup another WebAPI to let the client send message:

(get "/welcome/:whom/:what" #:websocket 'send-only
     (lambda (rc)
       (:websocket rc 'send (params rc "whom") (params rc "what"))
       "ok"))

The configure "#:websocket 'send-only" means this API is only for sending, so that it’s half-duplex. And another critical Websocket API here:

(:websocket <route-context> 'send <pipe-name> <data>)

Each time you send data from the WebAPI, for example:

curl localhost:3000/welcome/robot-1/nala

In our simple example of Websocket named-pipe test, you should run The named-pipe will receive the message "nala", and the page on the browser will print "hello welcome...nala" instantly.

24.4 Websocket APIs ¶

NOTE: The Websocket is in a very preliminary stage. It only support echo.

#:websocket `(proto ,protocol_name)

(:websocket <route-context> command)

24.5 Websocket frame ¶

GNU Artanis provides a Websocket data frame struct, as defined in RFC6455.

The frame will not be decoded or parsed into a record-type, but will be kept as the binary frame read from the client, and use bitwise operations to fetch fields. This kind of ‘lazy’ design saves much time as it doesn’t parse unused fields, and makes it easier to redirect without any serialization. If users want to get a certain field, Artanis provides APIs for fetching them. Users can decide how to parse the frames by themselves, which we think is more efficient.

Here are the APIs you can use:

(websocket-frame? <websocket-frame>)

;; parser: bytevector -> custom data frame
(websocket-frame-parser <websocket-frame>)

websocket-frame-parser is the registered reader for the protocol specified by the #:websocket configuration. The protocol is customizable based on protobuf. NOTE: Custom protocols support hasn’t been implemented yet.

(websocket-frame-head <websocket-frame>)
(websocket-frame-final-fragment? <websocket-frame>)
(websocket-frame-opcode <websocket-frame>)
(websocket-frame-payload <websocket-frame>)
(websocket-frame-mask <websocket-frame>)

To get the Websocket frame information. See Data framing for details.

• head returns the first 2 bytes in the data frame.
• final-fragment returns true if it’s the last frame in a session.
• opcode returns the opcode in the frame. Refer to Websocket opcode.
• payload returns the actual encoded data.
• mask returns the frame mask.

24.6 Websocket opcode ¶

Opcode defines the interpretation of "Payload data". If an unknown opcode is received, the receiving endpoint’s WebSocket connection will fail.

;;  check if it's a continuation frame
(is-continue-frame? opcode)

;;  check if it's text frame
(is-text-frame? opcode)

;; check if it's binary frame
(is-binary-frame? opcode)

;; check if it's control frame
(is-control-frame? opcode)
(is-non-control-frame? opcode)

;;  websocket requires closing
(is-close-frame? opcode)

;;  check if it's a ping frame
(is-ping-frame? opcode)

;;  check if it's a pong frame
(is-pong-frame? opcode)

;;  %xB-F are reserved for further control frames
(is-reserved-frame? opcode)

24.7 Websocket authentication ¶

The Websocket authentication hasn’t been unsupported yet.

25.1 Ragnarok introduction ¶

Since version 0.2, GNU Artanis has started to use a strong server core for high concurrency. Its name is Ragnarok. In the philosophy of the design of GNU Artanis, everything is meant to be flexible and customizable. The server core is no exception. In case Ragnarok doesn’t suit your needs, you’re free to use something else.

Ragnarok doesn’t use any popular library for handling events (libev/libuv etc …). It’s a brand new server core based on epoll and delimited continuations.

25.2 Principles ¶

A basic characteristic of Ragnarok is the use of co-routines. These co-routines are implemented with delimited continuations. There are no OS/kernel controlled threads, like pthread, for scheduling request-handlers in Ragnarok. All the tasks are scheduled by a userland scheduler. And the task is nothing but just a special continuation. The key difference between this and a regular full-stack continuation, is that you can set limits with precision, instead of having to capture the whole stack.

For researchers, there is a paper published on ICFP Scheme Workshop 2016 conference to explain the principle and the design of GNU Artanis:

Multi-purpose web framework design based on websockets over HTTP Gateway.

(to be continued …)

25.3 Features ¶

In Artanis, the request handling can be scheduled even when the socket buffer is full (depends on server.bufsize). And let other handlers deal with the requests. Just like the scheduling of an OS, but in userland.

If you have issues with the buffer when scheduling, there’s no way to flush before it breaks, since we can’t tell if the scheduling is caused by the buffering or the blocking.

Ragnarok takes advantage of SO_REUSEPORT introduced in GNU/Linux 3.9 to provide a feature named server.multi which can be enabled in the config. This feature allows users to start several Artanis instances listening to the same port, to take advantage of multi core CPUs, and the Linux Kernel managing the events.

(to be continued …)

25.4 Ragnarok APIs ¶

You can use these APIs to customize your own server core.

(to be continued …)

26.1 LPC ¶

LPC stands for Lightweight Persistent Cache. It’s the easiest way to use key-value DB in Artanis.

For example:

(get "/certain_rule" #:lpc <backend> (lambda (rc) ...))

The backend includes:

• #t or 'redis
• 'json

After configured, it’s easy to use it:

;; Setter
(:lpc rc 'set <key> <val>)

;; Getter
(:lpc rc 'get <key>)
;; or
(:lpc rc 'ref <key>)

In default, the key will be prefixed automatically:

(string-append "__artanis_lpc_" (get-conf '(db name)) "_" key)

27.1 String Template ¶

GNU Artanis provides Python3-like template strings:

(make-string-template tpl . vals)

• tpl stands for template string.
• vals is varg-list specifying default value to certain key.

For example:

(define st (make-string-template "hello ${name}"))
(st #:name "nala")
;; ==> "hello nala"

;; or you may specify a default value for ${name}
(define st (make-string-template "hello ${name}" #:name "unknown"))
(st)
;; ==> "hello unknown"
(st #:name "john")
;; ==> "hello john"

27.2 Random String Generator ¶

Get random string from /dev/urandom.

(get-random-from-dev #:length 8 #:uppercase #f)

27.3 Cryptographic hash functions ¶

;; hash a string with MD5
(string->md5 str)
;; hash a string with SHA-1
(string->sha-1 str)

SHA-2 hash functions are also supported from Artanis-0.2.5.

(string->sha-224 str)
(string->sha-384 str)
(string->sha-512 str)

27.4 Stack & Queue ¶

GNU Artanis provides simple interfaces for stack & queue:

;; stack operations
(new-stack)
(stack-pop! stk)
(stack-push! stk elem)
(stack-top stk)
(stack-remove! stk key)
(stack-empty? stk)

;; queue operations
(new-queue)
(queue-out! q)
(queue-in! q elem)
(queue-head q)
(queue-tail q)
(queue-remove! q key)
(queue-empty? q)

27.5 Useful string operation ¶

If you want to get all the contents from a file into a string, then don’t use get-string-all imported from rnrs. Because it will not detect the correct charset from the locale, and this may cause the length differ from the actual value. Although GNU Artanis can handle this length issue properly, you should use get-string-all-with-detected-charset when you need to do something like this. If you don’t care about the contents encoding but just want to get the them, it’s better to use get-bytevector-all imported from rnrs.

(get-string-all-with-detected-charset filename)

27.6 Time operation tool ¶

TODO