Skip to content
Snippets Groups Projects
Manual.rst 280 KiB
Newer Older
.. ==================================================
.. Header hierachy
.. ==
..  --
..   ^^
..    ''
..     ;;
..      ,,
..
.. --------------------------------------------------
.. Best Practice T3 reST  https://docs.typo3.org/typo3cms/drafts/github/xperseguers/RstPrimer/
.. External Links: `Bootstrap <http://getbootstrap.com/>`_:
.. Add Images: https://wiki.typo3.org/ReST_Syntax#Images
..
.. -*- coding: utf-8 -*- with BOM.


.. include:: Includes.txt

.. _general:

General
=======

* Project homepage: https://git.math.uzh.ch/typo3/qfq
* Latest relases: https://w3.math.uzh.ch/qfq/

The following features are only tested on linux hosts:

* HTML to PDF conversion - command `wkhtmltopdf`.
* Concatenation of PDF files - command `pdftk`.
* Mime type detection for uploads - command `file`.

Preparation
-----------

Report & Form
^^^^^^^^^^^^^

In PHP 5.x the QFQ extension needs  the PHP MySQL native driver. The following functions are used and are only available with the
native driver (see also: http://dev.mysql.com/downloads/connector/php-mysqlnd/):

* mysqli::get_result (important),
* mysqli::fetch_all (nice to use)

To normalize UTF8 input, the *php5-intl* resp. *php7.0-intl* package is needed by

* normalizer::normalize()

For the `download`_ function, the programs `pdftk` and `file` are necessary to concatenate PDF files.
	sudo apt-get install php5-mysqlnd php5-intl
	sudo apt-get install pdftk file                # for file upload and PDF
	sudo php5enmod mysqlnd
	sudo service apache2 restart

Preparation steps for Ubuntu 16.04::

	sudo apt install php7.0-intl
	sudo apt install pdftk libxrender1 file        # for file upload, PDF and 'HTML to PDF' (wkhtmltopdf)
`wkhtmltopdf <http://wkhtmltopdf.org/>`_ will be used by QFQ to offer 'website print' and 'HTML to PDF' conversion.
The program is not included in QFQ and has to be manually installed.
* The Ubuntu package `wkhtmltopdf` needs a running Xserver - this does not work on a headless webserver. Best is to
   install the QT version from the named website above.
In `config-qfq-ini`_ specify the:
* installed `wkhtmltopdf` binary:

  * `WKHTMLTOPDF = /.../wkhtmltopdf`

* the site base URL:

  * `BASE_URL_PRINT = http://example.com/`

**Important**: To access FE_GROUP protected pages or content, it's necessary to disable the `[FE][lockIP]` check! `wkhtml`
will access the Typo3 page locally (localhost) and that IP address is different from the client (=user) IP.

Configure via Typo3 Installtool `All configuration > $TYPO3_CONF_VARS['FE']`: ::
**Warning**: this disables an important anti-'session hijacking' protection. The security level of the whole installation
will be *lowered*! Again, this is only needed if `wkhtml` needs access to FE_GROUP protected pages & content. As an
alternative to lower the security level, create a separated page subtree which is only accessible (configured via
Typosript) from specific IPs **or** if a FE-User is logged in.

HTML to PDF conversion
''''''''''''''''''''''

`wkhtmltopdf` converts a website (local or remote) to a (multi)-page PDF file. It's mainly used in `download`_.

Print
'''''

Different browser prints the same page in different variations. To prevent this, QFQ implements a small PHP wrapper
`print.php` with uses `wkhtmltopdf` to convert HTML to PDF.
Provide a `print this page`-link (replace {current pageId})::

	<a href="typo3conf/ext/qfq/qfq/api/print.php?id={current pageId}">Print this page</a>

Any parameter specified after `print.php` will be delivered to `wkhtmltopdf` as part of the URL.

Typoscript code to implement a print link on every page::

	10 = TEXT
	10 {
		wrap = <a href="typo3conf/ext/qfq/qfq/api/print.php?id=...|&type=2"><span class="glyphicon glyphicon-print" aria-hidden="true"></span> Printview</a>
		data = page:uid
	}

Setup
-----

* Install the extension via the Extensionmanager.

  * If you install the extension by manual download/upload and get an error message
    "can't activate extension": rename the downloaded zip file to `qfq.zip` or `qfq_<version>.zip` (e.g. version: 0.9.1).

  * If the Extensionmanager stops after importing: check your memory limit in php.ini.

* Enable the online local-documentation_.
* Copy/rename the file *<Documentroot>/typo3conf/ext/<ext_dir>/config.example.qfq.ini* to
  *<Documentroot>/typo3conf/config.qfq.ini* and configure the necessary values: `config.qfq.ini`_
  The configuration file is outside the extension directory to not loose it during updates.
* When the QFQ Extension is called the first time on the Typo3 Frontend, the file *<ext_dir>/qfq/sql/formEditor.sql* will
  played and fills the database with the *FormEditor* records. This also happens automatically after each software update of QFQ.
* Configure Typoscript to include Bootstrap, jQuery, QFQ javascript and CSS files.

::

	page.meta {
	  X-UA-Compatible = IE=edge
	  X-UA-Compatible.attribute = http-equiv
	  viewport=width=device-width, initial-scale=1
	}

	page.includeCSS {

		file1 = typo3conf/ext/qfq/Resources/Public/Css/bootstrap.min.css
		file2 = typo3conf/ext/qfq/Resources/Public/Css/bootstrap-theme.min.css
		file3 = typo3conf/ext/qfq/Resources/Public/Css/jqx.base.css
		file4 = typo3conf/ext/qfq/Resources/Public/Css/jqx.bootstrap.css
		file5 = typo3conf/ext/qfq/Resources/Public/Css/qfq-bs.css
	}

	page.includeJS {

		file1 = typo3conf/ext/qfq/Resources/Public/JavaScript/jquery.min.js
		file2 = typo3conf/ext/qfq/Resources/Public/JavaScript/bootstrap.min.js
		file3 = typo3conf/ext/qfq/Resources/Public/JavaScript/validator.min.js
		file4 = typo3conf/ext/qfq/Resources/Public/JavaScript/jqx-all.js
		file5 = typo3conf/ext/qfq/Resources/Public/JavaScript/globalize.js
		file6 = typo3conf/ext/qfq/Resources/Public/JavaScript/tinymce.min.js
		file7 = typo3conf/ext/qfq/Resources/Public/JavaScript/EventEmitter.min.js
		file8 = typo3conf/ext/qfq/Resources/Public/JavaScript/qfq.min.js
	}

Setup a *report* to manage all *forms*:

* Create a Typo3 page.
* Set the 'URL Alias' to `form` (default) or the individual defined value in parameter EDIT_FORM_PAGE (config.qfq.ini).
* Insert a content record of type *qfq*.
* In the bodytext insert the following code:

::

	# If there is a form given by SIP: show
	form={{form:S}}

	10 {
		# List of Forms: Do not show this list of forms if there is a form given by SIP.
		# Table header.
		sql = SELECT CONCAT('{{pageId:T}}&form=Form&') as _Pagen, '#', 'Name', 'Title', 'Table', '' FROM (SELECT 1) AS fake WHERE  '{{form:SE}}'=''
		head = <table class="table table-hover qfq-table-50">
		tail = </table>
		rbeg = <thead><tr>
		rend = </tr></thead>
		fbeg = <th>
		fend = </th>

		10 {
			# All forms
			sql = SELECT CONCAT('{{pageId:T}}&form=Form&r=', f.id) as _Pagee, f.id, f.name, f.title, f.tableName, CONCAT('form=Form&r=', f.id) as _Paged FROM Form AS f  ORDER BY f.name
			rbeg = <tr>
			rend = </tr>
			fbeg = <td>
			fend = </td>
		}
	}

.. _config-qfq-ini:

config.qfq.ini
--------------

+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| Keyword                     | Example                                         | Description                                                                |
+=============================+=================================================+============================================================================+
| DB_USER                     | DB_USER=qfqUser                                 | Credentials configured in MySQL                                            |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| DB_PASSWORD                 | DB_PASSWORD=12345678                            | Credentials configured in MySQL                                            |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| DB_SERVER                   | DB_SERVER=localhost                             | Hostname of MySQL Server                                                   |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| DB_NAME                     | DB_NAME=qfq_db                                  | Database name                                                              |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| DB_NAME_TEST                | DB_NAME_TEST=qfq_db_test                        | Used during development of QFQ                                             |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| DB_INIT                     | DB_INIT=set names utf8                          | Global init for using the database.                                        |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| SQL_LOG                     | SQL_LOG=sql.log                                 | Filename to log SQL commands: relative to <ext_dir> or absolute.           |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| SQL_LOG_MODE                | SQL_LOG_MODE=modify                             | *all*: every statement will be logged - this is a lot                      |
|                             |                                                 | *modify*: log only statements who change data                              |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| SHOW_DEBUG_INFO             | SHOW_DEBUG_INFO=auto                            | Possible values: auto|yes|no. For 'auto': If a BE User is logged in,       |
|                             |                                                 | debug information will be shown on the fronend.                            |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| REDIRECT_ALL_MAIL_TO        | REDIRECT_ALL_MAIL_TO=john@doe.com               | If set, redirect all QFQ generated mails (Form, Report) to the specified.  |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| CSS_LINK_CLASS_INTERNA    L | CSS_LINK_CLASS_INTERNAL=internal                | CSS class name of links which points to internal tagets                    |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| CSS_LINK_CLASS_EXTERNAL     | CSS_LINK_CLASS_EXTERNAL=external                | CSS class name of links which points to internal tagets                    |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| CSS_CLASS_QFQ_CONTAINER     |CSS_CLASS_QFQ_CONTAINER=container                | QFQ with own Bootstrap: 'container'.                                       |
|                             |                                                 | QFQ already nested in Bootstrap of mainpage: <empty>                       |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| CSS_CLASS_QFQ_FORM_PILL     |CSS_CLASS_QFQ_FORM_PILL=qfq-color-grey-1         | Wrap around title bar for pills: CSS Class, typically a background color   |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| CSS_CLASS_QFQ_FORM_BODY     |CSS_CLASS_QFQ_FORM_BODY=qfq-color-grey-2         | Wrap around formelements: CSS Class, typically a background color          |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| DATE_FORMAT                 | DATE_FORMAT= yyyy-mm-dd                         | Possible options: yyyy-mm-dd, dd.mm.yyyy                                   |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| FORM_DATA_PATTERN_ERROR     |FORM_DATA_PATTERN_ERROR=please check pa.         | Customizable error message used in validator.js. 'pattern' violation       |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| FORM_DATA_REQUIRED_ERROR    |FORM_DATA_REQUIRED_ERROR=missing value           | Customizable error message used in validator.js. 'required' fields         |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| FORM_DATA_MATCH_ERROR       |FORM_DATA_MATCH_ERROR=type error                 | Customizable error message used in validator.js. 'match' retype mismatch   |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| FORM_DATA_ERROR             |FORM_DATA_ERROR=generic error                    | Customizable error message used in validator.js. 'no specific' given       |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| FORM_BS_COLUMNS             | FORM_BS_COLUMNS=12                              | The whole form will be wrapped in 'col-md-??'. Default is 12 for 100%      |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| FORM_BS_LABEL_COLUMNS       | FORM_BS_LABEL_COLUMNS = 3                       | Default number of BS columns for the 'label'-column                        |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| FORM_BS_INPUT_COLUMNS       | FORM_BS_INPUT_COLUMNS = 6                       | Default number of BS columns for the 'input'-column                        |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| FORM_BS_NOTE_COLUMNS        | FORM_BS_NOTE_COLUMNS = 3                        | Default number of BS columns for the 'note'-column                         |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| FORM_BUTTON_ON_CHANGE_CLASS | FORM_BUTTON_ON_CHANGE_CLASS=alert-info btn-info | Color for save button after modification                                   |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| BASE_URL_PRINT              | BASE_URL_PRINT=http://example.com               | URL where wkhtmltopdf will fetch the HTML (no parameter, those comes later)|
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| WKHTMLTOPDF                 | WKHTMLTOPDF=/usr/bin/wkhtmltopdf                | Binary where to find wkhtmltopdf.                                          |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| EDIT_FORM_PAGE              | EDIT_FORM_PAGE = form                           | T3 Pagealias to edit a form.                                               |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| LDAP_1_RDN                  | LDAP_1_RDN='ou=Admin,ou=example,dc=com'         | Credentials for non-anonymous LDAP access. At the moment only one set of   |
+-----------------------------+-------------------------------------------------+ crendentials is supported.                                                 |
| LDAP_1_PASSWORD             | LDAP_1_PASSWORD=mySecurePassword                |                                                                            |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| ESCAPE_TYPE_DEFAULT         | ESCAPE_TYPE_DEFAULT=s                           | All variables `{{...}}` get this escape class by default                   |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| SECURITY_VARS_HONEYPOT      | SECURITY_VARS_HONEYPOT = email,username,password| If empty: no check. All named variables will rendered as INPUT elements    |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| SECURITY_ATTACK_DELAY       | SECURITY_ATTACK_DELAY = 5                       | If an attack is detected, sleep 'x' seconds and exit PHP process           |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| SECURITY_SHOW_MESSAGE       | SECURITY_SHOW_MESSAGE = true                    | If an attack is detected, show a message                                   |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+
| SECURITY_GET_MAX_LENGTH     | SECURITY_GET_MAX_LENGTH = 32                    | Check that there are no GET vars longer than 'x' chars                     |
+-----------------------------+-------------------------------------------------+----------------------------------------------------------------------------+

Example: *typo3conf/config.qfq.ini*

::

	; To get internal default values, inactivate the option by commenting (= ';') it.
	DB_USER = qfqUser
	DB_SERVER = localhost
	DB_PASSWORD = 12345678
	DB_NAME = qfq_db
	DB_INIT = set names utf8
	SQL_LOG = sql.log
	SHOW_DEBUG_INFO = auto
	CSS_LINK_CLASS_INTERNAL = internal
	CSS_LINK_CLASS_EXT = external
	;CSS_CLASS_QFQ_CONTAINER =
	;CSS_CLASS_QFQ_FORM =
	CSS_CLASS_QFQ_FORM_PILL = qfq-color-grey-1
	CSS_CLASS_QFQ_FORM_BODY = qfq-color-grey-2
	;FORM_DATA_PATTERN_ERROR =
	;FORM_DATA_REQUIRED_ERROR =
	;FORM_DATA_MATCH_ERROR =
	;FORM_DATA_ERROR =
	;FORM_BS_LABEL_COLUMNS = 3
	;FORM_BS_INPUT_COLUMNS = 6
	;FORM_BS_NOTE_COLUMNS = 3
	BASE_URL_PRINT=http://example.com
	WKHTMLTOPDF=/usr/bin/wkhtmltopdf
	;LDAP_1_RDN='ou=Admin,dc=example,dc=com'
	;LDAP_1_PASSWORD=mySecurePassword
	;SECURITY_VARS_HONEYPOT=email,username,password
	;SECURITY_ATTACK_DELAY=5
	;SECURITY_SHOW_MESSAGE=true
	;SECURITY_GET_MAX_LENGTH=32

.. _local-documentation:

Local Documentation
-------------------

To render the QFQ reST documentation:

* Take care to have 'unzip' and 'Python setuptools' installed (necessary to run).

Preparation for Ubuntu 16.04::

	sudo apt install unzip python-setuptools python-pip

* Install the extension "Sphinx Python Documentation Generator and Viewer" (sphinx).

  * Execute the update script (symbol 'two arrows as a circle' behind the extension name)
  * Choose 'Sphinx 1.4.4' - click on 'Import'.

* In the Exension Manager open the configuration dialog of the extension 'sphinx'. Activate the 'Sphinx 1.4.4' option and save it.
* On top of the browser window click on the 'question mark' to open the menu, choose 'Sphinx'.
* Show doumentation 'QFQ Extension'

* If you have problems with the rendering, please check: http://mbless.de/blog/2015/01/26/sphinx-doc-installation-steps.html

.. _concept:

Concept
=======

SIPs
----

The following is a technical background information. Not needed to just use QFQ.

The SIPs are uniq timestamps, created/registered on the fly for a specific browser session (=user). Every SIP is
registered on the server (= stored in a PHP Session) and contains one or more key/value pairs. The key/value pairs never leave
the server. The SIPs will be used:

* to protect values not to be spoofed by anyone,
* to protect values not to be altered by anyone,
* to grant access, e.g.:

  * load and save forms,
  * upload files,
  * download files,
  * PHP AJAX pages.

SIPs becomes invalid, as soon as the current browser session is destroyed. The client (= user) can't manipulate the content
of SIPs - it's only possible to reuse already registered SIPs by the user, who already owns the session.

Access privileges
-----------------

The Typo3 FE Groups can be used to implement access privileges. Such groups are assigned to
* Typo3 FE users,
* Typo3 pages,
* and/or Typo3 content records (e.g. QFQ records).

This will be used for general page structure privileges.

A `record base` privileges controlling (e.g. which user can edit
which person record) will be implicit configured, by the way that records are viewable / editable (or not) through
SQL in the specific QFQ tt-content statements.

Typo3 QFQ content element
-------------------------

Insert one or more QFQ content elements on a Typo3 page. Specify column and language per content record as wished.

The title of the QFQ content element will not be rendered. It's only visible in the backend for orientation.

QFQ Keywords (Bodytext)
^^^^^^^^^^^^^^^^^^^^^^^

 +-------------------+---------------------------------------------------------------------------------+
 | Name              | Explanation                                                                     |
 +===================+=================================================================================+
 | form              | Formname defined in ttcontent record bodytext                                   |
 |                   | - Fix. E.g.: **form = person**                                                  |
 |                   | - by SIP: **form = {{form}}**                                                   |
 |                   | - by SQL: **form = {{SELECT c.form FROM conference AS c WHERE c.id={{a:C}} }}** |
 +-------------------+---------------------------------------------------------------------------------+
 | r                 | <record id> The form will load the record with the specified id                 |
 |                   | - Variants: **r = 123**, by SQL: **r = {{SELECT ...}}**                         |
 |                   | - If not specified, the default is '0'                                          |
 +-------------------+---------------------------------------------------------------------------------+
 | <level>.db        | Select a DB. Only necessary if a different than the standard DB should be used. |
 +-------------------+---------------------------------------------------------------------------------+
 | <level>.fbeg      | Start token for every field (=column)                                           |
 +-------------------+---------------------------------------------------------------------------------+
 | <level>.fend      | End token for every field (=column)                                             |
 +-------------------+---------------------------------------------------------------------------------+
 | <level>.head      | Start token for whole <level>                                                   |
 +-------------------+---------------------------------------------------------------------------------+
 | <level>.tail      | End token for whole <level>                                                     |
 +-------------------+---------------------------------------------------------------------------------+
 | <level>.rbeg      | Start token for row.                                                            |
 +-------------------+---------------------------------------------------------------------------------+
 | <level>.rbgd      | Alternating (per row) token                                                     |
 +-------------------+---------------------------------------------------------------------------------+
 | <level>.rend      | End token for row. Will be rendered **before** subsequent levels are processed  |
 +-------------------+---------------------------------------------------------------------------------+
 | <level>.renr      | End token for row. Will be rendered **after** subsequent levels are processed   |
 +-------------------+---------------------------------------------------------------------------------+
 | <level>.rsep      | Seperator token between rows                                                    |
 +-------------------+---------------------------------------------------------------------------------+
 | <level>.fsep      | Seperator token between fields (=columns)                                       |
 +-------------------+---------------------------------------------------------------------------------+
 | <level>.sql       | SQL Query                                                                       |
 +-------------------+---------------------------------------------------------------------------------+
 | <level>.althead   | If <level>.sql is empty, these token will be rendered                           |
 +-------------------+---------------------------------------------------------------------------------+
 | debugShowBodyText | If ='1' and config.qfq.ini:*showDebugInfo=yes* - shows a tooltip with bodytext  |
 +-------------------+---------------------------------------------------------------------------------+

.. _debug:

Debug
^^^^^


  * *yes*:

    * Form:

      * For every internal link/button, show tooltips with decoded SIP on mouseover.
      * Shows an 'Edit form'-button (wrench symbol) on a form. The link points to the T3 page with the :ref:`form-editor`.

    * Report: Will be configured per tt-content record.

      *debugShowBodyText = 1*

  * *no*: No debug info.

  * *auto*: Depending if there is a Typo3 BE session, set internally:

    * *SHOW_DEBUG_INFO = yes*  (BE session exist)
    * *SHOW_DEBUG_INFO = no*   (no BE session)

.. _REDIRECT_ALL_MAIL_TO:

* *REDIRECT_ALL_MAIL_TO=john@doe.com*

  * During the development, it might be helpful to configure a 'catch all' email address, which QFQ uses as the final receiver
    instead of the original intended one.

  * The setting will:

    * Replace the 'To' with the configured one.
    * Clear 'CC' and 'Bcc'
    * Write a note and the original configured receiver at the top of the email body.

.. _variables:

Variables
---------

Most fields of a form or report specification might contain:

* ''constants'' (=strings), this is the standard use case.
* ''variables'' retrieved from the stores (see below),
* ''SQL statements'' (limited set of),
* or any combination of the above.

* A variable (or SQL) statement is surrounded by curly braces:

  *{{VarName[:<store / prio>[:<sanitize class>[:<escape>]]]}}*

* Example:

  *{{r}}*

  *{{index:FS}}*

  *{{name:FS:alnumx:s}}*

  *{{SELECT name FROM person WHERE id=1234}}*

  *{{SELECT name FROM person WHERE id={{r}} }}*

  *{{SELECT name FROM person WHERE id={{key1:C:alnumx}} }}*

* Leading and trailing spaces inside curly braces are removed.

  * *{{ SELECT "Hello World"   }}* acts as *{{SELECT "Hello World"}}*
  * *{{ varname   }}* acts as *{{varname}}*


* There are several stores, from where to retrieve the value. If a value is not found in one store, the next store is searched,
  until a value is found or there are no more stores available.
* If anywhere along the line an empty string is found, this **is** a value: therefore, the search will stop.
* If no value is found, the value is an <empty string>.

URL Parameter
^^^^^^^^^^^^^

* URL (=GET) Parameter can be used in *forms* and *reports* as variables.
* If a value violates a parameter sanitize class, the value becomes an empty string.

Variables used in SQL Statements might cause trouble by using: NUL (ASCII 0), \\n, \\r, \\, ', ", and Control-Z.

To protect the web application the following `escape` types are available:
	* 'm' - `real_escape_string() <http://php.net/manual/en/mysqli.real-escape-string.php>`_ (m = mysql)
	* 'l' - LDAP search filter values will be escaped: `ldap-escape() <http://php.net/manual/en/function.ldap-escape.php>`_ (LDAP_ESCAPE_FILTER).
	* 'L' - LDAP DN values will be escaped. `ldap-escape() <http://php.net/manual/en/function.ldap-escape.php>`_ (LDAP_ESCAPE_DN).
	* 's' - single ticks will be escaped. str_replace() of ' against \\'.
	* 'd' - double ticks will be escaped: str_replace() of " against \\".
	* '-' - no escaping.
* The `escape` type is defined by the fourth parameter of the variable. E.g.: `{{name:FE:alnumx:m}}` (m = mysql).
* It's possible to combine different `escape` types, they will be processed in the order given. E.g. `{{name:FE:alnumx:Ls}}` (L, s).
* Escaping is typically necessary for SQL or LDAP queries.
* Be careful when escaping nested variables. Best is to escape **only** the most outer variable.
* In `config.qfq.ini`_ a global `ESCAPE_TYPE_DEFAULT` can be defined. The configured escape type applies to all substituted
  variables, who do not contain a *specific* escape type.
* Additionally a `defaultEscapeType` can be defined per `Form` (separate field in the Form Editor). This overwrites the
  global definition of `config.qfq.ini`. By default, every `Form.defaultEscapeType` = 'c' (=config), which means the settin
  in `config.qfq.ini`_.
* To suppress a default escape type, define the `escape type` = '-' on the specific variable. E.g.: `{{name:FE:alnumx:-}}`.
* All values in Store *C* (Client=Browser) and store *F* (Form) will be sanitized:
* All `predefined-variable-names`_ have a specific default sanitize class. For these variables, it's not necessary
  to specify a sanitize class.
* All other variables (Store: C, F) get by default the sanitize class defined in the corresponding form. If not defined
  the default class is 'digit'.
* A default sanitize class can be overwritten by individual definition: *{{a:C:all}}*

+------------------+------+-------+-----------------------------------------------------------------------------------------+
| Name             | Form | Query | Pattern                                                                                 |
+==================+======+=======+=========================================================================================+
| **alnumx**       | Form | Query | [A-Za-z][0-9]@-_.,;: /() ÀÈÌÒÙàèìòùÁÉÍÓÚÝáéíóúýÂÊÎÔÛâêîôûÃÑÕãñõÄËÏÖÜŸäëïöüÿç            |
+------------------+------+-------+-----------------------------------------------------------------------------------------+
| **digit**        | Form | Query | [0-9]                                                                                   |
+------------------+------+-------+-----------------------------------------------------------------------------------------+
| **numerical**    | Form | Query | [0-9.-+]                                                                                |
+------------------+------+-------+-----------------------------------------------------------------------------------------+
| **email**        | Form | Query | [a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}                                          |
+------------------+------+-------+-----------------------------------------------------------------------------------------+
| **min|max**      | Form |       | Compares the value against an lower and upper limit (numeric or string).                |
+------------------+------+-------+-----------------------------------------------------------------------------------------+
| **min|max date** | Form |       | Compares the value against an lower and upper date or datetime.                         |
+------------------+------+-------+-----------------------------------------------------------------------------------------+
| **pattern**      | Form |       | Compares the value against a regexp.                                                    |
+------------------+------+-------+-----------------------------------------------------------------------------------------+
| **allbut**       | Form | Query | All characters allowed, but not [ ]  { } % & \ #. The used regexp: '^[^\[\]{}%&\\#]+$', |
+------------------+------+-------+-----------------------------------------------------------------------------------------+
| **all**          | Form | Query | no sanitizing                                                                           |
+------------------+------+-------+-----------------------------------------------------------------------------------------+

Security
========

All values passed to QFQ will be:

* Checked against max. length and allowed content, on the client and on the server side. On the server side, the check
  happens before any further processing. The 'length' and 'allowed' content is specified per `FormElement`. 'alnumx' is the
  default allowed content for those. Violating the rules will stop the 'save record' process (Form) or result in an empty value (Report).

* Only elements defined in the `Form` definition or requested by `Report` will be processed.

* UTF8 normalized (normalizer::normalize) to unify different ways of composing characters. It's more a database interest
  to work with unified data.

SQL statements are typically fired as `prepared statements` with separated variables.
Further *custom* SQL statements will be defined by the webmaster - those do not use `prepared statements` and might be
affected by SQL injection. To prevent SQL injection, every variable can be escaped with `mysqli::real_escape_string` by
defining the `escape` modifier `m`.

**QFQ notice**:

* Variables passed by the client (=Browser) are untrusted and use the default sanatize class 'digit' (if nothing else is
  specified). If alpha characters are submitted, the content violates `digit` and becomes therefore empty - there is no
  error message. Best is to always use SIP or digits.

Get Parameter
-------------

**QFQ security restriction**:
* GET parameter might contain urlencoded content (%xx). Therefore all GET parameter will be processed by 'urldecode()'.
  As a result a text like '%nn' in GET variables will always be decoded. It's not possible to transfer '%nn' itself.
* GET variables are limited to SECURITY_GET_MAX_LENGTH chars - any violation will stop QFQ.

Post Parameter
--------------

Per `FormElement` (HTML input) the default is to `htmlspecialchars()` the input. This means &<>'" will be encoded as htmlentity
and saved as a htmlentity in the database. In case any of these characters (e.g. for HTML tags) are
required, the encoding can be disabled per FormElement: `encode=none` (default is `specialchar`).

During Form load, htmlentities are decoded again.
All $_SERVER vars are htmlentities encoded (all, not only specialchars!) .
Every QFQ Form contains 'honeypot'-HTML input elements (HTML: hidden & readonly). Which of them to use is configured in
`config.qfq.ini`_ (default:   'username', 'password' and 'email'). On every start of QFQ (form, report, save, ...),
these variables are tested if they are non-empty. In such a case a probably malicous bot has send the request and the
request will not be processed.

If any of the default configured variable names are needed (which will never be the case for QFQ), an explicit variable name
list have to be configured in `config.qfq.ini`_.
**QFQ security restriction**:
* The honeypot variables can't be used in GET or POST as regular HTML input elements - any values of them will terminate QFQ.
On any violation, QFQ will sleep SECURITY_ATTACK_DELAY seconds (`config.qfq.ini`_) and than exit the running PHP process.
A detected attack leads to a complete white (=empty) page.

If SECURITY_SHOW_MESSAGE = true (`config.qfq.ini`_), at least a message is displayed after the delay.
Client Parameter via SIP
------------------------

Links with URL parameters, targeting to the local website, are typically SIP encoded. Instead of transferring the parameter
as part of the URL, only one uniqe GET parameter 's' is appended at the link. The parameter 's' is uniq (equal to a
timestamp) for the user. Assigned variables are stored as a part of the PHP user session on the server.
Two users might have the same value of parameter 's', but the content is completely independet.

Variables needed by Typo3 remains on the link and are not 'sip-encoded'.

.. _`SecureDirectFileAccess`:

Secure direct fileaccess
------------------------

If the application uploads files, mostly it's not necessary and often a security issue, to offer a direct download of
the uploaded files. Best is to create a directory, e.g. `fileadmin/protected` and deny direct access via webbrowser to it.
E.g. for Apache set a htaccess rule: ::

		<Directory /var/www/html/fileadmin/protected>
			Require all denied
		</Directory>

**Important**: all QFQ uploads should then save files in or below such a directory.

To offer download of those files, use the reserved columnname '_download':`download`_ or variants.

**Important**: To protect the installation against executing of uploaded malicious script code, disable PHP for the final upload
directory. E.g. `fileadmin`: ::
		<Directory "/var/www/html/fileadmin>
Carsten  Rose's avatar
Carsten Rose committed
			php_admin_flag engine Off
		</Directory>

This is in general a good security improvement for directories with user supplied content.

File upload
-----------

By default the mime type of every uploaded file is checked against a whitelist of allowed mime types. The mime type of
a file can be (easily) faked by an attacker. This check is good to handle regular user file upload for specific file types. To
prevent attacks against uploading and executing malicous code this won't help.

Intstead prohibit the execution of user contributed files by the webserver config (`SecureDirectFileAccess`_).
Store
=====

Only variables that are known in a specified store can be substituted.

 +-----+----------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
 |Name |Description                                                                             | Content                                                                    |
 +=====+========================================================================================+============================================================================+
 | F   | :ref:`STORE_FORM`: data not saved in database yet.                                     | All native *FormElements*. Recent values from the Browser.                 |
 +-----+----------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
 | S   | :ref:`STORE_SIP`: Client parameter 's' will indicate the current SIP, which will be    | sip, r (recordId), form                                                    |
 |     | loaded from the SESSION repo to the SIP-Store.                                         |                                                                            |
 +-----+----------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
 | R   | :ref:`STORE_RECORD`: Record - the current record loaded in the form                    | All columns of the current record from the current table                   |
 +-----+----------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
 | B   | :ref:`STORE_BEFORE`: Record - the current record loaded in the form before any update  | All columns of the current record from the current table                   |
 +-----+----------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
 | P   | Parent record. E.g.: on multi forms the current record of the outer query              | All columns of the MultiSQL Statement from the table for the current row   |
 +-----+----------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
 | D   | Default values column : The *table.column* specified *default value*.                  |                                                                            |
 +-----+----------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
 | M   | Column type: The *table.column* specified *type*                                       |                                                                            |
 +-----+----------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
 | C   | :ref:`STORE_CLIENT`: POST variable, if not found: GET variable                         | Parameter sent from the Client (=Browser).                                 |
 +-----+----------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
 | T   | :ref:`STORE_TYPO3`: a) Bodytext (ttcontent record), b) Typo3 internal variables        | See Typo3 tt_content record configuration                                  |
 +-----+----------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
 | V   | :ref:`STORE_VARS`: Generic variables                                                   |                                                                            |
 +-----+----------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
 | L   | :ref:`STORE_LDAP`: Will be filled on demand during processing of a *FormElement*       | Custom specified list of LDAP attributes                                   |
 +-----+----------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
 | 0   | *Zero* - allways value: 0, might be helpful if a variable is empty or undefined and    | Any key                                                                    |
 |     | will be used in an SQL statement.                                                      |                                                                            |
 +-----+----------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
 | E   | *Empty* - allways an empty string, might be helpful if a variable is empty or undefined| Any key                                                                    |
 |     | and will be used in an SQL statement                                                   |                                                                            |
 +-----+----------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
 | Y   | :ref:`STORE_SYSTEM`: a) Database, b) helper vars for logging/debugging:                |                                                                            |
 |     | SYSTEM_SQL_RAW ... SYSTEM_FORM_ELEMENT_COLUMN, c) Any custom fields: CONTACT, HELP, ...|                                                                            |
 +-----+----------------------------------------------------------------------------------------+----------------------------------------------------------------------------+

* Default *<prio>*: *FSRVD* - Form / SIP / Record / Vars / Table definition.
* Hint: Preferable, parameter should be submitted by SIP, not by Client (=URL).

  * Warning: Data submitted via 'Client' can be easily spoofed and altered.
  * Best: Data submitted via SIP never leaves the server, cannot be spoofed or altered by the user.
  * SIPs can _only_ be defined by using *Report*. Inside of *Report* use columns 'Link' (with attribute 's'), 'page?' or 'Page?'.

.. _predefined-variable-names:

Predefined variable names
-------------------------

.. _STORE_FORM:

Store: *FORM* - F
^^^^^^^^^^^^^^^^^

* Sanatized: *yes*
* Represents the values in the form, typically before saving them.
* Used for:

  * *FormElements* who will be rerendered, after a parent *FormElement* has been changed by the user.
  * *FormElement* actions, before saving the form.
  * Values will be sanitized by the class configured in corresponding the *FormElement*. By default, the sanitize class is `alnumx`.

 +---------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
 | Name                            | Explanation                                                                                                                                |
 +=================================+============================================================================================================================================+
 | <FormElement name>              | Name of native *FormElement*. To get, exactly and only, the specified *FormElement* (for 'pId'): *{{pId:F}}*                               |
 +---------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+

.. _STORE_SIP:

Store: *SIP* - S
^^^^^^^^^^^^^^^^

* Sanatized: *no*
* Filled automatically by creating links. E.g.:

  * in `Report` by using `_page?` or `_link` (with active 's')
  * in `Form` by using subrecords: 'new', 'edit', 'delete' links (system) or by column type `_page?`, `_link`.

 +-------------------------+-----------------------------------------------------------+
 | Name                    | Explanation                                               |
 +=========================+===========================================================+
 | sip                     | 13 char uniqid                                            |
 +-------------------------+-----------------------------------------------------------+
 | r                       | current record id                                         |
 +-------------------------+-----------------------------------------------------------+
 | form                    | current form name                                         |
 +-------------------------+-----------------------------------------------------------+
 | table                   | current table name                                        |
 +-------------------------+-----------------------------------------------------------+
 | urlparam                | all non Typo3 paramter in one string                      |
 +-------------------------+-----------------------------------------------------------+
 | <user defined>          | additional user defined link parameter                    |
 +-------------------------+-----------------------------------------------------------+

.. _STORE_RECORD:

Store: *RECORD* - R
^^^^^^^^^^^^^^^^^^^

* Sanatized: *no*
* Current record loaded in Form.
* If r=0, all values are empty.

 +------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+
 | Name                   | Explanation                                                                                                                                      |
 +========================+==================================================================================================================================================+
 | <column name>          | Name of a column of the primary table (as defined in the current form). To get, exactly and only, the specified form *FormElement*: *{{pId:R}}*  |
 +------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+

.. _STORE_BEFORE:

Store: *BEFORE* - B
^^^^^^^^^^^^^^^^^^^

* Sanatized: *no*
* Current record loaded in Form without any modification.
* If r=0, all values are empty.

This store is handy to compare new and old values of a form.

 +------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+
 | Name                   | Explanation                                                                                                                                      |
 +========================+==================================================================================================================================================+
 | <column name>          | Name of a column of the primary table (as defined in the current form). To get, exactly and only, the specified form *FormElement*: *{{pId:R}}*  |
 +------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+

.. _STORE_CLIENT:

Store: *CLIENT* - C
^^^^^^^^^^^^^^^^^^^

* Sanatized: *yes*

 +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------+
 | Name                    | Explanation                                                                                                                              |
 +=========================+==========================================================================================================================================+
 | s                       | =SIP                                                                                                                                     |
 +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------+
 | r                       | record id. Typically stored in SIP, rarely specified on the URL                                                                          |
 +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------+
 | keySemId                | always current Semester Id                                                                                                               |
 +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------+
 | keySemIdUser            | *{{keySemIdUser}}*, may be changed by user                                                                                               |
 +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------+
 | HTTP_HOST               | current HTTP HOST                                                                                                                        |
 +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------+
 | REMOTE_ADDR             | Client IP address                                                                                                                        |
 +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------+
 | '$_SERVER[*]'           | All other variables accessable by *$_SERVER[]*. Only the often used have a pre-defined sanitize class.                                   |
 +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------+
 | form                    | Unique name of current form                                                                                                              |
 +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------+
 | ANREDE                  | *{{sex}}* == male >> Sehr geehrter Herr, *{{sex}}* == female  Sehr geehrte Frau                                                          |
 +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------+
 | EANREDE                 | *{{sex}}* == male >> Dear Mr., *{{sex}}* == female >> Dear Mrs.                                                                          |
 +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------+

.. _STORE_TYPO3:

Store: *TYPO3* (Bodytext) - T
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* Sanatized: *no*

 +-------------------------+-------------------------------------------------------------------+----------+
 | Name                    | Explanation                                                       | Note     |
 +=========================+===================================================================+==========+
 | form                    | Formname defined in ttcontent record bodytext                     | see note |
 |                         |                                                                   |          |
 |                         | * Fix. E.g. *form = person*                                       |          |
 |                         | * via SIP. E.g. *form = {{form}}*                                 |          |
 +-------------------------+-------------------------------------------------------------------+----------+
 | pageId                  | Record id of current Typo3 page                                   | see note |
 +-------------------------+-------------------------------------------------------------------+----------+
 | pageType                | Current selected page type (typically URL parameter 'type')       | see note |
 +-------------------------+-------------------------------------------------------------------+----------+
 | pageLanguage            | Current selected page language (typically URL parameter 'L')      | see note |
 +-------------------------+-------------------------------------------------------------------+----------+
 | ttcontentUid            | Record id of current Typo3 content element                        | see note |
 +-------------------------+-------------------------------------------------------------------+----------+
 | feUser                  | Logged in Typo3 FE User                                           |          |
 +-------------------------+-------------------------------------------------------------------+----------+
 | feUserUid               | Logged in Typo3 FE User uid                                       |          |
 +-------------------------+-------------------------------------------------------------------+----------+
 | feUserGroup             | FE groups of logged in Typo3 FE User                              |          |
 +-------------------------+-------------------------------------------------------------------+----------+

* **note**: not available
  * by *FormElement* class 'action' with type 'beforeSave', 'afterSave', 'beforeDelete', 'afterDelete'.

.. _STORE_VARS:

Store: *VARS* - V
^^^^^^^^^^^^^^^^^

* Sanatized: *no*

 +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
 | Name                    | Explanation                                                                                                                                |
 +=========================+============================================================================================================================================+
 | random                  | random string with length of 32 chars, alphanum                                                                                            |
 +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
 | slaveId                 | see *FormElement* `action`                                                                                                                 |
 +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
 | filename                | Original filename of an uploaded file via an 'upload'-FormElement. Valid only during processing of the current 'upload'-formElement.       |
 +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
 | fileDestinaton          | Destination (path & filename) for an uploaded file. Defined in an 'upload'-FormElement.parameter. Valid: same as 'filename'.               |
 +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+

.. _STORE_LDAP:

Store: *LDAP* - L
^^^^^^^^^^^^^^^^^

* Sanatized: *yes*

 +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
 | Name                    | Explanation                                                                                                                                |
 +=========================+============================================================================================================================================+
 | <custom defined>        | See *ldapAttributes*                                                                                                                       |
 +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+



.. _STORE_SYSTEM:

Store: *SYSTEM* - Y
^^^^^^^^^^^^^^^^^^^

* Sanatized: *no*

 +-------------------------+--------------------------------------------------------------------------+
 | Name                    | Explanation                                                              |
 +=========================+==========================================================================+
 | DB_USER                 | defined in config.ini                                                    |
 +-------------------------+--------------------------------------------------------------------------+
 | DB_SERVER               | defined in config.ini                                                    |
 +-------------------------+--------------------------------------------------------------------------+
 | DB_NAME                 | defined in config.ini                                                    |
 +-------------------------+--------------------------------------------------------------------------+
 | DB_INIT                 | defined in config.ini                                                    |
 +-------------------------+--------------------------------------------------------------------------+
 | SQL_LOG                 | defined in config.ini                                                    |
 +-------------------------+--------------------------------------------------------------------------+
 | SQL_LOG_MODE            | defined in config.ini                                                    |
 +-------------------------+--------------------------------------------------------------------------+
 | SHOW_DEBUG_INFO         | defined in config.ini                                                    |
 +-------------------------+--------------------------------------------------------------------------+
 | CSS_LINK_CLASS_INTERNAL | defined in config.ini                                                    |
 +-------------------------+--------------------------------------------------------------------------+
 | CSS_LINK_CLASS_EXTERNAL | defined in config.ini                                                    |
 +-------------------------+--------------------------------------------------------------------------+
 | CSS_CLASS_QFQ_CONTAINER | defined in config.ini                                                    |
 +-------------------------+--------------------------------------------------------------------------+
 | EXT_PATH                | computed during runtime                                                  |
 +-------------------------+--------------------------------------------------------------------------+
 | SITE_PATH               | computed during runtime                                                  |
 +-------------------------+--------------------------------------------------------------------------+
 | DATE_FORMAT             | defined in config.ini                                                    |
 +-------------------------+--------------------------------------------------------------------------+
 | class                   | defined in config.ini (CSS_CLASS_QFQ_FORM) or form definition            |
 +-------------------------+--------------------------------------------------------------------------+
 | classPill               | defined in config.ini (CSS_CLASS_QFQ_FORM_PILL) or form definition       |
 +-------------------------+--------------------------------------------------------------------------+
 | classBody               | defined in config.ini (CSS_CLASS_QFQ_FORM_BODY) or form definition       |
 +-------------------------+--------------------------------------------------------------------------+
 | data-pattern-error      | defined in config.ini or form definition                                 |
 +-------------------------+--------------------------------------------------------------------------+
 | data-require-error      | defined in config.ini or form definition                                 |
 +-------------------------+--------------------------------------------------------------------------+
 | data-match-error        | defined in config.ini or form definition                                 |
 +-------------------------+--------------------------------------------------------------------------+
 | data-error              | defined in config.ini or form definition                                 |
 +-------------------------+--------------------------------------------------------------------------+
 | bsColumns               | defined in config.ini (FORM_BS_COLUMNS) or form definition               |
 +-------------------------+--------------------------------------------------------------------------+
 | bsLabelColumns          | defined in config.ini (FORM_BS_LABEL_COLUMNS) or form definition         |
 +-------------------------+--------------------------------------------------------------------------+
 | bsInputColumns          | defined in config.ini (FORM_BS_INPUT_COLUMNS) or form definition         |
 +-------------------------+--------------------------------------------------------------------------+
 | bsNoteColumns           | defined in config.ini (FORM_BS_NOTE_COLUMNS) or form definition          |
 +-------------------------+--------------------------------------------------------------------------+