Skip to content
Snippets Groups Projects
Manual.rst 499.27 KiB

General

Installation

The following features are only tested / supported on linux hosts:

  • General: QFQ is coded to run on Linux hosts, preferable on Debian derivates like Ubuntu.
  • HTML to PDF conversion - command wkhtmltopdf.
  • Concatenation of PDF files - command pdfunite.
  • Mime type detection for uploads - command file.

Preparation

Report & Form

To normalize UTF8 input, php-intl package is needed by

  • normalizer::normalize()

For the download function, the programs pdfunite and file are necessary to concatenate PDF files.

Preparation for Ubuntu:

sudo apt install php-intl
sudo apt install poppler-utils libxrender1 file pdf2svg  # for file upload, PDF and 'HTML to PDF' (wkhtmltopdf), PDF split
sudo apt install inkscape imagemagick            # to render thumbnails

wkhtmltopdf

wkhtmltopdf will be used by QFQ to offer 'website print' and 'HTML to PDF' conversion. The program is not included in QFQ and has to be manually installed.

  • The Ubuntu package wkhtmltopdf needs a running Xserver - this does not work on a headless webserver.
    • Best is to install the QT version from the named website above.
    • In case of trouble with wkhtmltopdf, also install 'libxrender1'.
    • The current version 0.12.4 might have trouble with https connections. Version 0.12.5-dev (github master branch) seems more reliable. Please contact the QFQ authors if you need a compiled Ubuntu version of wkhtmltopdf.

In configuration specify:

config.cmdWkhtmltopdf:  /opt/wkhtmltox/bin/wkhtmltopdf
config.baseUrl: http://www.example.com/

If wkhtml has been compiled with dedicated libraries (not part of LD_LIBRARY_PATH), specify the LD_LIBRARY_PATH together with the path-filename:

config.cmdWkhtmltopdf: LD_LIBRARY_PATH=/opt/wkhtmltox/lib /opt/wkhtmltox/bin/wkhtmltopdf

Important

To access FE_GROUP protected pages or content, it's necessary to disable the [FE][lockIP] check! wkhtml will access the Typo3 page locally (localhost) and that IP address is different from the client (=user) IP.

Configure via Typo3 Installtool All configuration > $TYPO3_CONF_VARS['FE']:

[FE][lockIP] = 0

Warning

[FE][lockIP] = 0 disables an important anti-'session hijacking' protection. The security level of the whole installation will be lowered! Again, this is only needed if wkhtml needs access to FE_GROUP protected pages & content. As an alternative to lower the security level, create a separated page subtree which is only accessible (configured via Typoscript) from specific IPs or if a FE-User is logged in.

If there are problems with converting/downloading FE_GROUP protected pages, check configuration showDebugInfo = download to debug.

Note

Converting HTML to PDF gives no error message but RC=-1? Check carefully all includes of CSS, JS, images and so on! Typically some of them fails to load and wkhtml stops running! Verify the correct loading of all elements by calling the site via a regular browser and bypassing any browser cache (Ctrl F5).

Checklist wkhtml problems
  • config.baseUrl is configured and correct. The baseUrl has to be the same protocol as the website (http or https).
  • To track down problems:
    • In configuration set debug.showDebugInfo=auto,download.
    • Do the download.
    • Check QFQ_LOG for any output.
    • Grab the URL given in the QFQ_LOG, open a browser in private mode (no existing browser session) and open the URL.
    • Check the --cookie-jar '/tmp/qfq.cookie....' file for the cookie.
    • Call wkhtml manually on the webserver, with the same options as given in the QFQ_LOG.
HTML to PDF conversion

wkhtmltopdf converts a website (local or remote) to a (multi)-page PDF file. It's mainly used in download.

Print

Different browser prints the same page in different variations. To prevent this, QFQ implements a small PHP wrapper print.php with uses wkhtmltopdf to convert HTML to PDF.

Provide a print this page-link (replace 'current pageId' ):

<a href="typo3conf/ext/qfq/Classes/Api/print.php?id={current pageId}">Print this page</a>

Any parameter specified after print.php will be delivered to wkhtmltopdf as part of the URL.

Typoscript code to implement a print link on every page:

10 = TEXT
10 {
  wrap = <a href="typo3conf/ext/qfq/Classes/Api/print.php?id=|&type=99"><span class="glyphicon glyphicon-print" aria-hidden="true"></span> Printview</a>
  data = page:uid
}

Send Email

QFQ sends mail via sendEmail http://caspian.dotconf.net/menu/Software/SendEmail/ - a small perl script without a central configuration.

By default, sendEmail uses the local installed MTA, writes a logfile to fileadmin/protected/log/mail.log and handles attachments via commandline options. A basic HTML email support is implemented.

The latest version is v1.56, which has at least one bug. That one is patched in the QFQ internal version v1.56p1 (see QFQ GIT sources in directory 'patches/sendEmail.patch').

Nevertheless, on latest system the TLS support is broken - please check sendEmailProblem.

The Typo3 sendmail eco-system is not used at all by QFQ.

Thumbnail

Thumbnails will be rendered via ImageMagick (https://www.imagemagick.org/) 'convert' and 'inkscape' (https://inkscape.org). 'inkscape' is only used for '.svg' files.

The Typo3 graphic eco-system is not used at all by QFQ.

Usage: column-thumbnail.

Setup

  • Install the extension via the Extension Manager.
    • If you install the extension by manual download/upload and get an error message "can't activate extension": rename the downloaded zip file to qfq.zip or qfq_<version>.zip (e.g. version: 18.12.0).
    • If the Extension Manager stops after importing: check your memory limit in php.ini.
  • Copy/rename the file <site path>/typo3conf/ext/qfq/config.example.qfq.php to <site path>/typo3conf/config.qfq.php. Configure the necessary settings configuration The configuration file is outside of the extension directory, to not loose it during de-install and install again.
  • When the QFQ Extension is called the first time on the Typo3 frontend, the file <ext_dir>/Classes/Sql/formEditor.sql will played and fills the database with the Form editor records. This also happens automatically after each update of QFQ.
  • Configure Typoscript to include Bootstrap, jQuery, QFQ javascript and CSS files.

Setup CSS & JS

page.meta {
    X-UA-Compatible = IE=edge
    X-UA-Compatible.attribute = http-equiv
    viewport=width=device-width, initial-scale=1
}

page.includeCSS {
    file01 = typo3conf/ext/qfq/Resources/Public/Css/bootstrap.min.css
    file02 = typo3conf/ext/qfq/Resources/Public/Css/bootstrap-theme.min.css
    file03 = typo3conf/ext/qfq/Resources/Public/Css/jqx.base.css
    file04 = typo3conf/ext/qfq/Resources/Public/Css/jqx.bootstrap.css
    file05 = typo3conf/ext/qfq/Resources/Public/Css/qfq-bs.css
    file06 = typo3conf/ext/qfq/Resources/Public/Css/tablesorter-bootstrap.css
    file07 = typo3conf/ext/qfq/Resources/Public/Css/font-awesome.min.css
}

page.includeJS {
    file01 = typo3conf/ext/qfq/Resources/Public/JavaScript/jquery.min.js
    file02 = typo3conf/ext/qfq/Resources/Public/JavaScript/bootstrap.min.js
    file03 = typo3conf/ext/qfq/Resources/Public/JavaScript/validator.min.js
    file04 = typo3conf/ext/qfq/Resources/Public/JavaScript/jqx-all.js
    file05 = typo3conf/ext/qfq/Resources/Public/JavaScript/globalize.js
    file06 = typo3conf/ext/qfq/Resources/Public/JavaScript/tinymce.min.js
    file07 = typo3conf/ext/qfq/Resources/Public/JavaScript/EventEmitter.min.js
    file08 = typo3conf/ext/qfq/Resources/Public/JavaScript/typeahead.bundle.min.js
    file09 = typo3conf/ext/qfq/Resources/Public/JavaScript/qfq.min.js
    file10 = typo3conf/ext/qfq/Resources/Public/JavaScript/jquery.tablesorter.combined.min.js
    file11 = typo3conf/ext/qfq/Resources/Public/JavaScript/jquery.tablesorter.pager.min.js
    file12 = typo3conf/ext/qfq/Resources/Public/JavaScript/widget-columnSelector.min.js

    # Only needed in case FormElement 'annotate' is used.
    file20 = typo3conf/ext/qfq/Resources/Public/JavaScript/fabric.min.js
    file21 = typo3conf/ext/qfq/Resources/Public/JavaScript/qfq.fabric.min.js
}

FormEditor

Setup a report to manage all forms:

  • Create a Typo3 page.

  • Set the 'URL Alias' to form (recommended) or the individual defined value in parameter editFormPage (configuration).

  • Insert a content record of type qfq.

  • In the bodytext insert the following code:

    # If there is a form given by SIP: show
    form={{form:SE}}
    
    # In case indexQfq != indexData, set dbIndex=indexQfq.
    dbIndex = {{indexQfq:Y}}
    
    10 {
        # List of Forms: Do not show this list of forms if there is a form given by SIP.
        # Table header.
        sql = SELECT CONCAT('p:{{pageAlias:T}}&form=form|A:data-reference=newForm') as _pagen, '#', 'Name', 'Title', 'Table', '' FROM (SELECT 1) AS fake WHERE '{{form:SE}}'=''
        head = {{'b|p:id={{pageAlias:T}}&form=copyFormFromExt|t:Copy form from ExtForm|A:data-reference=copyForm' AS _link}}
               <table class="table table-hover qfq-table-50 tablesorter tablesorter-filter" id="{{pageAlias:T}}-form">
        tail = </table>
        rbeg = <thead class="qfq-sticky"><tr>
        rend = </tr></thead>
        fbeg = <th>
        fend = </th>
    
        10 {
            # All forms
            sql = SELECT CONCAT('p:{{pageAlias:T}}&form=form&r=', f.id, '|A:data-reference=editForm', f.name) as _pagee
                         , f.id, QMORE(f.name, 50), f.title AS '_striptags', f.tableName
                         , CONCAT('U:form=form&r=', f.id, '|A:data-reference=deletForm') as _paged
                      FROM Form AS f
                      ORDER BY f.name
            rbeg = <tr>
            rend = </tr>
            fbeg = <td>
            fend = </td>
        }
    }

To keep the overview about all forms, it's useful to know which form has been used, how often, on which page and when. Find these information included in the form-editor-usage report.

Installation: Check List

  • Protect the directory <T3 installation>/fileadmin/protected in Apache against direct file access.
    • <T3 installation>/fileadmin/protected/ should be used for confidential (uploaded / generated) data.
    • <T3 installation>/fileadmin/protected/log/... is the default place for QFQ log files.
  • Protect the directory <T3 installation>/fileadmin in Apache to not execute PHP Scripts - malicious uploads won't be executed.
  • Setup a log rotation rule for sqlLog.
  • Check that sqlLogMode is set to modify on productive sites. With none you have no chance to find out who changed which data and all really logs a mass of data.

Configuration

config.qfq.php

Keyword Example Description
DB_<n>_USER DB_1_USER=qfqUser Credentials configured in MySQL
DB_<n>_PASSWORD DB_1_PASSWORD=1234567890 Credentials configured in MySQL
DB_<n>_SERVER DB_1_SERVER=localhost Hostname of MySQL Server
DB_<n>_NAME DB_1_NAME=qfq_db Database name
LDAP_1_RDN LDAP_1_PASSWORD LDAP_1_RDN='ou=Admin,ou=example,dc=com' LDAP_1_PASSWORD='mySecurePassword' Credentials for non-anonymous LDAP access. Only one set supported.

Example: typo3conf/config.qfq.php:

<?php

// QFQ configuration
//
// Save this file as: <site path>/typo3conf/config.qfq.php

return [
    'DB_1_USER' => '<DBUSER>',
    'DB_1_SERVER' => '<DBSERVER>',
    'DB_1_PASSWORD' => '<DBPW>',
    'DB_1_NAME' => '<DB>',

    //DB_2_USER => <DBUSER>
    //DB_2_SERVER => <DBSERVER>
    //DB_2_PASSWORD => <DBPW>
    //DB_2_NAME => <DB>

    // DB_n ...
    // ...

    // LDAP_1_RDN => 'ou=Admin,ou=example,dc=com'
    // LDAP_1_PASSWORD => 'mySecurePassword'
];

Extension Manager: QFQ Configuration

Keyword Default / Example Description
Config
flagProduction yes yes|no: used to differentiate production and development site.
maxFileSize 10M If empty, take minimum of 'post_max_size' and 'upload_max_filesize'.
baseUrl http://example.com URL where wkhtmltopdf will fetch the HTML (no parameter, those comes later)
dateFormat yyyy-mm-dd Possible options: yyyy-mm-dd, dd.mm.yyyy.
thumbnailDirSecure fileadmin/protected/qfqThumbnail Important: secure directory 'protected' (recursive) against direct access.
thumbnailDirPublic typo3temp/qfqThumbnail Both thumbnail directories will be created if not existing.
cmdInkscape inkscape If inkscape is not available, specify an empty string.
cmdConvert convert GraphicsMagics 'convert' is recommended.
cmdWkhtmltopdf /usr/bin/wkhtmltopdf PathFilename of wkhtmltopdf. Optional variables like LD_LIBRARY_PATH=...
sendEMailOptions -o tls=yes General options. Check: http://caspian.dotconf.net/menu/Software/SendEmail
documentation http://docs.typo3.org... Link to the online documentation of QFQ. Every QFQ installation also contains a local copy: typo3conf/ext/qfq/Documentation/html/Manual.html
Dynamic
fillStoreSystemBySql1/2/3 SELECT s.id AS ... Specific values read from the database to fill the system store during QFQ load. See fillStoreSystemBySql for a usecase.
fillStoreSystemBySqlErrorMsg1/2/3 No current period found Only define an error message, if QFQ should stop running in case of an SQL error or not exact one record.
Debug
throwExceptionGeneralError auto
yes: 'general errors' in QFQ (PHP) will throw an exception.
auto: becomes 'yes', if 'flagProduction'!='yes', else 'no'.
no: 'general errors' in QFQ (PHP) will be silently ignored.
formSubmitLogMode all
all: every form submission will be logged.
none: no logging.
See form-submit-log-page for example QFQ code to display the log.
redirectAllMailTo john@doe.com If set, redirect all QFQ generated mails (Form, Report) to the specified.
sqlLogMode modify
all: every statement will be logged - this might be a lot.
modifyAll: log all statements which might change data, even if 0 rows
affected.
modify: log only statements which change data (affected rows > 0).
error: log only DB errors.
none: no SQL log at all.
sqlLogModeAutoCron error Applies only to AutoCron Jobs. For production 'error' should be fine.
sqlLog fileadmin/protected/log/sql.log Filename to log SQL commands: relative to <site path> or absolute. If the directory does not exist, create it.
qfqLog fileadmin/protected/log/qfq.log Filename to log general QFQ events:relative to <site path> or absolute. If the directory does not exist, create it.
mailLog fileadmin/protected/log/mail.log Filename to log sendEmail commands: relative to <site path> or absolute. If the directory does not exist, create it.
showDebugInfo auto FE - Possible values: yes|no|auto|download. For 'auto': If a BE User is logged in, a debug information will be shown on the FE.
Database
init init=SET names utf8; SET sql_mode = "NO_ENGINE_SUBSTITUTION" Global init for using the database. For 'sql_mode="NO_ENGINE_SUBSTITUTION"' see #7407.
update auto
auto: apply DB Updates only if there is a newer version.
always: apply DB Updates always, especially play formEditor.sql every time QFQ is called - not recommended!
never: never apply DB Updates.
indexData 1 Optional. Default: 1. Retrieve the current setting via {{dbNameData:Y}}.
indexQfq 1 Optional. Default: 1. Retrieve the current setting via {{dbNameQfq:Y}}.
Security
escapeTypeDefault m All variables {{...}} get this escape class by default. See variable-escape.
securityVarsHoneypot email,username,password If empty: no check. All named variables will rendered as INPUT elements.
securityAttackDelay 5 If an attack is detected, sleep 'x' seconds and exit PHP process.
securityShowMessage true If an attack is detected, show a message.
securityGetMaxLength 50 GET vars longer than 'x' chars triggers an attack-recognized. ExceptionMaxLength.
securityFailedAuthDelay 3 If REST authorization fails, sleep 'x' seconds before answering.
Form-Config
recordLockTimeoutSeconds 900 Timeout for record locking. After this time, a record will be replaced.
sessionTimeoutSeconds 1800 Timeout for FE User session. See sessionTimeoutSeconds
enterAsSubmit enterAsSubmit = 1 0: off, 1: Pressing enter in a form means save and close.
editFormPage form T3 Pagealias to edit a form.
formDataPatternError please check pattern error Customizable error message used in validator.js. 'pattern' violation.
formDataRequiredError missing value Customizable error message used in validator.js. 'required' fields.
formDataMatchError type error Customizable error message used in validator.js. 'match' retype mismatch.
formDataError generic error Customizable error message used in validator.js. 'no specific' given.
Form-Layout
labelAlign left Label align (left/center/right)/ Default: left. Will be inherited to Form.
cssClassQfqContainer container
QFQ with own Bootstrap: 'container'.
QFQ already nested in Bootstrap of mainpage: <empty>.
cssClassQfqForm qfq-color-base Wrap around QFQ 'Form'.
cssClassQfqFormPill qfq-color-grey-1 Wrap around title bar for pills: CSS Class, typically a background color.
cssClassQfqFormBody qfq-color-grey-2 Wrap around FormElements: CSS Class, typically a background color.
formBsColumns col-md-12 col-lg-10 The whole form will be wrapped. See bs-custom-field-width
formBsLabelColumns col-md-3 col-lg-3 The column get the width. See bs-custom-field-width
formBsInputColumns col-md-6 col-lg-6
formBsNoteColumns col-md-3 col-lg-3
extraButtonInfoInline <img src="info.png"> Image for extraButtonInfo (inline).
extraButtonInfoBelow <img src="info.png"> Image for extraButtonInfo (below).
extraButtonInfoPosition below 'auto' (default) or 'below'. See extraButtonInfo.
extraButtonInfoClass pull-right '' (default) or 'pull-right'. See extraButtonInfo.
Form-Language
formLanguage[ABCD]Id E.g.: 1 In Typo3 configured pageLanguage id. The number after the 'L' parameter.
formLanguage[ABCD]Label E.G.: english Label shown in Form editor, on the 'basic' tab.
saveButtonText   Text on the form save button. Typically none.
saveButtonTooltip Save Tooltip on the form save button.
saveButtonClass btn btn-default navbar-btn Bootstrap CSS class for save button on top of the form.
buttonOnChangeClass alert-info btn-info Bootstrap CSS class for save button showing 'data changed'.
saveButtonGlyphIcon glyphicon-ok Icon for the form save button.
closeButtonText   Text on the form close button. Typically none.
closeButtonTooltip close Tooltip on the form close button.
closeButtonClass btn btn-default navbar-btn Bootstrap CSS class for close button on top of the form.
closeButtonGlyphIcon glyphicon-remove Icon for the form close button.
deleteButtonText   Text on the form delete button. Typically none.
deleteButtonTooltip delete Tooltip on the form delete button.
deleteButtonClass btn btn-default navbar-btn Bootstrap CSS class for delete button on top of the form.
deleteButtonGlyphIcon glyphicon-trash Icon for the form delete button.
newButtonText   Text on the form new button. Typically none.
newButtonTooltip new Tooltip on the form new button.
newButtonClass btn btn-default navbar-btn Bootstrap CSS class for new button on top of the form.
newButtonGlyphIcon glyphicon-plus Icon for the form new button.
showIdInFormTitle 0 (off), 1 (on) Append at the form title the current record id.
cssClassColumnId text-muted A column in a subrecord with the name id|ID|Id gets this class.

After parsing the configuration, the following variables will be set automatically in STORE_SYSTEM:

Keyword Description
dbNameData Name of the 'data'-database. '{{dbNameData:Y}}
dbNameQfq Name of the 'QFQ'-database. '{{dbNameQfq:Y}}
dbNameT3 Name of the 'T3'-database. '{{dbNameT3:Y}}
sitePath Absolute path of the current T3 instance. '{{sitePath:Y}}
extPath Absolute path of the QFQ extension. '{{extPath:Y}}

Custom variables

Up to 30 custom variables can be defined in configuration.

E.g. to setup a contact address and reuse the information inside your installation do:

  custom1: ADMINISTRATIVE_CONTACT = john@doe.com
  custom2: ADMINISTRATIVE_ADDRESS = John Doe, Hollywood Blvd. 1, L.A.
  custom3: ADMINISTRATIVE_NAME = John Doe

* Somewhere in a `Form` or in `Report`::

     {{ADMINISTRATIVE_CONTACT:Y}}, {{ADMINISTRATIVE_ADDRESS:Y}}, {{ADMINISTRATIVE_NAME}}

It's also possible to configure such variables directly in config.qfq.php.

Fill STORE_SYSTEM by SQL

A specified SELECT statement in configuration in variable fillStoreSystemBySql1 (or 2, or 3) will be fired. The query should have 0 (nothing happens) or 1 row. All columns will be added to the existing STORE_SYSTEM. Existing variables will be overwritten. Be careful not to overwrite system values.

This option is useful to make generic custom values, saved in the database, accessible to all QFQ Report and Forms. Access such variables via {{<varname>:Y}}.

In case QFQ should stop working if a given query does not select exact one record (e.g. a missing period), define an error message:

fillStoreSystemBySql1: SELECT name FROM Person WHERE name='Doe'
fillStoreSystemBySqlErrorMsg1: Too many or to few "Doe's" in our database
periodId

This is

  • a usecase, implemented via fillStoreSystemBySql,
  • a way to access Period.id with respect to the current period (the period itself is custom defined).

After a full QFQ installation:

  • a table Period (extend / change it to your needs, fill them with your periods),
  • one sample record in table Period,

Websites, delivering semester data, school year schedules, or any other type or periods, often need an index to the current period.

In configuration:

fillStoreSystemBySql1: SELECT id AS periodId FROM Period WHERE start<=NOW() ORDER BY start DESC LIMIT 1

a variable 'periodId' will automatically computed and filled in STORE SYSTEM. Access it via {{periodId:Y0}}. To get the name and current period:

SELECT name, ' / ', start FROM Period WHERE id={{periodId:Y0}}

Typically, it's necessary to offer a 'previous' / 'next' link. In this example, the STORE SIP holds the new periodId:

SELECT CONCAT('p:{{pageAlias:T}}&periodId=', {{periodId:SY0}}-1, '|Next') AS _page, ' ', name, ' ',
  CONCAT('p:{{pageAlias:T}}&periodId=', {{periodId:SY0}}+1, '|Next') AS _page FROM Period AS s WHERE s.id={{periodId:SY0}}

Take care for minimum and maximum indexes (do not render the links if out of range).

DB USER privileges

The specified DB User needs privileges to the database of at least: SELECT / INSERT / UPDATE / DELETE / SHOW.

To apply automatically QFQ-'DB UPDATE' the following rights are mandatory too: CREATE / ALTER

To get access to the Typo3 installation, 'dbuser' should also have access to the Typo3 Database with at least SELECT / INSERT / UPDATE / DELETE.

Exception for SECURITY_GET_MAX_LENGTH

If it is necessary to use a GET variable which exceeds securityGetMaxLength limit, name the variable with '_<num>' at the end. E.g. my_long_variable_130. Such a variable has an allowed length of 130 chars. Access the variable as usual with the variable name: {{my_long_variable_130:C:...}}.

FE-User: Session timeout seconds

There is no timeout for website users who are not logged in (but typically those users don't have access to protected content).

For logged in users, the default timeout is the php.ini settings for session.cookie_lifetime and session.gc_maxlifetime (minimum of both). These timeout only affects QFQ related content and can be specified a) globally (QFQ configuration) and b) specific per Form.

The maximum timeout depends on the minimal value of php.ini session.cookie_lifetime and session.gc_maxlifetime. Specifying a higher value produces an error in the front end.

Every access to QFQ related content resets the timeout.

After FE login, the next access to QFQ related content starts the timeout counter.

Local Documentation

A HTML rendered version is available under: <your site>/typo3conf/ext/qfq/Documentation/html/Index.html

If you get a 'Page forbidden / not found' there might be some Webserver restrictions. E.g. the Typo3 example of .htaccess in the Typo3 installation folder will forbid access to any extension documentation (which is a good idea on a productive server). For a development server instead, deactivate the forbid rule of 'documentation'. In .htaccess (snippet from Typo3 7.6 _.htaccess):

production:   RewriteRule (?:typo3conf/ext|typo3/sysext|typo3/ext)/[^/]+/(?:Configuration|Resources/Private|Tests?|Documentation|docs?)/ - [F]
development:  RewriteRule (?:typo3conf/ext|typo3/sysext|typo3/ext)/[^/]+/(?:Configuration|Resources/Private|Tests?|docs?)/ - [F]

Concept

SIPs

The following is a technical background information. Not needed to just use QFQ.

The SIPs (=Server Id Pairs) are uniq timestamps, created/registered on the fly for a specific browser session (=user). Every SIP is registered on the server (= stored in a browser session) and contains one or more key/value pairs. The key/value pairs never leave the server. The SIPs will be used:

  • to protect values not to be spoofed by anyone,
  • to protect values not to be altered by anyone,
  • to grant access, e.g.:
    • load and save forms,
    • upload files,
    • download files,
    • PHP AJAX pages.

SIPs becomes invalid, as soon as the current browser session is destroyed. The client (= user) can't manipulate the content of SIPs - it's only possible to reuse already registered SIPs by the user, who already owns the session.

Access privileges

The Typo3 FE Groups can be used to implement access privileges. Such groups are assigned to

  • Typo3 FE users,
  • Typo3 pages,
  • and/or Typo3 content records (e.g. QFQ records).

This will be used for general page structure privileges.

A record base privileges controlling (e.g. which user can edit which person record) will be implicit configured, by the way that records are viewable / editable (or not) through SQL in the specific QFQ tt-content statements.

Typo3 QFQ content element

Insert one or more QFQ content elements on a Typo3 page. Specify column and language per content record as wished.

The title of the QFQ content element will not be rendered on the frontend. It's only visible to the webmaster in the backend for orientation.

QFQ Keywords (Bodytext)

All of the named parameter are optional.

Name Explanation
form
Formname.
Static: form = person
By SIP: form = {{form:SE}}
By SQL: form = {{SELECT c.form FROM config AS c WHERE c.id={{a:C}} }}
r
<record id>. The form will load the record with the specified id.
Static: r = 123
By SQL: r = {{SELECT ...}}
If not specified, the SIP parameter 'r' is used.
dbIndex E.g. dbIndex = {{indexQfq:Y}} Select a DB index. Only necessary if a different than the standard DB should be used.
debugShowBodyText If='1' and configuration:showDebugInfo: yes, shows a tooltip with bodytext
sqlLog Overwrites configuration: SQL_LOG . Only affects Report, not Form.
sqlLogMode Overwrites configuration: SQL_LOG_MODE . Only affects Report, not Form.
<level>.fbeg Start token for every field (=column)
<level>.fend End token for every field (=column)
<level>.fsep Separator token between fields (=columns)
<level>.fskipwrap Comma separated list of column id's. Skip wrapping of indexed columns.
<level>.shead Static start token for whole <level>, independent if records are selected Shown before head.
<level>.stail Static end token for whole <level>, independent if records are selected. Shown after tail.
<level>.head Dynamic start token for whole <level>. Only if at least one record is select.
<level>.tail Dynamic end token for whole <level>. Only if at least one record is select.
<level>.rbeg Start token for row.
<level>.rbgd Alternating (per row) token.
<level>.rend End token for row. Will be rendered before subsequent levels are processed
<level>.renr End token for row. Will be rendered after subsequent levels are processed
<level>.rsep Seperator token between rows
<level>.sql SQL Query
<level>.twig Twig Template
<level>.althead If <level>.sql has no rows selected (empty), these token will be rendered.
<level>.altsql If <level>.sql has no rows selected (empty) or affected (delete, update, insert) the <altsql> will be fired. Note: Sub queries of <level> are not fired, even if <altsql> selects some rows.
<level>.content
show (default): content of current and sub level are directly shown.
hide: content of current and sub levels are stored and not shown.
hideLevel: content of current and sub levels are stored and only sub levels
are shown.
store: content of current and sub levels are stored and shown.
To retrieve the content: {{<level>.line.content}}. See syntax-of-report

QFQ Database

Recommended setup for Typo3 & QFQ Installation is with two databases. One for the Typo3 installation and one for QFQ. A good practice is to name both databases equal, appending the suffix '_t3' and '_db'.

When QFQ is called, it checks for QFQ system tables. If they do not exist or have a lower version than the installed QFQ version, the system tables will be automatically installed or updated.

System tables

Name Use Database
Clipboard Temporary QFQ
Cron Persistent QFQ
Dirty Temporary QFQ | Data
Form Persistent QFQ
FormElement Persistent QFQ
FormSubmitLog Persistent QFQ | Data
MailLog Persistent QFQ | Data
Period Persistent Data
Split Persistent Data

See mail-log-page and form-submit-log-page for some Frontend views for these tables.

  • Check Bug #5459 / support of system tables in different DBs not supported.

Multi Database

Base: T3 & QFQ

QFQ typically interacts with one database, the QFQ database. The database used by Typo3 is typically a separate one. Theoretically it might be the same (never tested), but it's strongly recommended to use a separated QFQ database to have no problems on Typo3 updates and to have a clean separation between Typo3 and QFQ.

QFQ: System & Data

QFQ itself can be separated in 'QFQ system' (see system-tables) and 'QFQ data' databases (even more than one are possible). The 'QFQ system' stores the forms, record locking, log tables and so on - QFQ data is for the rest.

A Multi Database setup is given, if 'QFQ system' is different from 'QFQ data'.

Data: Data1, Data2, ..., Data n

Every database needs to be configured via configuration with it's own index.

QFQ data might switch between different 'data'-databases. In configuration one main QFQ data index will be specified in indexQfq. If specific forms or reports should use a different database than that, dbIndex might change indexData temporarily.

dbIndex: A Report (field dbIndex) as well as a Form (field parameter.`dbIndex`) can operate on a specific database.

A Form will:

  • load the form-definition from indexQfq (table Form and FormElement),
  • loads and save data from/in indexData (config.qfq.php) / dbIndex (form.parameter.dbIndex),
  • retrieve extra information via dbIndexExtra - this is useful to offer information from a database and save them in a different one.

The simplest setup, QFQ system & data in the same database, needs no indexQfq / indexData definition in configuration or one or both of them set to '1'

To separate QFQ system and data, indexQfq and indexData will have different indexes.

A Multi Database setup might be useful for:

  • several independent Typo3 QFQ installations (each have it's own form repository) and one central database, or
  • one QFQ installation which should display / load /save records from different databases, or
  • a combination of the above two.

Note:

  • Option 'A' is the most simple and commonly used.
  • Option 'B' separate the T3 and QFQ databases on two database hosts.
  • Option 'C' is like 'B' but with a shared 'QFQ data'-database between three 'Typo3 / QFQ' instances.
  • Further variants are possible.
  Domain Website Host T3 QFQ system QFQ data
A standalone.edu 'w' <dbHost>, <dbname>_t3, <dbnameSingle>_db
B appB1.edu 'wApp' <dbHostApp>, <dbnameB1>_t3 <dbHostB1>, <dbnameApp>_db
B appB2.edu 'wApp' <dbHostApp>, <dbnameB2>_t3 <dbHostB2>, <dbnameApp>_db
C appC1.edu 'wAppC' <dbHostAppC>, <dbnameC1>_t3 <dbHostC>, <dbnameSysC1>_db <dbHostData>_db, <dbNameData>_db
C appC2.edu 'wAppC' <dbHostAppC>, <dbnameC2>_t3 <dbHostC>, <dbnameSysC2>_db <dbHostData>_db, <dbNameData>_db
C appC3.edu 'wAppC3' <dbHostAppC3>, <dbnameC3>_t3 <dbHostC3>, <dbnameSysC3>_db <dbHostData>_db, <dbNameData>_db

In config-qfq-php mutliple database credentials can be prepared. Mandatory is at least one credential setup like DB_1_USER, DB_1_SERVER, DB_1_PASSWORD, DB_1_NAME. The number '1' indicates the dbIndex. Increment the number to specify further database credential setups.

Typically the credentials for DB_1 also have access to the T3 database.

Different QFQ versions, shared database

When using different QFQ versions and a shared 'QFQ data'-database, there is some risk of conflicting 'QFQ system' tables. Best is to always use the same QFQ version on all instances or use a Multi Database setup.

Debug

QFQ Log

Setup in configuration

  • qfqLog
    • Filename where to log QFQ debug and error messages.
    • File is relative to the <site path> or absolute (starting with '/').
    • Content: error situations of QFQ and debug, if enabled.

All non SQL related information will be logged to QFQ log file.

SQL Log

Setup in configuration

  • sqlLog
    • Filename where to log SQL queries and statistical data.
    • File is relative to the <site path> or absolute (starting with '/').
    • Content: SQL queries and timestamp, formName/formId, fe_user, success, affected rows, newly created record id's and accessed from IP.
    • The global setting can be overwritten by defining sqlLog inside of a QFQ tt-content record.
  • sqlLogMode: all|modify|error|none

    • all: logs every SQL statement.
    • modify: logs only statements who might potentially change data.
    • error: logs only queries which generate SQL errors.
    • none: no query logging at all.
    • The global setting can be overwritten by defining sqlLogMode inside of a QFQ tt-content record.
  • showDebugInfo = [yes|no|auto],[download]

    If active, displays additional information in the Frontend (FE). This is typically helpful during development.

    • yes:

      • Form:

        • For every internal link/button, show tooltips with decoded SIP on mouseover.
        • Shows an 'Edit form'-button (wrench symbol) on a form. The link points to the T3 page with the :ref:`form-editor`.
      • Report: Will be configured per tt-content record.

        debugShowBodyText = 1

    • no: No debug info.

    • auto: Depending if there is a Typo3 BE session, set internally:

      • showDebugInfo = yes (BE session exist)
      • showDebugInfo = no (no BE session)
    • download:

      • During a download (especially by using wkhtml), temporary files are not deleted automatically. Also the wkhtmltopdf and pdfunite command lines will be logged to QFQ_LOG. Use this only to debug problems on download.

MAIL Log

Setup in configuration

  • mailLog
    • File which sendEmail logs sending mail.
    • File is relative to the <site path> or absolute (starting with '/').

Mail Log page (Table MailLog)

For debugging purposes you may like to add a Mail Log page in the frontend. The following QFQ code could be used for that purpose (put it in a QFQ PageContent element):

# Page parameters
1.sql = SELECT @grId := '{{grId:C0:digit}}' AS _grId
2.sql = SELECT @summary := IF('{{summary:CE:alnumx}}' = 'true', 'true', 'false') AS _s

# Filters
10 {
  sql = SELECT "'", gr.id, IF(gr.id = @grId, "' selected>", "'>"), gr.value, ' (Id: ', gr.id, ')'
           FROM gGroup AS gr
           INNER JOIN MailLog AS ml ON ml.grId = gr.id
           GROUP BY gr.id
  head = <form onchange='this.submit();' class='form-inline'><input type='hidden' name='id' value='{{pageAlias:T0}}'>
           Filter By Group: <select name='grId' class='form-control'><option value=''></option>
  rbeg = <option value=
  rend = </option>
  tail = </select>
}

20 {
  sql = SELECT IF(@summary = 'true', ' checked', '')
  head = <div class='checkbox'><label><input type='checkbox' name='summary' value='true'
  tail = >Summary</label></div></form>
}

# Mail Log
50 {
  sql = SELECT id, '</td><td>', grId, '</td><td>', xId, '</td><td>',
            REPLACE(receiver, ',', '<br>'), '</td><td>', REPLACE(sender, ',', '<br>'), '</td><td>',
            DATE_FORMAT(modified, '%d.%m.%Y<br>%H:%i:%s'), '</td><td style="word-break:break-word;">',
            CONCAT('<b>', subject, '</b><br>', IF(@summary = 'true', CONCAT(SUBSTR(body, 1,
                   LEAST(IF(INSTR(body, '\n') = 0, 50, INSTR(body, '\n')), IF(INSTR(body, '<br>') = 0, 50,
                   INSTR(body, '<br>')))-1), ' ...'), CONCAT('<br>', REPLACE(body, '\n', '<br>'))) )
          FROM MailLog WHERE (grId = @grId OR @grId = 0)
          ORDER BY modified DESC
          LIMIT 100
  head = <table class="table table-condensed table-hover"><tr>
             <th>Id</th><th>grId</th><th>xId</th><th>To</th><th>From</th><th>Date</th><th>E-Mail</th></tr>
  tail = </table>
  rbeg = <tr><td>
  rend = </td></tr>
}

Redirect all mail to (catch all)

Setup in configuration

  • redirectAllMailTo=john@doe.com
    • During the development, it might be helpful to configure a 'catch all' email address, which QFQ uses as the final receiver instead of the original intended one.
    • The setting will:
      • Replace the 'To' with the configured one.
      • Clear 'CC' and 'Bcc'
      • Write a note and the original configured receiver at the top of the email body.

Show log files realtime

Display QFQ log files in realtime.

The following QFQ code could be used for that purpose (put it in a QFQ PageContent element):

#
# {{logfile:SU}}
#

# Show buttons to select log file.
10 {
  sql = SELECT '{{logfile:SU:::sql.log}}' AS '_=logfile'
  head = <p>
  tail = </p>

  20.sql = SELECT CONCAT('p:{{pageAlias:T}}&logfile=sql.log|t:sql.log|b:', IF('{{logfile:R}}'='sql.log','primary','')) AS _page, ' ',
                  CONCAT('p:{{pageAlias:T}}&logfile=qfq.log|t:qfq.log|b:', IF('{{logfile:R}}'='qfq.log','primary','')) AS _page, ' ',
                  CONCAT('p:{{pageAlias:T}}&logfile=mail.log|t:mail.log|b:', IF('{{logfile:R}}'='mail.log','primary','')) AS _page
}

# Show selected log file.
100 {
  sql = SELECT 'file:fileadmin/protected/log/{{logfile:R}}' AS _monitor
  head = <pre id="monitor-1">Please wait</pre>
}

Form Submit Log page

For debugging purposes you may like to add a Form Submit Log page in the frontend. The following QFQ code could be used for that purpose (put it in a QFQ PageContent element):

# Filters
20.shead = <form onchange='this.submit()' class='form-inline'><input type='hidden' name='id' value='{{pageAlias:T0}}'>
20 {
  sql = SELECT "'", id, IF(id = '{{formId:SC0}}', "' selected>", "'>"), name
        FROM Form ORDER BY name
  head = <label for='formId'>Form:</label> <select name='formId' id='formId' class='form-control'><option value=0></option>
  tail = </select>
  rbeg = <option value=
  rend = </option>
}
30 {
  sql = SELECT feUser, IF(feUser = '{{feUser:SCE:alnumx}}', "' selected>", "'>"), feUser
        FROM FormSubmitLog GROUP BY feUser ORDER BY feUser
  head = <label for='feUser'>FE User:</label> <select name='feUser' id='feUser' class='form-control'><option value=''></option>
  tail = </select>
  rbeg = <option value='
  rend = </option>
}
30.stail = </form>

# Show Log
50 {
  sql = SELECT l.id,
      CONCAT('<b>Form</b>: ', f.name,
        '<br><b>Record Id</b>: ', l.recordId,
        '<br><b>Fe User</b>: ', l.feUser,
        '<br><b>Date</b>: ', l.created,
        '<br><b>Page Id</b>: ', l.pageId,
        '<br><b>Session Id</b>: ', l.sessionId,
        '<br><b>IP Address</b>: ', l.clientIp,
        '<br><b>User Agent</b>: ', l.userAgent,
        '<br><b>SIP Data</b>: <div style="margin-left:20px;">', "<script>var data = JSON.parse('", l.sipData,
          "'); for (var key in data) {
          document.write('<b>' + key + '</b>: ' + data[key] + '<br>'); }</script>", '</div>'),
      CONCAT("<script>var data = JSON.parse('", l.formData,
        "'); for (var key in data) {
          document.write('<b>' + key + '</b>: ' + data[key] + '<br>'); }</script>")
      FROM FormSubmitLog AS l
      LEFT JOIN Form AS f ON f.id = l.formId
      WHERE (l.formId = '{{formId:SC0}}' OR '{{formId:SC0}}' = 0)
        AND (l.feUser = '{{feUser:SCE:alnumx}}' OR '{{feUser:SCE:alnumx}}' = '')
      ORDER BY l.created DESC LIMIT 100
  head = <table class="table table-hover">
         <tr><th>Id</th><th style="min-width:250px;">Environment</th><th>Submitted Data</th>
  tail = </table>
  rbeg = <tr>
  renr = </tr>
  fbeg = <td>
  fend = </td>
}

Variable

Variables in QFQ are surrounded by double curly braces. Four different types of variable substitution functionality is provided. Access to:

Some examples, including nesting:

# Store
#---------------------------------------------
{{r}}
{{index:FS}}
{{name:FS:alnumx:s:my default}}

# SQL
#---------------------------------------------
{{SELECT name FROM person WHERE id=1234}}

# Row columns
#---------------------------------------------
{{10.pId}}
{{10.20.pId}}

# Nesting
#---------------------------------------------
{{SELECT name FROM person WHERE id={{r}} }}
{{SELECT name FROM person WHERE id={{key1:C:alnumx}} }} # explained below
{{SELECT name FROM person WHERE id={{SELECT id FROM pf LIMIT 1}} }} # it's more efficient to use only one query

# Link Columns
{{p:form=Person&r=1|t:Edit Person|E|s AS link}}

Leading and trailing spaces inside curly braces are removed.

  • {{ SELECT "Hello World" }} becomes {{SELECT "Hello World"}}
  • {{ varname }} becomes {{varname}}

Types

Store variables

Syntax: {{VarName[:<store(s)[:<sanitize class>[:<escape>[:<default value>[:type violate message]]]]]}}

See also:

  • Example:

    {{pId}}
    {{pId:FSE}}
    {{pId:FSE:digit}}
    {{pId::digit}}
    {{name:FSE:alnumx:m}}
    {{name:::m}}
    {{name:FSE:alnumx:m:John Doe}}
    {{name::::John Doe}}
    {{name:FSE:alnumx:m:John Doe:forbidden characters}}
    {{name:::::forbidden characters}}
  • Zero or more stores might be specified to be searched for the given VarName.

  • If no store is specified, the default for the searched stores are: FSRVD (=FORM > SIP > RECORD > VARS > DEFAULT).

  • If the VarName is not found in one store, the next store is searched, up to the last specified store.

  • If the VarName is not found and a default value is given, the default is returned.

  • If no value is found, nothing is replaced - the string '{{<VarName>}}' remains.

  • If anywhere along the line an empty string is found, this is a value: therefore, the search will stop.

Sanitize class

Values in STORE_CLIENT C (Client=Browser) and STORE_FORM F (Form, HTTP 'post') are checked against a sanitize class. Values from other stores are not checked against any sanitize class, even if a sanitize class is specified.

  • Variables get by default the sanitize class defined in the corresponding FormElement. If not defined, the default class is 'digit'.
  • A default sanitize class can be overwritten by individual definition: {{a:C:alnumx}}
  • If a value violates the specific sanitize class, see variable-type-message-violate for default or customized message. By default the value becomes !!<name of sanitize class>!!. E.g. !!digit!!.

For QFQ variables and FormElements:

Name Form Query Pattern
alnumx Form Query [A-Za-z][0-9]@-_.,;: /() ÀÈÌÒÙàèìòùÁÉÍÓÚÝáéíóúýÂÊÎÔÛâêîôûÃÑÕãñõÄËÏÖÜŸäëïöüÿç
digit Form Query [0-9]
numerical Form Query [0-9.-+]
allbut Form Query All characters allowed, but not [ ] { } % #. The used regexp: '^[^[]{}%\#]+$',
all Form Query no sanitizing

Only in FormElement:

Rules for CheckType Auto (by priority):

  • TypeAheadSQL or TypeAheadLDAP defined: alnumx
  • Table definition * integer type: digit * floating point number: numerical
  • FE Type * 'password', 'note': all * 'editor', 'text' and encode = 'specialchar': all
  • None of the above: alnumx

Escape/Action class

The following escape & action types are available:

  • The escape/action class is defined by the fourth parameter of the variable. E.g.: {{name:FE:alnumx:m}} (m = mysql).
  • It's possible to combine multiple escape/action classes, they will be processed in the order given. E.g. {{name:FE:alnumx:Ls}} (L, s).
  • Escaping is typically necessary for all user supplied content, especially if they are processed via SQL or LDAP queries.
  • Be careful when escaping nested variables. Best is to escape only the most outer variable.
  • In configuration a global escapeTypeDefault can be defined. The configured escape/action class applies to all substituted variables, who do not contain a specific escape/action class.
  • Additionally a defaultEscapeType can be defined per Form (separate field in the Form editor). This overwrites the global definition of configuration. By default, every Form.defaultEscapeType = 'c' (=config), which means the setting in configuration.
  • To suppress an escape type, define the escape type = '-' on the specific variable. E.g.: {{name:FE:alnumx:-}}.

Escape

To 'escape' a character typically means: a character, which have a special meaning/function, should not treated as a special character. E.g. a string is surrounded by single ticks '. If such a string should contain the same single tick inside, the inside single tick has to be escaped - if not, the string end's at the second tick, not the third. This is typically done by a backlash: \

QFQ offers different ways of escaping. Which of them to use, depends on the situation.

Especially variables used in SQL statements might cause trouble when using: NUL (ASCII 0), \n, \r, \, ', ", or Control-Z.

Action

  • password - 'p': transforms the value of the variable into a Typo3 salted password hash. The hash function is the one used by Typo3 to encrypt and salt a password. This is useful to manipulate FE user passwords via QFQ. See setFeUserPassword
  • stop replace - 'S': typically QFQ will replace nested variables as long as there are variables to replace. This options stops this
  • exception - 'X': If a variable is not found in any given store, it's replace by a default value or an error message. In special situation it might be useful to do a full stop on all current actions (no further procession). A custom message can be defined via: variable-type-message-violate.
  • wipe - 'w': In special cases it might be useful to get a value via SIP only one time and after retrieving the value it will be deleted in STORE SIP . Further access to the variable will return 'variable undefined'. At time of writing only the STORE SIP supports the feature 'wipe'. This is useful to suppress any repeating events by using the browser history. The following example will send a mail only the first when it is called with a given SIP:

    10.sql = SELECT '...' AS _sendmail FROM Person AS p WHERE '{{action:S::w}}'='send' AND p.id={{pId:S}}

Default

  • Any string can be given to define a default value.
  • If a default value is given, it makes no sense to define more than one store: with a default value given, only the first store is considered.
  • If the default value contains a ':', that one needs to be escaped by \

Type message violate

If a value violates the sanitize class, the following actions are possible:

  • 'c' - The violated class will be set as content, surrounded by '!!'. E.g. '!!digit!!'. This is the default.
  • 'e' - Instead of the value an empty string will be set as content.
  • '0' - Instead of the value the string '0' will be set as content.
  • 'custom text ...' - Instead of the value, the custom text will be set as content. If the text contains a ':', that one needs to be escaped by \ . Check variable-escape qualifier 'C' to let QFQ do the colon escaping.

SQL variables

  • The detection of an SQL command is case insensitive.
  • Leading whitespace will be skipped.
  • The following commands are interpreted as SQL commands:
    • SELECT
    • INSERT, UPDATE, DELETE, REPLACE, TRUNCATE
    • SHOW, DESCRIBE, EXPLAIN, SET
  • An SQL Statement might contain variables, including additional SQL statements. Inner SQL queries will be executed first.
  • All variables will be substituted one by one from inner to outer.
  • The number of variables inside an input field or an SQL statement is not limited.
Result: string

A result of an SQL statement will be imploded over all: concat all columns of a row, concat all rows - there is no glue string.

Result: row

A few functions needs more than a returned string, instead separate columns are necessary. To indicate an array result, specify those with an '!':

{{!SELECT ...}}

This manual will specify the individual QFQ elements, who needs an array instead of a string. It's an error to return a string where an array is needed and vice versa.

Database index

To access different databases in a multi-database setup, the database index can be specified after the opening curly braces.

{{[1]SELECT ... }}

For using the indexData and indexQfq (configuration), it's a good practice to specify the variable name instead of the numeric index.

{{[{{indexData:Y}}]SELECT ...}}

If no dbIndex is given, {{indexData:Y}} is used.

Example
{{SELECT id, name FROM Person}}
{{SELECT id, name, IF({{feUser:T0}}=0,'Yes','No')  FROM Person WHERE id={{r:S}} }}
{{SELECT id, city FROM Address AS adr WHERE adr.accId={{SELECT id FROM Account AS acc WHERE acc.name={{feUser:T0}} }} }}
{{!SELECT id, name FROM Person}}
{{[2]SELECT id, name FROM Form}}
{{[{{indexQfq:Y}}]SELECT id, name FROM Form}}
Row column variables

Syntax: {{<level>.<column>}}

Only used in report to access outer columns. See access-column-values and syntax-of-report.

There might be name conflicts between VarName / SQL keywords and <line identifier>. QFQ checks first for '<level>', than for 'SQL keywords' and than for 'VarNames' in stores.

All types might be nested with each other. There is no limit of nesting variables.

Very specific: Also, it's possible that the content of a variable is again (including curly braces) a variable - this is sometimes used in text templates, where the template is retrieved from a record and specific locations in the text will be (automatically by QFQ) replaced by values from other sources.

General note: using this type of variables is only the second choice. First choice is {{column:R}} (see access-column-values) - using the STORE_RECORD is more portable cause no renumbering is needed if the level keys change.

Link column variables

These variables return a link, completely rendered in HTML. The syntax and all features of column-link are available. The following code will render a 'new person' button:

{{p:form&form=Person|s|N|t:new person AS link}}

For better reading, the format string might be wrapped in single or double quotes (this is optional):

{{"p:form&form=Person|s|N|t:new person" AS link}}

These variables are especially helpful in:

  • report, to create create links or buttons outside of an SQL statement. E.g. in head, rbeg, ...
  • form, to create links and buttons in labels or notes.

Security

All values passed to QFQ will be:

  • Checked against max. length and allowed content, on the client and on the server side. On the server side, the check happens before any further processing. The 'length' and 'allowed' content is specified per FormElement. 'digit' or 'alnumx' is the default. Violating the rules will stop the 'save record' process (Form) or result in an empty value (Report). If a variable is not replaced, check the default sanitize class.
  • Only elements defined in the Form definition or requested by Report will be processed.
  • UTF8 normalized (normalizer::normalize) to unify different ways of composing characters. It's more a database interest, to work with unified data.

SQL statements are typically fired as prepared statements with separated variables. Further custom SQL statements will be defined by the webmaster - those do not use prepared statements and might be affected by SQL injection. To prevent SQL injection, every variable is by default escaped with mysqli::real_escape_string.

QFQ notice:

  • Variables passed by the client (=Browser) are untrusted and use the default sanitize class 'digit' (if nothing else is specified). If alpha characters are submitted, the content violates digit and becomes therefore !!<name of sanitize class>!! - there is no error message. Best is to always use SIP (value is trustful) or at least digits for GET (=client) parameter (user might change those and therefore those are not trustful).

Get Parameter

QFQ security restriction:

  • GET parameter might contain urlencoded content (%xx). Therefore all GET parameter will be processed by 'urldecode()'. As a result a text like '%nn' in GET variables will always be decoded. It's not possible to transfer '%nn' itself.
  • GET values are limited to securityGetMaxLength (extension-manager-qfq-configuration) chars - any violation will stop QFQ. Individual exceptions are defined via ExceptionMaxLength.
  • GET parameter 'type' and 'L' might affected by (T3, configuration dependent) cache poisoning. If they contain non digit values, only the first character is used (if this is a digit) or completely cleaned (else).

Post Parameter

Per FormElement (HTML input) the default is to htmlspecialchars() the input. This means &<>'" will be encoded as htmlentity and saved as a htmlentity in the database. In case any of these characters (e.g. for HTML tags) are required, the encoding can be disabled per FormElement: encode=none (default is specialchar).

During Form load, htmlentities are decoded again.

$_SERVER

All $_SERVER vars are htmlentities encoded (all, not only specialchars!) .

Honeypot

Every QFQ Form contains 'honeypot'-HTML input elements (HTML: hidden & readonly). Which of them to use is configured in configuration (default: 'username', 'password' and 'email'). On every start of QFQ (form, report, save, ...), these variables are tested if they are non-empty. In such a case a probably malicious bot has send the request and the request will not be processed.

If any of the default configured variable names are needed (which will never be the case for QFQ), an explicit variable name list have to be configured in configuration.

QFQ security restriction:

  • The honeypot variables can't be used in GET or POST as regular HTML input elements - any values of them will terminate QFQ.

Violation

On any violation, QFQ will sleep securityAttackDelaySeconds (configuration) and than exit the running PHP process. A detected attack leads to a complete white (=empty) page.

If securityShowMessage: true (configuration), at least a message is displayed after the delay.

Client Parameter via SIP

Links with URL parameters, targeting to the local website, are typically SIP encoded. Instead of transferring the parameter as part of the URL, only one unique GET parameter 's' is appended at the link. The parameter 's' is unique (equal to a timestamp) for the user. Assigned variables are stored as a part of the PHP user session on the server. Two users might have the same value of parameter 's', but the content is completely independent.

Variables needed by Typo3 remains on the link and are not 'sip-encoded'.

Secure direct file access

If the application uploads files, mostly it's not necessary and often a security issue, to offer a direct download of the uploaded files. Best is to create a directory, e.g. <site path>/fileadmin/protected and deny direct access via webbrowser to it. E.g. for Apache set a rule:

<Directory "/var/www/html/fileadmin/protected">
    Require all denied
</Directory>

If you only have access to .htaccess, create a file <site path>/fileadmin/protected/.htaccess with:

<IfModule mod_authz_core.c>
     Require all denied
</IfModule>

Important

All QFQ uploads should save files only in/below such a protected directory.

To offer download of those files, use the reserved column name '_download' (see download) or variants.

Important

To protect the installation against executing of uploaded malicious script code, disable PHP for the final upload directory. E.g. fileadmin (Apache):

<Directory "/var/www/html/fileadmin">
  php_admin_flag engine Off
</Directory>

This is in general a good security improvement for directories with user supplied content.

File upload

By default the mime type of every uploaded file is checked against a white list of allowed mime types. The mime type of a file can be (easily) faked by an attacker. This check is good to handle regular user file upload for specific file types but won't help to prevent attacks against uploading and executing malicious code.

Instead prohibit the execution of user contributed files by the webserver config (SecureDirectFileAccess).

Typo3 Setup - best practice

  • Activate notification emails for every BE login (if there are only few BE users). In case the backend has been hacked, unusual login's (time or username) will appear:

    [BE][warning_email_addr] = <your email>
    [BE][warning_mode] = 1

Store

Only variables that are known in a specified store can be substituted.

Name Description Content
B :ref:`STORE_BEFORE`: Record - the current record loaded in the form before any update. All columns of the current record from the current table. See STORE_BEFORE.
C :ref:`STORE_CLIENT`: POST variable, if not found: GET variable. Parameter sent from the Client (=Browser). See STORE_CLIENT.
D Default values column : The table.column specified default value.  
E Empty - always an empty string, might be helpful if a variable is empty or undefined and will be used in an SQL statement. Any key
F :ref:`STORE_FORM`: data not saved in database yet. All native FormElements. Recent values from the Browser. See: STORE_FORM
L :ref:`STORE_LDAP`: Will be filled on demand during processing of a FormElement. Custom specified list of LDAP attributes. See STORE_LDAP.
M Column type: The table.column specified type.  
P Parent record. E.g.: on multi & copy forms the current record of the outer query. All columns of the MultiSQL Statement from the table for the current row
R :ref:`STORE_RECORD`: Record - the current record loaded in the form. All columns of the current record from the current table. See STORE_RECORD.
S :ref:`STORE_SIP`: Client parameter 's' will indicate the current SIP, which will be loaded from the SESSION repo to the SIP-Store. sip, r (recordId), form. See STORE_SIP.
T :ref:`STORE_TYPO3`: a) Bodytext (ttcontent record), b) Typo3 internal variables. See Typo3 tt_content record configuration. See STORE_TYPO3.
U :ref:`STORE_USER`: per user variables, valid as long as the browser session lives. Set via report: '...' AS '_=<var name>' See: STORE_USER, store_user_examples
V :ref:`STORE_VARS`: Generic variables. See STORE_VARS.
Y :ref:`STORE_SYSTEM`: a) Database, b) helper vars for logging/debugging: SYSTEM_SQL_RAW ... SYSTEM_FORM_ELEMENT_COLUMN, c) Any custom fields: CONTACT, HELP, ... See STORE_SYSTEM.
0 Zero - always value: 0, might be helpful if a variable is empty or undefined and will be used in an SQL statement. Any key
  • Default <prio>: FSRVD - Form / SIP / Record / Vars / Table definition.
  • Hint: Preferable, parameter should be submitted by SIP, not by Client (=URL).
    • Warning: Data submitted via 'Client' can be easily spoofed and altered.
    • Best: Data submitted via SIP never leaves the server, cannot be spoofed or altered by the user.
    • SIPs can _only_ be defined by using Report. Inside of Report use columns 'Link' (with attribute 's'), 'page?' or 'Page?'.

Store: FORM - F

  • Sanitized: yes
  • Represents the values in the form, typically before saving them.
  • Used for:
    • FormElements which will be rerendered, after a parent FormElement has been changed by the user.
    • FormElement actions, before saving the form.
    • Values will be sanitized by the class configured in corresponding the FormElement. By default, the sanitize class is alnumx.
Name Explanation
<FormElement name> Name of native FormElement. To get, exactly and only, the specified FormElement (for 'pId'): {{pId:F}}

Store: SIP - S

  • Sanitized: no
  • Filled automatically by creating links. E.g.:
    • in Report by using _page? or _link (with active 's')
    • in Form by using subrecords: 'new', 'edit', 'delete' links (system) or by column type _page?, _link.
Name Explanation
sip 13 char uniqid
r current record id
form current form name
table current table name
urlparam all non Typo3 parameter in one string
<user defined> additional user defined link parameter

Store: RECORD - R

  • Sanitized: no
  • Form: Current record.
  • Report: See access-column-values
  • If r=0, all values are empty.
Name Type | Explanation
<column name> Form Name of a column of the primary table (as defined in the current form). Example: {{pId:R}}
<column name> Report Name of a column of a previous fired SQL query. Example: {{pId:R}}
&<column name> Report (final) Name of a column of a previous fired SQL query, typically used by columns with a special-column-names. Final value. Example: '{{link:R}}' returns 'p:home&form=Person|s|b:success|t:Edit'. Whereas '{{&link:R}}' returns '<span class="btn btn-success"><a href="?home&s=badcaffee1234">Edit</a></span>

Store: BEFORE - B

  • Sanitized: no
  • Current record loaded in Form without any modification.
  • If r=0, all values are empty.

This store is handy to compare new and old values of a form.

Name Explanation
<column name> Name of a column of the primary table (as defined in the current form). To get, exactly and only, the specified form FormElement: {{pId:B}}

Store: CLIENT - C

  • Sanitized: yes
Name Explanation
s =SIP
r record id. Only if specified as GET parameter - typically stored in SIP (=STORE_SIP)
form Name of form to load. Only if specified as GET parameter - typically stored in SIP (=STORE_SIP)
HTTP_HOST current HTTP HOST
REMOTE_ADDR Client IP address
'$_SERVER[*]' All other variables accessible by $_SERVER[]. Only the often used have a pre-defined sanitize class.
Authorization Value of the HTTP Header 'Authorization'. This is typically not set. Mostly used for authentication of REST requests

Store: TYPO3 (Bodytext) - T

  • Sanitized: no
Name Explanation Note
form
Formname defined in ttcontent record bodytext
* Fix. E.g. form = person
* via SIP. E.g. form = {{form:SE}}
see note
pageId Record id of current Typo3 page see note
pageAlias Alias of current Typo3 page. If empty, take pageId. see note
pageTitle Title of current Typo3 page see note
pageType Current selected page type (typically URL parameter 'type') see note
pageLanguage Current selected page language (typically URL parameter 'L') see note
ttcontentUid Record id of current Typo3 content element see note
feUser Logged in Typo3 FE User  
feUserUid Logged in Typo3 FE User uid  
feUserGroup FE groups of logged in Typo3 FE User  
beUser Logged in Typo3 BE User  
beUserLoggedIn 'yes' | 'no' - Status if a BE-User is logged in  
  • note: not available:
    • in :ref:`dynamic-update` or
    • by FormElement class 'action' with type 'beforeSave', 'afterSave', 'beforeDelete', 'afterDelete'.

Store: VARS - V

  • Sanitized: no
Name Explanation
random Random string with length of 32 alphanum chars (lower & upper case). This is variable is always filled.
slaveId see `slaveId`_
  • FormElement 'upload':
Name Explanation
filename Original filename of an uploaded file via an 'upload'-FormElement. Valid only during processing of the current 'upload'-formElement.
filenameOnly Like filename, but without path.
filenameBase Like filename, but without an optional extension. E.g. filename='image.png' comes to filenameBase='image'
filenameExt Like filename, but only the optional extension. E.g. filename='image.png' comes to filenameExt='png'
fileDestination Destination (path & filename) for an uploaded file. Defined in an 'upload'-FormElement.parameter. Valid: same as 'filename'.
fileSize Size of the uploaded file.
mimeType Mime type of the uploaded file.

The directive fillStoreVar will fill the store VARS with custom values. Existing Store VARS values will be merged together with them. E.g.:

fillStoreVar = {{! SELECT p.name, p.email FROM Person AS p WHERE p.id={{pId:S}} }}
  • After filling the store, the values can be retrieved via {{name:V}} and {{email:V}}.
  • Be careful by specifying general purpose variables like id, r, pageId and so on. This might conflict with existing variables.
  • fillStoreVar can be used in form-parameter and fe-parameter-attributes

Store: LDAP - L

  • Sanitized: yes
  • See also :ref:`LDAP`:
Name Explanation
<custom defined> See ldapAttributes

Store: SYSTEM - Y

  • Sanitized: no

See configuration for a list of all settings.

Store: USER - U

  • Sanitized: no

At start of a new browser session (=user calls the website the first time or was logged out before) the store is empty. As soon as a value is set in the store, it remains as long as the browser session is alive (=until user logs out).

Values can be set via report '... AS "_=<var name>"'

See also: store_user_examples

LDAP

A form can retrieve data from LDAP server(s) to display or to save them. Configuration options for LDAP will be specified in the parameter field of the Form and/or the FormElement. Definitions of the FormElement will overwrite definitions of the Form. One LDAP Server can be configured per FormElement. Multiple FormElements might use individual LDAP Server configurations.

To decide which Parameter should be placed on Form.parameter and which on FormElement.parameter: If LDAP access is ...

  • only necessary in one FormElement, most useful setup is to specify all values in that specific FormElement,
  • needed on multiple FormElement*s (of the same *Form, e.g. one input with typeAhead, one note and one action), it's more efficient to specify the base parameter ldapServer, ldapBaseDn in Form.parameter and the rest on the current FormElement.
Parameter Example Description Form FormElement Used for
ldapServer directory.example.com Hostname. For LDAPS: ldaps://directory.example.com:636 x x TA, FSL
ldapBaseDn ou=Addressbook,dc=example,dc=com Base DN to start the search x x TA, FSL
ldapAttributes cn, email List of attributes to save in STORE_LDAP x x FSL
ldapSearch (mail=john.doe@example.com) Regular LDAP search expression x x FSL
ldapTimeLimit 3 (default) Maximum time to wait for an answer of the LDAP Server x x TA, FSL
ldapUseBindCredentials ldapUseBindCredentials=1 Use LDAP_1_* credentials from config-qfq-php for ldap_bind() x x TA, FSL
typeAheadLdap   Enable LDAP as 'Typeahead' data source   x TA
typeAheadLdapSearch (|(cn=*?*)(mail=*?*)) Regular LDAP search expression, returns upto typeAheadLimit x x TA
typeAheadLdapSearchPrefetch (mail=?) Regular LDAP search expression, typically return one record x x TA
typeAheadLdapSearchPerToken  
Split search value in token and OR-combine every search with
the individual tokens
x x TA
typeAheadLdapValuePrintf '%s / %s', cn, mail Custom format to display attributes, as value x x TA
typeAheadLdapIdPrintf '%s', mail Custom format to display attributes, as id x x TA
typeAheadLimit 20 (default) Result will be limited to this number of entries x x TA
typeAheadPedantic typeAheadPedantic=0 Turn off 'pedantic' mode - allow any values (see below) x x TA
typeAheadMinLength 2 (default) Minimum number of characters before starting the search x x TA
fillStoreLdap   Activate Fill STORE LDAP with the first retrieved record   x FSL