Manual.rst 454 KB
Newer Older
1
.. ==================================================
2
.. Header hierarchy
3
4
5
6
7
8
9
10
.. ==
..  --
..   ^^
..    ''
..     ;;
..      ,,
..
.. --------------------------------------------------
11
.. Best Practice T3 reST  https://docs.typo3.org/typo3cms/drafts/github/xperseguers/RstPrimer/
12
.. External Links: `Bootstrap <http://getbootstrap.com/>`_:
13
.. Add Images: https://wiki.typo3.org/ReST_Syntax#Images ...
14
..
15
16
17
.. Admonitions (https://docs.typo3.org/typo3cms/drafts/github/xperseguers/RstPrimer/#admonitions)
..       .. note::    .. important::     .. tip::      .. warning::
..
18
19
20
21
22
.. -*- coding: utf-8 -*- with BOM.


.. include:: Includes.txt

23
24
25
26
27
.. _general:

General
=======

28
* Project homepage: https://qfq.io
Carsten  Rose's avatar
Carsten Rose committed
29
* Latest releases: https://qfq.io/download
30
* Development: https://git.math.uzh.ch/typo3/qfq
31
* Slack: https://qfq-io.slack.com
32
33
34
35
36
37

.. _installation:

Installation
============

38
The following features are only tested / supported on linux hosts:
39

40
* General: QFQ is coded to run on Linux hosts, preferable on Debian derivates like Ubuntu.
41
42
43
44
* HTML to PDF conversion - command `wkhtmltopdf`.
* Concatenation of PDF files - command `pdftk`.
* Mime type detection for uploads - command `file`.

45

46
47
.. _`preparation`:

48
49
50
51
52
53
Preparation
-----------

Report & Form
^^^^^^^^^^^^^

54
To normalize UTF8 input, *php-intl* package is needed by
55
56
57

* normalizer::normalize()

58
For the `download`_ function, the programs `pdftk` and `file` are necessary to concatenate PDF files.
59

60
Preparation for Ubuntu::
61

62
  sudo apt install php-intl
63
64
  sudo apt install pdftk libxrender1 file pdf2svg  # for file upload, PDF and 'HTML to PDF' (wkhtmltopdf), PDF split
  sudo apt install inkscape imagemagick            # to render thumbnails
65

66
.. _wkhtml:
67

68
69
wkhtmltopdf
^^^^^^^^^^^
70

71
72
`wkhtmltopdf <http://wkhtmltopdf.org/>`_ will be used by QFQ to offer 'website print' and 'HTML to PDF' conversion.
The program is not included in QFQ and has to be manually installed.
73

74
75
76
77
* The Ubuntu package `wkhtmltopdf` needs a running Xserver - this does not work on a headless webserver.

  * Best is to install the QT version from the named website above.
  * In case of trouble with wkhtmltopdf, also install 'libxrender1'.
78
79
  * The current version 0.12.4 might have trouble with https connections. Version 0.12.5-dev (github master branch)
    seems more reliable. Please contact the QFQ authors if you need a compiled Ubuntu version of wkhtmltopdf.
80

81
In `configuration`_ specify the::
82

83
84
    config.cmdWkhtmltopdf=/opt/wkhtmltox/bin/wkhtmltopdf`.
    config.baseUrl=http://www.example.com/`.
85

86
87
If wkhtml has been compiled with dedicated libraries (not part of LD_LIBRARY_PATH), specify the LD_LIBRARY_PATH together
with the path-filename: ::
88
89

    cmdWkhtmltopdf=LD_LIBRARY_PATH=/opt/wkhtmltox/lib /opt/wkhtmltox/bin/wkhtmltopdf
90

91
92
.. important::

93
    To access FE_GROUP protected pages or content, it's necessary to disable the `[FE][lockIP]` check! `wkhtml`
94
    will access the Typo3 page locally (localhost) and that IP address is different from the client (=user) IP.
95
96

Configure via Typo3 Installtool `All configuration > $TYPO3_CONF_VARS['FE']`: ::
97
98
99

   [FE][lockIP] = 0

100
101
102
103
104
105
.. warning::

    ``[FE][lockIP] = 0`` disables an important anti-'session hijacking' protection. The security level of the whole installation
    will be *lowered*! Again, this is only needed if `wkhtml` needs access to FE_GROUP protected pages & content. As an
    alternative to lower the security level, create a separated page subtree which is only accessible (configured via
    Typoscript) from specific IPs **or** if a FE-User is logged in.
106

107
108
If there are problems with converting/downloading FE_GROUP protected pages, check `SHOW_DEBUG_INFO = download` to debug.

109
110
111
112
.. note::

    Converting HTML to PDF gives no error message but RC=-1? Check carefully all includes of CSS, JS, images
    and so on! Typically some of them fails to load and wkhtml stops running!
113
114


115
116
117
118
119
120
121
122
123
HTML to PDF conversion
''''''''''''''''''''''

`wkhtmltopdf` converts a website (local or remote) to a (multi)-page PDF file. It's mainly used in `download`_.

Print
'''''

Different browser prints the same page in different variations. To prevent this, QFQ implements a small PHP wrapper
Carsten  Rose's avatar
Carsten Rose committed
124
`print.php` with uses `wkhtmltopdf` to convert HTML to PDF.
125

126
Provide a `print this page`-link (replace 'current pageId' )::
127

128
  <a href="typo3conf/ext/qfq/Source/api/print.php?id={current pageId}">Print this page</a>
129
130
131
132
133

Any parameter specified after `print.php` will be delivered to `wkhtmltopdf` as part of the URL.

Typoscript code to implement a print link on every page::

134
135
136
137
138
  10 = TEXT
  10 {
    wrap = <a href="typo3conf/ext/qfq/Source/api/print.php?id=|&type=99"><span class="glyphicon glyphicon-print" aria-hidden="true"></span> Printview</a>
    data = page:uid
  }
139

140
141
142
143
144
145
Send Email
^^^^^^^^^^

QFQ sends mail via `sendEmail` http://caspian.dotconf.net/menu/Software/SendEmail/ - a small perl script without a central
configuration.

146
By default, `sendEmail` uses the local installed MTA, writes a logfile to `fileadmin/protected/log/mail.log` and handles attachments
147
148
149
150
151
via commandline options. A basic HTML email support is implemented.

The latest version is v1.56, which has at least one bug. That one is patched in the QFQ internal version v1.56p1 (see
QFQ GIT sources in directory 'patches/sendEmail.patch').

Carsten  Rose's avatar
Carsten Rose committed
152
Nevertheless, on latest system the TLS support is broken - please check sendEmailProblem_.
153

154
155
156
157
158
The Typo3 sendmail eco-system is not used at all by QFQ.

Thumbnail
^^^^^^^^^

159
Thumbnails will be rendered via ImageMagick (https://www.imagemagick.org/) 'convert' and 'inkscape' (https://inkscape.org).
160
161
'inkscape' is only used for '.svg' files.

162
The Typo3 graphic eco-system is not used at all by QFQ.
163

164
165
Usage: `column-thumbnail`_.

166
167
168
Setup
-----

Carsten  Rose's avatar
Carsten Rose committed
169
* Install the extension via the Extension Manager.
170
171

  * If you install the extension by manual download/upload and get an error message
Carsten  Rose's avatar
Carsten Rose committed
172
    "can't activate extension": rename the downloaded zip file to `qfq.zip` or `qfq_<version>.zip` (e.g. version: 18.12.0).
173

Carsten  Rose's avatar
Carsten Rose committed
174
  * If the Extension Manager stops after importing: check your memory limit in php.ini.
175

Carsten  Rose's avatar
Carsten Rose committed
176
* Copy/rename the file *<site path>/typo3conf/ext/qfq/config.example.qfq.php* to *<site path>/typo3conf/config.qfq.php*.
177
  Configure the necessary settings `configuration`_
Carsten  Rose's avatar
Carsten Rose committed
178
179
  The configuration file is outside of the extension directory, to not loose it during de-install and install again.
* When the QFQ Extension is called the first time on the Typo3 frontend, the file *<ext_dir>/qfq/sql/formEditor.sql* will
180
  played and fills the database with the *Form editor* records. This also happens automatically after each update of QFQ.
181
182
* Configure Typoscript to include Bootstrap, jQuery, QFQ javascript and CSS files.

183
184
185
186
.. _setup-css-js:

Setup CSS & JS
^^^^^^^^^^^^^^
187
188
::

189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
    page.meta {
        X-UA-Compatible = IE=edge
        X-UA-Compatible.attribute = http-equiv
        viewport=width=device-width, initial-scale=1
    }

    page.includeCSS {
        file01 = typo3conf/ext/qfq/Resources/Public/Css/bootstrap.min.css
        file02 = typo3conf/ext/qfq/Resources/Public/Css/bootstrap-theme.min.css
        file03 = typo3conf/ext/qfq/Resources/Public/Css/jqx.base.css
        file04 = typo3conf/ext/qfq/Resources/Public/Css/jqx.bootstrap.css
        file05 = typo3conf/ext/qfq/Resources/Public/Css/qfq-bs.css
        file06 = typo3conf/ext/qfq/Resources/Public/Css/tablesorter-bootstrap.css
    }

    page.includeJS {
205
206
207
208
209
210
211
        file01 = typo3conf/ext/qfq/Resources/Public/JavaScript/jquery.min.js
        file02 = typo3conf/ext/qfq/Resources/Public/JavaScript/bootstrap.min.js
        file03 = typo3conf/ext/qfq/Resources/Public/JavaScript/validator.min.js
        file04 = typo3conf/ext/qfq/Resources/Public/JavaScript/jqx-all.js
        file05 = typo3conf/ext/qfq/Resources/Public/JavaScript/globalize.js
        file06 = typo3conf/ext/qfq/Resources/Public/JavaScript/tinymce.min.js
        file07 = typo3conf/ext/qfq/Resources/Public/JavaScript/EventEmitter.min.js
Carsten  Rose's avatar
Carsten Rose committed
212
        file08 = typo3conf/ext/qfq/Resources/Public/JavaScript/typeahead.bundle.min.js
213
214
215
216
        file09 = typo3conf/ext/qfq/Resources/Public/JavaScript/qfq.min.js
        file10 = typo3conf/ext/qfq/Resources/Public/JavaScript/jquery.tablesorter.combined.min.js
        file11 = typo3conf/ext/qfq/Resources/Public/JavaScript/jquery.tablesorter.pager.min.js
        file12 = typo3conf/ext/qfq/Resources/Public/JavaScript/widget-columnSelector.min.js
217

218
219
220
        # Only needed in case FormElement 'annotate' is used.
        file20 = typo3conf/ext/qfq/Resources/Public/JavaScript/fabric.min.js
        file21 = typo3conf/ext/qfq/Resources/Public/JavaScript/qfq.fabric.min.js
221
    }
222

223

224
.. _form-editor:
225
226
227

FormEditor
----------
228

229
230
231
Setup a *report* to manage all *forms*:

* Create a Typo3 page.
232
* Set the 'URL Alias' to `form` (default) or the individual defined value in parameter `editFormPage` (configuration_).
233
* Insert a content record of type *qfq*.
Carsten  Rose's avatar
Carsten Rose committed
234
235
236
* In the bodytext insert the following code:

.. code-block:: mysql
237

238
239
    # If there is a form given by SIP: show
    form={{form:SE}}
240

241
242
    # In case indexQfq is different from indexData, set indexQfq.
    dbIndex = {{indexQfq:Y}}
243

244
245
246
    10 {
        # List of Forms: Do not show this list of forms if there is a form given by SIP.
        # Table header.
247
        sql = SELECT CONCAT('p:{{pageAlias:T}}&form=form') as _pagen, '#', 'Name', 'Title', 'Table', '' FROM (SELECT 1) AS fake WHERE '{{form:SE}}'=''
Carsten  Rose's avatar
Carsten Rose committed
248
        head = {{'b|p:id={{pageAlias:T}}&form=copyFormFromExt|t:Copy form from ExtForm' AS _link}}
249
               <table class="table table-hover qfq-table-50 tablesorter tablesorter-filter">
250
251
252
253
254
        tail = </table>
        rbeg = <thead><tr>
        rend = </tr></thead>
        fbeg = <th>
        fend = </th>
255

256
257
        10 {
            # All forms
258
            sql = SELECT CONCAT('p:{{pageAlias:T}}&form=form&r=', f.id) as _pagee
Carsten  Rose's avatar
Carsten Rose committed
259
                         , f.id, f.name, f.title AS '_striptags', f.tableName
260
261
262
                         , CONCAT('U:form=form&r=', f.id) as _paged
                      FROM Form AS f
                      ORDER BY f.name
263
264
265
266
267
268
            rbeg = <tr>
            rend = </tr>
            fbeg = <td>
            fend = </td>
        }
    }
269

270
271
.. _install-checklist:

Carsten  Rose's avatar
Carsten Rose committed
272
273
Installation: Check List
------------------------
274

275
276
277
* Protect the directory `<T3 installation>/fileadmin/protected` in Apache against direct file access.

  * `<T3 installation>/fileadmin/protected/` should be used for confidential (uploaded / generated) data.
Carsten  Rose's avatar
Carsten Rose committed
278
  * `<T3 installation>/fileadmin/protected/log/...` is the default place for QFQ log files.
279

280
281
* Protect the directory `<T3 installation>/fileadmin` in Apache to not execute PHP Scripts - malicious uploads won't be executed.
* Setup a log rotation rule for `sqlLog`.
282
283
* Check that `sqlLogMode` is set to `modify` on productive sites. With `none` you have no chance to find out who changed
  which data and `all` really logs a mass of data.
284

285
286
287
288
289
.. _configuration:

Configuration
-------------

290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
.. _config-qfq-php:

config.qfq.php
^^^^^^^^^^^^^^

+-------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| Keyword                       | Example                                               | Description                                                                |
+===============================+=======================================================+============================================================================+
| DB_<n>_USER                   | DB_1_USER=qfqUser                                     | Credentials configured in MySQL                                            |
+-------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| DB_<n>_PASSWORD               | DB_1_PASSWORD=1234567890                              | Credentials configured in MySQL                                            |
+-------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| DB_<n>_SERVER                 | DB_1_SERVER=localhost                                 | Hostname of MySQL Server                                                   |
+-------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| DB_<n>_NAME                   | DB_1_NAME=qfq_db                                      | Database name                                                              |
+-------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
Carsten  Rose's avatar
Carsten Rose committed
306
307
| LDAP_1_RDN                    | LDAP_1_RDN='ou=Admin,ou=example,dc=com'               | Credentials for non-anonymous LDAP access. Only one set supported.         |
| LDAP_1_PASSWORD               | LDAP_1_PASSWORD='mySecurePassword'                    |                                                                            |
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
+-------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+


Example: *typo3conf/config.qfq.php*: ::

    <?php

    // QFQ configuration
    //
    // Save this file as: <site path>/typo3conf/config.qfq.php

    return [
        'DB_1_USER' => '<DBUSER>',
        'DB_1_SERVER' => '<DBSERVER>',
        'DB_1_PASSWORD' => '<DBPW>',
        'DB_1_NAME' => '<DB>',

Carsten  Rose's avatar
Carsten Rose committed
325
326
327
328
        //DB_2_USER => <DBUSER>
        //DB_2_SERVER => <DBSERVER>
        //DB_2_PASSWORD => <DBPW>
        //DB_2_NAME => <DB>
329
330
331
332

        // DB_n ...
        // ...

Carsten  Rose's avatar
Carsten Rose committed
333
334
        // LDAP_1_RDN => 'ou=Admin,ou=example,dc=com'
        // LDAP_1_PASSWORD => 'mySecurePassword'
335
336
    ];

337
338
339
340
341
.. _extension-manager-qfq-configuration:

Extension Manager: QFQ Configuration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

342

343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| Keyword                           | Default / Example                                     | Description                                                                |
+===================================+=======================================================+============================================================================+
| Config                                                                                                                                                                 |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| documentation                     | http://docs.typo3.org...                              | Link to the online documentation of QFQ. Every QFQ installation also       |
|                                   |                                                       | contains a local copy: typo3conf/ext/qfq/Documentation/html/Manual.html    |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| flagProduction                    | yes                                                   | yes|no: used to differentiate production and development site.             |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| thumbnailDirSecure                | fileadmin/protected/qfqThumbnail                      | Important: secure directory 'protected' (recursive) against direct access. |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| thumbnailDirPublic                | typo3temp/qfqThumbnail                                | Both thumbnail directories will be created if not existing.                |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| cmdInkscape                       | inkscape                                              | If inkscape is not available, specify an empty string.                     |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| cmdConvert                        | convert                                               | GraphicsMagics 'convert' is recommended.                                   |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| cmdWkhtmltopdf                    | /usr/bin/wkhtmltopdf                                  | PathFilename of wkhtmltopdf. Optional variables like LD_LIBRARY_PATH=...   |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| baseUrl                           | http://example.com                                    | URL where wkhtmltopdf will fetch the HTML (no parameter, those comes later)|
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| sendEMailOptions                  | -o tls=yes                                            | General options. Check: http://caspian.dotconf.net/menu/Software/SendEmail |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| dateFormat                        | yyyy-mm-dd                                            | Possible options: yyyy-mm-dd, dd.mm.yyyy.                                  |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| Dynamic                                                                                                                                                                |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| fillStoreSystemBySql1/2/3         | SELECT s.id AS ...                                    | Specific values read from the database to fill the system store during QFQ |
|                                   |                                                       | load. See `fillStoreSystemBySql`_ for a usecase.                           |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| fillStoreSystemBySqlErrorMsg1/2/3 | No current period found                               | Only define an error message, if QFQ should stop running                   |
|                                   |                                                       | in case of an SQL error or not exact one record.                           |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| Debug                                                                                                                                                                  |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| throwExceptionGeneralError        | auto                                                  | | *yes*: 'general errors' in QFQ (PHP) will throw an exception.            |
|                                   |                                                       | | *auto*: becomes 'yes', if 'flagProduction'!='yes', else 'no'.            |
|                                   |                                                       | | *no*: 'general errors' in QFQ (PHP) will be silently ignored.            |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| formSubmitLogMode                 | all                                                   | | *all*: every form submission will be logged.                             |
|                                   |                                                       | | *none*: no logging.                                                      |
385
|                                   |                                                       | | See `form-submit-log-page`_ for example QFQ code to display the log.     |
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| redirectAllMailTo                 | john@doe.com                                          | If set, redirect all QFQ generated mails (Form, Report) to the specified.  |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| sqlLogMode                        | modify                                                | | *all*: every statement will be logged - this might a lot.                |
|                                   |                                                       | | *modify*: log only statements who change data. *error*: log only         |
|                                   |                                                       |   DB errors.                                                               |
|                                   |                                                       | | *none*: no SQL log at all.                                               |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| sqlLog                            | fileadmin/protected/log/sql.log                       | Filename to log SQL commands: relative to <site path> or absolute. If the  |
|                                   |                                                       | directory does not exist, create it.                                       |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| qfqLog                            | fileadmin/protected/log/qfq.log                       | Filename to log general QFQ events:relative to <site path> or absolute.    |
|                                   |                                                       | If the directory does not exist, create it.                                |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| mailLog                           | fileadmin/protected/log/mail.log                      | Filename to log `sendEmail` commands: relative to <site path> or absolute. |
|                                   |                                                       | If the directory does not exist, create it.                                |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| showDebugInfo                     | auto                                                  | FE - Possible values: yes|no|auto|download. For 'auto': If a BE User is    |
|                                   |                                                       | logged in, a debug information will be shown on the FE.                    |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| Database                                                                                                                                                               |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| init                              | init=SET names utf8; SET sql_mode =                   | Global init for using the database. For 'sql_mode="NO_ENGINE_SUBSTITUTION"'|
|                                   | "NO_ENGINE_SUBSTITUTION"                              | see #7407.                                                                 |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| update                            | auto                                                  | | *auto*: apply DB Updates only if there is a newer version.               |
|                                   |                                                       | | *always*: apply DB Updates always, especially play formEditor.sql every  |
|                                   |                                                       |   time QFQ is called - *not* recommended!                                  |
|                                   |                                                       | | *never*: never apply DB Updates.                                         |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| indexData                         | 1                                                     | Optional. Default: 1. Retrieve the current setting via {{dbNameData:Y}}.   |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| indexQfq                          | 1                                                     | Optional. Default: 1. Retrieve the current setting via {{dbNameQfq:Y}}.    |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| Security                                                                                                                                                               |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| escapeTypeDefault                 | m                                                     | All variables `{{...}}` get this escape class by default.                  |
|                                   |                                                       | See `variable-escape`_.                                                    |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| securityVarsHoneypot              | email,username,password                               | If empty: no check. All named variables will rendered as INPUT elements.   |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| securityAttackDelay               | 5                                                     | If an attack is detected, sleep 'x' seconds and exit PHP process.          |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| securityShowMessage               | true                                                  | If an attack is detected, show a message.                                  |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| securityGetMaxLength              | 50                                                    | GET vars longer than 'x' chars triggers an `attack-recognized`.            |
|                                   |                                                       | `ExceptionMaxLength`_.                                                     |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
434
435
| securityFailedAuthDelay           | 3                                                     | If authorization fails, sleep 'x' seconds before answering the request.    |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
| Form-Config                                                                                                                                                            |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| recordLockTimeoutSeconds          | 900                                                   | Timeout for record locking. After this time, a record will be replaced.    |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| sessionTimeoutSeconds             | 1800                                                  | Timeout for FE User session. See sessionTimeoutSeconds_                    |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| enterAsSubmit                     | enterAsSubmit = 1                                     | 0: off, 1: Pressing *enter* in a form means *save* and *close*.            |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| editFormPage                      | form                                                  | T3 Pagealias to edit a form.                                               |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| formDataPatternError              | please check pattern error                            | Customizable error message used in validator.js. 'pattern' violation.      |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| formDataRequiredError             | missing value                                         | Customizable error message used in validator.js. 'required' fields.        |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| formDataMatchError                | type error                                            | Customizable error message used in validator.js. 'match' retype mismatch.  |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| formDataError                     | generic error                                         | Customizable error message used in validator.js. 'no specific' given.      |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| Form-Layout                                                                                                                                                            |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| labelAlign                        | left                                                  | Label align (left/center/right)/ Default: left. Will be inherited to Form. |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| cssClassQfqContainer              | container                                             | | QFQ with own Bootstrap: 'container'.                                     |
|                                   |                                                       | | QFQ already nested in Bootstrap of mainpage: <empty>.                    |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| cssClassQfqForm                   | qfq-color-base                                        | Wrap around QFQ 'Form'.                                                    |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| cssClassQfqFormPill               | qfq-color-grey-1                                      | Wrap around title bar for pills: CSS Class, typically a background color.  |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| cssClassQfqFormBody               | qfq-color-grey-2                                      | Wrap around FormElements: CSS Class, typically a background color.         |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| formBsColumns                     | col-md-12 col-lg-10                                   | The whole form will be wrapped. See `bs-custom-field-width`_               |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| formBsLabelColumns                | col-md-3 col-lg-3                                     | The column get the width. See `bs-custom-field-width`_                     |
+-----------------------------------+-------------------------------------------------------+                                                                            |
| formBsInputColumns                | col-md-6 col-lg-6                                     |                                                                            |
+-----------------------------------+-------------------------------------------------------+                                                                            |
| formBsNoteColumns                 | col-md-3 col-lg-3                                     |                                                                            |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| extraButtonInfoInline             | <img src="info.png">                                  | Image for `extraButtonInfo`_ (inline).                                     |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| extraButtonInfoBelow              | <img src="info.png">                                  | Image for `extraButtonInfo`_ (below).                                      |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| extraButtonInfoPosition           | below                                                 | 'auto' (default) or 'below'. See `extraButtonInfo`_.                       |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| extraButtonInfoClass              | pull-right                                            | '' (default) or 'pull-right'. See `extraButtonInfo`_.                      |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| Form-Language                                                                                                                                                          |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| formLanguage[ABCD]Id              | -  E.g.: 1                                            | In Typo3 configured pageLanguage id. The number after the 'L' parameter.   |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| formLanguage[ABCD]Label           | -  E.G.: english                                      | Label shown in *Form editor*, on the 'basic' tab.                          |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| saveButtonText                    | -                                                     | Text on the form save button. Typically none.                              |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| saveButtonTooltip                 | Save                                                  | Tooltip on the form save button.                                           |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| saveButtonClass                   | btn btn-default navbar-btn                            | Bootstrap CSS class for save button on top of the form.                    |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| saveButtonClassOnChange           | alert-info btn-info                                   | Bootstrap CSS class for save button showing 'data changed'.                |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| saveButtonGlyphIcon               | glyphicon-ok                                          | Icon for the form save button.                                             |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| closeButtonText                   | -                                                     | Text on the form close button. Typically none.                             |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| closeButtonTooltip                | close                                                 | Tooltip on the form close button.                                          |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| closeButtonClass                  | btn btn-default navbar-btn                            | Bootstrap CSS class for close button on top of the form.                   |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| closeButtonGlyphIcon              | glyphicon-remove                                      | Icon for the form close button.                                            |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| deleteButtonText                  | -                                                     | Text on the form delete button. Typically none.                            |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| deleteButtonTooltip               | delete                                                | Tooltip on the form delete button.                                         |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| deleteButtonClass                 | btn btn-default navbar-btn                            | Bootstrap CSS class for delete button on top of the form.                  |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| deleteButtonGlyphIcon             | glyphicon-trash                                       | Icon for the form delete button.                                           |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| newButtonText                     | -                                                     | Text on the form new button. Typically none.                               |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| newButtonTooltip                  | new                                                   | Tooltip on the form new button.                                            |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| newButtonClass                    | btn btn-default navbar-btn                            | Bootstrap CSS class for new button on top of the form.                     |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| newButtonGlyphIcon                | glyphicon-plus                                        | Icon for the form new button.                                              |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| showIdInFormTitle                 | 0 (off), 1 (on)                                       | Append at the form title the current record id.                            |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| cssClassColumnId                  | text-muted                                            | A column in a subrecord with the name id|ID|Id gets this class.            |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
527

528
After parsing the configuration, the following variables will be set automatically in STORE_SYSTEM:
529
530
531
532

+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------+
| Keyword                       | Description                                                                                                                        |
+===============================+====================================================================================================================================+
533
| dbNameData                    | Name of the 'data'-database. '{{dbNameData:Y}}                                                                                     |
534
+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------+
535
| dbNameQfq                     | Name of the 'QFQ'-database. '{{dbNameQfq:Y}}                                                                                       |
536
+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------+
537
538
| dbNameT3                      | Name of the 'T3'-database. '{{dbNameT3:Y}}                                                                                         |
+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------+
539
| sitePath                      | Absolute path of the current T3 instance. '{{sitePath:Y}}                                                                          |
540
+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------+
541
| extPath                       | Absolute path of the QFQ extension. '{{extPath:Y}}                                                                                 |
542
+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------+
543

544

545
546
547
548
.. _`CustomVariables`:

Custom variables
^^^^^^^^^^^^^^^^
549

550
Up to 30 custom variables can be defined in `configuration`_.
551

552
E.g. to setup a contact address and reuse the information inside your installation do: ::
553

554
555
556
   custom1: ADMINISTRATIVE_CONTACT = john@doe.com
   custom2: ADMINISTRATIVE_ADDRESS = John Doe, Hollywood Blvd. 1, L.A.
   custom3: ADMINISTRATIVE_NAME = John Doe
557
558
559
560
561

 * Somewhere in a `Form` or in `Report`::

      {{ADMINISTRATIVE_CONTACT:Y}}, {{ADMINISTRATIVE_ADDRESS:Y}}, {{ADMINISTRATIVE_NAME}}

562
563
It's also possible to configure such variables directly in `config.qfq.php`_.

Carsten  Rose's avatar
Carsten Rose committed
564
.. _`fillStoreSystemBySql`:
565

Carsten  Rose's avatar
Carsten Rose committed
566
567
Fill STORE_SYSTEM by SQL
^^^^^^^^^^^^^^^^^^^^^^^^
568

569
570
A specified SELECT statement in `configuration`_ in variable `fillStoreSystemBySql1` (or `2`,
or `3`) will be fired. The query should have 0 (nothing happens) or 1 row. All columns will be
Carsten  Rose's avatar
Carsten Rose committed
571
**added** to the existing STORE_SYSTEM. Existing variables will be overwritten. Be careful not to overwrite system values.
572

Carsten  Rose's avatar
Carsten Rose committed
573
574
575
576
577
578
This option is useful to make generic custom values, saved in the database, accessible to all QFQ Report and Forms.
Access such variables via `{{<varname>:Y}}`.

In case QFQ should stop working if a given query does not select exact one record (e.g. a missing period), define an
error message: ::

579
580
  fillStoreSystemBySql1: SELECT name FROM Person WHERE name='Doe'
  fillStoreSystemBySqlErrorMsg1: Too many or to few "Doe's" in our database
581

582
.. _`periodId`:
583
584
585
586
587
588

periodId
''''''''

This is

Carsten  Rose's avatar
Carsten Rose committed
589
* a usecase, implemented via `fillStoreSystemBySql`_,
590
* a way to access `Period.id` with respect to the current period (the period itself is custom defined).
591

592
After a full QFQ installation:
593
594
595
596

* a table `Period` (extend / change it to your needs, fill them with your periods),
* one sample record in table `Period`,

597
Websites, delivering semester data, school year schedules, or any other type or periods, often need an index to the
Carsten  Rose's avatar
Carsten Rose committed
598
*current* period.
599

600
In configuration_: ::
601

602
  fillStoreSystemBySql1: SELECT id AS periodId FROM Period WHERE start<=NOW() ORDER BY start DESC LIMIT 1
603
604
605
606
607
608
609
610

a variable 'periodId' will automatically computed and filled in STORE SYSTEM. Access it via `{{periodId:Y0}}`.
To get the name and current period: ::

  SELECT name, ' / ', start FROM Period WHERE id={{periodId:Y0}}

Typically, it's necessary to offer a 'previous' / 'next' link. In this example, the STORE SIP holds the new periodId: ::

611
612
  SELECT CONCAT('p:{{pageAlias:T}}&periodId=', {{periodId:SY0}}-1, '|Next') AS _page, ' ', name, ' ',
    CONCAT('p:{{pageAlias:T}}&periodId=', {{periodId:SY0}}+1, '|Next') AS _page FROM Period AS s WHERE s.id={{periodId:SY0}}
613
614
615
616

Take care for minimum and maximum indexes (do not render the links if out of range).

.. _`DbUserPrivileges`:
617
618
619
620
621
622
623
624

DB USER privileges
^^^^^^^^^^^^^^^^^^

The specified DB User needs privileges to the database of at least: SELECT / INSERT / UPDATE / DELETE / SHOW.

To apply automatically QFQ-'DB UPDATE' the following rights are mandatory too: CREATE / ALTER

625
To get access to the Typo3 installation, 'dbuser' should also have access to the Typo3 Database with at least SELECT / INSERT / UPDATE / DELETE.
626
627
628



629
630
631
632
633
.. _`ExceptionMaxLength`:

Exception for SECURITY_GET_MAX_LENGTH
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

634
If it is necessary to use a GET variable which exceeds `securityGetMaxLength` limit, name the variable with '_<num>' at
635
the end. E.g. `my_long_variable_130`. Such a variable has an allowed length of 130 chars. Access the variable as
636
637
638
usual with the variable name: `{{my_long_variable_130:C:...}}`.


639
640
641
642
643
644
645
.. _`sessionTimeoutSeconds`:

FE-User: Session timeout seconds
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

There is no timeout for website users who are not logged in (but typically those users don't have access to protected content).

646
647
For logged in users, the default timeout is the php.ini settings for `session.cookie_lifetime` and `session.gc_maxlifetime`
(minimum of both). These timeout only affects QFQ related content and can be
648
649
650
651
652
653
654
655
656
specified a) globally (QFQ configuration) and b) specific per Form.

The maximum timeout depends on the minimal value of php.ini `session.cookie_lifetime` and `session.gc_maxlifetime`.
Specifying a higher value produces an error in the front end.

Every access to QFQ related content resets the timeout.

After FE login, the next access to QFQ related content starts the timeout counter.

657
658
659
660
661
.. _local-documentation:

Local Documentation
-------------------

662
A HTML rendered version is available under: <your site>/typo3conf/ext/qfq/Documentation/html/Index.html
663

664
665
If you get a 'Page forbidden / not found' there might be some Webserver restrictions. E.g. the Typo3 example of `.htaccess`
in the Typo3 installation folder will forbid access to any extension documentation (which is a good idea on a productive
666
667
server). For a development server instead, deactivate the forbid rule of 'documentation'. In `.htaccess` (snippet from
Typo3 7.6 _.htaccess): ::
668

669
670
  production:   RewriteRule (?:typo3conf/ext|typo3/sysext|typo3/ext)/[^/]+/(?:Configuration|Resources/Private|Tests?|Documentation|docs?)/ - [F]
  development:  RewriteRule (?:typo3conf/ext|typo3/sysext|typo3/ext)/[^/]+/(?:Configuration|Resources/Private|Tests?|docs?)/ - [F]
671
672
673
674
675
676

.. _concept:

Concept
=======

677
678
679
680
681
SIPs
----

The following is a technical background information. Not needed to just use QFQ.

682
The SIPs (=Server Id Pairs) are uniq timestamps, created/registered on the fly for a specific browser session (=user). Every SIP is
683
registered on the server (= stored in a browser session) and contains one or more key/value pairs. The key/value pairs never leave
684
685
686
687
688
689
690
691
692
693
694
695
696
the server. The SIPs will be used:

* to protect values not to be spoofed by anyone,
* to protect values not to be altered by anyone,
* to grant access, e.g.:

  * load and save forms,
  * upload files,
  * download files,
  * PHP AJAX pages.

SIPs becomes invalid, as soon as the current browser session is destroyed. The client (= user) can't manipulate the content
of SIPs - it's only possible to reuse already registered SIPs by the user, who already owns the session.
697
698
699
700
701

Access privileges
-----------------

The Typo3 FE Groups can be used to implement access privileges. Such groups are assigned to
702

703
704
705
706
707
708
709
* Typo3 FE users,
* Typo3 pages,
* and/or Typo3 content records (e.g. QFQ records).

This will be used for general page structure privileges.

A `record base` privileges controlling (e.g. which user can edit
710
711
which person record) will be implicit configured, by the way that records are viewable / editable (or not) through
SQL in the specific QFQ tt-content statements.
712
713
714
715
716
717

Typo3 QFQ content element
-------------------------

Insert one or more QFQ content elements on a Typo3 page. Specify column and language per content record as wished.

718
719
The title of the QFQ content element will not be rendered on the frontend. It's only visible to the webmaster in the
backend for orientation.
720
721
722
723

QFQ Keywords (Bodytext)
^^^^^^^^^^^^^^^^^^^^^^^

724
725
**All of the named parameter are optional.**

726
727
728
 +-------------------+---------------------------------------------------------------------------------+
 | Name              | Explanation                                                                     |
 +===================+=================================================================================+
729
730
731
732
733
734
735
736
737
 | form              | | Formname.                                                                     |
 |                   | | Static: **form = person**                                                     |
 |                   | | By SIP: **form = {{form:SE}}**                                                |
 |                   | | By SQL: **form = {{SELECT c.form FROM config AS c WHERE c.id={{a:C}} }}**     |
 +-------------------+---------------------------------------------------------------------------------+
 | r                 | | <record id>. The form will load the record with the specified id.             |
 |                   | | Static: **r = 123**                                                           |
 |                   | | By SQL: **r = {{SELECT ...}}**                                                |
 |                   | | If not specified, the SIP parameter 'r' is used.                              |
738
 +-------------------+---------------------------------------------------------------------------------+
739
740
741
742
 | dbIndex           | E.g. `dbIndex = {{indexQfq:Y}}` Select a DB index. Only necessary if a          |
 |                   | different than the standard DB should be used.                                  |
 +-------------------+---------------------------------------------------------------------------------+
 | debugShowBodyText | If='1' and configuration_:*showDebugInfo: yes*, shows a tooltip with bodytext   |
743
 +-------------------+---------------------------------------------------------------------------------+
744
745
746
 | sqlLog            | Overwrites configuration_: `SQL_LOG`_ . Only affects `Report`, not `Form`.      |
 +-------------------+---------------------------------------------------------------------------------+
 | sqlLogMode        | Overwrites configuration_: `SQL_LOG_MODE`_ . Only affects `Report`, not `Form`. |
747
748
749
750
751
 +-------------------+---------------------------------------------------------------------------------+
 | <level>.fbeg      | Start token for every field (=column)                                           |
 +-------------------+---------------------------------------------------------------------------------+
 | <level>.fend      | End token for every field (=column)                                             |
 +-------------------+---------------------------------------------------------------------------------+
752
 | <level>.shead     | Static start token for whole <level>, independent if records are selected       |
753
 |                   | Shown before `head`.                                                            |
754
 +-------------------+---------------------------------------------------------------------------------+
755
 | <level>.stail     | Static end token for whole <level>, independent if records are selected.        |
756
 |                   | Shown after `tail`.                                                             |
757
758
759
760
 +-------------------+---------------------------------------------------------------------------------+
 | <level>.head      | Dynamic start token for whole <level>. Only if at least one record is select.   |
 +-------------------+---------------------------------------------------------------------------------+
 | <level>.tail      | Dynamic end token for whole <level>. Only if at least one record is select.     |
761
762
763
 +-------------------+---------------------------------------------------------------------------------+
 | <level>.rbeg      | Start token for row.                                                            |
 +-------------------+---------------------------------------------------------------------------------+
764
 | <level>.rbgd      | Alternating (per row) token.                                                    |
765
766
767
768
769
770
771
772
773
774
775
 +-------------------+---------------------------------------------------------------------------------+
 | <level>.rend      | End token for row. Will be rendered **before** subsequent levels are processed  |
 +-------------------+---------------------------------------------------------------------------------+
 | <level>.renr      | End token for row. Will be rendered **after** subsequent levels are processed   |
 +-------------------+---------------------------------------------------------------------------------+
 | <level>.rsep      | Seperator token between rows                                                    |
 +-------------------+---------------------------------------------------------------------------------+
 | <level>.fsep      | Seperator token between fields (=columns)                                       |
 +-------------------+---------------------------------------------------------------------------------+
 | <level>.sql       | SQL Query                                                                       |
 +-------------------+---------------------------------------------------------------------------------+
776
777
778
 | <level>.althead   | If <level>.sql is empty, these token will be rendered.                          |
 +-------------------+---------------------------------------------------------------------------------+
 | <level>.altsql    | If <level>.sql is empty, these query will be fired. No sub queries.             |
779
 |                   | Shown after `althead`                                                           |
780
 +-------------------+---------------------------------------------------------------------------------+
781
782
783
784
 | <level>.content   | | *show* (default): content of current and sub level are directly shown.        |
 |                   | | *hide*: content of current and sub levels are stored and not shown.           |
 |                   | | *store*: content of current and sub levels are stored and shown.              |
 |                   | | To retrieve the content: `{{<level>.line.content}}`. See `syntax-of-report`_  |
785
 +-------------------+---------------------------------------------------------------------------------+
786

787
788
789
790
791
792
793
794
.. _`qfq-database`:

QFQ Database
------------

Recommended setup for Typo3 & QFQ Installation is with *two* databases. One for the Typo3 installation and one for QFQ.
A good practice is to name both databases equal, appending the suffix '_t3' and '_db'.

Carsten  Rose's avatar
Carsten Rose committed
795
When QFQ is called, it checks for QFQ system tables. If they do not exist or have a lower version than the installed QFQ
796
797
798
799
800
801
802
version, the system tables will be automatically installed or updated.

.. _`system-tables`:

System tables
^^^^^^^^^^^^^

803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
+---------------+------------+------------+
| Name          | Use        | Database   |
+===============+============+============+
| Clipboard     | Temporary  | QFQ        |
+---------------+------------+------------+
| Cron          | Persistent | QFQ        |
+---------------+------------+------------+
| Dirty         | Temporary  | QFQ | Data |
+---------------+------------+------------+
| Form          | Persistent | QFQ        |
+---------------+------------+------------+
| FormElement   | Persistent | QFQ        |
+---------------+------------+------------+
| FormSubmitLog | Persistent | QFQ | Data |
+---------------+------------+------------+
| MailLog       | Persistent | QFQ | Data |
+---------------+------------+------------+
| Period        | Persistent | Data       |
+---------------+------------+------------+
| Split         | Persistent | Data       |
+---------------+------------+------------+

825
See `mail-log-page`_ and `form-submit-log-page`_ for some Frontend views for these tables.
826
827

* Check Bug #5459 - support of system tables in different DBs not supported.
828

829
830
831
832
.. _`multi-database`:

Multi Database
^^^^^^^^^^^^^^
833

834
835
Base: T3 & QFQ
''''''''''''''
836

837
838
QFQ typically interacts with one database, the QFQ database. The database used by Typo3 is typically a separate one.
Theoretically it might be the same (never tested), but it's strongly recommended to use a separated QFQ database to have
Carsten  Rose's avatar
Carsten Rose committed
839
no problems on Typo3 updates and to have a clean separation between Typo3 and QFQ.
840
841
842
843

QFQ: System & Data
''''''''''''''''''

Carsten  Rose's avatar
Carsten Rose committed
844
845
QFQ itself can be separated in 'QFQ system' (see `system-tables`_) and 'QFQ data' databases (even more than one are
possible). The 'QFQ system' stores the forms, record locking, log tables and so on - `QFQ data` is for the rest.
846
847
848
849
850
851

A `Multi Database` setup is given, if 'QFQ system' is different from 'QFQ data'.

Data: Data1, Data2, ..., Data n
'''''''''''''''''''''''''''''''

852
Every database needs to be configured via `configuration`_ with it's own `index`.
853

854
855
856
`QFQ data` might switch between different 'data'-databases. In configuration_ one main `QFQ data` index will be specified
in `indexQfq`. If specific forms or reports should use a different database than that, `dbIndex` might change
`indexData` temporarily.
857
858

`dbIndex`: A `Report` (field `dbIndex`) as well as a `Form` (field `parameter`.`dbIndex`) can operate on a specific database.
859

860
861
862

A `Form` will:

Carsten  Rose's avatar
Carsten Rose committed
863
* load the form-definition from `indexQfq` (table `Form` and `FormElement`),
864
* loads and save data from/in `indexData` (config.qfq.php) / `dbIndex` (form.parameter.dbIndex),
865
* retrieve extra information via `dbIndexExtra` - this is useful to offer information from a database and save them in a
866
  different one.
867

868
869
The simplest setup, QFQ system & data in the same database, needs no `indexQfq / indexData` definition in
configuration_ or one or both of them set to '1'
870

871
To separate QFQ system and data, indexQfq and indexData will have different indexes.
872
873
874
875
876
877
878


A Multi Database setup might be useful for:

* several independent Typo3 QFQ installations (each have it's own form repository) and one central database, or
* one QFQ installation which should display / load /save records from different databases, or
* a combination of the above two.
879

880
Note:
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902

* Option 'A' is the most simple and commonly used.
* Option 'B' separate the T3 and QFQ databases on two database hosts.
* Option 'C' is like 'B' but with a shared 'QFQ data'-database between three 'Typo3 / QFQ' instances.
* Further variants are possible.

+---+----------------+--------------+-------------------------------+------------------------------+----------------------------------+
|   | Domain         | Website Host | T3                            | QFQ system                   | QFQ data                         |
+===+================+==============+===============================+==============================+==================================+
| A | standalone.edu | 'w'          | <dbHost>, <dbname>_t3, <dbnameSingle>_db                                                        |
+---+----------------+--------------+-------------------------------+------------------------------+----------------------------------+
| B | appB1.edu      | 'wApp'       | <dbHostApp>, <dbnameB1>_t3    | <dbHostB1>, <dbnameApp>_db                                      |
+---+----------------+--------------+-------------------------------+------------------------------+----------------------------------+
| B | appB2.edu      | 'wApp'       | <dbHostApp>, <dbnameB2>_t3    | <dbHostB2>, <dbnameApp>_db                                      |
+---+----------------+--------------+-------------------------------+------------------------------+----------------------------------+
| C | appC1.edu      | 'wAppC'      | <dbHostAppC>, <dbnameC1>_t3   | <dbHostC>, <dbnameSysC1>_db  | <dbHostData>_db, <dbNameData>_db |
+---+----------------+--------------+-------------------------------+------------------------------+----------------------------------+
| C | appC2.edu      | 'wAppC'      | <dbHostAppC>, <dbnameC2>_t3   | <dbHostC>, <dbnameSysC2>_db  | <dbHostData>_db, <dbNameData>_db |
+---+----------------+--------------+-------------------------------+------------------------------+----------------------------------+
| C | appC3.edu      | 'wAppC3'     | <dbHostAppC3>, <dbnameC3>_t3  | <dbHostC3>, <dbnameSysC3>_db | <dbHostData>_db, <dbNameData>_db |
+---+----------------+--------------+-------------------------------+------------------------------+----------------------------------+

903
In config-qfq-php_ mutliple database credentials can be prepared. Mandatory is at least one credential setup like
904
905
906
`DB_1_USER`, `DB_1_SERVER`, `DB_1_PASSWORD`, `DB_1_NAME`. The number '1' indicates the `dbIndex`. Increment the number
to specify further database credential setups.

907
Typically the credentials for `DB_1`  also have access to the T3 database.
908
909
910
911
912
913


Different QFQ versions, shared database
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

When using different QFQ versions and a shared 'QFQ data'-database, there is some risk of conflicting
Carsten  Rose's avatar
Carsten Rose committed
914
'QFQ system' tables. Best is to always use the same QFQ version on all instances or use a Multi Database setup.
915

916
917
918
.. _debug:

Debug
919
920
921
922
=====

SQL Logging
-----------
923

924
Setup in configuration_
925
926
927

.. _SQL_LOG:

928
* *sqlLog*
929
930

  * Filename where to log SQL queries and statistical data.
931
  * File is relative to the `<site path>` or absolute (starting with '/').
932
933
934
  * Content: SQL queries and timestamp, formName/formId, fe_user, success, affected rows, newly created record
    id's and accessed from IP.
  * The global setting can be overwritten by defining `sqlLog` inside of a QFQ tt-content record.
935

936
937
938

.. _SQL_LOG_MODE:

939
* *sqlLogMode: all|modify|error|none*
940
941
942
943
944
945
946

  * *all*: logs every SQL statement.
  * *modify*: logs only statements who might potentially change data.
  * *error*: logs only queries which generate SQL errors.
  * *none*: no query logging at all.
  * The global setting can be overwritten by defining `sqlLogMode` inside of a QFQ tt-content record.

947
* *showDebugInfo = [yes|no|auto],[download]*
948
949

  If active, displays additional information in the Frontend (FE). This is typically helpful during development.
950
951
952
953
954
955

  * *yes*:

    * Form:

      * For every internal link/button, show tooltips with decoded SIP on mouseover.
956
      * Shows an 'Edit form'-button (wrench symbol) on a form. The link points to the T3 page with the :ref:`form-editor`.
957
958
959
960
961
962
963
964
965

    * Report: Will be configured per tt-content record.

      *debugShowBodyText = 1*

  * *no*: No debug info.

  * *auto*: Depending if there is a Typo3 BE session, set internally:

966
967
    * *showDebugInfo = yes*  (BE session exist)
    * *showDebugInfo = no*   (no BE session)
968

969
970
971
  * *download*:

    * During a download (especially by using wkhtml), temporary files are not deleted automatically. Also the
972
      ``wkhtmltopdf`` and ``pdftk`` commandlines will be logged to `SQL_LOG`_. Use this only to debug problems on download.
973

974
975
.. _REDIRECT_ALL_MAIL_TO:

976
977
978
Redirect all mail to (catch all)
--------------------------------

979
Setup in configuration_
980

981
* *redirectAllMailTo=john@doe.com*
982
983
984
985
986
987
988
989
990

  * During the development, it might be helpful to configure a 'catch all' email address, which QFQ uses as the final receiver
    instead of the original intended one.

  * The setting will:

    * Replace the 'To' with the configured one.
    * Clear 'CC' and 'Bcc'
    * Write a note and the original configured receiver at the top of the email body.
991

992
_`mail-log-page`
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009

Mail Log page
-------------

For debugging purposes you may like to add a Mail Log page in the frontend.
The following QFQ code could be used for that purpose (put it in a QFQ PageContent element): ::

    # Page parameters
    1.sql = SELECT @grId := '{{grId:C0:digit}}' AS _grId
    2.sql = SELECT @summary := IF('{{summary:CE:alnumx}}' = 'true', 'true', 'false') AS _s

    # Filters
    10 {
      sql = SELECT gr.id, IF(gr.id = @grId, "' selected>", "'>"), gr.value, ' (Id: ', gr.id, ')'
               FROM gGroup AS gr
               INNER JOIN MailLog AS ml ON ml.grId = gr.id
               GROUP BY gr.id
1010
1011
      head = <form onchange='this.submit();' class='form-inline'><input type='hidden' name='id' value='{{pageAlias:T0}}'>
               Filter By Group: <select name='grId' class='form-control'><option value=''></option>
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
      rbeg = <option value='
      rend = </option>
      tail = </select>
    }
    20 {
      sql = SELECT IF(@summary = 'true', ' checked', '')
      head = <div class='checkbox'><label><input type='checkbox' name='summary' value='true'
      tail = >Summary</label></div></form>
    }

    # Mail Log
    50 {
      sql = SELECT id, '</td><td>', grId, '</td><td>', xId, '</td><td>',
                REPLACE(receiver, ',', '<br>'), '</td><td>', REPLACE(sender, ',', '<br>'), '</td><td>',
                DATE_FORMAT(modified, '%d.%m.%Y<br>%H:%i:%s'), '</td><td style="word-break:break-word;">',
1027
1028
1029
                CONCAT('<b>', subject, '</b><br>', IF(@summary = 'true', CONCAT(SUBSTR(body, 1,
                       LEAST(IF(INSTR(body, '\n') = 0, 50, INSTR(body, '\n')), IF(INSTR(body, '<br>') = 0, 50,
                       INSTR(body, '<br>')))-1), ' ...'), CONCAT('<br>', REPLACE(body, '\n', '<br>'))) )
1030
1031
1032
              FROM MailLog WHERE (grId = @grId OR @grId = 0)
              ORDER BY modified DESC
              LIMIT 100
1033
1034
      head = <table class="table table-condensed table-hover"><tr>
                 <th>Id</th><th>grId</th><th>xId</th><th>To</th><th>From</th><th>Date</th><th>E-Mail</th></tr>
1035
1036
1037
1038
1039
      tail = </table>
      rbeg = <tr><td>
      rend = </td></tr>
    }

1040
_`form-submit-log-page`
1041
1042
1043
1044
1045
1046
1047
1048

Form Submit Log page
--------------------

For debugging purposes you may like to add a Form Submit Log page in the frontend.
The following QFQ code could be used for that purpose (put it in a QFQ PageContent element): ::

    # Filters
1049
    20.shead = <form onchange='this.submit()' class='form-inline'><input type='hidden' name='id' value='{{pageAlias:T0}}'>
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
    20 {
      sql = SELECT id, IF(id = '{{formId:SC0}}', "' selected>", "'>"), name
            FROM Form ORDER BY name
      head = <label for='formId'>Form:</label> <select name='formId' id='formId' class='form-control'><option value=0></option>
      tail = </select>
      rbeg = <option value='
      rend = </option>
    }
    30 {
      sql = SELECT feUser, IF(feUser = '{{feUser:SCE:alnumx}}', "' selected>", "'>"), feUser
            FROM FormSubmitLog GROUP BY feUser ORDER BY feUser
      head = <label for='feUser'>FE User:</label> <select name='feUser' id='feUser' class='form-control'><option value=''></option>
      tail = </select>
      rbeg = <option value='
      rend = </option>
    }
    30.stail = </form>

    # Show Log
    50 {
      sql = SELECT l.id,
          CONCAT('<b>Form</b>: ', f.name,
            '<br><b>Record Id</b>: ', l.recordId,
            '<br><b>Fe User</b>: ', l.feUser,
            '<br><b>Date</b>: ', l.created,
            '<br><b>Page Id</b>: ', l.pageId,
            '<br><b>Session Id</b>: ', l.sessionId,
            '<br><b>IP Address</b>: ', l.clientIp,
            '<br><b>User Agent</b>: ', l.userAgent,
            '<br><b>SIP Data</b>: <div style="margin-left:20px;">', "<script>var data = JSON.parse('", l.sipData,
              "'); for (var key in data) {
              document.write('<b>' + key + '</b>: ' + data[key] + '<br>'); }</script>", '</div>'),
          CONCAT("<script>var data = JSON.parse('", l.formData,
            "'); for (var key in data) {
              document.write('<b>' + key + '</b>: ' + data[key] + '<br>'); }</script>")
          FROM FormSubmitLog AS l
          LEFT JOIN Form AS f ON f.id = l.formId
          WHERE (l.formId = '{{formId:SC0}}' OR '{{formId:SC0}}' = 0)
            AND (l.feUser = '{{feUser:SCE:alnumx}}' OR '{{feUser:SCE:alnumx}}' = '')
          ORDER BY l.created DESC LIMIT 100
      head = <table class="table table-hover">
             <tr><th>Id</th><th style="min-width:250px;">Environment</th><th>Submitted Data</th>
      tail = </table>
      rbeg = <tr>
      renr = </tr>
      fbeg = <td>
      fend = </td>
    }

1099
1100
.. _variables:

1101
1102
Variable
========
1103

1104
1105
Variables in QFQ are surrounded by double curly braces. Four different types of variable substitution functionality is
provided. Access to:
1106

1107
1108
* `store-variables`_
* `sql-variables`_
1109
1110
* `row-column-variables`_
* `link-column-variables`_
1111

1112
Some examples, including nesting::
1113

1114
1115
1116
1117
  # Store
  #---------------------------------------------
  {{r}}
  {{index:FS}}
1118
  {{name:FS:alnumx:s:my default}}
1119

1120
1121
1122
  # SQL
  #---------------------------------------------
  {{SELECT name FROM person WHERE id=1234}}
1123

1124
  # Row columns
1125
1126
1127
  #---------------------------------------------
  {{10.pId}}
  {{10.20.pId}}
1128

1129
1130
1131
1132
1133
  # Nesting
  #---------------------------------------------
  {{SELECT name FROM person WHERE id={{r}} }}
  {{SELECT name FROM person WHERE id={{key1:C:alnumx}} }} # explained below
  {{SELECT name FROM person WHERE id={{SELECT id FROM pf LIMIT 1}} }} # it's more efficient to use only one query
1134

1135
1136
  # Link Columns
  {{p:form=Person&r=1|t:Edit Person|E|s AS link}}
1137

1138
Leading and trailing spaces inside curly braces are removed.
1139

1140
1141
  * *{{ SELECT "Hello World"   }}* becomes *{{SELECT "Hello World"}}*
  * *{{ varname   }}* becomes *{{varname}}*
1142

1143
1144
Types
-----
1145

1146
.. _`store-variables`:
1147

1148
1149
Store variables
^^^^^^^^^^^^^^^
1150

1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
Syntax:  *{{VarName[:<store(s)[:<sanitize class>[:<escape>[:<default value>[:type violate message]]]]]}}*

See also:

 * `store`_
 * `sanitize-class`_
 * `variable-escape`_
 * `variable-default`_
 * `variable-type-message-violate`_

1161

1162
* Example::
1163

1164
1165
1166
  {{pId}}
  {{pId:FSE}}
  {{pId:FSE:digit}}
1167
  {{pId::digit}}
1168
  {{name:FSE:alnumx:m}}
1169
  {{name:::m}}
1170
  {{name:FSE:alnumx:m:John Doe}}
1171
1172
1173
  {{name::::John Doe}}
  {{name:FSE:alnumx:m:John Doe:forbidden characters}}
  {{name:::::forbidden characters}}
1174

1175
* Zero or more stores might be specified to be searched for the given VarName.
1176
* If no store is specified, the default for the searched stores are: **FSRVD** (=FORM > SIP > RECORD > VARS > DEFAULT).
1177
* If the VarName is not found in one store, the next store is searched,  up to the last specified store.
1178
1179
* If the VarName is not found and a default value is given, the default is returned.
* If no value is found, nothing is replaced - the string '{{<VarName>}}' remains.
1180
* If anywhere along the line an empty string is found, this **is** a value: therefore, the search will stop.
1181

1182
.. _`sanitize-class`:
1183

1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
Sanitize class
--------------

Values in STORE_CLIENT *C* (Client=Browser) and STORE_FORM *F* (Form, HTTP 'post') are checked against a
sanitize class. Values from other stores are *not* checked against any sanitize class, even if a sanitize class is specified.

* Variables get by default the sanitize class defined in the corresponding `FormElement`. If not defined,
  the default class is 'digit'.
* A default sanitize class can be overwritten by individual definition: *{{a:C:alnumx}}*
* If a value violates the specific sanitize class, see `variable-type-message-violate`_ for default or customized message.
  By default the value becomes `!!<name of sanitize class>!!`. E.g. `!!digit!!`.

For QFQ variables and FormElements:

+------------------+------+-------+-----------------------------------------------------------------------------------------+
| Name             | Form | Query | Pattern                                                                                 |
+==================+======+=======+=========================================================================================+
| **alnumx**       | Form | Query | [A-Za-z][0-9]@-_.,;: /() ÀÈÌÒÙàèìòùÁÉÍÓÚÝáéíóúýÂÊÎÔÛâêîôûÃÑÕãñõÄËÏÖÜŸäëïöüÿç            |
+------------------+------+-------+-----------------------------------------------------------------------------------------+
| **digit**        | Form | Query | [0-9]                                                                                   |
+------------------+------+-------+-----------------------------------------------------------------------------------------+
| **numerical**    | Form | Query | [0-9.-+]                                                                                |
+------------------+------+-------+-----------------------------------------------------------------------------------------+
| **allbut**       | Form | Query | All characters allowed, but not [ ]  { } % \ #. The used regexp: '^[^\[\]{}%\\#]+$',    |
+------------------+------+-------+-----------------------------------------------------------------------------------------+
| **all**          | Form | Query | no sanitizing                                                                           |
+------------------+------+-------+-----------------------------------------------------------------------------------------+


Only in FormElement:

+------------------+------+-------+-------------------------------------------------------------------------------------------+
| **auto**         | Form |       | Only supported for FormElements. Most suitable checktype is dynamically evaluated based   |
|                  |      |       | on native column definition, the FormElement type, and other info. See below for details. |
+------------------+------+-------+-------------------------------------------------------------------------------------------+
| **email**        | Form | Query | [a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}                                           |
+------------------+------+-------+-------------------------------------------------------------------------------------------+
| **pattern**      | Form |       | Compares the value against a regexp.                                                      |
+------------------+------+-------+-------------------------------------------------------------------------------------------+


Rules for CheckType Auto (by priority):

* TypeAheadSQL or TypeAheadLDAP defined: **alnumx**
* Table definition
1229
1230
  * integer type: **digit**
  * floating point number: **numerical**
1231
* FE Type
1232
1233
  * 'password', 'note': **all**
  * 'editor', 'text' and encode = 'specialchar': **all**
1234
1235
1236
1237
1238
1239
1240
1241
* None of the above: **alnumx**


.. _`variable-escape`:

Escape
------

1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
To 'escape' a character typically means: a character, which have a special meaning/function, should not treated as a special
character.
E.g. a string is surrounded by single ticks '. If such a string should contain a single tick inside (like 'Miller's'),
the inside single tick has to be escaped. This is typically done by a backlash: 'Millers\\'s'.

QFQ offers different ways of escaping. Which of them to use, depends on the situation.

Especially variables used in SQL Statements might cause trouble when using: NUL (ASCII 0), \\n, \\r, \\, ', ", or Control-Z.

Additional there is the escape class 'p' (password hash) which is not 'escape' but hashing. It transforms the value of
the variable into a hash. The hash function is the one used by Typo3 to encrypt and salt a password. This is useful to
manipulate FE user passwords via QFQ. See `setFeUserPassword`_
1254

Carsten  Rose's avatar
Carsten Rose committed
1255
The following `escape` and `hashing` types are available:
1256

1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
  * 'm' - `real_escape_string() <http://php.net/manual/en/mysqli.real-escape-string.php>`_ (m = mysql)
  * 'l' - LDAP search filter values: `ldap-escape() <http://php.net/manual/en/function.ldap-escape.php>`_ (LDAP_ESCAPE_FILTER).
  * 'L' - LDAP DN values. `ldap-escape() <http://php.net/manual/en/function.ldap-escape.php>`_ (LDAP_ESCAPE_DN).
  * 's' - Single ticks ' will be escaped against \\'.
  * 'd' - double ticks " will be escaped against \\".
  * 'C' - colon ':' will be escaped against \\:.
  * 'c' - config - the escape type configured in `configuration`_.
  * 'p' - password hashing: depends on the hashing type in the Typo3 installation, includes salting if configured.
  * ''  - the escape type configured in `configuration`_.
  * '-' - no escaping.
1267
1268
1269

* The `escape` type is defined by the fourth parameter of the variable. E.g.: `{{name:FE:alnumx:m}}` (m = mysql).
* It's possible to combine different `escape` types, they will be processed in the order given. E.g. `{{name:FE:alnumx:Ls}}` (L, s).
Carsten  Rose's avatar
Carsten Rose committed
1270
* Escaping is typically necessary for all user supplied content, especially if they are processed via SQL or LDAP queries.
1271
1272
1273
1274
1275
1276
* Be careful when escaping nested variables. Best is to escape **only** the most outer variable.
* In configuration_ a global `escapeTypeDefault` can be defined. The configured escape type applies to all substituted
  variables, who *do not* contain a *specific* escape type.
* Additionally a `defaultEscapeType` can be defined per `Form` (separate field in the *Form editor*). This overwrites the
  global definition of `configuration`. By default, every `Form.defaultEscapeType` = 'c' (=config), which means the setting
  in `configuration`_.
Carsten  Rose's avatar
Carsten Rose committed
1277
* To suppress an escape type, define the `escape type` = '-' on the specific variable. E.g.: `{{name:FE:alnumx:-}}`.
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293

.. _`variable-default`:

Default
-------

* Any string can be given to define a default value.
* If a default value is given, it makes no sense to define more than one store: with a default value given, only the
  first store is considered.
* If the default value contains a ':', that one needs to be escaped by '\'.

.. _`variable-type-message-violate`:

Type message violate
--------------------

1294
1295
1296
1297
1298
1299
If a value violates the sanitize class, instead of content on of the following texts will be set.
 * 'c' - The violated class will be set as content, surrounded by '!!'. E.g. '!!digit!!'. This is the default.
 * 'e' - Instead of the value an empty string will be set as content.
 * '0' - Instead of the value the string '0' will be set as content.
 * 'custom text ...' - Instead of the value the custom text will be set as content. If the text contains a ':', that one needs to
    be escaped by '\'. Check `variable-escape`_ qualifier 'C' to let QFQ do the colon escaping.
1300

1301
.. _`sql-variables`:
1302

1303
1304
SQL variables
^^^^^^^^^^^^^
1305

1306
1307
1308
* The detection of an SQL command is case *insensitive*.
* Leading  whitespace will be skipped.
* The following commands are interpreted as SQL commands:
1309

1310
1311
1312
  * SELECT
  * INSERT, UPDATE, DELETE, REPLACE, TRUNCATE
  * SHOW, DESCRIBE, EXPLAIN, SET
1313

1314
* A SQL Statement might contain variables, including additional SQL statements. Inner SQL queries will be executed first.
1315
1316
* All variables will be substituted one by one from inner to outer.
* The number of variables inside an input field or a SQL statement is not limited.
1317