Commit 75a9cff8 authored by Marc Egger's avatar Marc Egger
Browse files

Merge branch 'develop' into marcT3-10-FAF

# Conflicts:
#	extension/Classes/Core/Constants.php
#	extension/Classes/Core/Save.php
parents 67594662 1e03320f
Pipeline #4826 passed with stages
in 6 minutes and 33 seconds
......@@ -54,6 +54,7 @@ selenium:
script:
- unzip -q build/qfq.zip -d qfq
- cd docker/
# Required docker images: https://systemvcs.math.uzh.ch/megger/qfq_docker.git
- ./run_qfq_docker.sh -no-deploy
- ./deploy_to_container.sh ../qfq
- ./run_selenium_tests_docker.sh
......
......@@ -46,6 +46,8 @@ The following features are only tested / supported on linux hosts:
* PDF decrypt (used for merge with pdfunite) - command `qpdf`.
* PDF decrypt (used for merge with pdfunite) - command `gs` - in case `qpdf` is not successful.
* Mime type detection for uploads - command `file`.
* Split PDF into JPG - command `convert`.
* Repair PDF - command `pdftocairo`.
.. _`preparation`:
......@@ -428,7 +430,11 @@ Extension Manager: QFQ Configuration
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| cmdInkscape | inkscape | If inkscape is not available, specify an empty string. |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| cmdConvert | convert | GraphicsMagics 'convert' is recommended. |
| cmdConvert | convert | Image-/GraphicsMagics 'convert' is recommended. |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| cmdPdf2svg | pdf2svg | Convert PDF to SVG. |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| cmdPdftocairo | pdftocairo | Poppler based. |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| cmdWkhtmltopdf | /usr/bin/wkhtmltopdf | PathFilename of wkhtmltopdf. Optional variables like LD_LIBRARY_PATH=... |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
......
......@@ -73,10 +73,9 @@ Date: 06.09.2020
Notes
^^^^^
New table 'uniq' to guarantee uniq random strings.
New SIP protected AJAX calls.
Report now fires calls to websocket (remote hosts).
Report now fires REST calls to remote hosts.
* New SIP protected AJAX calls.
* Report now fires calls to websocket (remote hosts).
* Report now fires REST calls to remote hosts.
Features
^^^^^^^^
......@@ -209,9 +208,9 @@ Notes
* Add new keyword 'render' in tt-content Report and QFQ config. 'render' will control if a) only Form OR Report will be
rendered (render=single) or b) as previous Form AND Report together (render=both).
Advantage: with 'render=single' no more SELECT ... FROM (SELECT '') AS fake WHERE '{{form:SE}}'=''.
* Advantage: with 'render=single' no more SELECT ... FROM (SELECT '') AS fake WHERE '{{form:SE}}'=''.
Attention: NEW default behaviour in new QFQ installations - render=single. Behaviour in old installations is unchanged.
* Attention: NEW default behaviour in new QFQ installations - render=single. Behaviour in old installations is unchanged.
* tt-content records with 'render = api' can stay on the same page as the link to get the content (e.g. Excel Export).
* Change default doc page to qfq.io/doc.
......@@ -338,9 +337,6 @@ Version 19.11.3
Date: 29.11.2019
Notes
^^^^^
Features
^^^^^^^^
......@@ -388,7 +384,7 @@ Notes
* Enhance formModeGlobal=requiredOff/-ButMark (temporarily skipRequiredCheck): fill's '{{allRequiredGiven:V}}' before
save to 1 (all given) else 0.
Offers user to save form, even if not all required data are given and offers application logic to check easily if all
* Offers user to save form, even if not all required data are given and offers application logic to check easily if all
required fields has been filled.
Features
......@@ -575,9 +571,6 @@ Version 19.8.0
Date: 28.08.2019
Notes
^^^^^
Features
^^^^^^^^
......@@ -1082,9 +1075,6 @@ Version 19.1.0
Date: 03.01.2019
Notes
^^^^^
Features
^^^^^^^^
......@@ -1697,7 +1687,7 @@ Features
Bug Fixes
^^^^^^^^^
* #5706 / Fixed that problematic characters in 'fileDestination' has not been sanatized.
* #5706 / Fixed that problematic characters in 'fileDestination' has not been sanitized.
* Fixed problem with buttons clipping trough alert.
* Client: wrong variable, updated CSS for long errors.
......@@ -1845,7 +1835,7 @@ Notes
Features
^^^^^^^^
* #5022 / Variable violates sanatize class: 'msg' instead of empty string - new identifier "!!<sanitize class>!!".
* #5022 / Variable violates santize class: 'msg' instead of empty string - new identifier "!!<sanitize class>!!".
* #4813 / Exception during form load: show 'form edit link' if editor is logged in.
* formEditor.sql: Increas size of Form.title to give more room for SQL statements in.
* Manual.rst: enhance debug tipps.
......@@ -2418,7 +2408,7 @@ Bug Fixes
^^^^^^^^^
* #3953 / Radio buttons: Auswahl nicht angezeigt, wenn per itemList definiert.
* #3982 / Filename Sanatize: remove spaces. Filename not properbly enclosed by double ticks.
* #3982 / Filename Santize: remove spaces. Filename not properbly enclosed by double ticks.
Version 0.18.6
--------------
......@@ -2612,7 +2602,7 @@ Features
* #3218, #3600 / download.php / export: QFQ is now able to create PDFs and ZIPs on the fly. The sources might be
uploaded PDFs or Websites (local or remote) which will be converted to PDFs.
* Implement 'encode=specialchar' - new option per FormElement which is now the default for every FormElement.
* Sanatize.php: New function urlDecodeArr(). Decode all _GET vars.
* Santize.php: New function urlDecodeArr(). Decode all _GET vars.
* Implemented max GET parameter lenght. Default: 50.
* Implemented new escape class 'mysql' (realEscapeString).
* LICENSE.txt: Add GPLv3.
......@@ -2699,8 +2689,8 @@ Bug Fixes
new `explodeTemplateGroupElements()`
* TypeAhead.js: Handle <ENTER> key properly.
* #3462 / FormElement.parameter: requiredList not ok for non numeric content. STORE_FORM had been called without 'sanatize class'.
Therefore, all non numeric values has been sanatized by default. New: SANATIZE_ALLOW_ALL.
* #3462 / FormElement.parameter: requiredList not ok for non numeric content. STORE_FORM had been called without 'santize class'.
Therefore, all non numeric values has been sanitized by default. New: SANITIZE_ALLOW_ALL.
* Corrected error message to use 'itemList' instead of 'itemValues'. Renamed constant too.
* #2542 / FormElement-Typ 'note' funktioniert nicht mit dynamic update. 'Label' and 'note' are fixed - 'value' is still not updated, open.
......@@ -2911,7 +2901,7 @@ Features
* Multiple / Advanced Upload: new logic implements slaveId, sqlInsert, sqlUpdate, sqlDelete.
* FormElement.parameter: sqlBefore / sqlAfter fired during 'Form' save for action elements.
* STORE FORM: variable 'filename' moved to STORE VAR - sanatize class needs no longer specified.
* STORE FORM: variable 'filename' moved to STORE VAR - sanitize class needs no longer specified.
* STORE VAR: two new variables 'filename' and 'fileDestination' valid during processing of current upload FormElement.
* Default store priority list changed. Old: 'FSRD', New: 'FSRVD'.
* CODING.md: update doc for FormElement 'upload' and general 'Form' rendering & save (recursive rendering).
......@@ -2971,7 +2961,7 @@ Features
* Added STORE_BEFORE, #3146 - Mainly used to compare old and new values during a form 'save' action.
* Added 'best practice' for defining and using of 'Central configure values' in UserManual.
* Added accent characters to sanatize class 'alnumx', #3183.
* Added accent characters to sanitize class 'alnumx', #3183.
* Set default all QFQ send mails to 'auto-submit'.
* Added possibility to customize error messages ('data-pattern-error', 'data-rquired-error', 'data-match-error',
'data-error') if validation fails. Customization can be done on global level (config.qfq.ini), per Form or per FormElement.
......
......@@ -1322,15 +1322,15 @@ Renders images. Allows to define an alternative text and a title attribute for t
10.sql = SELECT "<path to image>|[<alt text>]|[<title text>]" AS _img
+-------------+-------------------------------------------------------------------------------------------+---------------------------+
|**Parameter**|**Description** |**Default value/behaviour**|
+=============+===========================================================================================+===========================+
|<pathtoimage>|The path to the image file. |none |
+-------------+-------------------------------------------------------------------------------------------+---------------------------+
|<alttext> |Alternative text. Will be displayed if image can't be loaded (alt attribute of img tag). |empty string |
+-------------+-------------------------------------------------------------------------------------------+---------------------------+
|<titletext> |Text that will be set as image title in the title attribute of the img tag. |no title attribute rendered|
+-------------+-------------------------------------------------------------------------------------------+---------------------------+
+--------------+-------------------------------------------------------------------------------------------+-----------------------------+
|**Parameter** |**Description** | **Default value/behaviour** |
+==============+===========================================================================================+=============================+
|<pathtoimage> |The path to the image file. | none |
+--------------+-------------------------------------------------------------------------------------------+-----------------------------+
|<alttext> |Alternative text. Will be displayed if image can't be loaded (alt attribute of img tag). | empty string |
+--------------+-------------------------------------------------------------------------------------------+-----------------------------+
|<titletext> |Text that will be set as image title in the title attribute of the img tag. | no title attribute rendered |
+--------------+-------------------------------------------------------------------------------------------+-----------------------------+
**Minimal Example** ::
......
......@@ -260,7 +260,7 @@ const ERROR_IO_WRITE_FILE = 1316;
const ERROR_PATH_INVALID = 1317;
const ERROR_PDF2SVG = 1320;
const ERROR_PDF_SPLIT = 1320;
const ERROR_PDF2JPEG = 1321;
//Report
......@@ -641,6 +641,8 @@ const SYSTEM_SHOW_ID_IN_FORM_TITLE = 'showIdInFormTitle';
const SYSTEM_CMD_WKHTMLTOPDF = 'cmdWkhtmltopdf';
const SYSTEM_CMD_INKSCAPE = 'cmdInkscape';
const SYSTEM_CMD_CONVERT = 'cmdConvert';
const SYSTEM_CMD_PDF2SVG = 'cmdPdf2svg';
const SYSTEM_CMD_PDFTOCAIRO = 'cmdPdftocairo';
const SYSTEM_CMD_QPDF = 'cmdQpdf';
const SYSTEM_CMD_GS = 'cmdGs';
const SYSTEM_CMD_PDFUNITE = 'cmdPdfunite';
......@@ -769,7 +771,7 @@ const TOKEN_FOUND_STOP_REPLACE = 'stopReplace';
const VAR_INDEX_VALUE = 0;
const VAR_INDEX_STORE = 1;
const VAR_INDEX_SANATIZE = 2;
const VAR_INDEX_SANITIZE = 2;
const VAR_INDEX_ESCAPE = 3;
const VAR_INDEX_DEFAULT = 4;
const VAR_INDEX_MESSAGE = 5;
......
......@@ -393,7 +393,7 @@ class Evaluate {
$typeMessageViolate = ($arrToken[VAR_INDEX_MESSAGE] === null || $arrToken[VAR_INDEX_MESSAGE] === '') ? SANITIZE_TYPE_MESSAGE_VIOLATE_CLASS : $arrToken[VAR_INDEX_MESSAGE];
// search for value in stores
$value = $this->store::getVar($arrToken[VAR_INDEX_VALUE], $arrToken[VAR_INDEX_STORE], $arrToken[VAR_INDEX_SANATIZE],
$value = $this->store::getVar($arrToken[VAR_INDEX_VALUE], $arrToken[VAR_INDEX_STORE], $arrToken[VAR_INDEX_SANITIZE],
$foundInStore, $typeMessageViolate, $arrToken[VAR_INDEX_DEFAULT]);
// escape ticks
......
......@@ -399,7 +399,7 @@ class FormAction {
*
* @param string $listOfFormElementNames E.g.: 'city, street, number'
*
* @return bool true if at lease one of the named elements is non empty on STORE_FORM (use SANATIZE_ALLOW_ALL to
* @return bool true if at lease one of the named elements is non empty on STORE_FORM (use SANITIZE_ALLOW_ALL to
* perform the check)
* @throws \CodeException
* @throws \UserFormException
......
......@@ -318,7 +318,7 @@ class HelperFile {
public static function rename($oldname, $newname) {
if (false === @rename($oldname, $newname)) {
$msg = self::errorGetLastAsString() . " - rename($oldname ,$newname)";
$msg = self::errorGetLastAsString() . " - rename($oldname, $newname)";
throw new \UserFormException(json_encode([ERROR_MESSAGE_TO_USER => 'unlink failed', ERROR_MESSAGE_TO_DEVELOPER => $msg]), ERROR_IO_RENAME);
}
......
......@@ -7,7 +7,6 @@
*/
namespace IMATHUZH\Qfq\Core\Helper;
/**
......@@ -289,14 +288,19 @@ class Sanitize {
*/
public static function digitCheckAndCleanGet($key) {
if (isset($_GET[$key]) && !ctype_digit($_GET[$key])) {
if (!empty($_GET[$key]) && ctype_digit($_GET[$key][0])) {
$_GET[$key] = $_GET[$key][0];
} else {
$_GET[$key] = '';
}
if (!isset($_GET[$key])) {
$_GET[$key] = '';
return;
}
}
if (ctype_digit($_GET[$key])) {
return;
}
if (ctype_digit($_GET[$key][0] ?? '')) {
$_GET[$key] = $_GET[$key][0];
} else {
$_GET[$key] = '';
}
}
}
\ No newline at end of file
......@@ -1246,13 +1246,10 @@ class Save {
switch ($fileSplitType) {
case FE_FILE_SPLIT_SVG:
$cmd = 'pdf2svg "' . $newSrc . '" "' . $fileNameDest . '" all';
$cmd = $this->buildPdf2svg($newSrc, $fileNameDest);
break;
case FE_FILE_SPLIT_JPEG:
if ($fileSplitTypeOptions == '') {
$fileSplitTypeOptions = FE_FILE_SPLIT_OPTIONS_JPEG;
}
$cmd = "convert $fileSplitTypeOptions '$newSrc' '$fileNameDest'";
$cmd = $this->buildConvertSplit($newSrc, $fileNameDest, $fileSplitTypeOptions);
break;
default:
throw new \UserFormException("Unknown 'fileSplit' type: " . $formElement[FE_FILE_SPLIT], ERROR_UNKNOWN_TOKEN);
......@@ -1260,14 +1257,33 @@ class Save {
// Split PDF
$cwdToApp = Path::cwdToApp();
// TODO: use full paths (with Path class functions) instead of chdir
// NOTE: don't use Path class functions until next chdir since cwdToApp is wrong!
HelperFile::chdir($tempDir, $cwdToApp);
$output = Support::qfqExec($cmd, $rc);
$cnt = 0;
do {
$output = Support::qfqExec($cmd, $rc);
$cnt++;
if ($rc != 0) {
// Split failed
$cmdRepair = $this->buildPdftocairo($newSrc, $newSrc . '.1');
$output .= PHP_EOL . $cmdRepair . PHP_EOL;
$output .= Support::qfqExec($cmdRepair, $rc1);
if ($rc1 != 0) {
// Repair failed too: stop here.
break;
}
HelperFile::rename($newSrc . '.1', $newSrc);
}
} while ($rc != 0 && $cnt < 2);
HelperFile::chdir($cwd, $cwdToApp);
if ($rc != 0) {
throw new \UserFormException(
json_encode([ERROR_MESSAGE_TO_USER => 'pdf2svg failed', ERROR_MESSAGE_TO_DEVELOPER => "[$cwd][cmd=$cmd]$output"]),
ERROR_PDF2SVG);
json_encode([ERROR_MESSAGE_TO_USER => 'pdf split failed', ERROR_MESSAGE_TO_DEVELOPER => "[$cwd][cmd=$cmd]$output"]),
ERROR_PDF_SPLIT);
}
$files = Helperfile::getSplitFileNames($tempDir);
......@@ -1311,6 +1327,54 @@ class Save {
HelperFile::rmdir($tempDir);
}
/**
* @param $src
* @param $dest
* @return string
* @throws \CodeException
* @throws \DbException
* @throws \UserFormException
* @throws \UserReportException
*/
private function buildPdf2svg($src, $dest) {
$cmd = $this->store->getVar(SYSTEM_CMD_PDF2SVG, STORE_SYSTEM);
return "$cmd '$src' '$dest' all";
}
/**
* @param $src
* @param $dest
* @return string
* @throws \CodeException
* @throws \DbException
* @throws \UserFormException
* @throws \UserReportException
*/
private function buildPdftocairo($src, $dest) {
$cmd = $this->store->getVar(SYSTEM_CMD_PDFTOCAIRO, STORE_SYSTEM);
return "$cmd -pdf '$src' '$dest'";
}
/**
* @param $src
* @param $dest
* @param $options
* @return string
* @throws \CodeException
* @throws \DbException
* @throws \UserFormException
* @throws \UserReportException
*/
private function buildConvertSplit($src, $dest, $options) {
if ($options == '') {
$options = FE_FILE_SPLIT_OPTIONS_JPEG;
}
$cmd = $this->store->getVar(SYSTEM_CMD_CONVERT, STORE_SYSTEM);
return "$cmd $options '$src' '$dest'";
}
/**
* Create/update or delete the slave record.
*
......
......@@ -447,6 +447,8 @@ class Config {
SYSTEM_CMD_WKHTMLTOPDF => '/opt/wkhtmltox/bin/wkhtmltopdf',
SYSTEM_CMD_INKSCAPE => 'inkscape',
SYSTEM_CMD_CONVERT => 'convert',
SYSTEM_CMD_PDF2SVG => 'pdf2svg',
SYSTEM_CMD_PDFTOCAIRO => 'pdftocairo',
SYSTEM_CMD_QPDF => 'qpdf',
SYSTEM_CMD_GS => 'gs',
SYSTEM_CMD_PDFUNITE => 'pdfunite',
......
......@@ -343,7 +343,7 @@ class EvaluateTest extends AbstractDatabaseTest {
$this->assertEquals(false, $eval->substitute('a:V', $foundInStore));
$this->assertEquals('', $foundInStore);
// Sanatize Class: digits
// Sanitize Class: digits
$this->assertEquals('1234567', $eval->substitute('a:F:digit', $foundInStore));
$this->assertEquals(STORE_FORM, $foundInStore);
......@@ -356,7 +356,7 @@ class EvaluateTest extends AbstractDatabaseTest {
$this->assertEquals('1234567', $eval->substitute('a:F:all', $foundInStore));
$this->assertEquals(STORE_FORM, $foundInStore);
// Sanatize Class: text
// Sanitize Class: text
$this->store->setVar('a', 'Hello world @-_.,;: /()', STORE_FORM, true);
$this->assertEquals('!!digit!!', $eval->substitute('a:F:digit', $foundInStore));
......
......@@ -349,7 +349,7 @@ class SanitizeTest extends TestCase {
unset ($_GET[CLIENT_PAGE_LANGUAGE]);
Sanitize::digitCheckAndCleanGet(CLIENT_PAGE_LANGUAGE);
$this->assertEquals(false, isset($_GET[CLIENT_PAGE_LANGUAGE]));
$this->assertEquals('', $_GET[CLIENT_PAGE_LANGUAGE]);
$_GET[CLIENT_PAGE_LANGUAGE] = '';
Sanitize::digitCheckAndCleanGet(CLIENT_PAGE_LANGUAGE);
......
......@@ -346,6 +346,8 @@ class StoreTest extends TestCase {
SYSTEM_THUMBNAIL_DIR_PUBLIC_REL_TO_APP => Path::APP_TO_SYSTEM_THUMBNAIL_DIR_PUBLIC_DEFAULT,
SYSTEM_CMD_INKSCAPE => 'inkscape',
SYSTEM_CMD_CONVERT => 'convert',
SYSTEM_CMD_PDF2SVG => 'pdf2svg',
SYSTEM_CMD_PDFTOCAIRO => 'pdftocairo',
SYSTEM_CMD_WKHTMLTOPDF => '/opt/wkhtmltox/bin/wkhtmltopdf',
SYSTEM_DATE_FORMAT => 'yyyy-mm-dd',
SYSTEM_THROW_GENERAL_ERROR => 'no',
......
......@@ -22,9 +22,15 @@ thumbnailDirPublic = typo3temp/qfqThumbnail
# cat=config/config; type=string; label=Command 'inkscape':Default is 'inkscape'. Will be used to convert SVG to images (png). An empty string disables `inkscape`. If it is not available, `convert` will be used instead.
cmdInkscape = inkscape
# cat=config/config; type=string; label=Command 'convert':Default is 'convert'. Will be used to convert images to thumbnails. GraphicsMagics 'convert' is recommended.
# cat=config/config; type=string; label=Command 'convert':Default is 'convert'. Will be used to convert images to thumbnails and to split PDF to JPEG.
cmdConvert = convert
# cat=config/config; type=string; label=Command 'pdf2svg':Default is 'pdf2svg'. Will be used to convert PDF to SVG.
cmdPdf2svg = pdf2svg
# cat=config/config; type=string; label=Command 'pdftocairo':Default is 'pdftocairo'. Will be used to repair broken PDFs to be splitted via convert.
cmdPdftocairo = pdftocairo
# cat=config/config; type=string; label=Command 'wkhtmltopdf':Default is '/opt/wkhtmltox/bin/wkhtmltopdf'. Command to convert a HTML page to a PDF.
cmdWkhtmltopdf = /opt/wkhtmltox/bin/wkhtmltopdf
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment