Commit c36b513d authored by Carsten  Rose's avatar Carsten Rose
Browse files

Merge branch 'marcT3-10-FAF' into 'develop'

Marc: Form/Report As File, Path class, Config class, Typo3 v9 compatability

See merge request !296
parents 1e03320f c0018213
Pipeline #4874 passed with stages
in 5 minutes and 37 seconds
......@@ -5,6 +5,7 @@ before_script:
variables:
SELENIUM_LOGS_PATH: "/scratch/tmp/7/"
GIT_STRATEGY: clone
stages:
- before
......@@ -63,7 +64,7 @@ selenium:
- cd docker; ./remove-containers.sh <<< "y"
- cd ..
- umask 002
- mkdir "$SELENIUM_LOGS_PATH/$CI_COMMIT_SHORT_SHA"
- mkdir -p "$SELENIUM_LOGS_PATH/$CI_COMMIT_SHORT_SHA"
- cp extension/Tests/selenium/selenium_logs/* "$SELENIUM_LOGS_PATH/$CI_COMMIT_SHORT_SHA/"
- echo "Selenium Logs copied to $SELENIUM_LOGS_PATH/$CI_COMMIT_SHORT_SHA/"
- echo "Or download result (log/screenshot) in gitlab under CI/CD > Pipelines <job> > right side 'Artifacts'"
......
......@@ -92,6 +92,8 @@ General
* With the `Dynamic` option, it's easily possible to use one Typo3 page and display different forms on that specific
page.
* Forms are synced between the database and form files. See :ref:`formAsfile`.
Form process order
------------------
......@@ -3459,6 +3461,8 @@ Table: Person
Import/merge form
-----------------
* ATTENTION * : This currently does not work. To copy merge forms you may just copy/merge the form files. See :ref:`formAsFile`.
The form `copyFormFromExt` copies a form from table `ExtForm / ExtFormElement` to `Form / FormElement`. The import/merge
form:
......@@ -3481,4 +3485,13 @@ If there are several T3/QFQ instances and if forms should be imported frequently
'import Forms from db xyz' like: ::
10.sql = CREATE OR REPLACE table ExtForm SELECT * FROM <db xyz>.Form
20.sql = CREATE OR REPLACE table ExtFormElement SELECT * FROM <db xyz>.FormElement
\ No newline at end of file
20.sql = CREATE OR REPLACE table ExtFormElement SELECT * FROM <db xyz>.FormElement
.. _`formAsFile`
Form As File
------------
* Forms are synced between the database and form files located in the form directory contained in the qfq project directory. See: :ref:`qfq-project-path-php`
* **ATTENTION** : Form and FormElement changes in the database are only registered if they are performed by the form editor. Otherwise they might get overwritten during the next file sync!
......@@ -37,6 +37,13 @@
General Tips
============
Typo3 Debug Mode
----------------
Manual to always display Exceptions and Errors in the backend and frontend of Typo3:
`Typo3: Debugging and Development Setup <https://docs.typo3.org/m/typo3/reference-coreapi/master/en-us/ApiOverview/ErrorAndExceptionHandling/Examples/Index.html#debugging-and-development-setup>`_
Errors
------
......
......@@ -204,9 +204,10 @@ Setup
* If the Extension Manager stops after importing: check your memory limit in php.ini.
* Copy/rename the file *<site path>/typo3conf/ext/qfq/config-example.qfq.php* to *<site path>/typo3conf/config.qfq.php*.
* Copy/rename the file *<site path>/typo3conf/ext/qfq/example.qfq.json* to *<site path>/fileadmin/protected/qfqProject/qfq.json*.
Configure the necessary settings :ref:`configuration`
The configuration file is outside of the extension directory, to not loose it during de-install and install again.
* When the QFQ Extension is called the first time on the Typo3 frontend, the file *<ext_dir>/Classes/Sql/formEditor.sql* will
played and fills the database with the *Form editor* records. This also happens automatically after each update of QFQ.
* Configure Typoscript to include Bootstrap, jQuery, QFQ javascript and CSS files.
......@@ -327,10 +328,38 @@ Installation: Check List
Configuration
-------------
.. _config-qfq-php:
.. _qfq-project-path-php
config.qfq.php
^^^^^^^^^^^^^^
qfq.project.path.php
^^^^^^^^^^^^^^^^^^^^
* The file `qfq.project.path.php` is located/created in the root directory of the application. (Where index.php is located)
* This file only returns the path to the qfq project directory where logs, config (`qfq.json`), report files and form files are located.
* If the file does not exist, it is created and the project path is set as follows:
* Does the deprecated config file typo3conf/config.qfq.php exist? Then set the qfq project path to `fileadmin/protected/qfqProject` (config is migrated automatically to qfqProject).
* If not, does `fileadmin/protected exist`? Then set the qfq project path to `fileadmin/protected/qfqProject`.
* If not, then set qfq project path to `../` (i.e. outside the app directory where index.php is located)
Example: *typo3conf/config.qfq.php*: ::
<?php
/**
QFQ project path configuration
!! ATTENTION !!: The files in the project directory should NOT be served by your http server!
Only exception: The app directory inside the project directory may be served.
*/
return 'fileadmin/protected/qfqProject'; // path relative to app directory (i.e. location of this file).
.. _qfq.json
qfq.json
^^^^^^^^
* Additionally to the keywords bellow one can also override the configuration values defined in the Typo3 extension manager: :ref:`extension-manager-qfq-configuration`
* e.g. if `qfq.json` contains `"flagProduction":"no"` then this value is taken instead of the one set in the extension manager.
+-------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| Keyword | Example | Description |
......@@ -347,38 +376,36 @@ config.qfq.php
| LDAP_1_PASSWORD | LDAP_1_PASSWORD='mySecurePassword' | |
+-------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
Example: *fileadmin/protected/qfqProject/qfq.json*: ::
Example: *typo3conf/config.qfq.php*: ::
<?php
{
"DB_1_USER": "<DBUSER>",
"DB_1_SERVER": "<DBSERVER>",
"DB_1_PASSWORD": "<DBPW>",
"DB_1_NAME": "<DB>",
// QFQ configuration
//
// Save this file as: <site path>/typo3conf/config.qfq.php
"DB_2_USER": "<OPTIONAL DBUSER>",
"DB_2_SERVER": "<OPTIONAL DBSERVER>",
"DB_2_PASSWORD": "<OPTIONAL DBPW>",
"DB_2_NAME": "<OPTIONAL DB>",
return [
'DB_1_USER' => '<DBUSER>',
'DB_1_SERVER' => '<DBSERVER>',
'DB_1_PASSWORD' => '<DBPW>',
'DB_1_NAME' => '<DB>',
"LDAP_1_RDN": "<OPTIONAL> ou=Admin,ou=example,dc=com",
"LDAP_1_PASSWORD": "<OPTIONAL> mySecurePassword"
}
//DB_2_USER => <DBUSER>
//DB_2_SERVER => <DBSERVER>
//DB_2_PASSWORD => <DBPW>
//DB_2_NAME => <DB>
.. _config-qfq-php:
// DB_n ...
// ...
config.qfq.php
^^^^^^^^^^^^^^
// LDAP_1_RDN => 'ou=Admin,ou=example,dc=com'
// LDAP_1_PASSWORD => 'mySecurePassword'
];
**DEPRECATED** : use `qfq.json` as described above. :ref:`qfq.json`
.. _extension-manager-qfq-configuration:
Extension Manager: QFQ Configuration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* These configuration values can be overwritten by `qfq.json`. :ref:`qfq.json`
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| Keyword | Default / Example | Description |
......@@ -424,6 +451,8 @@ Extension Manager: QFQ Configuration
| documentation | http://docs.typo3.org... | Link to the online documentation of QFQ. Every QFQ installation also |
| | | contains a local copy: typo3conf/ext/qfq/Documentation/html/Manual.html |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| reportAsFileAutoExport | no | Auto export of qfq reports to files. See :ref:`reportAsFile` |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| Dynamic |
+-----------------------------------+-------------------------------------------------------+----------------------------------------------------------------------------+
| fillStoreSystemBySql1/2/3 | SELECT s.id AS ... | Specific values read from the database to fill the system store during QFQ |
......
......@@ -1452,7 +1452,7 @@ Run a php function defined in an external script.
+-------------------+----------------------------------------------------+------------------------------------------------------------------------------------+
| call | call:my_function | PHP function to call |
+-------------------+----------------------------------------------------+------------------------------------------------------------------------------------+
| arg | arg:a1=Hello&a2=World" | Arguments are parsed and passed to the function together with the other parameters |
| arg | arg:a1=Hello&a2=World | Arguments are parsed and passed to the function together with the other parameters |
+-------------------+----------------------------------------------------+------------------------------------------------------------------------------------+
**Example**
......@@ -2824,6 +2824,34 @@ Example::
]}'>
</div>
.. _reportAsFile:
Report As File
--------------
* If the toplevel token `file` is present inside the body of a QFQ tt-content element then the given report file is loaded and rendered.
* The tt-content body is ignored in that case.
* The path to the report file must be given relative to the report directory inside the qfq project directory. See :ref:`qfq-project-path-php`
* If the QFQ setting `reportAsFileAutoExport` (see :ref:`extension-manager-qfq-configuration`) is enabled, then every QFQ tt-content element which does not contain the `file` keyword is exported automatically when the report is rendered the first time.
* The path of the created file is given by the typo3 page structure
* The tt-content element body is replaced with `file=<path-to-new-file>`
Example tt-content body::
file=Home/myPage/qfq-report.qfqr
# Everything else is ignored!!
10.sql = SELECT 'This is ignored!!'
Example Home/myPage/qfq-report.qfqr::
# Some comment
10.sql = SELECT 'The file content is executed.'
Example of rendered report::
The file content is executed.
Report Examples
---------------
......
......@@ -25,13 +25,13 @@ Quick Form Query Extension
2017-2020
:Authors:
Carsten Rose, Benjamin Baer, Marc Egger
Carsten Rose, Benjamin Baer
:Further Contributors:
Rafael Ostertag, Elias Villiger, Nicola Chiapolini
:Email:
carsten.rose@math.uzh.ch, benjamin.baer@math.uzh.ch, marc.egger@uzh.ch
carsten.rose@math.uzh.ch, benjamin.baer@math.uzh.ch
:License:
This document is published under the Open Publication License
......
......@@ -74,6 +74,12 @@ basic: .npmpackages .virtual_env
touch $@
.npmpackages:
echo "CURRENT USER '${USER}'"
npm --version
which npm
node --version
which node
echo "${PATH}"
npm ls -g grunt-cli 2>/dev/null || { echo "Please install grunt-cli npm package using 'npm install -g grunt-cli'" 1>&2 ; exit 1; }
# update npm at persistent location and copy node_modules (to speed up process)
......@@ -110,14 +116,14 @@ phpunit:
# update composer with dev to install phpunit package
cd extension; composer update
# create test config files and get test database password from environment variable
cp -v extension/Tests/phpunit_config.qfq.php extension/config.qfq.php; sed -i "s/#PHPUNIT_PASSWORD#/$(PHPUNIT_MYSQL_PASSWORD)/" extension/config.qfq.php
cp -v extension/Tests/phpunit_LocalConfiguration.php extension/LocalConfiguration.php
# mock typo3 directory structure
mkdir -p typo3conf/ext/qfq
mv -v extension/* typo3conf/ext/qfq/
# create new kind of config (qfq.json)
cp -v typo3conf/ext/qfq/Tests/phpunit_qfq.json ../conf/qfq.json; sed -i "s/#PHPUNIT_PASSWORD#/$(PHPUNIT_MYSQL_PASSWORD)/" ../conf/qfq.json
cp -v typo3conf/ext/qfq/Tests/phpunit_LocalConfiguration.php typo3conf/LocalConfiguration.php
# run phpunit
cd typo3conf/ext/qfq/; pwd; vendor/bin/phpunit --configuration phpunit.xml
......@@ -131,5 +137,5 @@ doc-local:
doc-qfqio:
rsync -av "Documentation/_build/html/" root@w16.math.uzh.ch:/var/www/html/qfq/doc/
......@@ -11,6 +11,7 @@
"ext-fileinfo": "*",
"ext-ldap": "*",
"ext-intl": "*",
"ext-iconv": "*"
"ext-iconv": "*",
"ext-curl": "*"
}
}
\ No newline at end of file
This diff is collapsed.
......@@ -2,119 +2,68 @@
source run_qfq_docker.output
# this function prints a separator
function print_separator {
# prints the separator
echo -e "----------------------------------------------------------------------"
}
# prints the starting separator
print_separator
# checks that a file named geckodriver doesn't already exist
if [ ! -f "geckodriver" ]; then
# stores the current version of the driver
gecko_version=$(curl --silent "https://api.github.com/repos/mozilla/geckodriver/releases/latest" | grep -Po '"tag_name": "\K.*?(?=")') &> /dev/null
# downloads the geckodriver from github
wget -O /tmp/geckodriver.tar.gz https://github.com/mozilla/geckodriver/releases/download/${gecko_version}/geckodriver-${gecko_version}-linux64.tar.gz &> /dev/null
# unzips the downloaded geckodriver
tar xzf /tmp/geckodriver.tar.gz geckodriver &> /dev/null
# makes the geckodriver executable
chmod +x geckodriver &> /dev/null
# prints a success output
echo -e "Successfully downloaded geckodriver"
fi
SELENIUM_URL=$1 # url to test from the input variable
ENGINE=$2 # engine (gecko/chrome) to use. default "chrome"
HEADLESS=$3 # run tests without opening browser GUI. (yes/no, default "no")
SLOWDOWN=$4 # add a pause in seconds after each selenium action (default 0)
# checks that a file named chromedriver doesn't already exist
if [ ! -f "chromedriver" ]; then
# downloads the newest version of the chromedriver from google
wget -O /tmp/chromedriver.zip http://chromedriver.storage.googleapis.com/`curl -sS chromedriver.storage.googleapis.com/LATEST_RELEASE`/chromedriver_linux64.zip &> /dev/null
# unzips the downloaded chromedriver
unzip /tmp/chromedriver.zip chromedriver &> /dev/null
# makes the chromedriver executable
chmod +x chromedriver &> /dev/null
# prints a success output
echo -e "Successfully downloaded chromedriver"
fi
# reads the url to test from the input variable
SELENIUM_URL=$1
# checks if the selenium url is not given
# Set default URL
if [ -z $SELENIUM_URL ]; then
SELENIUM_URL="http://127.0.0.1:${T3_PORT}"
fi
export SELENIUM_URL=$SELENIUM_URL # make URL accessible to selenium script
# defines the url to be used during testing
export SELENIUM_URL=$SELENIUM_URL
# stores the default engine
# set engine and driver
DEFAULT_ENGINE="chrome"
# reads the engine from the 2nd input variable
ENGINE=$2
# checks if an engine is not specified
if [ -z $ENGINE ]; then
# defines the default engine to use during tests
export SELENIUM_BROWSER=$DEFAULT_ENGINE
# defines the path to the drivers of the engine
export SELENIUM_DRIVER_PATH="${PWD}/${DEFAULT_ENGINE}driver"
else
# defines the engine to use during tests
export SELENIUM_BROWSER=$ENGINE
# defines the path to the drivers of the engine
export SELENIUM_DRIVER_PATH="${PWD}/${ENGINE}driver"
fi
# stores the default headless option
# set headless
DEFAULT_HEADLESS="no"
# reads the headless option from the 3rd input variable
HEADLESS=$3
# checks if the headless parameter is not specified
if [ -z $HEADLESS ]; then
# defines if the browser gui should open
export SELENIUM_HEADLESS=$DEFAULT_HEADLESS
else
# defines if the browser gui should open
export SELENIUM_HEADLESS=$HEADLESS
fi
# stores the default slowdown
# set slowdown
DEFAULT_SLOWDOWN="0"
# reads the slowdown from the 4th input variable
SLOWDOWN=$4
# checks if the slowdown parameter is not specified
if [ -z $SLOWDOWN ]; then
# defines what slowdown should be applied
export SELENIUM_SLOWDOWN=$DEFAULT_SLOWDOWN
else
# defines what slowdown should be applied
export SELENIUM_SLOWDOWN=$SLOWDOWN
fi
cd ../extension/Tests/selenium
# prints running tests
echo -e -n "Running tests: "
function print_separator {
echo -e "----------------------------------------------------------------------"
}
print_separator
# runs the python tests
python3 -W ignore -m unittest discover
# download geckodriver (Firefox)
if [ ! -f "geckodriver" ]; then
gecko_version=$(curl --silent "https://api.github.com/repos/mozilla/geckodriver/releases/latest" | grep -Po '"tag_name": "\K.*?(?=")') &> /dev/null
wget -O /tmp/geckodriver.tar.gz https://github.com/mozilla/geckodriver/releases/download/${gecko_version}/geckodriver-${gecko_version}-linux64.tar.gz &> /dev/null
tar xzf /tmp/geckodriver.tar.gz geckodriver &> /dev/null
chmod +x geckodriver &> /dev/null
echo -e "Successfully downloaded geckodriver"
fi
# checks if the tests were successful
if [ $? -eq 0 ]; then
print_separator
# prints a success message
echo -e "Successfully tested ${SELENIUM_URL}"
else
print_separator
# prints a success message
echo -e "Failed while testing ${SELENIUM_URL}"
# download chromedriver (Chrome)
if [ ! -f "chromedriver" ]; then
wget -O /tmp/chromedriver.zip http://chromedriver.storage.googleapis.com/`curl -sS chromedriver.storage.googleapis.com/LATEST_RELEASE`/chromedriver_linux64.zip &> /dev/null
unzip /tmp/chromedriver.zip chromedriver &> /dev/null
chmod +x chromedriver &> /dev/null
echo -e "Successfully downloaded chromedriver"
fi
# prints the trailing separator
print_separator
# run tests
cd ../extension/Tests/selenium
echo -e -n "Running tests: "
python3 -W ignore -m unittest discover
# exits the program
exit 0
......@@ -10,6 +10,7 @@ namespace IMATHUZH\Qfq\Api;
require_once(__DIR__ . '/../../vendor/autoload.php');
use IMATHUZH\Qfq\Core\Helper\Path;
use IMATHUZH\Qfq\Core\QuickFormQuery;
use IMATHUZH\Qfq\Core\Store\Store;
......@@ -37,6 +38,7 @@ $status = HTTP_400_BAD_REQUEST;
try {
try {
Path::setMainPaths(Path::API_TO_APP);
$qfq = new QuickFormQuery(['bodytext' => '']);
$data = $qfq->dataReport();
......
......@@ -10,11 +10,10 @@ namespace IMATHUZH\Qfq\Api;
require_once(__DIR__ . '/../../vendor/autoload.php');
use IMATHUZH\Qfq\Core\Helper\Path;
use IMATHUZH\Qfq\Core\QuickFormQuery;
use IMATHUZH\Qfq\Core\Store\Store;
/**
* delete: success
* SIP_MODE_ANSWER: MODE_HTML
......@@ -74,6 +73,7 @@ $flagSuccess = false;
try {
try {
Path::setMainPaths(Path::API_TO_APP);
$qfq = new QuickFormQuery(['bodytext' => '']);
$answer = $qfq->delete();
......
......@@ -11,13 +11,14 @@ namespace IMATHUZH\Qfq\Api;
require_once(__DIR__ . '/../../vendor/autoload.php');
use IMATHUZH\Qfq\Core\Form\Dirty;
use IMATHUZH\Qfq\Core\Helper\Path;
/**
* Return JSON encoded answer
*
*/
try {
Path::setMainPaths(Path::API_TO_APP);
$dirty = new Dirty();
$answer = $dirty->process();
......
......@@ -12,6 +12,7 @@ namespace IMATHUZH\Qfq\Api;
require_once(__DIR__ . '/../../vendor/autoload.php');
use IMATHUZH\Qfq\Core\Helper\Path;
use IMATHUZH\Qfq\Core\Report\Download;
......@@ -21,6 +22,7 @@ $output = '';
try {
try {
Path::setMainPaths(Path::API_TO_APP);
$download = new Download();
// If all is fine: process() will output file via print() !!
......
......@@ -10,8 +10,9 @@ namespace IMATHUZH\Qfq\Api;
require_once(__DIR__ . '/../../vendor/autoload.php');
use IMATHUZH\Qfq\Core\Helper\Path;
use IMATHUZH\Qfq\Core\QuickFormQuery;
/**
* Return JSON encoded answer
......@@ -50,6 +51,7 @@ $answer[API_MESSAGE] = '';
try {
try {
Path::setMainPaths(Path::API_TO_APP);
$qfq = new QuickFormQuery(['bodytext' => '']);
$data = $qfq->dragAndDrop();
......
......@@ -11,7 +11,7 @@ namespace IMATHUZH\Qfq\Api;
require_once(__DIR__ . '/../../vendor/autoload.php');
use IMATHUZH\Qfq\Core\File;
use IMATHUZH\Qfq\Core\Helper\Path;
/**
* Process File Upload - immediately when the the user selects a file.
......@@ -38,7 +38,7 @@ $answer[API_MESSAGE] = '';
try {
try {
Path::setMainPaths(Path::API_TO_APP);
$fileUpload = new File();
$fileUpload->process();
......
......@@ -10,9 +10,9 @@ namespace IMATHUZH\Qfq\Api;
require_once(__DIR__ . '/../../vendor/autoload.php');
use IMATHUZH\Qfq\Core\Helper\Path;
use IMATHUZH\Qfq\Core\Store\Store;
use IMATHUZH\Qfq\Core\QuickFormQuery;
/**
......@@ -52,6 +52,7 @@ $answer[API_MESSAGE] = '';
try {
try {
Path::setMainPaths(Path::API_TO_APP);
$qfq = new QuickFormQuery(['bodytext' => '']);
$data = $qfq->updateForm();
......
......@@ -11,13 +11,17 @@ namespace IMATHUZH\Qfq\Api;
require_once(__DIR__ . '/../../vendor/autoload.php');
use IMATHUZH\Qfq\Core\Helper\Path;
use IMATHUZH\Qfq\Core\Report\Html2Pdf;
use IMATHUZH\Qfq\Core\Store\Config;
/**
* Main
*/
try {
$html2pdf = new Html2Pdf();
Path::setMainPaths(Path::API_TO_APP);
$html2pdf = new Html2Pdf(Config::getConfigArray());
$html2pdf->outputHtml2Pdf();
......
......@@ -10,6 +10,7 @@ namespace IMATHUZH\Qfq\Api;
require_once(__DIR__ . '/../../vendor/autoload.php');
use IMATHUZH\Qfq\Core\Helper\Path;
use IMATHUZH\Qfq\Core\QuickFormQuery;
use IMATHUZH\Qfq\Core\Helper\OnString;
......@@ -21,6 +22,7 @@ $data = array();
try {
try {
Path::setMainPaths(Path::API_TO_APP);
$form = OnString::splitPathInfoToIdForm($_SERVER['PATH_INFO'] ?? '', $restId, $restForm);
// get latest `ìd`
......
......@@ -10,12 +10,11 @@ namespace IMATHUZH\Qfq\Api;
require_once(__DIR__ . '/../../vendor/autoload.php');
use IMATHUZH\Qfq\Core\Helper\Path;
use IMATHUZH\Qfq\Core\QuickFormQuery;
use IMATHUZH\Qfq\Core\Store\Store;
use IMATHUZH\Qfq\Core\Helper\Support;
/**
* Return JSON encoded answer
*
......@@ -53,6 +52,7 @@ $answer[API_MESSAGE] = '';
try {
try {
Path::setMainPaths(Path::API_TO_APP);
$qfq = new QuickFormQuery(['bodytext' => ""]);