Commit de14d11e authored by Carsten  Rose's avatar Carsten Rose
Browse files

F5885: implement 'append' mode.

parent 4ee6228a
......@@ -399,40 +399,59 @@ A download might be:
* a converted HTML page to PDF,
* a PDF file, concatenated on single PDF files and/or converted HTML page to PDF.
* a thumbnail, streamed from cache dir of if not present/recent rendered on request.
* a file to monitor constantly.
'api/download.php' will be called with a SIP (no other vars used). The SIP contains:
* DOWNLOAD_EXPORT_FILENAME - any target filename, if none given take DOWNLOAD_OUTPUT_PDF ('output.pdf').
* DONWLOAD_MODE - file / pdf / excel / zip / thumbnail. If not specified:
* DONWLOAD_MODE - file / pdf / excel / zip / thumbnail / monitor. If not specified:
a) 'file' is the default, if only one source is given and if that is a file.
b) 'pdf' is the default, if there are multiple TOKEN_URL, TOKEN_URL_PARAM, TOKEN_FILE in SIP_DOWNLOAD_PARAMETER found.
* SIP_DOWNLOAD_PARAMETER (base64 encoded): all DONWLOAD_MODE but 'thumbnail' - contains all parameter to source elements.
Format: <format 1>:<element 1>|<format 2>:<element 2>|...|<format n>:<element n>|
<format>: TOKEN_URL, TOKEN_URL_PARAM, TOKEN_FILE, TOKEN_THUMBNAIL_DIMENSION
<element>: depending on the token - see below
URL: a) 'u:http://w3c.org', b) 'u:w3c.org/', c) 'u:w3c.org/2017/index.php?issue=23'
URL_PARAM: a) 'U:id=export&r=123', b) 'U:id=export&r=123&_orientation=landscape&_page-size=a3'
FILE: a) 'F:fileadmin/example.png'
* DONWLOAD_MODE: file / pdf / excel / zip
* SIP_DOWNLOAD_PARAMETER (base64 encoded) - contains all parameter to source elements.
Format: <format 1>:<element 1>|<format 2>:<element 2>|...|<format n>:<element n>|
<format>: TOKEN_URL, TOKEN_URL_PARAM, TOKEN_FILE, TOKEN_THUMBNAIL_DIMENSION
<element>: depending on the token - see below
URL: a) 'u:http://w3c.org', b) 'u:w3c.org/', c) 'u:w3c.org/2017/index.php?issue=23'
URL_PARAM: a) 'U:id=export&r=123', b) 'U:id=export&r=123&_orientation=landscape&_page-size=a3'
FILE: a) 'F:fileadmin/example.png'
* In URL_PARAM extra parameter used by `wkhtmltopdf` can be specified. All Parameter, starting with '-'
will be extracted from the regular URL_PARAM and instead forwarded as options to `wkhtmlpdf`
* DONWLOAD_MODE: thumbnail
* SIP_DOWNLOAD_PARAMETER (base64 encoded)
* T:<pathFilename Source>
* W:<dimension>
* r:<render mode>
* Render the thumbnail
Download.php will be called with the SIP. After decoding the SIP, the base64 encoded parameter are used with
DONWLOAD_MODE=file and SIP_DOWNLOAD_PARAMETER=F:<thumbnail>
* DONWLOAD_MODE: monitor
SIP encoded parameter
* In URL_PARAM extra parameter used by `wkhtmltopdf` can be specified. All Parameter, starting with '-'
will be extracted from the regular URL_PARAM and instead forwarded as options to `wkhtmlpdf`
* SIP_DOWNLOAD_PARAMETER (base64 encoded): *DONWLOAD_MODE:thumbnail*
* T:<pathFilename Source>
* W:<dimension>
* r:<render mode>
* Render the thumbnail
Download.php will be called with the SIP. After decoding the SIP, the base64 encoded parameter are used with
DONWLOAD_MODE=file and SIP_DOWNLOAD_PARAMETER=F:<thumbnail>
* file: <filename>
* tail: <number of last lines>
* append: 0|1
The retrieved lines are outputted without any conversion.
* The base64 encoding is necessary:
* to deliver multiple elements with the same token (e.g. multiple PDF files to concatenate).
* special parameter names, like 'id', should not force the regular interpretation of 'id' during conversion to a SIP.
During preparing and delivering the download, a popup shows a spinning gear by default. The popup itself will display an
individual message. The popup needs some HTML code (only once per T3 page).
During preparing and delivering the download ()file / pdf / excel / zip), a popup shows a spinning gear by default. The
popup itself will display an individual message. The popup needs some HTML code (only once per T3 page).
Download links might be generated in `report` as well as in `subrecords of forms`. To trigger the generation of the HTML
popup code, a variable DOWNLOAD_POPUP_REQUEST in STORE_SYSTEM will be set to 'true' (string) in class Link(), as soon as
the first download link is rendered. During internal rendering of the download link, the const text token
......
This diff is collapsed.
......@@ -25,17 +25,19 @@ set_error_handler("\\qfq\\ErrorHandler::exception_error_handler");
$data = '';
try {
$download = new \qfq\Download();
// If all is fine - 'process()' never returns! The output file is delivered and PHP is stopped after that.
$data = $download->process(STORE_SIP, OUTPUT_MODE_DIRECT);
} catch (qfq\CodeException $e) {
$data = $e->formatMessage();
} catch (qfq\DbException $e) {
$data = $e->formatMessage();
} catch (qfq\DownloadException $e) {
$data = $e->formatMessage();
try {
$download = new \qfq\Download();
// If all is fine - 'process()' never returns! The output file is delivered and PHP is stopped after that.
$data = $download->process(STORE_SIP, OUTPUT_MODE_DIRECT);
} catch (qfq\CodeException $e) {
$data = $e->formatMessage();
} catch (qfq\DbException $e) {
$data = $e->formatMessage();
} catch (qfq\DownloadException $e) {
$data = $e->formatMessage();
}
} catch (\Exception $e) {
$data = "Exception: " . $e->getMessage();
}
......
......@@ -2769,7 +2769,7 @@ abstract class AbstractBuildForm {
* @param string $mode
* * mode=RETURN_URL: return complete URL
* * mode=RETURN_SIP: returns only the sip
* * mode=RETURN_ARRAY: returns array with url ('_url') and all decoded and created
* * mode=RETURN_ARRAY: returns array with url ('sipUrl') and all decoded and created
* parameters.
*
* @return string String: "API_DIR/delete.php?sip=...."
......
......@@ -146,11 +146,11 @@ const ERROR_UNKNOWN_MODE = 1032;
const ERROR_NOT_IMPLEMENTED = 1033;
const ERROR_RESERVED_KEY_NAME = 1034;
const ERROR_MISSING_FORM = 1035;
const ERROR_MISSING_PARAMETER_FILE = 1035;
const ERROR_UNKNOWN_FORWARD_MODE = 1036;
const ERROR_MISSING_MESSAGE_FAIL = 1037;
const ERROR_MISSING_EXPECT_RECORDS = 1038;
const ERROR_MISSING_HIDDEN_FIELD_IN_SIP = 1039;
const ERROR_MISSING_PARAMETER_FILE = 1040;
const ERROR_UNKNOWN_CHECKTYPE = 1042;
const ERROR_PATTERN_VIOLATION = 1043;
......@@ -579,6 +579,7 @@ const SIP_MODE_ANSWER = '_modeAnswer'; // Mode how delete() will answer to clien
const SIP_FORM = CLIENT_FORM;
const SIP_TABLE = 'table'; // delete a record from 'table'
const SIP_URLPARAM = 'urlparam';
const SIP_SIP_URL = 'sipUrl';
const SIP_MAKE_URLPARAM_UNIQ = '_makeUrlParamUniq'; // SIPs for 'new records' needs to be uniq per TAB! Therefore add a uniq parameter
const SIP_DOWNLOAD_PARAMETER = '_b64_download'; // Parametername, filled in SIP, to hold all download element parameter.
......@@ -1298,6 +1299,11 @@ const DOWNLOAD_SIP_ENCODE_PARAMETER = '_sip';
const OUTPUT_MODE_DIRECT = 'direct';
const OUTPUT_MODE_FILE = 'file';
const MONITOR_TAIL_DEFAULT = 30;
const MONITOR_APPEND_DEFAULT = 0;
const MONITOR_INTERVAL_DEFAULT = 1000;
const MONITOR_HTML_ID_DEFAULT = 'monitor-1';
// HTML2PDF
const HTML2PDF_PAGEID = 'id';
const HTML2PDF_PARAM_GET = 'paramGet';
......@@ -1358,6 +1364,10 @@ const TOKEN_L_APPEND = 'append';
const TOKEN_L_INTERVAL = 'interval';
const TOKEN_L_HTML_ID = 'htmlId';
const MONITOR_MODE_APPEND_0 = '0';
const MONITOR_MODE_APPEND_1 = '1';
const MONITOR_SESSION_FILE_SEEK = 'monitor-seek-file';
const RENDER_MODE_1 = '1';
const RENDER_MODE_2 = '2';
......
......@@ -21,7 +21,7 @@ require_once(__DIR__ . '/../helper/Sanitize.php');
require_once(__DIR__ . '/../helper/HelperFile.php');
require_once(__DIR__ . '/../report/Html2Pdf.php');
require_once(__DIR__ . '/Thumbnail.php');
//require_once(__DIR__ . '/Sendmail.php');
require_once(__DIR__ . '/Monitor.php');
require_once(__DIR__ . '/../exceptions/DownloadException.php');
//require_once(__DIR__ . '/../Evaluate.php');
//require_once(__DIR__ . '/../helper/KeyValueStringParser.php');
......@@ -71,6 +71,7 @@ class Download {
* @throws CodeException
* @throws DbException
* @throws UserFormException
* @throws UserReportException
*/
public function __construct($phpUnit = false) {
......@@ -300,7 +301,8 @@ class Download {
if($downloadMode == DOWNLOAD_MODE_MONITOR){
$monitor = new Monitor();
return $monitor->tailCustom($vars[TOKEN_L_FILE], $vars[TOKEN_L_TAIL]);
return $monitor->dump($vars[TOKEN_L_FILE], $vars[TOKEN_L_TAIL], $vars[TOKEN_L_APPEND]);
}
if ($downloadMode == DOWNLOAD_MODE_THUMBNAIL) {
......
......@@ -835,8 +835,7 @@ class Link {
if ($vars[NAME_SIP] === "1") {
$paramArray = $this->sip->queryStringToSip($urlNParam, RETURN_ARRAY);
//TODO: Index '_url' umbnennen
$urlNParam = $paramArray['_url'];
$urlNParam = $paramArray[SIP_SIP_URL];
if (Support::findInSet(SYSTEM_SHOW_DEBUG_INFO_YES, $this->store->getVar(SYSTEM_SHOW_DEBUG_INFO, STORE_SYSTEM))) {
$vars[NAME_TOOL_TIP] .= PHP_EOL . PHP_EOL . $this->sip->debugSip($paramArray);
......
......@@ -10,10 +10,14 @@
namespace qfq;
use PHPUnit\Exception;
require_once(__DIR__ . '/../store/Session.php');
require_once(__DIR__ . '/../store/Store.php');
require_once(__DIR__ . '/../helper/Support.php');
require_once(__DIR__ . '/../helper/Token.php');
require_once(__DIR__ . '/../helper/OnString.php');
require_once(__DIR__ . '/../exceptions/DownloadException.php');
//use qfq;
......@@ -28,18 +32,26 @@ class Monitor {
*/
private $store = null;
/**
* @var Session
*/
private $session = null;
/**
* @param bool|false $phpUnit
* @throws CodeException
* @throws UserFormException
* @throws UserReportException
*/
public function __construct($phpUnit = false) {
$this->store = Store::getInstance();
$this->session = Session::getInstance($phpUnit);
}
/**
* Renders some JS to monitor a file on the server.
* $str: 'file:typo3conf/sql.log|tail:1000|append:1|interval:1000|htmlId:monitor1'
* Report: $str: 'file:typo3conf/sql.log|tail:1000|append:1|interval:1000|htmlId:monitor1'
* Fetch: download.php?s=<sip> with sip: file=<filename>&tail=<number of lines>
*
* @param string $str
*
......@@ -58,20 +70,29 @@ class Monitor {
}
// Set defaults
Support::setIfNotSet($control, TOKEN_L_TAIL, 1000);
$isContinuous = Support::setIfNotSet($control, TOKEN_L_APPEND, '1') == '1' ? 'true' : 'false';
$interval = Support::setIfNotSet($control, TOKEN_L_INTERVAL, 100);
$htmlId = Support::setIfNotSet($control, TOKEN_L_HTML_ID, 'monitor1');
Support::setIfNotSet($control, TOKEN_L_TAIL, MONITOR_TAIL_DEFAULT);
$isContinuous = Support::setIfNotSet($control, TOKEN_L_APPEND, MONITOR_APPEND_DEFAULT) == MONITOR_MODE_APPEND_1 ? 'true' : 'false';
$interval = Support::setIfNotSet($control, TOKEN_L_INTERVAL, MONITOR_INTERVAL_DEFAULT);
$htmlId = Support::setIfNotSet($control, TOKEN_L_HTML_ID, MONITOR_HTML_ID_DEFAULT);
// Setup query to generate SIP
$queryString = DOWNLOAD_MODE . '=' . DOWNLOAD_MODE_MONITOR . '&' . TOKEN_L_FILE . '=' . $control[TOKEN_L_FILE] . '&' . TOKEN_L_TAIL . '=' . $control[TOKEN_L_TAIL];
$url = store::getSipInstance()->queryStringToSip(API_DIR . '/' . API_DOWNLOAD_PHP . '?' . $queryString, RETURN_URL);
$queryString = DOWNLOAD_MODE . '=' . DOWNLOAD_MODE_MONITOR .
'&' . TOKEN_L_FILE . '=' . $control[TOKEN_L_FILE] .
'&' . TOKEN_L_TAIL . '=' . $control[TOKEN_L_TAIL] .
'&' . TOKEN_L_APPEND . '=' . $control[TOKEN_L_APPEND];
// $url = store::getSipInstance()->queryStringToSip(API_DIR . '/' . API_DOWNLOAD_PHP . '?' . $queryString, RETURN_URL);
$arr = store::getSipInstance()->queryStringToSip('../../../qfq/api/' . API_DOWNLOAD_PHP . '?' . $queryString, RETURN_ARRAY);
$url = $arr[SIP_SIP_URL];
// On page reload, take care to remove optional exsiting old seek position.
$key = $this->getSeekSessionKey($arr[CLIENT_SIP]);
$this->session::unsetItem($key);
$code = <<<EOF
<script type="text/javascript">
$(document).ready(function () {
var getFile = new QfqNS.DisplayFile({
webworker: "Worker/GetFileContent.js",
webworker: "typo3conf/ext/qfq/Resources/Public/JavaScript/GetFileContent.js",
filePath: "$url",
interval: $interval,
targetId: "$htmlId",
......@@ -85,17 +106,92 @@ EOF;
return $code;
}
/**
* @param $filepath
* @param int $lines
* @param $modeAppend
* @return string
* @throws CodeException
* @throws DownloadException
* @throws UserFormException
*/
public function dump($filepath, $lines, $modeAppend) {
// Retrieve always the last $lines
if ($modeAppend == MONITOR_MODE_APPEND_0) {
return $this->tailCustom($filepath, $lines);
}
// First call to this file in this session.
$f = $this->open($filepath);
// Check if there is already a seek position.
$key = $this->getSeekSessionKey($this->store->getVar(SIP_SIP, STORE_SIP));
$seek = $this->session::get($key);
if ($seek === false) {
// Get seek position EOF
fseek($f, 0, SEEK_END);
$seek = ftell($f);
// Store the position in session.
$this->session::set($key, $seek);
// On initial call, get the specified last $lines.
return $this->tailCustom($filepath, $lines, true, $f);
}
fseek($f, $seek, SEEK_SET);
$output = fread($f, 8192);
$this->session::set($key, ftell($f));
return $output;
}
/**
* @param $sip
* @return string
*/
private function getSeekSessionKey($sip){
return MONITOR_SESSION_FILE_SEEK . '-' . $sip;
}
/**
* @param string $filepath
* @param string $mode
* @return bool|resource
* @throws DownloadException
*/
private function open($filepath, $mode = 'rb') {
// Open file
$f = @fopen($filepath, $mode);
if ($f === false) {
throw new DownloadException ("Error open '" . $filepath . "': " . error_get_last(), ERROR_IO_CHDIR);
}
return $f;
}
/**
* Slightly modified version of http://www.geekality.net/2011/05/28/php-tail-tackling-large-files/
* @author Torleif Berger, Lorenzo Stanco
* @link http://stackoverflow.com/a/15025877/995958
* @license http://creativecommons.org/licenses/by/3.0/
*
* @param string $filepath
* @param int $lines
* @param bool $adaptive
* @return string
* @throws DownloadException
*/
public function tailCustom($filepath, $lines = 1, $adaptive = true) {
private function tailCustom($filepath, $lines = 1, $adaptive = true, $f = null) {
// Open file
$f = @fopen($filepath, "rb");
if ($f === false) return false;
if ($f === null) {
$f = $this->open($filepath);
}
// Sets buffer size, according to the number of lines to retrieve.
// This gives a performance boost when reading a few lines from the file.
......
......@@ -36,6 +36,7 @@ require_once(__DIR__ . '/../Evaluate.php');
require_once(__DIR__ . '/../helper/KeyValueStringParser.php');
require_once(__DIR__ . '/../helper/Token.php');
require_once(__DIR__ . '/Thumbnail.php');
require_once(__DIR__ . '/Monitor.php');
const DEFAULT_QUESTION = 'question';
const DEFAULT_ICON = 'icon';
......
......@@ -48,7 +48,7 @@ class Sip {
* @return string/array
* * mode=RETURN_URL: return complete URL
* * mode=RETURN_SIP: returns only the sip
* * mode=RETURN_ARRAY: returns array with url ('_url') and all decoded and created parameters.
* * mode=RETURN_ARRAY: returns array with url ('sipUrl') and all decoded and created parameters.
* @throws CodeException
* @throws UserFormException
*/
......@@ -91,11 +91,11 @@ class Sip {
$script = $phpScriptName . $script;
}
$clientArray['_url'] = $script . OnArray::toString($clientArray) . $anchor;
$clientArray[SIP_SIP_URL] = $script . OnArray::toString($clientArray) . $anchor;
switch ($mode) {
case RETURN_URL:
$rc = $clientArray['_url'];
$rc = $clientArray[SIP_SIP_URL];
break;
case RETURN_SIP:
$rc = $s;
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment