Commit ca0b2137 authored by Carsten  Rose's avatar Carsten Rose
Browse files

Manual.rst: replace '{{form:S}}' against '{{form:SE}}'. Work on 'Auto Cron' description

AutoCron.php: Fix problem with array in checkForOldJobs(). Implement check that re-trigger asynchronous cron jobs are handled correctly.
parent 432db1b4
......@@ -705,7 +705,7 @@ QFQ Keywords (Bodytext)
+===================+=================================================================================+
| form | Formname defined in ttcontent record bodytext |
| | - Fix. E.g.: **form = person** |
| | - by SIP: **form = {{form}}** |
| | - by SIP: **form = {{form:SE}}** |
| | - by SQL: **form = {{SELECT c.form FROM conference AS c WHERE c.id={{a:C}} }}** |
+-------------------+---------------------------------------------------------------------------------+
| r | <record id> The form will load the record with the specified id |
......@@ -1507,7 +1507,7 @@ Store: *TYPO3* (Bodytext) - T
| form | Formname defined in ttcontent record bodytext | see note |
| | | |
| | * Fix. E.g. *form = person* | |
| | * via SIP. E.g. *form = {{form}}* | |
| | * via SIP. E.g. *form = {{form:SE}}* | |
+-------------------------+-------------------------------------------------------------------+----------+
| pageId | Record id of current Typo3 page | see note |
+-------------------------+-------------------------------------------------------------------+----------+
......@@ -1904,7 +1904,7 @@ General
* Inside the QFQ record: `form = <formname>`. E.g.:
* Static: `form = Person`
* Dynamic: `form = {{form:S}}` (the left `form` is a keyword for QFQ, the right `form` is a free chooseable variable name)
* Dynamic: `form = {{form:SE}}` (the left `form` is a keyword for QFQ, the right `form` is a free chooseable variable name)
* With the `Dynamic` option, it's easily possible to use one Typo3 page and display different forms on that specific
page. This is nice to configure few Typo 3 pages. The disadvantage is that the user might loose the navigation.
......@@ -6597,13 +6597,13 @@ SYSTEM
Auto Cron
---------
The `autocron` service fires periodically jobs like `send a mail` or `open a webpage`.
The `autocron` service fires periodically jobs like `open a webpage` or `send mail`.
* The frequency can be configured.
* Minimal time distance is 1 minute.
* Will be triggered via system cron. Minimal time distance is 1 minute.
* Starttime and frequency configureable.
* Per job:
* If a job's runs and receives the next trigger, the running job will be completed first.
* If a job still runs and receives the next trigger, the running job will be completed first.
* If more than one trigger arrives during a run, only one trigger will be processed.
* If the system misses a run, it will be played as soon as the system is online again.
* If multiple runs are missed, only one run is fired as soon as the system is online again.
......@@ -6620,14 +6620,15 @@ Setup a cron entry, typically as the webserver user ('www-data' on debian): ::
Create / edit `autocron` jobs
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Create a T3 page with a QFQ record. Such page should be access restricted and is only needed to edit `autocron` jobs: ::
Create a T3 page with a QFQ record (similar to the formeditor). Such page should be access restricted and is only needed
to edit `autocron` jobs: ::
form={{form:S}}
dbIndex={{DB_INDEX_QFQ:Y}}
form={{form:S}}
10 {
# Table header.
sql = SELECT CONCAT('p:{{pageId:T}}&form=cron') AS _pagen, 'id', 'Next run','Frequency','Comment','Last run','Status' FROM (SELECT 1) AS fake WHERE '{{form:SE}}'=''
sql = SELECT CONCAT('p:{{pageId:T}}&form=cron') AS _pagen, 'id', 'Next run','Frequency','Comment','Last run','In progress', 'Status' FROM (SELECT 1) AS fake WHERE '{{form:SE}}'=''
head = <table class='table table-hover qfq-table-50'>
tail = </table>
rbeg = <thead><tr>
......@@ -6637,34 +6638,60 @@ Create a T3 page with a QFQ record. Such page should be access restricted and is
10 {
# All Cron Jobs
sql = SELECT CONCAT('<tr class="', IF(c.lastStatus LIKE 'Error%','danger',''),
IF(c.inProgress!=0 AND DATE_ADD(c.inProgress, INTERVAL 10 MINUTE)<NOW(),' warning',''),
IF(c.status='enable','',' text-muted'), '">'),
sql = SELECT CONCAT('<tr class="',
IF(c.lastStatus LIKE 'Error%','danger',''),
IF(c.inProgress!=0 AND DATE_ADD(c.inProgress, INTERVAL 10 MINUTE)<NOW(),' warning',''),
IF(c.status='enable','',' text-muted'),'" ',
IF(c.inProgress!=0 AND DATE_ADD(c.inProgress, INTERVAL 10 MINUTE)<NOW(),'title="inProgress > 10mins"',
IF(c.lastStatus LIKE 'Error%','title="Status: Error"','')),
'>'),
'<td>', CONCAT('p:{{pageId:T}}&form=cron&r=', c.id) AS _pagee, '</td><td>',
c.id, '</td><td>',
IF(c.nextrun=0,"", DATE_FORMAT(c.nextrun, "%d.%m.%y %H:%i:%s")), '</td><td>',
c.frequency, '</td><td>',
c.comment, '</td><td>',
IF(c.lastrun=0,"", DATE_FORMAT(c.lastrun,"%d.%m.%y %H:%i:%s")), '</td><td>',
IF(c.inProgress=0,"", DATE_FORMAT(c.inProgress,"%d.%m.%y %H:%i:%s")), '</td><td>',
LEFT(c.laststatus,40) AS '_+pre', '</td><td>',
CONCAT('U:form=cron&r=', c.id) AS _paged, '</td></tr>'
FROM Cron AS c
ORDER BY c.id
FROM Cron AS c
ORDER BY c.id
}
}
Usage
^^^^^
The OS `cron` service will call the `QFQ autocron` every minute. `QFQ autocron` checks if there is a pending job, by looking
for jobs with `Next run`<=NOW(). All found jobs will be fired - depending on their type, such jobs will send mail(s) or
The system `cron` service will call the `QFQ autocron` every minute. `QFQ autocron` checks if there is a pending job, by looking
for jobs with `nextRun`<=NOW(). All found jobs will be fired - depending on their type, such jobs will send mail(s) or
open a `webpage`. A `webpage` will mostly be a local T3 page with at least one QFQ record on it. Such a QFQ record might
do some manipulation on the database or any other wished task.
do some manipulation on the database or any other task.
A jop with `nextRun`=0 won't never be started.
Job: repeating
''''''''''''''
* frequency: '1 MINUTE', '2 DAY', '1 MONTH', ....
After finishing a job, `nextRun` will be increased by `frequency`. If `nextRun` still points in the past, it will be
increased by `frequency` again, until it points to the future.
Job: asynchronous
'''''''''''''''''
* frequency: <empty>
Auto repeating is switched off. Such jobs are called `asynchronous`.
If `nextRun` is > 0 and in the past, the job will be fired. After the job has been done, `nextRun` will be set to 0.
After finishing a job, `Next run` will be increased by `Frequency`. If `Next run` still points in the past, it will be
increased by Frequency again, until it points to the future.
This is useful for jobs which have to be fired from time to time, and should not be triggered in parallel.
With `Next run`=0 the auto repeating is switched off.
To fire such an asynchronous just set `nextRun`=NOW() and wait for the next system cron run.
Type: Mail
''''''''''
......@@ -6673,8 +6700,7 @@ At the moment there is a special sendmail notation - this will change in the fut
* `Mail`: ::
{{!SELECT 'john@doe.com' AS sendMailTo, 'Custom subject' AS sendMailSubject, 'jane@doe.com' AS sendMailFrom,
123 AS sendMailGrId, 456 AS sendMailXId}}
{{!SELECT 'john@doe.com' AS sendMailTo, 'Custom subject' AS sendMailSubject, 'jane@doe.com' AS sendMailFrom, 123 AS sendMailGrId, 456 AS sendMailXId}}
Autocron will send as many mails as records are selected by the SQL query in field `Mail`. Field `Mail body` provides
the mail text.
......
......@@ -10,6 +10,6 @@ $EM_CONF[$_EXTKEY] = array(
'dependencies' => 'fluid,extbase',
'clearcacheonload' => true,
'state' => 'alpha',
'version' => '0.25.15a'
'version' => '0.25.15'
);
......@@ -73,12 +73,12 @@ class AutoCron {
*/
private function checkForOldJobs($ageMaxMinutes) {
$sql = "SELECT CONCAT('cron.id=', c.id, ' Started at: ', c.inProgress) FROM Cron AS c WHERE DATE_ADD(c.inProgress, INTERVAL $ageMaxMinutes MINUTE)<NOW() AND c.status='enable' ";
$sql = "SELECT CONCAT('AutoCron: cron.id=', c.id, ' in progress since: ', c.inProgress, ' - older than 10 mins') FROM Cron AS c WHERE DATE_ADD(c.inProgress, INTERVAL $ageMaxMinutes MINUTE)<NOW() AND c.status='enable' ";
// If there are too long running jobs: throw an exception
$rows = $this->dbArray[$this->dbIndexQfq]->sql($sql, ROW_REGULAR);
if (!empty($rows)) {
echo implode(PHP_EOL, $rows);
echo implode(PHP_EOL, $rows[0]);
}
}
......@@ -264,6 +264,7 @@ class AutoCron {
foreach ($jobs as $job) {
$this->store->setStore($job, STORE_PARENT_RECORD, true);
$nextRun = $job[AUTOCRON_NEXT_RUN];
// Start progress counter
$this->dbArray[$this->dbIndexQfq]->sql("UPDATE Cron SET inProgress=NOW() WHERE id=? LIMIT 1", ROW_REGULAR, [$job[COLUMN_ID]]);
......@@ -279,9 +280,20 @@ class AutoCron {
throw new ShellException('Unknown cron.type value: ' . $job[AUTOCRON_TYPE], ERROR_UNKNOWN_MODE);
}
// Finish Job
$job = $this->calcNextRun($job);
$sql = "UPDATE Cron SET lastRun=NOW(), lastStatus=?, nextRun=?, inProgress=0 WHERE id=? LIMIT 1";
// Check 'asynchronous' job if they have been triggered during processing: prepare to be fired during the next cron turn again.
if ($job[AUTOCRON_FREQUENCY] == '') {
// Get latest c.nextRun to compare and check for an update in between
$rowCheckAsynchronous = $this->dbArray[$this->dbIndexQfq]->sql("SELECT c.nextRun FROM Cron AS c WHERE id=? LIMIT 1", ROW_EXPECT_1, [$job[COLUMN_ID]]);
if (strcmp($nextRun, $rowCheckAsynchronous[AUTOCRON_NEXT_RUN]) < 0) {
// There was an update on the current job in between: preserve 'nextRun' to fire the job on the next system cron turn.
$job[AUTOCRON_NEXT_RUN] = $rowCheckAsynchronous[AUTOCRON_NEXT_RUN];
}
}
// Finish Job
$sql = "UPDATE Cron SET lastRun=inProgress, lastStatus=?, nextRun=?, inProgress=0 WHERE id=? LIMIT 1";
$this->dbArray[$this->dbIndexQfq]->sql($sql, ROW_REGULAR, [$job[AUTOCRON_LAST_STATUS], $job[AUTOCRON_NEXT_RUN], $job[COLUMN_ID]]);
}
......
......@@ -108,7 +108,7 @@ $UPDATE_ARRAY = array(
"ALTER TABLE `FormElement` CHANGE `label` `label` VARCHAR(511) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL DEFAULT ''",
],
'0.25.15a' => [
'0.25.16' => [
"ALTER TABLE `Cron` ADD `outputFile` VARCHAR(255) NOT NULL AFTER `comment`",
"ALTER TABLE `Cron` ADD `outputMode` ENUM('overwrite','append') NOT NULL DEFAULT 'append' AFTER `outputFile`",
"ALTER TABLE `Cron` ADD `outputPattern` VARCHAR(255) NOT NULL AFTER `outputMode`",
......
......@@ -423,32 +423,33 @@ VALUES
'none');
# FormElements: AutoCron
INSERT INTO FormElement (formId, name, label, mode, modeSql, type, encode, checkType, ord, parameter, size, note, dynamicUpdate)
INSERT INTO FormElement (formId, name, label, mode, modeSql, type, encode, checkType, ord, parameter, size, note, dynamicUpdate, bsLabelColumns, bsInputColumns, bsNoteColumns)
VALUES
(4, 'status', 'Enabled', 'show', '', 'checkbox', 'specialchar', 'alnumx', 10, '', '', '', 'no'),
(4, 'type', 'Type', 'show', '', 'radio', 'specialchar', 'alnumx', 20, 'buttonClass=btn-default', '', '', 'yes'),
(4, 'status', 'Enabled', 'show', '', 'checkbox', 'specialchar', 'alnumx', 10, '', '', '', 'no', '', '', ''),
(4, 'type', 'Type', 'show', '', 'radio', 'specialchar', 'alnumx', 20, 'buttonClass=btn-default', '', '', 'yes', '', '', ''),
(4, 'nextRun', 'Next run', 'show', '', 'text', 'specialchar', 'alnumx', 30,
'extraButtonInfo = Cronjob will be started if specified timestamp is over. If timestamp=0: Job will never be started<br>Every time the jobs runs, this timestamp will be increased automatically by "frequency".',
'', '', 'no'),
'', '', 'no', '', '', ''),
(4, 'frequency', 'Frequency', 'show', '', 'text', 'specialchar', 'alnumx', 40,
'extraButtonInfo = Repeat AutoCron-job with the specified interval. If empty: no repeating.<br>E.g.: "1 DAY", "15 MINUTE'', "6 MONTH" - used directly in SQL-Function "DATE_ADD(&lt;nextrun&gt;, INTERVAL &lt;frequency&gt;)"',
'', '', 'no'),
(4, 'comment', 'Comment', 'show', '', 'text', 'specialchar', 'allbut', 50, '', '', '', 'no'),
'', '', 'no', '', '', ''),
(4, 'comment', 'Comment', 'show', '', 'text', 'specialchar', 'allbut', 50, '', '', '', 'no', '', '', ''),
(4, 'sql1', 'Mail', 'show', '{{SELECT IF("{{type:FR:alnumx}}"="mail","show","hidden") }}', 'text', 'none', 'all', 60,
'extraButtonInfo = Query: &#123;&#123;!SELECT ... as sendMailTo...&#125;&#125;<br><b>sendMailTo / sendMailCc / sendMailBcc</b>: Separate multiple by comma.<br><b>sendMailFrom</b><br><b>sendMailSubject</b><br><b>sendMailReplyTo</b>: Optional<br><b>sendMailFlagAutoSubmit</b>: Optional. on|off. Default on - if "on", suppresses OoO answers from receivers.<br><b>sendMailGrId</b>: Optional<br><b>sendMailXId</b>: Optional',
'60,4', '', 'yes'),
'60,4', '', 'yes', '', '', ''),
(4, 'content', '{{SELECT IF("{{type:FR:alnumx}}"="mail","Mail body","URL") }}', 'show', '', 'text', 'none', 'all', 70,
'extraButtonInfo = Website: URL<br>Mail: Static Body or &#123;{SELECT ...&#125;}', '40,4', '', 'yes'),
(4, 'outputFile', 'Log output to file', 'show', '{{SELECT IF("{{type:FR:alnumx}}"="mail","hidden","show") }}', 'text', 'none', 'all', 80, '', '', '', 'yes'),
(4, 'outputMode', 'Mode output', 'show', '{{SELECT IF("{{type:FR:alnumx}}"="mail","hidden","show") }}', 'radio', 'specialchar', 'alnumx', 90, 'buttonClass=btn-default', '', '', 'yes'),
(4, 'outputPattern', 'Pattern to look for on output', 'show', '{{SELECT IF("{{type:FR:alnumx}}"="mail","hidden","show") }}', 'text', 'none', 'all', 100, '', '', 'If pattern isn\'t found, return an error.<br>Check <a href="https://secure.php.net/manual/en/pcre.pattern.php">pcre</a> / <a href="https://regexp101.com">regexp101.com</a> ', 'yes'),
(4, 'lastRun', 'Last run', 'readonly', '', 'text', 'specialchar', 'alnumx', 120, '', '', '', 'no'),
(4, 'lastStatus', 'Laststatus', 'readonly', '', 'text', 'specialchar', 'alnumx', 130, '', '', '', 'no'),
(4, 'inProgress', 'Running', 'show', '', 'text', 'specialchar', 'alnumx', 140,
'extraButtonInfo = Starttime of running job. When job is finished, will be set back to 0. A new job will only be started, if this is 0.',
'', '', 'no');
'extraButtonInfo = Website: URL absolute like "http://..." or relative like "?id=pagealias..."<br>Mail: Static Body or &#123;{SELECT ...&#125;}', '40,4', '', 'yes', '', '', ''),
(4, 'outputFile', 'Log output to file', 'show', '{{SELECT IF("{{type:FR:alnumx}}"="mail","hidden","show") }}', 'text', 'none', 'all', 80, '', '', '', 'yes', '', '', ''),
(4, 'outputMode', 'Mode output', 'show', '{{SELECT IF("{{type:FR:alnumx}}"="mail","hidden","show") }}', 'radio', 'specialchar', 'alnumx', 90, 'buttonClass=btn-default', '', '', 'yes', '', '', ''),
(4, 'outputPattern', 'Pattern to look for on output', 'show', '{{SELECT IF("{{type:FR:alnumx}}"="mail","hidden","show") }}', 'text', 'none', 'all', 100, '', '', 'If pattern isn\'t found, return an error.<br>Check <a href="https://secure.php.net/manual/en/pcre.pattern.php">pcre</a> / <a href="https://regexp101.com">regexp101.com</a> ', 'yes', '', '', ''),
(4, 'lastRun', 'Last run', 'readonly', '', 'text', 'specialchar', 'alnumx', 120, '', '', '', 'no', '', '', ''),
(4, 'lastStatus', 'Laststatus', 'readonly', '', 'text', 'specialchar', 'alnumx', 130, '', '50,6', '', 'no', '3', '9',
'0'),
(4, 'inProgress', 'In progress since', 'show', '', 'text', 'specialchar', 'alnumx', 140,
'extraButtonInfo = Start time of a running job. When job is finished, this will be set back to 0. A new job will only be started, if this is 0. A progress duration >10mins will be treated as an error.',
'', '', 'no', '', '', '');
CREATE TABLE IF NOT EXISTS `Split` (
`id` INT(11) NOT NULL AUTO_INCREMENT,
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment