#3706 / Check File Upload: a) mime type, b) max file size

Implemented: file upload check for mime type and max file size.
File.php, AbstractBuildForm.php: Implement FE_FILE_MIME_TYPE_ACCEPT and FE_FILE_MAX_FILE_SIZE

.gitignore

+/.python_virtualenv
 /.plantuml_install
 /doc/*.pdf
 /.doc_plantuml

doc/CODING.md

@@ -176,7 +176,7 @@ Upload to server, before 'save'
      [STORE_EXTRA][<uploadSip>][FILES_FLAG_DELETE]='1'
   * An optional previous upload file (still not saved on the final place) will be deleted.
   * An optional existing variable [STORE_EXTRA][<uploadSip>][FILES_TMP_NAME] will be deleted. The 'flagDelete' must not 
-    be change - it's later needed to detect to delete earlier uploaded files.
+    be change - it's later needed to detect to delete, earlier uploaded files.
 
 Form save
 .........

extension/Documentation/Manual.rst

@@ -51,21 +51,21 @@ For the `download`_ function, the program `pdftk` is necessary to concatenate PD
 
 Preparation for Ubuntu 14.04::
 
-	sudo apt-get install php5-mysqlnd php5-intl pdftk
+	sudo apt-get install php5-mysqlnd php5-intl
+	sudo apt-get install pdftk file                # for file upload and PDF
 	sudo php5enmod mysqlnd
 	sudo service apache2 restart
 
 Preparation steps for Ubuntu 16.04::
 
 	sudo apt install php7.0-intl
-	sudo apt install pdftk libxrender1        # for PDF and 'HTML to PDF' (wkhtmltopdf)
+	sudo apt install pdftk libxrender1 file        # for file upload, PDF and 'HTML to PDF' (wkhtmltopdf)
 
 .. _wkhtmltopdf:
 
 wkhtmltopdf
 ^^^^^^^^^^^
 
-
 `wkhtmltopdf`  `<http://wkhtmltopdf.org/>`_ will be used by QFQ to offer 'website print' and 'HTML to PDF' conversion.
 The converter is not included in QFQ and has to be manually installed.
 
@@ -2237,20 +2237,40 @@ An upload element is based on a 'file browse'-button and a 'trash'-button (=dele
 The 'file browse'-button is displayed, if there is no file uploaded already.
 The 'trash'-button is displayed, if there is a file uploaded already.
 
-After clicking on the browse brutton , the user can select a file from the local filesystem.
+After clicking on the browse brutton, the user select a file from the local filesystem.
 After choosing the file, the upload starts immediately, shown by a turning wheel. When the server received the whole file
-and accepts the file, the 'file browse'-button dissappears and the filename is shown, followed by a 'trash'-button.
+and accepts (see below) the file, the 'file browse'-button disappears and the filename is shown, followed by a 'trash'-button.
 Either the user is satisfied now or the user can delete the uploaded file (and maybe upload another one).
 
 Until this point, the file is cached on the server but not copied to the `fileDestination`. The user have to save the
-current record, either to finalize the upload or to delete a previous uploaded file.
+current record, either to finalize the upload and/or to delete a previous uploaded file.
+
+The FormElement behaves like a
+
+* 'native FormElement' (showing controls/text on the form) as well as an
+* 'action FormElement' by fireing queries and doing some additional actions during form save.
 
-The FormElement behaves like a 'native FormElement' (showing controls/text on the form) as well as an 'action FormElement'
-by fireing queries and doing some additional actions during form save. Inside the *Form editor* it's shown as a 'native FormElement'.
+Inside the *Form editor* it's shown as a 'native FormElement'.
+During saving the current record, it behaves like an action FormElement
+and will be processed after saving the primary record and before any action FormElements are processed.
 
 * *FormElement.parameter*:
 
-  * *accept*: `image/*,video/*,audio/*,.doc,.docx,.pdf,<mime type>`
+  * *accept*: `<mime type>,image/*,video/*,audio/*,.doc,.docx,.pdf`
+
+    * List of mime types (also known as 'media types'): http://www.iana.org/assignments/media-types/media-types.xhtml
+    * If none is specified, 'application/pdf' is set. This forces that always (!) one type is specified.
+    * One or more media types might be specified, seperated by ','.
+    * Different browser respect the given definitions in different ways. Typically the 'file choose' dialog offer:
+
+      * the specified mime type (some browers only show 'custom', if more than one mime type is given),
+      * the option 'All files' (the user is always free to **try** to upload other filetypes),
+      * the 'file choose' dialog only offers files of the selected (in the dialog) type.
+
+    * If for a specific filetype is no mime type available, the definition of file extension(s) is possible. This is **less
+      secure**, cause there is no *content* check on the server after the upload.
+
+  * *maxFileSize*: max filesize in Bytes for an uploaded file. Default: 10485760 (=10MB)
 
   * *fileDestination*: Destination where to copy the file. A good practice is to specify a relative `fileDestination` -
     such an installation (filesystem and database) are moveable.
@@ -2275,6 +2295,14 @@ by fireing queries and doing some additional actions during form save. Inside th
 
   * *fileReplace=always*: If `fileDestination` exist - replace it by the new one.
 
+Immediately after the upload finished (before the user press save), the file will be checked on the server for it's
+content or file extension (see 'accept').
+
+The maximum size is defined by the minimum of `upload_max_filesize`, `post_max_size`and `memory_limit` (PHP script) in the php.ini.
+
+In case of broken uploads, please also check `max_input_time` in php.ini.
+
+
 Deleting a record and the referenced file
 '''''''''''''''''''''''''''''''''''''''''
 
@@ -3748,7 +3776,7 @@ Parameter and (element) sources
 
     If there is no `exportFilename` defined and `mode=file`, than the original filename is taken.
 
-    If the mimetype is different from the `exportFilename` extension, then the mimetype extension will be added to
+    If the mime type is different from the `exportFilename` extension, then the mime type extension will be added to
     `exportFilename`. This guarantees that a filemanager will open the file with the correct application.
 
     The user typically expect meaningful and distinct filenames for different download links.
@@ -3761,7 +3789,7 @@ Parameter and (element) sources
   * *mode* = <file | pdf | zip> - This parameter is optional and can be skipped in most situations. Mandatory
     for 'zip'.
 
-      * If `m:file`, the mimetype is derived dynamically from the specified file. In this mode, only one element source
+      * If `m:file`, the mime type is derived dynamically from the specified file. In this mode, only one element source
         is allowed per download link (no concatenation).
 
       * In case of multiple element sources, only `pdf` or `zip` is supported.

extension/qfq/qfq/AbstractBuildForm.php

@@ -2486,6 +2486,10 @@ abstract class AbstractBuildForm {
     public function buildFile(array $formElement, $htmlFormElementName, $value, array &$json, $mode = FORM_LOAD) {
         $attribute = '';
 
+        if (empty($formElement[FE_FILE_MIME_TYPE_ACCEPT])) {
+            $formElement[FE_FILE_MIME_TYPE_ACCEPT] = UPLOAD_DEFAULT_MIME_TYPE;
+        }
+
         # Build param array for uniq SIP
         $arr = array();
         $arr['fake_uniq_never_use_this'] = uniqid(); // make sure we get a new SIP. This is needed for multiple forms (same user) with r=0
@@ -2495,6 +2499,9 @@ abstract class AbstractBuildForm {
         $arr[CLIENT_RECORD_ID] = $this->store->getVar(SIP_RECORD_ID, STORE_SIP);
         $arr[CLIENT_PAGE_ID] = 'fake';
         $arr[EXISTING_PATH_FILE_NAME] = $value;
+        $arr[FE_FILE_MIME_TYPE_ACCEPT] = $formElement[FE_FILE_MIME_TYPE_ACCEPT];
+        $arr[FE_FILE_MAX_FILE_SIZE] = empty($formElement[FE_FILE_MAX_FILE_SIZE]) ? UPLOAD_DEFAULT_MAX_SIZE : $formElement[FE_FILE_MAX_FILE_SIZE];
+
         $sipUpload = $this->sip->queryStringToSip(OnArray::toString($arr), RETURN_SIP);
 
         $hiddenSipUpload = $this->buildNativeHidden($htmlFormElementName, $sipUpload);
@@ -2504,7 +2511,7 @@ abstract class AbstractBuildForm {
 //        $attribute .= Support::doAttribute('class', 'form-control');
         $attribute .= Support::doAttribute('type', 'file');
         $attribute .= Support::doAttribute('title', $formElement['tooltip']);
-        $attribute .= $this->getAttributeList($formElement, ['autofocus', 'accept']);
+        $attribute .= $this->getAttributeList($formElement, [FE_AUTOFOCUS, FE_FILE_MIME_TYPE_ACCEPT]);
         $attribute .= Support::doAttribute('data-load', ($formElement[FE_DYNAMIC_UPDATE] === 'yes') ? 'data-load' : '');
         $attribute .= Support::doAttribute('data-sip', $sipUpload);
 

extension/qfq/qfq/Constants.php

@@ -202,8 +202,11 @@ const ERROR_TOO_MANY_PARAMETER = 1409;
 
 // Upload
 const ERROR_UPLOAD = 1500;
-const ERROR_UNKNOWN_ACTION = 1502;
-const ERROR_NO_TARGET_PATH_FILE_NAME = 1503;
+const ERROR_UPLOAD_TOO_BIG = 1501;
+const ERROR_UPLOAD_FILE_TYPE = 1502;
+const ERROR_UPLOAD_GET_MIME_TYPE = 1503;
+const ERROR_UNKNOWN_ACTION = 1504;
+const ERROR_NO_TARGET_PATH_FILE_NAME = 1505;
 
 // LDAP
 const ERROR_LDAP_CONNECT = 1600;
@@ -687,6 +690,8 @@ const FE_SHOW_ZERO = 'showZero';  // value: 0|1
 const FE_FILE_DESTINATION = 'fileDestination'; // Target pathFilename for an uploaded file.
 const FE_FILE_REPLACE_MODE = 'fileReplace'; // Target pathFilename for an uploaded file.
 const FE_FILE_REPLACE_MODE_ALWAYS = 'always'; // Target pathFilename for an uploaded file.
+const FE_FILE_MIME_TYPE_ACCEPT = 'accept'; // Target pathFilename for an uploaded file.
+const FE_FILE_MAX_FILE_SIZE = 'maxFileSize'; // Target pathFilename for an uploaded file.
 const FE_SQL_VALIDATE = 'sqlValidate'; // Action: Query to validate form load
 const FE_EXPECT_RECORDS = 'expectRecords'; // Action: expected number of rows of FE_SQL_VALIDATE
 const FE_MESSAGE_FAIL = 'messageFail'; // Action: Message to display, if FE_SQL_VALIDATE fails.
@@ -817,7 +822,8 @@ const UPLOAD_MODE_UNCHANGED = 'unchanged';
 const UPLOAD_MODE_NEW = 'new';
 const UPLOAD_MODE_DELETEOLD = 'deleteOld';
 const UPLOAD_MODE_DELETEOLD_NEW = 'deleteOld+new';
-
+const UPLOAD_DEFAULT_MAX_SIZE = 10485760; /* 10MB */
+const UPLOAD_DEFAULT_MIME_TYPE = 'application/pdf';
 // $_FILES
 const FILES_NAME = 'name';
 const FILES_TMP_NAME = 'tmp_name';

extension/qfq/qfq/File.php

@@ -85,8 +85,15 @@ class File {
             throw new UserFormException($this->uploadErrMsg[$newArr[FILES_ERROR]], ERROR_UPLOAD);
         }
 
-        //TODO: do necessary checks with uploaded file HERE!!!
+        $maxFileSize = $this->store->getVar(FE_FILE_MAX_FILE_SIZE, STORE_SIP);
+        if ($statusUpload['size'] >= $maxFileSize) {
+            throw new UserFormException('File to big. Max size allowed: ' . $maxFileSize, ERROR_UPLOAD_TOO_BIG);
+        }
 
+        $accept = $this->store->getVar(FE_FILE_MIME_TYPE_ACCEPT, STORE_SIP);
+        if (!$this->checkFileType($statusUpload['tmp_name'], $statusUpload['name'], $accept)) {
+            throw new UserFormException('Filetype not allowed. Allowed: ' . $accept, ERROR_UPLOAD_FILE_TYPE);
+        }
 
         // rename uploaded file: ?.cached
         $filenameCached = Support::extendFilename($statusUpload[FILES_TMP_NAME], UPLOAD_CACHED);
@@ -96,9 +103,70 @@ class File {
     }
 
     /**
-     * @param string $keyStoreExtra
+     * Checks the file filetype against the allowed mimetype definition. Return true as soon as one match is found.
+     * Types recognized:
+     *   * 'mime type' as delivered by `file` which matches a definition on http://www.iana.org/assignments/media-types/media-types.xhtml
+     *   * Joker based: audio/*, video/*, image/*
+     *   * Filename extension based: .pdf,.doc,..
+     *
+     * @param string $tmp_name
+     * @param string $name
+     * @param string $accept
+     * @return bool
+     * @throws UserFormException
+     */
+    private function checkFileType($tmp_name, $name, $accept) {
+
+        $return_var = 0;
+
+        // E.g.: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet; charset=binary'
+        $fileMimeType = exec('file --brief --mime ' . $tmp_name, $output, $return_var);
+        if ($return_var != 0) {
+            throw new UserFormException('Error get mime type of upload.', ERROR_UPLOAD_GET_MIME_TYPE);
+        }
+        // Strip optional '; charset=binary'
+        $arr = explode(';', $fileMimeType, 2);
+        $fileMimeType = $arr[0];
+
+        // Split between 'Media Type' and 'Media Subtype'
+        $fileMimeTypeSplitted = explode('/', $arr[0], 2);
+
+        $path_parts = pathinfo($name); // to extract the filename extension of the uploaded file.
+
+        // Process all listed mimetypes (incl. filename extension and joker)
+        // $accept e.g.: 'image/*,application/pdf,.pdf'
+        $arr = explode(',', $accept); // Split multiple defined mimetypes/extensions in single chunks.
+        foreach ($arr as $listElementMimeType) {
+            $listElementMimeType = trim($listElementMimeType);
+            if ($listElementMimeType == '') {
+                continue; // will be skipped
+            } elseif ($listElementMimeType[0] == '.') { // Check for defintion 'filename extension'
+                if ('.' . $path_parts['extension'] == $listElementMimeType) {
+                    return true;
+                }
+            } else {
+                // Check for Joker, e.g.: 'image/*'
+                $splitted = explode('/', $listElementMimeType, 2);
+
+                if ($splitted[1] == '*') {
+                    if ($splitted[0] == $fileMimeTypeSplitted[0]) {
+                        return true;
+                    }
+                } elseif ($fileMimeType == $listElementMimeType) {
+                    return true;
+                }
+            }
+        }
+
+        return false;
+    }
+
+    /**
+     * @param $sipUpload
+     * @param $statusUpload
      * @throws CodeException
      * @throws UserFormException
+     * @internal param string $keyStoreExtra
      */
     private function doDelete($sipUpload, $statusUpload) {
 

extension/qfq/tests/phpunit/DeleteTest.php

@@ -21,19 +21,7 @@ class DeleteTest extends \AbstractDatabaseTest {
         $delete = new Delete(true);
 
         // empty 'form' not allowed
-        $delete->process(array(), 123);
-    }
-
-    /**
-     * @expectedException \qfq\CodeException
-     */
-    public function testProcessException1() {
-
-        $delete = new Delete(true);
-
-        // 'form' with empty tablename not allowed
-        $form[F_TABLE_NAME] = '';
-        $delete->process($form, 123);
+        $delete->process('', 123);
     }
 
     /**
@@ -44,8 +32,8 @@ class DeleteTest extends \AbstractDatabaseTest {
         $delete = new Delete(true);
 
         // empty record id not allowed
-        $form[F_TABLE_NAME] = 'Person';
-        $delete->process($form, '');
+        $formName = 'Person';
+        $delete->process($formName, '');
     }
 
     /**
@@ -54,10 +42,10 @@ class DeleteTest extends \AbstractDatabaseTest {
     public function testProcessException3() {
 
         $delete = new Delete(true);
-        $form[F_TABLE_NAME] = 'Person';
+        $formName = 'Person';
 
         // record id = 0 not allowed
-        $delete->process($form, 0);
+        $delete->process($formName, 0);
     }
 
     /**