[Backport] Prevent layout cache corruption in edge case

CHANGELOG.md

@@ -1,3 +1,60 @@
+2.1.14
+=============
+* GitHub issues:
+    * [#7723](https://github.com/magento/magento2/issues/7723) -- Catalog rule contains-condition not saving multiple selection in 2.1.2 (fixed in [magento/magento2#13546](https://github.com/magento/magento2/pull/13546))
+    * [#13214](https://github.com/magento/magento2/issues/13214) -- Not a correct displaying for Robots.txt (fixed in [magento/magento2#13550](https://github.com/magento/magento2/pull/13550))
+    * [#13315](https://github.com/magento/magento2/issues/13315) -- Mobile "Payment Methods" step looks bad on mobile (fixed in [magento/magento2#13980](https://github.com/magento/magento2/pull/13980))
+    * [#13474](https://github.com/magento/magento2/issues/13474) -- [2.1.10] Swagger not working for multistore installs?  (fixed in [magento/magento2#13486](https://github.com/magento/magento2/pull/13486))
+    * [#4173](https://github.com/magento/magento2/issues/4173) -- Cron schedule bug (fixed in [magento/magento2#14096](https://github.com/magento/magento2/pull/14096))
+    * [#5808](https://github.com/magento/magento2/issues/5808) -- [2.1.0] Problem on mobile when catalog gallery allowfullscreen is false (fixed in [magento/magento2#14098](https://github.com/magento/magento2/pull/14098))
+    * [#6694](https://github.com/magento/magento2/issues/6694) -- Override zip_codes.xml (fixed in [magento/magento2#14117](https://github.com/magento/magento2/pull/14117))
+    * [#10559](https://github.com/magento/magento2/issues/10559) -- Extending swatch functionality using javascript mixins does not work in Safari and MS Edge (fixed in [magento/magento2#12928](https://github.com/magento/magento2/pull/12928))
+    * [#3489](https://github.com/magento/magento2/issues/3489) -- CURL Json POST (fixed in [magento/magento2#14151](https://github.com/magento/magento2/pull/14151))
+    * [#5463](https://github.com/magento/magento2/issues/5463) -- The ability to store passwords using different hashing algorithms is limited (fixed in [magento/magento2#13886](https://github.com/magento/magento2/pull/13886))
+    * [#3882](https://github.com/magento/magento2/issues/3882) -- An XML comment node as parameter in widget.xml fails with fatal error (fixed in [magento/magento2#14219](https://github.com/magento/magento2/pull/14219))
+    * [#1931](https://github.com/magento/magento2/issues/1931) -- Can't cancel removal of a block or container in layout by setting remove attribute value to false (fixed in [magento/magento2#14198](https://github.com/magento/magento2/pull/14198))
+    * [#7403](https://github.com/magento/magento2/issues/7403) -- JS Translation Regex leads to unexpected results and untranslatable strings (fixed in [magento/magento2#14349](https://github.com/magento/magento2/pull/14349))
+    * [#7816](https://github.com/magento/magento2/issues/7816) -- Customer_account.xml file abused (fixed in [magento/magento2#14323](https://github.com/magento/magento2/pull/14323))
+    * [#10700](https://github.com/magento/magento2/issues/10700) -- Magento 2 Admin panel show loading on each page (fixed in [magento/magento2#14417](https://github.com/magento/magento2/pull/14417))
+    * [#11930](https://github.com/magento/magento2/issues/11930) -- setup:di:compile's generated cache files inaccessible by the web-server user (fixed in [magento/magento2#14417](https://github.com/magento/magento2/pull/14417))
+    * [#14572](https://github.com/magento/magento2/issues/14572) -- Specify the table when adding field to filter for the collection Eav/Model/ResourceModel/Entity/Attribute/Option/Collection.php (fixed in [magento/magento2#14596](https://github.com/magento/magento2/pull/14596))
+* GitHub pull requests:
+    * [magento/magento2#13949](https://github.com/magento/magento2/pull/13949) -- Fix misnamed namespace (by @Ethan3600)
+    * [magento/magento2#13545](https://github.com/magento/magento2/pull/13545) -- Backport of PR-5028 for Magento 2.1: Load jquery using requirejs to p… (by @hostep)
+    * [magento/magento2#13546](https://github.com/magento/magento2/pull/13546) -- Backport of PR-8246 for Magento 2.1: Fixes #7723 - saving multi selec… (by @hostep)
+    * [magento/magento2#13550](https://github.com/magento/magento2/pull/13550) -- Backport of MAGETWO-84006 for Magento 2.1: Fix robots.txt content typ… (by @hostep)
+    * [magento/magento2#13896](https://github.com/magento/magento2/pull/13896) -- MAGETWO-59112 Backport 2.1.x  (by @Ctucker9233)
+    * [magento/magento2#13812](https://github.com/magento/magento2/pull/13812) -- [Backport 2.1] Add RewriteBase directive template in .htaccess file into pub/static folder (by @ccasciotti)
+    * [magento/magento2#13658](https://github.com/magento/magento2/pull/13658) -- [Backport 2.1-develop] Show redirect_to_base config in store scope (by @JeroenVanLeusden)
+    * [magento/magento2#13980](https://github.com/magento/magento2/pull/13980) -- Backport of PR-13777. Mobile 'Payments methods' step looks bad on mobile (by @Frodigo)
+    * [magento/magento2#13987](https://github.com/magento/magento2/pull/13987) -- Backport of PR-13750 for Magento 2.1: Less clean up (by @Karlasa)
+    * [magento/magento2#14022](https://github.com/magento/magento2/pull/14022) -- fix catalog_rule_promo_catalog_edit.xml layout (by @Karlasa)
+    * [magento/magento2#13806](https://github.com/magento/magento2/pull/13806) -- [Backport 2.1] Add quoting for base path in DI compile command (by @simpleadm)
+    * [magento/magento2#13486](https://github.com/magento/magento2/pull/13486) -- [Backport 2.1-develop] Change the store code in Swagger based on a param (by @JeroenVanLeusden)
+    * [magento/magento2#14096](https://github.com/magento/magento2/pull/14096) -- [Backport 2.1] Schedule generation was broken (by @simpleadm)
+    * [magento/magento2#14098](https://github.com/magento/magento2/pull/14098) -- [Backport 2.1] MAGETWO-64250 Problem on mobile when catalog gallery allowfullscreen is false (by @simpleadm)
+    * [magento/magento2#14115](https://github.com/magento/magento2/pull/14115) -- [Backport 2.1] MAGETWO-71697: Fix possible bug when saving address with empty street line (by @simpleadm)
+    * [magento/magento2#14117](https://github.com/magento/magento2/pull/14117) -- [Backport 2.1]  MAGETWO-59258: Override module-directory/etc/zip_codes.xml only the last code of a country gets include (by @simpleadm)
+    * [magento/magento2#12928](https://github.com/magento/magento2/pull/12928) -- Issues #10559 - Extend swatch using mixins (M2.1) (by @srenon)
+    * [magento/magento2#14151](https://github.com/magento/magento2/pull/14151) -- [Backport 2.1] 8373: Fix CURL Json POST (by @simpleadm)
+    * [magento/magento2#13886](https://github.com/magento/magento2/pull/13886) -- #5463 - Use specified hashing algo in \Magento\Framework\Encryption\Encryptor::getHash (by @k4emic)
+    * [magento/magento2#14168](https://github.com/magento/magento2/pull/14168) -- [Backport 2.1] Added mage/translate component to customers's ajax login (by @ccasciotti)
+    * [magento/magento2#13654](https://github.com/magento/magento2/pull/13654) -- [Backport 2.1-develop] Update Store getConfig() to respect valid false return value (by @JeroenVanLeusden)
+    * [magento/magento2#14219](https://github.com/magento/magento2/pull/14219) -- Backport of PR-8772 for Magento 2.1: magento/magento2#3882 (by @hostep)
+    * [magento/magento2#14198](https://github.com/magento/magento2/pull/14198) -- [Backport] Can't cancel removal of a block or container in layout by setting remove attribute value to false (by @quisse)
+    * [magento/magento2#14349](https://github.com/magento/magento2/pull/14349) -- Backport of PR-10445 for Magento 2.1: Fix JS translation search (by @hostep)
+    * [magento/magento2#14332](https://github.com/magento/magento2/pull/14332) -- Backport: Fix for broken navigation menu on IE11 #14230 (by @sergiy-v)
+    * [magento/magento2#14323](https://github.com/magento/magento2/pull/14323) -- #7816: Customer_account.xml file abused (2.1) (by @mikewhitby)
+    * [magento/magento2#14417](https://github.com/magento/magento2/pull/14417) -- [BACKPORT 2.1] Removed cache backend option which explicitly set file permissions (by @xtremeperf)
+    * [magento/magento2#14436](https://github.com/magento/magento2/pull/14436) -- Fix HTML tags in meta description (by @vseager)
+    * [magento/magento2#14480](https://github.com/magento/magento2/pull/14480) -- [Backport 2.1] Return status in console commands (by @simpleadm)
+    * [magento/magento2#14497](https://github.com/magento/magento2/pull/14497) -- [backport] fix for button color in email template  (by @Karlasa)
+    * [magento/magento2#14348](https://github.com/magento/magento2/pull/14348) -- [Backport 2.1] Add json and xml support to the post method in socket client (by @simpleadm)
+    * [magento/magento2#14479](https://github.com/magento/magento2/pull/14479) -- [Backport 2.1] Configurable product price options by store (by @simpleadm)
+    * [magento/magento2#14505](https://github.com/magento/magento2/pull/14505) -- [Backport] Check if store id is not null instead of empty (by @quisse)
+    * [magento/magento2#14524](https://github.com/magento/magento2/pull/14524) -- [backport] fix translation issue with rating stars (by @Karlasa)
+    * [magento/magento2#14596](https://github.com/magento/magento2/pull/14596) -- Specify the table when adding field to filter (by @PierreLeMaguer)
+
 2.1.13
 =============
 * GitHub issues:

app/code/Magento/Backend/composer.json

@@ -22,7 +22,7 @@
         "magento/framework": "100.1.*"
     },
     "type": "magento2-module",
-    "version": "100.1.8",
+    "version": "100.1.9",
     "license": [
         "OSL-3.0",
         "AFL-3.0"

app/code/Magento/Catalog/Model/ImageUploader.php

@@ -64,6 +64,18 @@ class ImageUploader
      */
     protected $allowedExtensions;
 
+    /**
+     * List of allowed image mime types.
+     *
+     * @var array
+     */
+    private $allowedMimeTypes = [
+        'image/jpg',
+        'image/jpeg',
+        'image/gif',
+        'image/png',
+    ];
+
     /**
      * ImageUploader constructor
      *
@@ -218,6 +230,7 @@ public function moveFileFromTmp($imageName)
      * @return string[]
      *
      * @throws \Magento\Framework\Exception\LocalizedException
+     * @throws \Exception
      */
     public function saveFileToTmpDir($fileId)
     {
@@ -228,6 +241,10 @@ public function saveFileToTmpDir($fileId)
         $uploader->setAllowedExtensions($this->getAllowedExtensions());
         $uploader->setAllowRenameFiles(true);
 
+        if (!$uploader->checkMimeType($this->allowedMimeTypes)) {
+            throw new \Magento\Framework\Exception\LocalizedException(__('File validation failed.'));
+        }
+
         $result = $uploader->save($this->mediaDirectory->getAbsolutePath($baseTmpPath));
         unset($result['path']);
 

app/code/Magento/Catalog/Model/Product/Gallery/UpdateHandler.php

@@ -28,6 +28,9 @@ protected function processDeletedImages($product, array &$images)
         foreach ($images as &$image) {
             if (!empty($image['removed'])) {
                 if (!empty($image['value_id']) && !isset($picturesInOtherStores[$image['file']])) {
+                    if (preg_match('/\.\.(\\\|\/)/', $image['file'])) {
+                        continue;
+                    }
                     $recordsToDelete[] = $image['value_id'];
                     $catalogPath = $this->mediaConfig->getBaseMediaPath();
                     $isFile = $this->mediaDirectory->isFile($catalogPath . $image['file']);

app/code/Magento/Catalog/Test/Unit/Model/ImageUploaderTest.php

@@ -0,0 +1,166 @@
+<?php
+/**
+ * Copyright © Magento, Inc. All rights reserved.
+ * See COPYING.txt for license details.
+ */
+namespace Magento\Catalog\Test\Unit\Model;
+
+/**
+ * Magento\Catalog\Model\ImageUploader unit tests.
+ */
+class ImageUploaderTest extends \PHPUnit_Framework_TestCase
+{
+    /**
+     * @var \Magento\Catalog\Model\ImageUploader
+     */
+    private $imageUploader;
+
+    /**
+     * Core file storage database.
+     *
+     * @var \Magento\MediaStorage\Helper\File\Storage\Database|\PHPUnit_Framework_MockObject_MockObject
+     */
+    private $coreFileStorageDatabaseMock;
+
+    /**
+     * Media directory object (writable).
+     *
+     * @var \Magento\Framework\Filesystem|\PHPUnit_Framework_MockObject_MockObject
+     */
+    private $mediaDirectoryMock;
+
+    /**
+     * Media directory object (writable).
+     *
+     * @var \Magento\Framework\Filesystem\Directory\WriteInterface|\PHPUnit_Framework_MockObject_MockObject
+     */
+    private $mediaWriteDirectoryMock;
+
+    /**
+     * Uploader factory.
+     *
+     * @var \Magento\MediaStorage\Model\File\UploaderFactory|\PHPUnit_Framework_MockObject_MockObject
+     */
+    private $uploaderFactoryMock;
+
+    /**
+     * Store manager.
+     *
+     * @var \Magento\Store\Model\StoreManagerInterface|\PHPUnit_Framework_MockObject_MockObject
+     */
+    private $storeManagerMock;
+
+    /**
+     * @var \Psr\Log\LoggerInterface|\PHPUnit_Framework_MockObject_MockObject
+     */
+    private $loggerMock;
+
+    /**
+     * Base tmp path.
+     *
+     * @var string
+     */
+    private $baseTmpPath;
+
+    /**
+     * Base path.
+     *
+     * @var string
+     */
+    private $basePath;
+
+    /**
+     * Allowed extensions.
+     *
+     * @var string
+     */
+    private $allowedExtensions;
+
+    /**
+     * @inheritdoc
+     */
+    protected function setUp()
+    {
+        $this->coreFileStorageDatabaseMock = $this->getMockBuilder(
+            \Magento\MediaStorage\Helper\File\Storage\Database::class
+        )
+            ->disableOriginalConstructor()
+            ->getMock();
+        $this->mediaDirectoryMock = $this->getMockBuilder(
+            \Magento\Framework\Filesystem::class
+        )
+            ->disableOriginalConstructor()
+            ->getMock();
+        $this->mediaWriteDirectoryMock = $this->getMockBuilder(
+            \Magento\Framework\Filesystem\Directory\WriteInterface::class
+        )
+            ->disableOriginalConstructor()
+            ->getMock();
+        $this->mediaDirectoryMock->expects($this->any())->method('getDirectoryWrite')->willReturn(
+            $this->mediaWriteDirectoryMock
+        );
+        $this->uploaderFactoryMock = $this->getMockBuilder(
+            \Magento\MediaStorage\Model\File\UploaderFactory::class
+        )
+            ->disableOriginalConstructor()
+            ->getMock();
+        $this->storeManagerMock = $this->getMockBuilder(
+            \Magento\Store\Model\StoreManagerInterface::class
+        )
+            ->disableOriginalConstructor()
+            ->getMock();
+        $this->loggerMock = $this->getMockBuilder(\Psr\Log\LoggerInterface::class)
+            ->disableOriginalConstructor()
+            ->getMock();
+        $this->baseTmpPath = 'base/tmp/';
+        $this->basePath =  'base/real/';
+        $this->allowedExtensions = ['.jpg'];
+
+        $this->imageUploader =
+            new \Magento\Catalog\Model\ImageUploader(
+                $this->coreFileStorageDatabaseMock,
+                $this->mediaDirectoryMock,
+                $this->uploaderFactoryMock,
+                $this->storeManagerMock,
+                $this->loggerMock,
+                $this->baseTmpPath,
+                $this->basePath,
+                $this->allowedExtensions
+            );
+    }
+
+    public function testSaveFileToTmpDir()
+    {
+        $fileId = 'file.jpg';
+        $allowedMimeTypes = [
+            'image/jpg',
+            'image/jpeg',
+            'image/gif',
+            'image/png',
+        ];
+        /** @var \Magento\MediaStorage\Model\File\Uploader|\PHPUnit_Framework_MockObject_MockObject $uploader */
+        $uploader = $this->getMockBuilder(\Magento\MediaStorage\Model\File\Uploader::class)
+            ->disableOriginalConstructor()
+            ->getMock();
+        $this->uploaderFactoryMock->expects($this->once())->method('create')->willReturn($uploader);
+        $uploader->expects($this->once())->method('setAllowedExtensions')->with($this->allowedExtensions);
+        $uploader->expects($this->once())->method('setAllowRenameFiles')->with(true);
+        $this->mediaWriteDirectoryMock->expects($this->once())->method('getAbsolutePath')->with($this->baseTmpPath)
+            ->willReturn($this->basePath);
+        $uploader->expects($this->once())->method('save')->with($this->basePath)
+            ->willReturn(['tmp_name' => $this->baseTmpPath, 'file' => $fileId, 'path' => $this->basePath]);
+        $uploader->expects($this->atLeastOnce())->method('checkMimeType')->with($allowedMimeTypes)->willReturn(true);
+        $storeMock = $this->getMockBuilder(\Magento\Store\Model\Store::class)
+            ->disableOriginalConstructor()
+            ->setMethods(['getBaseUrl'])
+            ->getMock();
+
+        $this->storeManagerMock->expects($this->once())->method('getStore')->willReturn($storeMock);
+        $storeMock->expects($this->once())->method('getBaseUrl');
+        $this->coreFileStorageDatabaseMock->expects($this->once())->method('saveFile');
+
+        $result = $this->imageUploader->saveFileToTmpDir($fileId);
+
+        $this->assertArrayNotHasKey('path', $result);
+    }
+}

app/code/Magento/Catalog/composer.json

@@ -33,7 +33,7 @@
         "magento/module-catalog-sample-data": "Sample Data version:100.1.*"
     },
     "type": "magento2-module",
-    "version": "101.0.13",
+    "version": "101.0.14",
     "license": [
         "OSL-3.0",
         "AFL-3.0"

app/code/Magento/CatalogImportExport/Model/Import/Uploader.php

@@ -7,10 +7,12 @@
 
 use Magento\Framework\App\Filesystem\DirectoryList;
 use Magento\Framework\Filesystem\DriverPool;
+use Magento\Framework\App\ObjectManager;
 
 /**
  * Import entity product model
  *
+ * @SuppressWarnings(PHPMD.CouplingBetweenObjects)
  * @author      Magento Core Team <[email protected]>
  */
 class Uploader extends \Magento\MediaStorage\Model\File\Uploader
@@ -85,6 +87,11 @@ class Uploader extends \Magento\MediaStorage\Model\File\Uploader
      */
     protected $_coreFileStorage;
 
+    /**
+     * @var \Magento\Framework\App\Filesystem\DirectoryResolver
+     */
+    private $directoryResolver;
+
     /**
      * @param \Magento\MediaStorage\Helper\File\Storage\Database $coreFileStorageDb
      * @param \Magento\MediaStorage\Helper\File\Storage $coreFileStorage
@@ -93,6 +100,7 @@ class Uploader extends \Magento\MediaStorage\Model\File\Uploader
      * @param \Magento\Framework\Filesystem $filesystem
      * @param \Magento\Framework\Filesystem\File\ReadFactory $readFactory
      * @param null $filePath
+     * @param \Magento\Framework\App\Filesystem\DirectoryResolver|null $directoryResolver
      * @throws \Magento\Framework\Exception\LocalizedException
      */
     public function __construct(
@@ -102,7 +110,8 @@ public function __construct(
         \Magento\MediaStorage\Model\File\Validator\NotProtectedExtension $validator,
         \Magento\Framework\Filesystem $filesystem,
         \Magento\Framework\Filesystem\File\ReadFactory $readFactory,
-        $filePath = null
+        $filePath = null,
+        \Magento\Framework\App\Filesystem\DirectoryResolver $directoryResolver = null
     ) {
         if ($filePath !== null) {
             $this->_setUploadFile($filePath);
@@ -113,6 +122,8 @@ public function __construct(
         $this->_validator = $validator;
         $this->_directory = $filesystem->getDirectoryWrite(DirectoryList::ROOT);
         $this->_readFactory = $readFactory;
+        $this->directoryResolver = $directoryResolver
+            ?: ObjectManager::getInstance()->get(\Magento\Framework\App\Filesystem\DirectoryResolver::class);
     }
 
     /**
@@ -217,6 +228,7 @@ protected function _validateFile()
 
         $fileExtension = pathinfo($filePath, PATHINFO_EXTENSION);
         if (!$this->checkAllowedExtension($fileExtension)) {
+            $this->_directory->delete($filePath);
             throw new \Exception('Disallowed file type.');
         }
         //run validate callbacks
@@ -262,7 +274,10 @@ public function getTmpDir()
      */
     public function setTmpDir($path)
     {
-        if (is_string($path) && $this->_directory->isReadable($path)) {
+        if (is_string($path)
+            && $this->_directory->isReadable($path)
+            && $this->directoryResolver->validatePath($this->_directory->getAbsolutePath($path), DirectoryList::ROOT)
+        ) {
             $this->_tmpDir = $path;
             return true;
         }