project loading and adding/removing layers is slow if project has composers #19203

qgib · 2014-07-06T23:50:51Z

Author Name: Per Gammerath (Per Gammerath)
Original Redmine Issue: 10828
Affected QGIS version: 2.18.4
Redmine category:project_loading/saving

I don't know whether to report this as a bug or feature request, but QGIS is horribly slow at loading projects. What takes 19 seconds in Dufour, takes 32 seconds in Chugiak.

see also:
#12125

QGIS.zip (Per Gammerath)
With_layouts.qgs (Per Gammerath)

Related issue(s): #23423 (duplicates)
Redmine related issue(s): 15499

qgib · 2014-07-07T00:01:02Z

Author Name: Giovanni Manghi (@gioman)

it may be a bug, but anyway without attaching a project (and data) to replicate the issue it would be hard to troubleshoot it.

category_id was configured as Project Loading/Saving
status_id was changed from Open to Feedback

qgib · 2014-07-07T01:41:23Z

Author Name: Per Gammerath (Per Gammerath)

Here is my project.

qgib · 2014-07-07T01:44:30Z

Author Name: Nathan Woodrow (@NathanW2)

Have you enabled multicore rendering?

Settings -> Options -> Rendering

qgib · 2014-07-07T01:57:27Z

Author Name: Per Gammerath (Per Gammerath)

I can't upload the project. The file is 12MB.

qgib · 2014-07-07T01:59:41Z

Author Name: Per Gammerath (Per Gammerath)

Yes, Multicore rendering is enabled. Rendering is very good once the project is loaded.

qgib · 2014-07-07T02:23:29Z

Author Name: Giovanni Manghi (@gioman)

Per Gammerath wrote:

I can't upload the project. The file is 12MB.

dropbox (or similar)?

qgib · 2014-07-07T02:49:54Z

Author Name: Per Gammerath (Per Gammerath)

https://drive.google.com/file/d/0B8lAQWa28EprMVJSdDhVazNJLVE/edit?usp=sharing

qgib · 2014-07-07T03:44:12Z

Author Name: Giovanni Manghi (@gioman)

I had to change the data source path in your project but after that I can confirm your observation:

in qgis 2.2 qgis loads the layers, at the same time builds the layers tree and shortly after in renders the canvas
in qgis 2.4 it seems to be sequential: qgis laods the layers, then it stays busy, then the TOC tree is rendered all together, and then the layers are rendered in the canvas.

I don't know if this depends on multi-threading, the legend refactory and/or other factors. If this has a potential fix then we should tag this as a regression.

status_id was changed from Feedback to Open
subject was changed from Slow loading to Slow project loading in qgis 2.4/master when compared to 2.2
version was changed from 2.4.0 to master

qgib · 2014-07-07T22:53:45Z

Author Name: Per Gammerath (Per Gammerath)

I have been suspecting our project file of being messy and siltet, so I built a new project file and styled it to look like the original. The result was amazing, now the project loads in 16 seconds. This tells me, that there must be something wrong in the process of loading and saving project files. Our original project file was initially built in 2.0.1.
In the proces of building the new project file, QGIS crashed more than 20 times, just from organizing layers and loading styles. It's highly unstable.

qgib · 2014-07-07T23:34:22Z

Author Name: Per Gammerath (Per Gammerath)

Hmmm, just added all the composer layouts, and now everything is slow again. So maybe this is all about composer layouts.

qgib · 2014-07-08T00:37:11Z

Author Name: Giovanni Manghi (@gioman)

In the proces of building the new project file, QGIS crashed more than 20 times, just from organizing layers and loading styles.

something that can be replicated every time?

qgib · 2014-07-08T00:37:54Z

Author Name: Giovanni Manghi (@gioman)

Per Gammerath wrote:

Hmmm, just added all the composer layouts, and now everything is slow again. So maybe this is all about composer layouts.

it is just enough to add one layout? does it need to contain a particular object to make slow the loading of the project?

status_id was changed from Open to Feedback

qgib · 2014-07-08T02:25:50Z

Author Name: Per Gammerath (Per Gammerath)

Giovanni Manghi wrote:

In the proces of building the new project file, QGIS crashed more than 20 times, just from organizing layers and loading styles.

something that can be replicated every time?

It seems to be random. It crashed so many times that I had to save the project with each change to make sure I didn't have to start all over.

qgib · 2014-07-08T02:30:44Z

Author Name: Per Gammerath (Per Gammerath)

Giovanni Manghi wrote:

Per Gammerath wrote:

Hmmm, just added all the composer layouts, and now everything is slow again. So maybe this is all about composer layouts.

it is just enough to add one layout? does it need to contain a particular object to make slow the loading of the project?

I have 8 layouts, A4, A3, A2, A1, all with landscape and portrait orientation. I don't see how I can do without them.

qgib · 2014-07-08T04:23:53Z

Author Name: Per Gammerath (Per Gammerath)

If I'm right about the fact that Layouts slow down the loading of projects, would it be reasonable to let QGIS skip them and only load them when needed?

qgib · 2014-07-09T06:23:19Z

Author Name: Giovanni Manghi (@gioman)

Per Gammerath wrote:

I have 8 layouts, A4, A3, A2, A1, all with landscape and portrait orientation. I don't see how I can do without them.

I didn't mean this.

As you said that you built the projects from scratch and that at some point you realized that it depends on layouts, then I would like to know if it was enough to add 1 layout (to get a slow to load project) and if the content of the layout made some difference.

qgib · 2014-07-09T06:46:40Z

Author Name: Per Gammerath (Per Gammerath)

For each layout I deleted, QGIS got faster. So I think it's about the number of layouts and not the content.
My layouts include a map, legend, small logo, date field and scale field.

qgib · 2014-07-09T20:57:10Z

Author Name: Nyall Dawson (@nyalldawson)

One thing that would help track this down would be if you could try removing different item types from the compositions and testing whether a single item type is causing the slow down. For instance, if you remove all the scale bars from all the compositions does this make any difference to the project load speed?

qgib · 2014-07-09T22:56:52Z

Author Name: Per Gammerath (Per Gammerath)

Nyall Dawson wrote:

One thing that would help track this down would be if you could try removing different item types from the compositions and testing whether a single item type is causing the slow down. For instance, if you remove all the scale bars from all the compositions does this make any difference to the project load speed?

I have tested some different scenariois, and load speeds are:

Project file with 8 layouts including a map, legend, scalebar, logo
30 seconds
Project file with 8 layouts including a map, scalebar, logo
24 seconds
Project file with 8 layouts including a map
23 seconds
Project file with 2 layouts including a map, scalebar, legend, logo
14 seconds

I think this indicates that the number of layouts and load speed is tied together.

qgib · 2014-07-10T00:56:06Z

Author Name: Nyall Dawson (@nyalldawson)

Are they the only elements in your layouts? ( map, scalebar, legend, logos ?). No other items at all, like arrows or shapes?

qgib · 2014-07-10T01:45:42Z

Author Name: Per Gammerath (Per Gammerath)

Nyall Dawson wrote:

Are they the only elements in your layouts? ( map, scalebar, legend, logos ?). No other items at all, like arrows or shapes?

No other objects.

qgib · 2014-07-16T12:32:12Z

Author Name: Martin Dobias (@wonder-sk)

Fixed in changeset "18a1c8a6360faf69a9b1bb9620b75baa7f1319cd".

status_id was changed from Feedback to Closed

qgib · 2014-07-16T21:22:29Z

Author Name: Paolo Cavallini (@pcav)

Better backport it to 2.4 branch?

status_id was changed from Closed to Feedback

qgib · 2014-07-17T02:13:14Z

Author Name: Giovanni Manghi (@gioman)

resolution was changed from to fixed/implemented

qgib · 2014-07-17T02:47:28Z

Author Name: Giovanni Manghi (@gioman)

Paolo Cavallini wrote:

Better backport it to 2.4 branch?

#18a1c8a6360faf69a9b1bb9620b75baa7f1319cd#commitcomment-7044552

status_id was changed from Feedback to Closed

qgib · 2014-07-17T06:37:36Z

Author Name: Jürgen Fischer (@jef-n)

Giovanni Manghi wrote:

Paolo Cavallini wrote:

Better backport it to 2.4 branch?

#18a1c8a6360faf69a9b1bb9620b75baa7f1319cd#commitcomment-7044552

done in 904443f

qgib · 2014-07-17T06:39:28Z

Author Name: Jürgen Fischer (@jef-n)

tag was changed from to backport

qgib · 2015-06-22T22:28:06Z

Author Name: Per Gammerath (Per Gammerath)

I've just downloaded and tested QGIS 2.8, and it strikes me that the issue with slow loading still exists, though the resolution is set to fixed. How come?

status_id was changed from Closed to Reopened

qgib · 2015-06-22T22:50:43Z

Author Name: Saber Razmjooei (@saberraz)

Do you have some sample projects with data?

status_id was changed from Reopened to Feedback
version was changed from master to 2.8.2

qgib · 2015-06-22T23:26:41Z

Author Name: Per Gammerath (Per Gammerath)

Try this these projects

8872 was configured as QGIS.zip

QGIS.zip (Per Gammerath)

qgib · 2015-06-22T23:31:46Z

Author Name: Saber Razmjooei (@saberraz)

It opens and saves pretty much instantly for wihtout layout.qgs and a couple of seconds with layout.qgs project on my QGIS 2.8.2.

qgib · 2015-06-22T23:36:37Z

Author Name: Per Gammerath (Per Gammerath)

On my computer, there is a 11 seconds difference. To emphasize the problem, try and the duplicate layouts, and you will see that it takes even longer.

qgib · 2015-06-22T23:39:22Z

Author Name: Per Gammerath (Per Gammerath)

Try this map

8873 was configured as With_layouts.qgs

With_layouts.qgs (Per Gammerath)

qgib · 2015-06-23T00:26:24Z

Author Name: Saber Razmjooei (@saberraz)

With 22 print layouts it loads in ~10 seconds for me now. With this no. of layout, I'd say it is acceptable!

qgib · 2015-06-23T00:36:10Z

Author Name: Per Gammerath (Per Gammerath)

You must have a really fast computer then. I'm running QGIS on a quad core 2.40 GHz, 4 GB ram. With 30 layouts, it takes about 30 seconds to load.
Nevertheless, I would like to know why QGIS has to process these layouts upon opening and not on demand?

qgib · 2015-06-23T02:50:40Z

Author Name: Saber Razmjooei (@saberraz)

status_id was changed from Feedback to Open

qgib · 2015-09-01T00:59:27Z

Author Name: Per Gammerath (Per Gammerath)

I see that this issue persists in QGIS Pisa. I still would like to know why the layouts can't be loaded on demand?

fixed_version_id was configured as Version 2.12

qgib · 2015-11-06T02:55:19Z

Author Name: Lene Fischer (@LeneFischer)

Times for saving project on local C: or network H:

Projekt with composers (1-2 composers) on C: 1-2 sec
Projekt with composers (1-2 composers)on H: 24 sec
Projekt without composers on c: < 1 sec
Projekt without composers on h: 9 sec

Test times are from save until mouse react again. Test in 2.12

qgib · 2015-11-06T03:40:24Z

Author Name: Giovanni Manghi (@gioman)

Per Gammerath wrote:

I see that this issue persists in QGIS Pisa. I still would like to know why the layouts can't be loaded on demand?

not sure about that, anyway I retested all the projects available in this ticket and loading time for projects with composers is somwhow slower than the projects without, but it is always in the few seconds range, really nothing that would led me to think about an issue to solve. I retested on master on both linux and windows.

qgib · 2015-11-06T03:40:30Z

Author Name: Giovanni Manghi (@gioman)

status_id was changed from Open to Feedback

qgib · 2015-11-07T10:25:59Z

Author Name: Giovanni Manghi (@gioman)

fixed_version_id removed Version 2.12

qgib · 2015-12-11T00:24:30Z

Author Name: Per Gammerath (Per Gammerath)

It seems that this issue is more acute when projects are loaded from a network drive, as Lene Fischer stated in comment #10098. Subjecting users to such slow speeds is quite unfortunate, especially when QGIS becomes unresponsive in Windows.

assigned_to_id was configured as Giovanni Manghi

qgib · 2015-12-11T00:36:38Z

Author Name: Per Gammerath (Per Gammerath)

I copied my project to a local drive and I realized that it didn't change the matter.

Project with 6 composers: 26 seconds
Project with 0 composers: 6 seconds

Please consider a solution for this annoyance.

qgib · 2015-12-26T06:38:56Z

Author Name: Giovanni Manghi (@gioman)

see also #20319 and in particular #20319 (comment)

resolution was changed from fixed/implemented to
subject was changed from Slow project loading in qgis 2.4/master when compared to 2.2 to project loading and adding/removing layers is slow if project has composers
status_id was changed from Feedback to Open
assigned_to_id removed Giovanni Manghi

qgib · 2016-05-31T00:21:05Z

Author Name: Kilian Hagemann (Kilian Hagemann)

OMG, can't believe I only found this bug now.

We deal with very large and complex (>150 layer) gis project files and they take forever to open. Now that I found this bug I was curious and did some benchmarks of our own:

test file: 193 layers and 106 print composers
test environment: QGIS 64bit on Windows 10, 8GB RAM, recent i3 dual core CPU (4 logical processors)
measured from end of missing layer dialogue (artificially inserted to remove file loading times which is disk/network dependent) to the time the canvas is rendered
with QGIS 2.8.8:
** timing WITH print composers: 00:16:25 with 30-45% CPU usage
** timing WITHOUT print composers: 00:00:03
with QGIS 2.14.3:
** timing WITH print composers: 01:09:56 with 30-45% CPU usage
** timing WITHOUT print composers: 00:00:04

So QGIS 2.8 takes about 9 seconds and QGIS 2.14 40 seconds (!) for each print composer saved inside the .qgs file. I hope this helps to stress the severity of this bug, so please could the QGIS devs look into the code that loads and parses the print composers and remove the bottlenecks? If the info provided by the OP and others isn't sufficient, I could help in reproducing the issue.

Until then I guess we will all remove print composers from our projects and manage them outside QGIS...

qgib · 2016-08-29T11:13:14Z

Author Name: Anita Graser (@anitagraser)

Also confirmed by http://gis.stackexchange.com/questions/208507/qgis-desktop-2-14-5-and-2-16-projects-with-multiple-composers-loading-very-slowl

version was changed from 2.8.2 to master

qgib · 2016-10-21T00:09:45Z

Author Name: Bjoern Gessler (Bjoern Gessler)

Got the same problem as well under Linux, as under Windows 7 and 10. QGIS VErsion 16.3 and 16.1.
I got a project with about 15 Print composers, some of them quite large (about 60 x 90 cm). The QGIS-project has risen to a size of about 16MB, loading the project takes about 20 Minutes on a i5-3,4 GHz computer with 8 GB of RAM.
Also saving the project sometimes takes up to 5 minutes.
Makes working sometimes really annoying...
Guess this might be a major issue witch then could be more or less simply fixed by not loading all map layouts into the memory at start up.

qgib · 2017-03-07T00:38:59Z

Author Name: Giovanni Manghi (@gioman)

There have been several improvements in both 2.18.* and master (qgis3) and all the provided projects with/without layouts now open in a very reasonable amount of time, especially on master.

status_id was changed from Open to Closed
resolution was changed from to fixed/implemented
version was changed from master to 2.18.4

qgib · 2017-03-20T19:18:38Z

Author Name: Nyall Dawson (@nyalldawson)

With the changes merged in #4279 all sample projects attached here load instantly.

qgib · 2017-11-10T09:50:27Z

Author Name: Ábel Kőszeghy (Ábel Kőszeghy)

My project files with many layouts still load painfully slow from network drives. I'm on version 2.18.14.

qgib added Bug Either a bug report, or a bug fix. Let's hope for the latter! Project labels May 25, 2019

qgib closed this as completed May 25, 2019

This was referenced May 25, 2019

Very poor performance with complex projects (saving, adding/removing layers) #20319

Closed

Print composer performance at startup #23132

Closed

Qgis Desktop 2.14.5 and 2.16 Projects with Multiple Composers Loading Very Slowly #23423

Closed

project loading and adding/removing layers is slow if project has composers #19203

project loading and adding/removing layers is slow if project has composers #19203

Comments

qgib commented Jul 6, 2014 • edited Loading

qgib commented Jul 7, 2014

qgib commented Jul 7, 2014

qgib commented Jul 7, 2014

qgib commented Jul 7, 2014

qgib commented Jul 7, 2014

qgib commented Jul 7, 2014

qgib commented Jul 7, 2014

qgib commented Jul 7, 2014

qgib commented Jul 7, 2014

qgib commented Jul 7, 2014

qgib commented Jul 8, 2014

qgib commented Jul 8, 2014

qgib commented Jul 8, 2014

qgib commented Jul 8, 2014

qgib commented Jul 8, 2014

qgib commented Jul 9, 2014

qgib commented Jul 9, 2014

qgib commented Jul 9, 2014

qgib commented Jul 9, 2014

qgib commented Jul 10, 2014

qgib commented Jul 10, 2014

qgib commented Jul 16, 2014

qgib commented Jul 16, 2014

qgib commented Jul 17, 2014

qgib commented Jul 17, 2014

qgib commented Jul 17, 2014

qgib commented Jul 17, 2014

qgib commented Jun 22, 2015

qgib commented Jun 22, 2015

qgib commented Jun 22, 2015

qgib commented Jun 22, 2015

qgib commented Jun 22, 2015

qgib commented Jun 22, 2015

qgib commented Jun 23, 2015

qgib commented Jun 23, 2015

qgib commented Jun 23, 2015

qgib commented Sep 1, 2015

qgib commented Nov 6, 2015

qgib commented Nov 6, 2015

qgib commented Nov 6, 2015

qgib commented Nov 7, 2015

qgib commented Dec 11, 2015 • edited Loading

qgib commented Dec 11, 2015

qgib commented Dec 26, 2015 • edited Loading

qgib commented May 31, 2016

qgib commented Aug 29, 2016

qgib commented Oct 21, 2016

qgib commented Mar 7, 2017

qgib commented Mar 20, 2017

qgib commented Nov 10, 2017

qgib commented Jul 6, 2014 •

edited

Loading

qgib commented Dec 11, 2015 •

edited

Loading

qgib commented Dec 26, 2015 •

edited

Loading