NC16.0.3 Upload fails with files 2GiB in 32bit instances (again) #16431

nachoparker · 2019-07-17T00:36:23Z

NextCloudPi 1.15.0, NC16.0.3, Odroid HC2. Tested by uploading a 5GiB file through the web interface.

# uname -a
Linux nextcloudpi 4.14.127-odroidxu4 #61 SMP PREEMPT Mon Jul 1 20:41:09 CEST 2019 armv7l GNU/Linux

I know this was fixed by using chunking in NC13 but it is back now.

I can see the file being fully uploaded and then reconstructed in the destination folder (increasing size until full size).

Then the interface fails with this error and the file gets removed

NCP ticket

nextcloud/nextcloudpi#948

The text was updated successfully, but these errors were encountered:

F1orian · 2019-07-17T10:29:11Z

I get this error message when uploading through the web interface:
Odroid HC2 NC 16.0.3

And this when uploading through WebDAV:

[no app in context] Error: Sabre\DAV\Exception\BadRequest: expected filesize 2147483647 got 1861091328 at <>

/var/www/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php line 156
put(null)
/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 1096
createFile("Snap.7z", null)
/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php line 525
createFile("Snap.7z", null, null)
<>
httpPut(Sabre\HTTP\Reque ... "}, Sabre\HTTP\Response {})
/var/www/nextcloud/3rdparty/sabre/event/lib/EventEmitterTrait.php line 105
call_user_func_array([Sabre\DAV\CorePlugin {},"httpPut"], [Sabre\HTTP\Requ ... }])
/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 479
emit("method:PUT", [Sabre\HTTP\Requ ... }])
/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 254
invokeMethod(Sabre\HTTP\Reque ... "}, Sabre\HTTP\Response {})
/var/www/nextcloud/apps/dav/appinfo/v1/webdav.php line 80
exec()
/var/www/nextcloud/remote.php line 163
require_once("/var/www/nextcl ... p")

PUT /remote.php/webdav/Snap.7z
from 192.168.178.104 by ncp at 2019-07-17T10:41:35+00:00

I just did a new clean installation and tried the web Interface upload again:

The second error appeared approximately 100 times in the log.

zedocrob · 2019-07-17T19:36:03Z

Same issue here on Odroid HC2:
1- i can drag'ndrop a 4Gb file in my Browser and download it back correctly, but always got this error in logs:
`[PHP] Error: file_put_contents(): content truncated from 3915768813 to 2147483647 bytes at /var/www/nextcloud/lib/private/Files/Storage/Local.php#488

MOVE /remote.php/dav/uploads/xxx/web-file-upload-31c5297bca63498f17fedc95638e6c6f-1563389990100/.file
from 192.168.0.254 by xxx at 2019-07-17T19:05:36+00:00``

2-Using a webdav client (cyberduck, transmit...) i can't upload it, file chucking bug...
3-Also, I can't delete the 4Gb file in trash, i have this error:
[remote] Error: TypeError: Return value of OCA\Files_Trashbin\Sabre\AbstractTrash::getSize() must be of the type int, float returned at <<closure>>

i have to force trashbin:cleanup to delete it

violoncelloCH · 2019-07-21T21:44:50Z

cc @nextcloud/server-triage

kesselb · 2019-07-21T23:15:34Z

@nachoparker @F1orian #15095 #16138

@zedocrob #13160

F1orian · 2019-07-22T07:20:19Z

@kesselb May I ask why this issue was closed? I didn't see a strong resemblance in the issues you linked.
Thanks anyways for looking into this!

kesselb · 2019-07-22T14:23:35Z

expected filesize 2147483647 got 1861091328

Same as #15095. You tried to disable mod_reqtimeout and http2?

F1orian · 2019-07-25T09:29:58Z

@kesselb I disabled both mod_reqtimeout and http2. Still getting an error message and the upload fails

[webdav] Fatal: Sabre\DAV\Exception\BadRequest: expected filesize 2147483647 got 1025335296 at <>

/var/www/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php line 156
put(null)
/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 1096
createFile("Lightroom.7z", null)
/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php line 525
createFile("Lightroom.7z", null, null)
<>
httpPut(Sabre\HTTP\Reque ... "}, Sabre\HTTP\Response {})
/var/www/nextcloud/3rdparty/sabre/event/lib/EventEmitterTrait.php line 105
call_user_func_array([Sabre\DAV\CorePlugin {},"httpPut"], [Sabre\HTTP\Requ ... }])
/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 479
emit("method:PUT", [Sabre\HTTP\Requ ... }])
/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 254
invokeMethod(Sabre\HTTP\Reque ... "}, Sabre\HTTP\Response {})
/var/www/nextcloud/apps/dav/appinfo/v1/webdav.php line 80
exec()
/var/www/nextcloud/remote.php line 163
require_once("/var/www/nextcl ... p")

PUT /remote.php/webdav/Lightroom.7z
from 192.168.178.104 by ncp at 2019-07-25T09:24:10+00:00

kesselb · 2019-10-09T14:15:24Z

Confirmed for 17: #17485

delfer · 2019-10-09T16:04:02Z

Partial solution (file uploaded, but was truncated to 2GiB):

--- lib/private/Files/Storage/Local.php.orig	2019-10-09 15:40:47.571461212 +0000
+++ lib/private/Files/Storage/Local.php	2019-10-09 15:50:39.208620584 +0000
@@ -485,6 +485,12 @@
 	}

 	public function writeStream(string $path, $stream, int $size = null): int {
-		return (int)file_put_contents($this->getSourcePath($path), $stream);
+		$fz = fopen($this->getSourcePath($path), 'wb');
+		$count = 0;
+		while(($l=fread($stream, 65536))) {
+    			$count += fwrite($fz, $l);
+		}
+		fclose($fz);
+		return (int)$count;
 	}
 }

kesselb · 2019-10-09T16:18:54Z

Thanks for digging @delfer 👍

You could try to

server/lib/public/Files/Storage/IWriteStreamStorage.php

Line 39 in 93de637

public function writeStream(string $path, $stream, int $size = null): int;

remove : int there, in your above patch and return $count without the typecast.

delfer · 2019-10-09T18:25:45Z

@kesselb thank you for help. File is still truncated to 2GiB

-- lib/public/Files/Storage/IWriteStreamStorage.php.orig	2019-10-09 17:56:19.065499527 +0000
+++ lib/public/Files/Storage/IWriteStreamStorage.php	2019-10-09 17:56:36.965524710 +0000
@@ -36,5 +36,5 @@
 	 * @return int the number of bytes written
 	 * @since 15.0.0
 	 */
-	public function writeStream(string $path, $stream, int $size = null): int;
+	public function writeStream(string $path, $stream, int $size = null);
 }

--- lib/private/Files/Storage/Common.php.orig	2019-10-09 18:00:34.313848529 +0000
+++ lib/private/Files/Storage/Common.php	2019-10-09 17:58:42.645698305 +0000
@@ -820,7 +820,7 @@
 	 * @param int $size
 	 * @return int
 	 */
-	public function writeStream(string $path, $stream, int $size = null): int {
+	public function writeStream(string $path, $stream, int $size = null) {
 		$target = $this->fopen($path, 'w');
 		if (!$target) {
 			return 0;

--- lib/private/Files/Storage/Local.php.orig	2019-10-09 15:40:47.571461212 +0000
+++ lib/private/Files/Storage/Local.php	2019-10-09 17:55:53.401463198 +0000
@@ -484,7 +484,13 @@
 		}
 	}

-	public function writeStream(string $path, $stream, int $size = null): int {
-		return (int)file_put_contents($this->getSourcePath($path), $stream);
+	public function writeStream(string $path, $stream, int $size = null) {
+		$fz = fopen($this->getSourcePath($path), 'wb');
+		$count = 0;
+		while(($l=fread($stream, 65536))) {
+    			$count += fwrite($fz, $l);
+		}
+		fclose($fz);
+		return $count;
 	}
 }

delfer · 2019-10-17T09:52:34Z

It can be fixed only by changes in PHP image. docker-library/php#901

delfer · 2019-10-24T15:31:17Z

Fixed by docker-library/php#901
Tested in Docker image nextcloud:17-apache (nextcloud@sha256:58836d449406b396c1c119c2661577fd7d8284ff41a7d181a0e2be232d2757fc, 2019-10-24T05:04:49.946856094Z)
Can be closed.

kesselb · 2019-10-24T16:07:50Z

Can be closed.

nextcloud/docker ✔️
nextcoud/nextcloudpi ❌ (does not inherit from library/php)

Probably they need to recompile php with the flags -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64. @nachoparker is it okay with you to move this issue to nextcloudpi?

Thanks @delfer for the research 👍

nachoparker · 2019-10-25T03:01:56Z

already there nextcloud/nextcloudpi#948

kesselb · 2019-10-25T13:13:58Z

Okay with you to close this issue then? @nachoparker @F1orian

nachoparker · 2019-10-27T04:18:01Z

I am not sure, rebuilding PHP is not always feasible, so if NC doesn't do anything about it the bug is going to remain there.

Ideally this would be worked around as it has been in the past.

kesselb · 2019-10-27T18:10:58Z

Index: lib/private/Files/Storage/Local.php
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- lib/private/Files/Storage/Local.php	(revision 83af640780fd1532316b07bdfc68fd570c487a2c)
+++ lib/private/Files/Storage/Local.php	(date 1572199369066)
@@ -218,7 +218,15 @@
 	}
 
 	public function file_put_contents($path, $data) {
-		return file_put_contents($this->getSourcePath($path), $data);
+		$handle = $this->fopen($path, 'w');
+		if ($handle === false) {
+			return false;
+		}
+
+		$result = fwrite($handle, $data);
+		fclose($handle);
+
+		return $result !== false;
 	}
 
 	public function unlink($path) {
Index: apps/dav/lib/Connector/Sabre/File.php
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- apps/dav/lib/Connector/Sabre/File.php	(revision 83af640780fd1532316b07bdfc68fd570c487a2c)
+++ apps/dav/lib/Connector/Sabre/File.php	(date 1572197457791)
@@ -172,7 +172,7 @@
 				$this->changeLock(ILockingProvider::LOCK_EXCLUSIVE);
 			}
 
-			if ($partStorage->instanceOfStorage(Storage\IWriteStreamStorage::class)) {
+			if (PHP_INT_SIZE === 8 && $partStorage->instanceOfStorage(Storage\IWriteStreamStorage::class)) {
 
 				if (!is_resource($data)) {
 					$tmpData = fopen('php://temp', 'r+');

fopen and fwrite is able to write files bigger than PHP_INT_MAX (update: if php is compiled with -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64) . Additionally we need to disable IWriteStreamStorage for 32bit because the methods are type hinted. Don't know if such a patch is acceptable.

We probably could write some own storage adapter Local32 without the IWriteStreamStorage and all the knowledge how to read / write / determine file size / etc. for 32bit systems. Local would also benefit because some 32bit specific code could be removed.

Perhaps https://wiki.php.net/rfc/bigint will make it's way to php sometime.

delfer · 2019-10-30T15:00:14Z

@kesselb fopen and fwrite is NOT able to write files bigger than MAX_INT.
It is limitation of any program build with gcc, not only php.
You should look at my test algorithm and results in docker-library/php#901

kesselb · 2019-10-30T15:07:48Z

Tested the above code on my rpi3 (with nextcloudpi) and were able to write files > 2147483647. Have you tried the above patch? Is there a way to figure out if php is compiled with -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64?

delfer · 2019-10-31T07:23:05Z

This code:

<?php
$f = fopen('largefile.tmp', 'wb');
for ($i = 0; $i < 2048+2; $i++) {
	$c = fwrite($f, pack('x1048576'));
	if ($c == 0) {
		trigger_error("Can not write large file", E_USER_ERROR);
	}
}
fclose ($f);

Command pack('x1048576') generates 1M of zeroes.

On Pi3 PHP compiled without flags after writing 2147483647 bytes writes 0 bytes every call. When compiled with flags - it writes as much bytes as in argument.

kesselb · 2019-10-31T14:22:30Z

Above code writes a 2149580800 bytes file. Added echo $i . PHP_EOL and it iterates till the end.

<?php
$f = fopen('largefile.tmp', 'wb');
for ($i = 0; $i < 4096+2; $i++) {
        echo $i . PHP_EOL;
	$c = fwrite($f, pack('x1048576'));
	if ($c == 0) {
		echo "zer0 bytes written" . PHP_EOL;
	}
}
fclose ($f);

Changed your code like above and run it again: Writes a 4297064448 bytes file.

pi@nextcloudpi:~ $ php -r "var_dump(PHP_INT_MAX);"
int(2147483647)
pi@nextcloudpi:~ $ php -r "var_dump(PHP_INT_SIZE);"
int(4)

delfer · 2019-10-31T15:42:08Z

@kesselb can you please check php -r "phpinfo();" | grep PHP_CFLAGS

thrubovc · 2020-01-08T06:35:59Z

Yesterday I opened bug #18720 which was referenced to this one. What's funny is that I can upload/download anything below 4GB, not just 2GB as this bug reports. (dockerized Nextcloud on RPI4). Or am I just missing something?

StableNarwhal · 2020-03-16T21:01:13Z

Yesterday I opened bug #18720 which was referenced to this one. What's funny is that I can upload/download anything below 4GB, not just 2GB as this bug reports. (dockerized Nextcloud on RPI4). Or am I just missing something?

Did you fix it? Having the same problem

thrubovc · 2020-03-16T21:33:30Z

Nope, never got it to work on 32bit raspbian. I have, however, switched to Manjaro which is fully 64bit. Nextcloud is running on it flawlessly

StableNarwhal · 2020-03-16T21:35:23Z

Thank your for your answer. Guess I'll tinker around a bit then. Cheers!

encodeering · 2020-05-17T08:30:38Z

Hi,

also run into the trunction issue on a armhf odroid hc1 and just wanted to share my configuration, as it might help to solve this.

Versions

PHP 7.3.18 (cli) (built: May 16 2020 14:51:26) ( NTS )
Nextcloud 18.0.4

php -r "phpinfo();" | grep PHP_CFLAGS
PHP_CFLAGS => -fstack-protector-strong -fpic -fpie -O2 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
$_SERVER['PHP_CFLAGS'] => -fstack-protector-strong -fpic -fpie -O2 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64

php -r "var_dump(PHP_INT_MAX);"
int(2147483647)

php -r "var_dump(PHP_INT_SIZE);"
int(4)

uname -m -r
4.14.176-1-ARCH armv7l

Upload of a ~3.5GB is fine at browser level, but the following erros occurs later

file_put_contents(): content truncated from 3706877836 to 2147483647 bytes at /var/www/html/lib/private/Files/Storage/Local.php#503
--

Nextcloud is behind a nginx proxy and uses a php-fpm configuration.

Anything else that could be useful?

ghost · 2020-08-03T14:41:01Z

hi guys, i have the same problem and i can't resolve it :(

Hardware: Raspberry pi4 2gb
Software: 32bit raspbian
Nextcloud 19.0.1 running on apache with php7.3

I already changed the php.ini settings, request timeout in apache, etc.. I saw that it was resolved in nextcloud 13, so it may be possible to fix it again? A cloud solution with a 2gb upload limit would be strange... :(

thrubovc · 2020-08-03T14:55:51Z

hi guys, i have the same problem and i can't resolve it :(

Hardware: Raspberry pi4 2gb
Software: 32bit raspbian
Nextcloud 19.0.1 running on apache with php7.3

I already changed the php.ini settings, request timeout in apache, etc.. I saw that it was resolved in nextcloud 13, so it may be possible to fix it again? A cloud solution with a 2gb upload limit would be strange... :(

get ubuntu. it's fully 64bit. it's a little heavier, but very similar to raspbian.

ghost · 2020-08-03T15:08:39Z

I saw your comment about Manjaro, and I was thinking about changing OS for 64 bit. But I still have to confirm, that everything I use works well on ubuntu too (pihole, smtp, etc..)

BTW I'm still hoping to figure it out for Raspbian, as I did put a lot of energy in my current solution 😄 Speaking of Raspbian, Rpi OS 64 bit is still in a very early stage, I would not risk it for a daily use yet.

I did a little search, but didn't find the issue where uploading more than 2gb was solved in NC13 for 32 bit devices. I hope to find something in that version to get our problem solved again.

ghost · 2020-08-05T19:05:53Z

I spent the last 2 days fixing this problem. The result was:

There is no way to get nextcloud 19 working with >2gb files on Raspbian (or any 32bit arm based os basically).
I tried to install an ubuntu arm64 version, but it is too much for the Raspberry pi4. After a few freezes, a lot of compatibility issues (no working vnc solution, lot of missing arm packages) i gave up, ubuntu is not the way to go.
I came back to Raspbian 32 bit, and tried to downgrade my php version from 7.3 to 7.2. This caused a loop on the login screen of nextcloud.
I tried php 7.3 with my owncloud instance. The login was fine, but file uploading behaved the same as in nextcloud (was not able to upload files bigger than 2gb).
I downgraded php to version 7.2 on my owncloud instance. Suddenly, the problem was solved, i had no issues uploading my 2.5 gb test file.

Regarding the original nextcloud problem, the solution will be around php changes between version 7.2 and 7.3. I'll not dig deeper into this now, i'll stick with my owncloud for a while. I'll be happy to come back to nextcloud, if the problem will be solved :)

thrubovc · 2020-08-05T19:10:10Z

That's a bummer to hear. Ubuntu did feel a little heavier for rpi4, slower logins, services running by default that I didn't need, and the darn disks would never spin down... but I didn't have any major issues on it. I reverted to manjaro and its fast and I think the community is really good, including the upstream arch linux one.

skjnldsv · 2020-09-09T11:30:39Z

Hey! What is the status here?

ghost · 2020-09-09T11:56:17Z

Hi,
it's all the same. the uploader doesn't like big files on any 32 bit operating system. :(
Tried it with Manjaro 32 bit and Raspbian 32 bit on a Raspberry pi4.

thrubovc · 2020-09-09T11:59:42Z

I wouldn't count on the Nextcloud team to fix this as 32bit OSes are slowly dying out and there are higher priorities than working on 32bit-related bugs. Thankfully you have a handful of fully 64bit OSes to replace Raspbian with (I presume you're running Raspbian).

skjnldsv · 2020-09-09T12:07:16Z

Yeah, unfortunately, I think I'll close this as wontfix. :/
Even arm are now almost all 64bits 🤷

pjft · 2020-09-22T10:37:45Z

Even though this was closed recently, I just wanted to say that the code shared by @kesselb solves the problem perfectly on #22936 . Thank you.

githule · 2020-09-24T10:41:01Z

I wouldn't count on the Nextcloud team to fix this as 32bit OSes are slowly dying out and there are higher priorities than working on 32bit-related bugs. Thankfully you have a handful of fully 64bit OSes to replace Raspbian with (I presume you're running Raspbian).

Should tell this to lots of user that build/bought Pi based offers around nextcloud. Maybe we should update requierments or catch the exception...

thrubovc · 2020-09-24T10:43:03Z

I wouldn't count on the Nextcloud team to fix this as 32bit OSes are slowly dying out and there are higher priorities than working on 32bit-related bugs. Thankfully you have a handful of fully 64bit OSes to replace Raspbian with (I presume you're running Raspbian).

Should tell this to lots of user that build/bought Pi based offers around nextcloud. Maybe we should update requierments or catch the exception...

I am running a fully 64bit OS on my RPI. Nobody said you're stuck with 32bit if you bought a RPI.

githule · 2020-09-24T10:56:14Z

I wouldn't count on the Nextcloud team to fix this as 32bit OSes are slowly dying out and there are higher priorities than working on 32bit-related bugs. Thankfully you have a handful of fully 64bit OSes to replace Raspbian with (I presume you're running Raspbian).

Should tell this to lots of user that build/bought Pi based offers around nextcloud. Maybe we should update requierments or catch the exception...

I am running a fully 64bit OS on my RPI. Nobody said you're stuck with 32bit if you bought a RPI.

Ok if you have a recent pi. There are also alternate boards.
Maybe at least the UI must be aware of a max upload size / download setting ? Indeed, however a recent Pi or alternate board can technically handle big files, Its not a good thing to let people upload very big file considering the storage/computing capacity.

Draco-Zero · 2022-03-13T11:19:09Z

I had the same problem on PI. Switching to arm64 and changing the PHP version to 8.0 fixed it.

nachoparker added bug 0. Needs triage Pending check for reproducibility or if it fits our roadmap labels Jul 17, 2019

nachoparker mentioned this issue Jul 17, 2019

Can't upload files larger than 2Gb on Odroid HC2 nextcloud/nextcloudpi#948

Closed

violoncelloCH added the feature: dav label Jul 21, 2019

kesselb closed this as completed Jul 21, 2019

kesselb reopened this Jul 25, 2019

delfer mentioned this issue Oct 9, 2019

Can not upload file to 32-bit host #17485

Closed

kesselb mentioned this issue Jan 7, 2020

"Error while copying file to target location (copied bytes: 0, expected filesize: -1 )" #18720

Closed

nachoparker mentioned this issue Mar 20, 2020

Content Truncated reported (but wrong report ?!) nextcloud/nextcloudpi#1103

Closed

skjnldsv added the needs info label Sep 9, 2020

skjnldsv closed this as completed Sep 9, 2020

skjnldsv added feature: filesystem and removed 0. Needs triage Pending check for reproducibility or if it fits our roadmap needs info labels Sep 9, 2020

macearl mentioned this issue Sep 9, 2020

Wrong or no sizes of files/folders #5031

Closed

Vigasto mentioned this issue Sep 22, 2020

Error assembling chunks from large uploads (>4GB) #22936

Closed

kesselb mentioned this issue Feb 24, 2021

Uploading large files fails with nextcloud 20.0.6 on raspberry pi 2 with 32bit cpu #25782

Closed

ZachHandley mentioned this issue Jul 22, 2024

Bring back 32 bit support on Nextcloud 26 #36072

Closed

20 tasks

NC16.0.3 Upload fails with files 2GiB in 32bit instances (again) #16431

NC16.0.3 Upload fails with files 2GiB in 32bit instances (again) #16431

Comments

nachoparker commented Jul 17, 2019 • edited Loading

F1orian commented Jul 17, 2019 • edited Loading

zedocrob commented Jul 17, 2019 • edited Loading

violoncelloCH commented Jul 21, 2019

kesselb commented Jul 21, 2019

F1orian commented Jul 22, 2019

kesselb commented Jul 22, 2019

F1orian commented Jul 25, 2019 • edited Loading

kesselb commented Oct 9, 2019

delfer commented Oct 9, 2019 • edited Loading

kesselb commented Oct 9, 2019

delfer commented Oct 9, 2019

delfer commented Oct 17, 2019 • edited Loading

delfer commented Oct 24, 2019 • edited Loading

kesselb commented Oct 24, 2019 • edited Loading

nachoparker commented Oct 25, 2019

kesselb commented Oct 25, 2019

nachoparker commented Oct 27, 2019

kesselb commented Oct 27, 2019 • edited Loading

delfer commented Oct 30, 2019 • edited Loading

kesselb commented Oct 30, 2019

delfer commented Oct 31, 2019

kesselb commented Oct 31, 2019

delfer commented Oct 31, 2019

thrubovc commented Jan 8, 2020

StableNarwhal commented Mar 16, 2020

thrubovc commented Mar 16, 2020

StableNarwhal commented Mar 16, 2020

encodeering commented May 17, 2020

ghost commented Aug 3, 2020

thrubovc commented Aug 3, 2020

ghost commented Aug 3, 2020 • edited by ghost Loading

ghost commented Aug 5, 2020 • edited by ghost Loading

thrubovc commented Aug 5, 2020

skjnldsv commented Sep 9, 2020

ghost commented Sep 9, 2020 • edited by ghost Loading

thrubovc commented Sep 9, 2020

skjnldsv commented Sep 9, 2020

pjft commented Sep 22, 2020

githule commented Sep 24, 2020

thrubovc commented Sep 24, 2020

githule commented Sep 24, 2020 • edited Loading

Draco-Zero commented Mar 13, 2022

nachoparker commented Jul 17, 2019 •

edited

Loading

F1orian commented Jul 17, 2019 •

edited

Loading

zedocrob commented Jul 17, 2019 •

edited

Loading

F1orian commented Jul 25, 2019 •

edited

Loading

delfer commented Oct 9, 2019 •

edited

Loading

delfer commented Oct 17, 2019 •

edited

Loading

delfer commented Oct 24, 2019 •

edited

Loading

kesselb commented Oct 24, 2019 •

edited

Loading

kesselb commented Oct 27, 2019 •

edited

Loading

delfer commented Oct 30, 2019 •

edited

Loading

ghost commented Aug 3, 2020 •

edited by ghost

Loading

ghost commented Aug 5, 2020 •

edited by ghost

Loading

ghost commented Sep 9, 2020 •

edited by ghost

Loading

githule commented Sep 24, 2020 •

edited

Loading