Logs are not sent in Laravel queues when use async log

[genivaldosilva]

Summary of problem or feature request

Logs from an asynchronous job in a Laravel queue are not sent to Elasticsearch until you shutdown the queue process.

This is because the php artisan queue:work does not close the process. Queue workers do not "reboot" the framework before processing each job.

Code snippet of problem

How to reproduce

1. Create logging config (config/logging.php).

'elasticsearch' => [
    'driver' => 'custom',
    'via' => \App\Logging\CreateElasticsearchLogger::class,
    'level' => env('LOG_LEVEL', 'debug'),
],

2. Create class \App\Logging\CreateElasticsearchLogger

<?php

declare(strict_types=1);

namespace App\Logging;

use Elastic\Elasticsearch\ClientBuilder;
use Monolog\Handler\ElasticsearchHandler;
use Monolog\Logger;

class CreateElasticsearchLogger
{
    /**
     * Create a custom Monolog instance.
     */
    public function __invoke(array $config): Logger
    {
        $hosts = ['host-to-elastic:9200'];
        $username = 'elastic';
        $password = 'password';
        $index = 'your-index-name';
        $type = '_doc';

        $client = ClientBuilder::create()
            ->setHosts($hosts)
            ->setBasicAuthentication($username, $password)
            ->setHttpClient(new \Symfony\Component\HttpClient\Psr18Client)
            ->build()
            ->setAsync(true);

        $options = [
            'index' => $index,
            'type' => $type,
            'ignore_error' => true,
        ];

        $handler = new ElasticsearchHandler($client, $options, $config['level'], true);

        $logger = new Logger(config('app.env'));
        $logger->setHandlers([$handler]);

        return $logger;
    }
}

3. Create a test job - App\Jobs\TestJob

<?php

declare(strict_types=1);

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Log;

class TestJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public function handle(): void
    {
        Log::info('The job has been executed.');
    }
}

4. Add test routes - routes/web.php

Route::get('test-log', function () {
    \Illuminate\Support\Facades\Log::info('The test log has been executed');
});

Route::get('test-log-job', function () {
    dispatch(new App\Jobs\TestJob());
});

5. Config .env

LOG_CHANNEL=elasticsearch
LOG_LEVEL=debug

QUEUE_CONNECTION=redis

6. Run the queue:

php artisan queue:work redis

7. Call GET /test-log
   You'll see:
   

8. Call GET /test-log-job
   You won't see, but when you kill the process:

kill 667

You will receive the log:

System details

• Operating System: Docker - PHP-8.2-alpine
• PHP Version: 8.2
• ES-PHP client version: ^8.10
• Elasticsearch version: docker image elasticsearch:7.17.13

[ezimuel]

@genivaldosilva sorry for my late reply. I noticed that you are using Elasticsearch server version 7.17.13 and elasticsearch-php version 8.10. You should use the latest version 7.17.2 of elasticsearch-php since you are using servion 7 of the server.
Please try and let me know, thanks.

[genivaldosilva]

Hi @ezimuel!
Same problem.