Add benchmarks

[autaut03]

Hey.

Some context: I'm the author of https://github.com/good-php/reflection, which is a reflection library like this one. As I don't have the resources needed to maintain it, I'm looking into replacing it with an open-source solution like yours :) My only use case for it right now is https://github.com/good-php/serialization, which is a serialization library. That requires good performance of the reflection.

So my proposal is to add benchmarks here, so we can know how performant this is and whether there's room for improvement or not :)

[vudaltsov]

Hi, @autaut03, first of all thank you for the interest in the Typhoon project.

I've made a simple benchmark via TheDragonCode/benchmark using Typhoon 0.3.3 and Good PHP v1.0.0-alpha.4:

<?php

use DragonCode\Benchmark\Benchmark;
use GoodPhp\Reflection\ReflectorBuilder;
use Symfony\Component\Filesystem\Filesystem;
use Typhoon\OPcache\TyphoonOPcache;
use Typhoon\Reflection\Cache\NullCache;
use Typhoon\Reflection\TyphoonReflector;

require_once __DIR__ . '/../../vendor/autoload.php';

$typhoonInMemory = TyphoonReflector::build();

$opcache = new TyphoonOPcache(__DIR__ . '/../../var/benchmark/cache');
$opcache->clear();
$typhoonFile = TyphoonReflector::build(cache: $opcache);

$goodInMemory = (new ReflectorBuilder())
    ->withMemoryCache()
    ->build();

(new Filesystem())->remove(__DIR__ . '/../../var/benchmark/cache');
$goodFile = (new ReflectorBuilder())
    ->withFileCache(__DIR__.'/../../var/benchmark/good')
    ->build();

Benchmark::start()
    ->compare([
        'native reflection' => static fn(): array => (new ReflectionClass(AppendIterator::class))->getMethods(),
        'typhoon, in-memory' => static fn(): array => $typhoonInMemory->reflectClass(AppendIterator::class)->getMethods(),
        'good, in-memory' => static fn(): mixed => $goodInMemory->forType(AppendIterator::class)->declaredMethods(),
        'typhoon, file' => static fn(): array => $typhoonFile->reflectClass(AppendIterator::class)->getMethods(),
        'good, file' => static fn(): mixed => $goodFile->forType(AppendIterator::class)->declaredMethods(),
    ]);

 ------- ------------------------ ------------------------ ------------------------ ------------------------ ------------------------ 
  #       native reflection        typhoon, in-memory       good, in-memory          typhoon, file            good, file              
 ------- ------------------------ ------------------------ ------------------------ ------------------------ ------------------------ 
  min     0.001042 ms - 0 bytes    0.012834 ms - 0 bytes    0.022625 ms - 0 bytes    2.137084 ms - 0 bytes    0.403083 ms - 0 bytes   
  max     0.008625 ms - 0 bytes    74.884041 ms - 6.00 MB   13.324542 ms - 2.00 MB   178.58125 ms - 2.00 MB   0.697625 ms - 0 bytes   
  avg     0.0011292 ms - 0 bytes   0.0140292 ms - 0 bytes   0.0252542 ms - 0 bytes   2.7485916 ms - 0 bytes   0.4597041 ms - 0 bytes  
  total   4.193458 ms              100.619458 ms            40.642917 ms             504.577959 ms            77.499084 ms            
 ------- ------------------------ ------------------------ ------------------------ ------------------------ ------------------------ 
  Order   - 1 -                    - 2 -                    - 3 -                    - 5 -                    - 4 -                   
 ------- ------------------------ ------------------------ ------------------------ ------------------------ ------------------------ 

Seems like Typhoon is on average faster with in-memory cache and worse with the file one. We already know, why file cache is performing worse (some architectural issues), it is already being fixed in the upcoming 0.4.0. I will conduct a new benchmark upon release.

0.4.x will also have it's own convenient API and a set of Adapters for native reflection. So if you are planning to release tools based on Typhoon, please, wait for 0.4.0 to avoid huge refactoring. I hope we'll land it it in early May. Then we are also planning to finish Hydrator (it is 80% complete in a private repo).

I'm sorry to hear that you don't have enough time to work on your project, some healthy competition is always good. We'll be happy if you use Typhoon in your projects and give us feedback and/or contribute.