How is it so fast

[pullandfork]

At a high level, when UBO starts up, it loads hundreds of rules into memory. At runtime, when each web page is loaded, each of the URLs are compared with each of the rules and user preferences to determine whether to block or allow. All this happens in milliseconds!

How does this happen so fast? Is there some flowchart that covers the technical design and data/logic flow?

[gwarser]

https://www.reddit.com/r/uBlockOrigin/comments/17301xs/how_is_it_so_fast/

[pullandfork]

Yes, that's my thread. I did not get clarity so I asked here. Thanks

[stephenhawk8054]

What's not clear with the answer in this?

https://www.reddit.com/r/uBlockOrigin/comments/mdh9jz/order_of_complexity_for_network_filters/gsa7gd2/

[pullandfork]

That's very helpful for someone writing filter rules. I'm looking for an overview of the implementation, the design approach. How long do the various steps take e.g. load a list, process a specific URL, etc.

[gothic-bum]

Is there some flowchart that covers the technical design and data/logic flow?

I'm looking for an overview of the implementation, the design approach.

https://github.com/gorhill/uBlock/wiki/Overview-of-uBlock's-network-filtering-engine

https://github.com/gorhill/uBlock/wiki/Overview-of-uBlock's-network-filtering-engine:-details

https://github.com/gorhill/uBlock/wiki/Does-uBlock-Origin-block-ads-or-just-hide-them%3F

https://github.com/gorhill/uBlock/wiki/Doesn't-uBlock-Origin-add-overhead-to-page-load%3F

---

How long do the various steps take e.g. load a list, process a specific URL, etc.

https://github.com/gorhill/uBlock/wiki/Tools

https://github.com/gorhill/uBlock/wiki/Dashboard%3A-Support > Troubleshooting Information > More

https://profiler.firefox.com/

https://developer.chrome.com/docs/devtools/#performance

[pullandfork]

Lovely! Thanks, that should be sufficient.

[gorhill]

each of the URLs are compared with each of the rules

There are two main performance-critical components in uBO, which is the static filtering engine and cosmetic filtering engine.

The main one which is critical performance-wise is the static filtering engine, which has been highly optimized over the years. The cosmetic filtering engine is something completely different, working on the DOM, and there is no need to match against all network requests.

In the Support pane, there is a More button which opens a tool page and which purpose is to assist in debugging filtering engine issues, but you can use it to have a sense of how the static filtering engine is structured internally by clicking the SNFE: Dump button.

There is also a benchmark button, which will benchmark your current filterset against a collection of URLs (here is the way to feed uBO with a set of URLs from an external source).

When I use the built-in benchmark, after having setup the advanced setting benchmarkDatasetURL, I get an average of 8 µs per request on my side when using default lists (the result will depend on your CPU performance and the enabled lists).

If I had to describe why it's so fast in as few words as possible, I would say it's because it mostly just deals with integer numbers at the core -- i.e. it avoids as much as possible using JS data structures (other than typed arrays) to encode the static network filters in memory.

How the filters are encoded in those typed arrays is also another optimization level.

For example, filters which are just plain hostnames are the most common sort of filter, and are encoded in typed array-based tries (a single typed array is used for all tries). This is one of the major optimization, and to demonstrate this, just import StevenBlack/hosts as a list (which contains over 130K hostnames), and run the benchmark again: no difference, I still get an average of 8 µs per request.

But not all static network filters are just plain hostname, and various optimized representations are used according to the most common type of filters found in filter lists, which you can see under Filter class stats when you dump the content of the static network filtering engine using the SNFE: Dump button.

[pullandfork]

Thank you, appreciate you jumping in!